Cognipeer Console

Appearance

1.2 line ​

June 2026 – present. The current line.

This is where Console grew from a gateway with a dashboard into a platform: the Agent Sandbox, the open-source split, a long security pass on tenant isolation, Web Search, MCP Hubs, and Cost & Optimization.

v1.2.26-community ​

16 August 2026

Traces now record what shape the answer was required to take. A model call has two halves that decide its output: the tools it was offered, and the structured-output contract it was held to. Only the first was ever captured. So a reply that was not valid JSON looked identical whether a schema had been enforced and the model failed it, or nothing had ever asked for JSON — and that is the difference between a bug and a design choice.

Every model-call event now carries a response_format section: the contract type, the schema name, whether it was strict, and the schema itself. It is recorded per call rather than per session, because an agent can enforce a schema on its final turn only. The tracing screen shows it next to the tool menu.

It arrives by every route. The Cognipeer agent-sdk emits it directly — for the native response_format path and for providers without one, where the SDK enforces the same contract through a forced tool call and now says so. OTLP ingestion synthesises it from OpenInference's llm.invocation_parameters and the OTel GenAI gen_ai.output.type + gen_ai.request.structured_output_schema pair. The @cognipeer/observability packages (JS and Python) map it from LangChain, the Vercel AI SDK, and OpenAI Agents.

Two logging fields that explain a bad answer. finishReason — length, meaning the model hit its output ceiling, is the single most common cause of truncated and unparseable JSON, and without it that failure is indistinguishable from a model that simply answered badly. It is shown as a badge on the event when the stop was abnormal. And reasoningTokens, a subset of the output count that is routinely most of the output bill on a reasoning model while being invisible in the response text. It is recorded for attribution only and deliberately not added to the bill — the tokens are already inside outputTokens, and counting them again would overcharge.

Gateway request logs keep the JSON schema, not just its name. The logged request contract recorded that a schema existed; a replay needs the schema itself. It is now kept under its own size budget, so an enormous schema drops without taking the messages down with it.

Why this matters beyond the trace screen

The captured contract flows onward. Traffic Snapshots copy it onto each dataset item, and evaluation suites and prompt-optimizer runs send it back on the wire. A test that replays production traffic without production's schema is measuring a looser system than the one you run — which is exactly how a JSON-shape regression survives a green suite.

Evaluation can replay a conversation turn by turn. A suite has always sent the whole recorded prefix and graded one answer. That isolates each decision against a known-good history — cheap, and the cleanest regression signal — but it structurally cannot catch drift: an agent that answers every turn correctly in isolation and still loses the thread once it is reading its own output passes with full marks. The new turn by turn mode drives the conversation and feeds the model its own answers back, at one model call per user turn. Recorded tool exchanges still replay verbatim — a tool result is a fact about the environment a test cannot regenerate — and only the final answer is graded, so scores stay comparable between the two modes. Single-turn datasets behave identically under both.

A JSON-shape scorer. Similarity scoring happily passes a reply that kept the gist and dropped required fields. This one grades structural conformance against the reference output.

Evaluation targets can override the system prompt. Snapshot items embed the prompt they were recorded with, so without an override every run re-tests the prompt already in production. A target now takes a literal prompt or a promptKey resolved per run — promote a new version and the next run tests it. Targets also carry their own response_format and output-token ceiling.

Datasets can be labeled by AI, and sliced by those labels. An analysis definition's field set is a label taxonomy; pointing a run at a dataset instead of the conversation corpus turns the same engine into a labeler. Labels land on each item as queryable key/value pairs, with a distribution panel and filters over them. A human edit always wins: reviewer corrections are stamped as such and later AI runs leave them alone.

Datasets can be cloned into golden sets. Label a captured corpus, correct what matters, then copy the reviewed slice into a new dataset — filtered by segment, by whether a human confirmed it, and by whether the item has anything to grade against. It is a copy rather than a view on purpose: a golden set has to be stable, or tomorrow's traffic capture silently changes what the regression suite means and two runs a week apart stop being comparable.

Prompt optimizer inherits the captured contract. With no response_format configured, a run now optimizes under the contract the captured traffic actually ran with, and records which contract it used — that fact changes what the scores mean, and nothing else on the run said it.

Catalog price fill is reachable from the model screens. The component shipped in 1.2.25 but was never wired into create or edit.

agent-sdk upgraded from 0.7.0 to 0.9.3

Console's own agent runtime moves two minor versions. The only behavioural change that reaches it is maxParallelTools on the balanced profile, rescaled 2 → 5 in agent-sdk 0.8.0 — Console pins the other rescaled limits explicitly. Agents with several tools may now execute more of them concurrently.

v1.2.25-community ​

16 August 2026

Edition change

Cost & Optimization is now an enterprise module in full. It had been shipping in the community edition; it should not have been. The screens, the services and the APIs all moved to the enterprise overlay — not gated at the API layer with the code left behind, but moved.

A community installation keeps the service-catalog entry with an upsell page behind it. /api/cost/* and /api/prescriptions/* answer HTTP 402 naming the module they require, rather than rendering a broken page.

If you are running the community edition and using these screens, this release removes them. Nothing else changes: spend is still recorded, usage_daily is still written, quotas and Model Hub figures are unaffected. Two pieces stayed behind deliberately, because other things depend on them — price resolution for externally-priced models, which runs on the ingest path that writes usage_daily, and the /api/model-price-catalog* route, which also backs the Model Hub.

Prescriptions stops calling itself Analysis. The report screens spoke entirely in the language of the Analysis workbench sitting one click away — "Analysis: <subject>", "New analysis", "Analysis window" — which made the two indistinguishable. "Analysis" now means the workbench; these are reports and say so.

Prescription narratives render as markdown. The narrator writes headings, lists and emphasis. Rendering that as preformatted text turned every report into one grey wall.

Reports can be focused on a single model or agent. A new Everything / By model / By agent control scopes the trend, the totals and both breakdowns together, so a report about one agent no longer sits above tables describing everything else.

Analysis no longer prescribes an SDK upgrade when no per-turn tool menu was recorded. Plenty of people trace through OpenTelemetry or their own ingest; it states the observation instead.

Dense tables became readable. Badge clusters were laid out without wrapping, which shrank each badge until its label ellipsised to something like "P.." — coloured, present and completely unreadable. They wrap now and refuse to shrink.

Release notes. This section. The old Changelog page had not been touched since June and documented a version numbering that was retired; it now points here.

v1.2.24-community ​

15 August 2026

The container image build stops dying silently. Builds had begun failing with no error output at all — the step just ended and the log came back empty. The cause was the build process sizing its memory from the host machine rather than from the container it was running in, growing past what was actually available, and being killed rather than reporting a problem. The build now declares an explicit memory ceiling below the container's limit, so it collects garbage before it hits the wall instead of dying at it.

No application code changed in this release.

v1.2.23-community ​

15 August 2026

Automated prescriptions. Cost & Optimization gained a recommendation engine that runs on a schedule instead of waiting to be asked. It reads your recorded traffic and produces a written report: which agents are costing more than their peers, where a prompt is carrying dead weight, which tool definitions are being paid for on every turn and never called. Seventeen detectors, robust statistics so a single outlier day does not drive a recommendation, and a narrator that writes the findings in prose rather than leaving you a table to interpret.

Workload signals. The same analysis surfaces what a workload actually demands — how long the inputs run, whether responses are structured, whether tools are involved, how much of the traffic is non-English. These are what make a model recommendation defensible rather than a guess based on price alone.

Documentation. Cost & Optimization and the Cost & Prescriptions API reference were written for this release.

v1.2.22-community ​

15 August 2026

Cost & Optimization: from production traffic to a measured model decision. The loop closes. You can take a deterministic sample of real production traffic, put it through a mandatory PII gate before it is stored, and then replay that sample against candidate models and score the results. The point is that the answer comes from a measurement, not a projection: a cheaper model that a spreadsheet says will save you 60% will show up here failing the tests, if that is what it does.

The PII gate is not optional and not a checkbox. Traffic is masked or replaced with pseudonyms before it lands in a snapshot; the pseudonym form preserves the fact that two mentions refer to the same entity, so a conversation still makes sense after redaction, and the salt used to generate them is never written down. Sampling is deterministic — a hash of each row — so the same window produces the same sample and a result can be reproduced.

Dataset import from OpenAI, gateway, Bedrock and Langfuse exports, for evaluation sets you already have.

A tool-call scorer. Scoring the trajectory — which tools the model chose and in what order — not only the final text.

Defects found by running the documented workflow ​

Writing the guides meant executing them, and executing them found five things that had shipped without working.

  • Model Hub spend read zero on SQLite. The aggregate never selected any cost field, while the MongoDB implementation summed it correctly. Two database implementations, one of them silently returning a different answer.
  • A snapshot from a LangChain agent had no reference answer. The recorded response was read from one envelope shape only, so serialized LangChain messages produced dataset items with nothing to compare against — silently.
  • The tool-call scorer could not be selected. The scorer, its tests and the snapshot builder's expected-tool-call output all shipped, but the endpoint's list of valid scorers was a hand-kept copy that omitted it. It now derives from the supported list rather than duplicating it.
  • An assertion scorer with no assertions passed everything. It reported success on every item rather than reporting that it had nothing to check.
  • A collapsible section header was keyboard-inaccessible — it announced itself as a button but could not be focused or activated from the keyboard.

A How-To section. How-To is a set of task-shaped guides written for people who arrived wanting to do something specific rather than to read a reference page — connecting an OpenAI-compatible client, tracing an existing agent, crawling a site, automating a browser task, and a long walkthrough of optimizing token usage end to end.

Broken documentation links are now build failures. Dead-link checking had been disabled, and it was quietly hiding an entire sidebar section pointing at pages that did not exist. It is on, and the pages exist.

v1.2.21-community ​

15 August 2026

A defect-fix release, and a bigger one than its size suggests. Several of these were settings that appeared to work and did nothing.

Request parameters ​

The gateway narrowed every chat request down to four fields before sending it on. Two consequences: models that reject a particular parameter returned a 400 with no way to express that, and anything outside the OpenAI schema could not be sent at all.

  • A registry of parameters each model rejects, per provider driver and model id, unioned with a manual per-model list and switchable off per model.
  • top_p, presence_penalty and frequency_penalty are actually forwarded. They were collected from the request and logged as if they had been applied, then dropped — the Playground's Top P slider was a shipped no-op.
  • The token budget survives when a model rejects max_tokens under that name and wants a different key.
  • Per-model default parameters and opt-in caller passthrough, so fields specific to vLLM and SGLang can reach the upstream.
  • The agent path uses the same resolver. It had hard-coded temperature 0.7, ignored the model's settings entirely, and passed top_p and max_tokens under key names nothing read — so agent Top P and Max Tokens had never taken effect.
  • Retries stop multiplying. The client library's own retry budget nested inside Console's allowed up to 21 upstream calls for one request, 18 of them invisible to the circuit breaker.

Streaming ​

The OpenAI-compatible stream was non-compliant enough that strict clients could not consume it, and it was losing data on the way.

  • Streamed requests were logged with zero tokens. Usage was being read from keys the streaming path does not populate — and because budget and rate-limit updates are gated on usage being present, they were skipped entirely for every streamed request.
  • Tool calls appeared to be called with empty parameters whenever streaming was on. Argument fragments are now emitted as deltas as they arrive.
  • One completion id across all frames, the assistant role on the opening frame, always a terminal finish reason, and string content.
  • A mid-stream failure is delivered as an error frame followed by [DONE] instead of destroying the socket, and the upstream request is aborted when the client disconnects.

If you bill or rate-limit on recorded usage and you serve streamed traffic, this is the release that makes those numbers correct.

v1.2.20-community ​

14 August 2026

Support handoff. Console had no route to Support, so operators fell back to email. A signed-in user can now be handed into Cognipeer Support with their identity already established. The server exchanges its own secret for a single-use code; the browser only ever receives the resulting URL. On-premises deployments that know the Support URL but hold no credentials fall back to the Support login rather than failing the action.

A release runbook. Releasing documents the order the two channels are released in and why it cannot be reversed.

v1.2.19-community ​

4 August 2026

A version alignment release. No application changes.

v1.2.16-community ​

3 August 2026

Vision inputs are normalized, and image data is redacted from logs. Images arriving in different shapes are normalized to one internal form, and the image payloads themselves no longer end up in persisted request logs — a base64 image in a log line is both a privacy problem and a storage problem.

v1.2.15-community ​

3 August 2026

OpenAI-compatible inference errors are normalized. Errors from OpenAI-compatible providers were reaching callers in whatever shape the upstream happened to use. They now come back in one consistent form.

v1.2.14-community ​

31 July 2026

The largest release in the line. It covers everything from the initial open-source release through the end of July, so it is grouped by theme rather than listed chronologically.

Breaking change

The JS Sandbox module has been removed. The in-process JavaScript execution runtime and its API endpoints are gone. Code execution moved to the Agent Sandbox, which runs work in a real container instead of inside the server process. If you were calling the JS Sandbox endpoints, migrate to the Agent Sandbox.

Alongside the removal, outbound HTTP from tools and integrations now goes through a shared SSRF guard — requests to private network ranges are refused unless the integration explicitly opts in.

Tenant isolation ​

The single largest thread in this release, and the reason to upgrade if you run more than one tenant. A series of defects allowed a request to be served against the wrong tenant's database under concurrency — not a broken permission check, but a request that never bound itself to a tenant in the first place and fell through to whatever the last global binding happened to be.

  • Every client-API request, dashboard request, model route and inference plugin now binds its tenant explicitly and per request.
  • Tenant binding was unified onto one canonical wrapper, and three copies that had drifted — one of which skipped the role check entirely — were deleted.
  • Schedulers and queue consumers bind tenant scope too, which is what was letting background jobs write results into the wrong tenant.
  • API-token access and the model/provider creation flows were hardened separately.
  • A legacy data-backfill routine was assigning a default project to providers that were explicitly scoped to other projects, making them visible where they should not have been. Fixed, with a self-heal for records it had already touched.
  • Secrets echoed back by upstream services are scrubbed before request logs are persisted.

A new Web Search service: project-scoped search instances with pluggable engines, its own client API, and a dashboard. Six engines ship with it — Bing, Brave, Serper, Tavily, SearXNG and DuckDuckGo.

Knowledge Engine ​

RAG was renamed Knowledge Engine everywhere it is user-facing — the concept had outgrown the acronym. Module creation also gained reranker selection, so a knowledge module can rerank its own results without a separate step.

MCP ​

  • MCP Hubs groundwork — the types, schema and licence rules for curated catalogs of MCP servers.
  • Per-tool toggles — individual tools on an MCP server can be switched off rather than all-or-nothing.
  • Import from OpenAPI and Postman — point Console at a spec and get a tool or MCP server out of it, with an import UI and a playground for trying calls.
  • Editing a server no longer wipes its stored upstream secrets, spec re-import works from the edit form, and streamable HTTP is served correctly on /sse.
  • Sandbox-backed MCP execution and Aegis policy enforcement are gated behind Enterprise, with a clear message in the UI rather than a silent failure.

Client API ​

  • Admin-surface endpoints for providers, models, projects, members and licence.
  • Observability and authoring endpoints, with per-token scopes so a token can be issued for exactly what it needs.
  • The complete OpenAPI specification is published.

Agents and interoperability ​

  • Runtime context header passthrough — headers from the caller can be forwarded to downstream tools, default-deny and explicitly allowlisted, so a tool can act with the end user's authorization rather than a service account's.
  • Inbound A2A server — Console can be addressed as an agent by other agent-to-agent clients, with a published agent card.
  • Tool-call progress — agent runs surface tool activity as it happens instead of going quiet until the answer arrives.
  • Reasoning content is surfaced across streaming, the API, agents and the UI for models that emit it.
  • Models declare explicit discovery capabilities rather than being probed.

Crawler ​

An extended hardening pass, driven by real crawls failing in production.

  • Structure-preserving markdown conversion (@cognipeer/to-markdown 3.1.0) — tables and nested lists survive the conversion instead of collapsing.
  • Anti-bot challenge pages are detected rather than being stored as if they were content, and the headless fingerprint was reduced.
  • SSRF-safe redirect handling, atomic job claiming, and a persisted cancel signal so a cancel survives a restart.
  • Cancel actually stops the job, within about a second, aborting in-flight requests.
  • Errors no longer consume the page budget, results are no longer dropped when the page limit is reached mid-batch, and all results are shown rather than the first 200.
  • Jobs orphaned by a server restart are reconciled at boot.
  • Page, webhook and Knowledge Engine errors are logged and shown in the Runs UI instead of failing invisibly.
  • Markdown conversion is bounded by a timeout, so one pathological page cannot hold a job open forever.
  • TLS chain and download fallbacks for sites that serve incomplete certificate chains or files the HTTP client cannot fetch directly.

Guardrails ​

Evaluation results were never being written — a guardrail could run for weeks with nothing to show for it. Logging works, and the release adds word filtering, a redact action, a configurable fail mode, and a more robust LLM evaluation path. The model selector is always visible, disabled with an explanation when the project has no model configured, rather than absent.

Red teaming and compliance ​

  • An EU AI Act compliance report and the EU risk taxonomy.
  • A system-prompt-leakage probe.
  • A "Policy Probes" entry for red-teaming Aegis policies.

Providers and storage ​

  • Zero-config built-in vector and file providers — a fresh install has working vector storage and file storage without configuring anything.
  • S3-compatible custom endpoints, so MinIO and similar work with the aws-s3-files provider.

Platform ​

  • The container runs the server directly as PID 1. Previously the process manager sat in front of it and did not reliably forward the shutdown signal, so graceful shutdown never ran and any crawl in progress was hard-killed partway through.
  • A per-request legacy backfill that was running a dozen full collection scans on nearly every authenticated request is gone; database indexes are created from a manifest at boot instead.
  • Auth pages and transactional emails were rebuilt on the design system.
  • Registration defaults to open rather than invitation-only.
  • Agent tracing: a session's totals are no longer reset by its own later legs, and the recently-active-agents panel became a real table.

Earlier in the 1.2 line ​

These changes shipped between the open-source release and v1.2.14-community, and are included in it.

Agent Sandbox and GPU pools ​

1 June 2026

The Agent Sandbox arrived — a container runtime with its own APIs and dashboard, giving agents a real machine to work on. GPU pool management and terminal access over WebSocket landed at the same time.

The open-source split ​

5 June 2026

Console was split into a community edition under AGPL-3.0 and an enterprise overlay. See v1.0.0-community for what the first public release contained. A prompt optimizer module shipped alongside.

Realtime, groups and external auth ​

18 June 2026

  • Realtime voice and chat — the schema and gateway routing landed in the community edition; the realtime service itself is an enterprise module.
  • User groups with tenant and project grants, and permissions resolved as a union across a user's groups.
  • An external authentication seam for directory and SSO providers, so LDAP and SSO can be attached without forking the login flow.
  • Client API expansion — batch, moderation, spend and realtime endpoints.
  • Red-team probes and an overview screen, and evaluation run comparison.
  • Browser automation hardening — stale element references fall back to selectors, action and navigation timeouts are bounded, and a browser_pdf tool was added.
  • Azure's v1 audio endpoint does not route /audio/* by model; requests are now deployment-scoped so audio works against Azure.

Provider scoping ​

19 June 2026

Providers became strictly project-scoped, with error messages that say so. A provider that is not assigned to your project cannot be used from it — and now tells you that instead of failing obscurely.

Community edition is AGPL-3.0. Commercial licensing and support are available separately.

Copyright © 2026 Cognipeer