Agentic UI — Rich, Interactive Surfaces Beyond Plain Text

The visual counterpart to voice-agentic-ux. Agents that reason, plan, and act in parallel need surfaces beyond a chat bubble: artifacts, generative UI, option-pickers, receipts dashboards. This page scaffolds the corpus, the patterns, and what agentic UI means for Ozzy Dev’s own harness.

Context

• Chat is a data channel, not a UI. Long-running, non-deterministic, tool-using agents produce work that is easier to look at and interact with than to read line by line — diffs, dashboards, option matrices, receipts, live progress.
• Agentic UI is the visual counterpart to voice-agentic-ux: voice carries narrative at ambient volume; agentic UI carries structure, data, and choice at focus.
• Our own harness is already an agentic UI surface: the Kilo `question` / `suggest` tools render into the terminal as interactive prompts, and the Ozzy Dev console renders receipts, roadmap, /memory, and /research as agent-touched pages.
• Scope: the surfaces where agents present work to humans (and back). General LLM UX and prompt engineering live elsewhere.

Prior art (linked, primary sources)

• Claude Code artifacts — Session output published as a live, interactive web page on claude.ai/code — PR walkthroughs, dashboards, side-by-side option matrices, investigation timelines. The reference agentic-UI-from-a-terminal-agent pattern.
• Anthropic Claude artifacts (chat) — The original artifact concept — HTML, React (JSX), SVG, Mermaid, Markdown, PDF renderable inline. Established the split-window “chat + artifact” layout that everyone else copied.
• OpenAI ChatGPT Canvas — Side-panel document/code editor with targeted-selection editing, length/tone/reading-level shortcuts, and version history. The collaborative-editing take on agentic UI.
• Anthropic computer use — Inverts the agentic-UI arrow: instead of the agent rendering UI, the agent uses UI — screenshot + click/type/scroll loop. Frames the human-UI as a general-purpose tool surface.
• Vercel AI SDK generative UI (streamUI / streamText+tools) — Tool calls whose return values are typed React components; each invocation moves through partial-call → call → result states and renders inline. Canonical “tools are components” pattern.
• AG-UI — Agent-User Interaction Protocol — Open, event-based protocol (~16 event types over SSE/WebSockets) standardising how agent backends stream text, tool calls, state deltas, and generative UI to frontends. Complements MCP (agent↔tools) and A2A (agent↔agent).
• CopilotKit — React/Angular frontend stack over AG-UI: CopilotChat, headless chat hooks, `useCopilotAction` for frontend-tool + generative UI, human-in-the-loop rendering across states. The reference “add an agent to my existing app” integration.
• assistant-ui — shadcn-style React primitives for chat + tool UIs — message parts, tool renderers, composer, thread. Design-system-friendly alternative to CopilotKit’s all-in-one components.
• Thesys C1 (Generative UI API) — OpenAI-compatible API that streams live UI components (charts, forms, tables, dashboards, artifacts, slides) instead of markdown. Pushes generative UI from a library pattern to a hosted service.
• CopilotKit “3 patterns of Generative UI” (AG-UI, A2UI/Open-JSON-UI, MCP Apps) — Names the taxonomy the field is settling on: Controlled (dev-registered components the agent picks), Declarative (JSON UI specs), Open-ended (MCP Apps / sandboxed HTML). Vocabulary we should adopt.
• Ink & Switch — Malleable Software — The long-arc vision: end-users — with LLM help — continuously reshape the tools they use. Frames agentic UI not as chat with rendered widgets but as an ongoing negotiation of the workspace itself.
• Malleable Overview-Detail Interfaces (Min, Chen, Cao, Xia — CHI ’25) — Peer-reviewed HCI on user-customisable overview/detail (“Fluid Attributes”). Cites the exact pattern our /memory Repos tab and /research index already implement — and points at how AI can generate/reformat attributes on demand.
• Xia et al. — Generative UI with evolving task-driven data models — Argues generative UIs must ride on generative task/data models, not just widget generation. Directly relevant to how our ADRs / receipts / roadmap should shape the surfaces that render them.

Patterns so far

• Artifacts — the session’s work as a live page — The agent publishes a shareable, interactive page (PR walkthrough, dashboard, timeline, option matrix). Chat remains the conversation; the artifact holds the structure.
  Terminal text is the wrong medium for anything wider than ≈80 chars or richer than prose. Artifacts move structured work to a surface designed for it.
• Generative UI — tools return components — A tool call’s result is not a data blob but a typed component the client renders inline. States: partial-call (streaming args) → call (executing) → result. Three flavours: controlled (registered components), declarative (JSON UI spec), open-ended (sandboxed HTML/MCP Apps).
  Removes the “paragraphs-that-describe-a-table” anti-pattern. The model chooses the widget; the developer bounds the widget set.
• Option-picker — open-question as interactive UI — When the agent has an open decision, it renders each option with a brief analysis and a recommendation, then waits. The human picks; the decision (and its rationale) is captured as an artifact.
  The default “answer this vague question in prose” loop is expensive for both sides. A structured picker forces the agent to precommit its analysis and gives the human a single click to steer.
• Streaming progress + tool-driven state — The UI reflects tool execution as it happens — file being read, patch being applied, receipt being emitted — not just the final message. AG-UI’s TEXT_MESSAGE_CONTENT / TOOL_CALL_START / STATE_DELTA event vocabulary is the reference.
  Long-running agent work without visible progress is indistinguishable from a hang. Streaming state is the situational-trust dial.
• Confirmation gate as UI, not prose — Destructive or expensive actions render an inline widget (diff, cost estimate, dry-run summary) with explicit Approve/Reject/Amend controls, instead of “reply YES to continue”.
  Text approval is easy to auto-answer and easy to misread. A widget makes the choice visible and auditable in the transcript.
• Receipts and telemetry dashboards — Every routed search/write emits a signed receipt; dashboards render fleets of them as filterable, drillable tables and time-series (per-agent, per-tool, per-wave).
  Agentic UI’s reflective surface. The transcript is the story; the dashboard is the evidence.
• Malleable surfaces — The human tweaks the surface itself (“hide this column”, “group by wave”, “add a chart of this field”) and the agent regenerates the layout. The UI is a moving target, not a fixed screen.
  The long-arc bet. Ink & Switch’s malleable-software work and CHI’25 malleable-overview-detail results say the win is not “better widgets” but “users reshape widgets in the flow of work”.

Relation to Ozzy Dev

Agentic UI is not aspirational for us — the harness already renders several surfaces. The question is which patterns to formalise and which to leave bespoke.

• Kilo `question` / `suggest` tools — our live option-picker — After the initial session audit, the orchestrator surfaces open questions to the operator as option-pickers: each option gets a brief analysis and an explicit recommendation, rendered by the harness as an interactive prompt (not free prose). The AGENTS.md convention codifies this; the tools are the surface.
  A concrete, already-shipping example of agentic UI inside our own harness. Voice carries the arrival of the picker; the picker is the visual moment where the human decides.
• Console `/research` and `/roadmap` — canonical living surfaces — Long-lived pages (this one included) are the durable log; agents update them at the end of every wave. Not chat-rendered widgets — first-class routes in the app.
  Establishes the split: ephemeral generative UI for a single decision; durable, curated pages for standing knowledge.
• Console `/memory` (Status + Repos tabs) — fleet health as agentic UI — Per-repo coverage matrix (codesearch / group / serena / semantic + ignore-slurp hazards), backed by `GET /api/doctor` and `GET /api/doctor/repos`. An overview-detail surface exactly of the shape CHI’25 describes; today it is designer-fixed, tomorrow malleable.
  Our own answer to “where does the fleet stand?” rendered as UI, not a wall of jq output.
• Receipts + telemetry (ADR-007 / ADR-008 / `telemetry_emit`) — Every routed search and write emits a signed receipt to D1; the console renders session reports, per-tool events, and experiment scoreboards. This IS the agentic UI of the flywheel.
  Agentic UI’s value is only auditable when the underlying operation is receiptable. Our routed-tool convention makes agentic-UI dashboards trustworthy by construction.
• DevFlow wizard (shared with voice-agentic-ux) — The wizard is the integration point where voice, generative UI, and option-pickers meet. Step 2 (voice + agent flow) already claims this surface; agentic UI extends it to the other steps.
  Keeps the two research threads aligned rather than colliding. Voice narrates; UI structures; both feed the same wizard.

Open questions

• Where do we sit on the Controlled ↔ Declarative ↔ Open-ended axis? — Controlled (registered components) gives us design-system discipline; open-ended (sandboxed HTML) gives us maximum reach. Our tokens.css/ADR-003 stance argues Controlled by default, sandboxed HTML only inside artifact-style surfaces.
• AG-UI adoption — do we speak the protocol or stay bespoke? — AG-UI is coalescing as the agent↔frontend standard (SSE, ~16 event types, TS + Python SDKs). Adopting it would give our console interop with LangGraph/CrewAI/Mastra/etc.; staying bespoke keeps us leaner. Decision belongs alongside our MCP-first stance.
• Option-picker taxonomy — which decisions get a picker vs prose vs typed approval? — Pickers are cheap once, expensive over time (interruption cost). Some decisions want a picker (mid-wave routing), some a diff-widget (destructive change), some a typed gate (merge). Needs an explicit rubric.
• Artifact durability — do we host, or are artifacts session-ephemeral? — Claude hosts artifacts under claude.ai/code URLs. Our equivalent could be console routes (`/artifact/:id`) backed by D1 or R2. Durability changes what “share with the team” means — and what the receipt has to point to.
• Malleable surfaces — do we build for user re-shaping now or later? — CHI’25 evidence is compelling but the design cost is real. Deferring risks the same lock-in Ink & Switch critiques; jumping early risks over-general primitives that never earn back their complexity.
• Two-surface coherence — how do voice and UI stay aligned mid-session? — A picker arrives while a spoken update is in the queue. Does the picker interrupt speech, wait for it, or narrate itself in one line? Cross-cuts with voice-agentic-ux §interruption semantics.
• Computer-use inversion — do we expose our own UI as an agent tool? — If the console is agentic UI, screen-level access lets other agents drive it. Interesting for E2E tests; risky for prod. Probably a research prototype in a Cubicle, not a shipping surface.

Future work

1. Prototype an option-picker component in the console (Controlled generative UI over tokens.css primitives) and wire it to the Kilo `question`/`suggest` tools.
2. Bench AG-UI vs bespoke event stream on our console: latency, interop cost, receipt fidelity.
3. Decision-gate taxonomy — picker vs diff-widget vs typed approval — with worked examples from recent waves.
4. Artifact-durability design note: URL scheme, storage tier (D1/R2), receipt back-reference.
5. Malleable /memory Repos tab pilot: user-controlled column set + AI-generated derived attributes.
6. Cross-page contract with `mem:voice-agentic-ux` on interruption semantics when a picker and a spoken update collide.
7. KPI registration on the experimental-framework matrix: pickers-per-wave, picker-latency, artifact-open-rate, generative-UI vs prose task-completion.

How this page stays current

Living draft. v0.1 scaffolds the corpus and outline; v0.2 lands the option-picker prototype and the AG-UI adoption decision; later revisions add the artifact-durability design note and the malleable-surface pilot writeup.