Chat vs Structured UI

Pros and cons of a structured UI

Structured UI is good for the following types of tasks: repeatable, high signal, reviewable, and high consequence workflows. We can use our existing patterns and present users with interfaces that are easy to learn due to consistency across sessions, which reduces the need to remember the right way to formulate an effective prompt.

For agents in particular, a defined UI also allows us to control what we feed into the agent and makes the response from the LLM more predictable. We don’t have to necessarily handle someone using the wrong word, or an underbaked prompt. While this doesn’t yield determinism from the output, it helps to control the variables. It is best when you have these characteristics:

• Known inputs and outputs
  • You know what you are selecting, and what you want the outcome to be. You come prepared with a list of workspaces, for example.
• Repeatable steps
  • Doing the same flow via chat is ineffective, because you often have to give it intent and context via your starting prompt every time. If you are doing a task often, a structured UI you can learn well over time is usually better.
  • Can be represented as a wizard.
• Clear completion criteria
  • Chat is good because you can iterate at low cost until you reach a state of completion, but if I know a workflow is complete when X is done and Y is done, it becomes clearer to use a structured UI.
• High consequence
  • AI tools can carry the connotation that they’re less trustworthy. We can work around that by showing reasoning, but dropping high-consequence workflows into chat can make them feel more ephemeral.
• Need for comparison, review, or approval
  • Chat windows are more ephemeral than a detail page with a visitable UI.

Downsides include:

• Higher cost of design and implementation
  • We need to design each flow in a bespoke way.
• Inconsistency with existing flows
  • There might end up being two ways to accomplish the same task.
• Long-term scaling cost
  • Are we going to be replacing the UI down the line?
• Adds UI sprawl
  • If each agent had its own UI surface, the product would get cluttered and dense.

Examples of when to use structured UI

A good UI for a specific task will depend on a case-by-case basis, but some examples that could make sense for agents:

• I am doing a bulk action
  • You go through a set amount of inputs you already have an idea of, and you want to be able to select from those directly.
• The CTA is something that lives in the UI and can be served contextually
  • I have an error in my Terraform run, and I want the agent to remediate the issue.

Pros and cons of a chat interface

Naturally, chat has similar benefits, more specifically toward exploration and discovery. Within chat flows, the cost of figuring out what you should do is very low compared to a UI. This makes it incredibly effective for open-ended requests and follow-ups that refine a task until it’s in a state that’s actionable for an agent.

Chat workflows are often better at some of the following:

• Intent is unclear or needs further refining
  • You know the general direction, but you need guidance on where to go.
• The task is exploratory
  • You can ask an agent to figure out what you need to do when you see a signal you don’t know how to act on.
• Follow-ups and gradual learning
  • The agent can learn from multiple queries, as opposed to one rigid set of metadata that can’t be added to.
  • Allows for clarification steps.
• When the agent is giving insights or recommendations rather than executing
  • Where chat really shines is layering on top of the core UI rather than replacing it — giving the user additional information they can use in their decision making.
• Less costly to implement
  • Long wizards and structured flows take a lot of engineering capacity; some sort of generative UI can dramatically reduce that cost.

But chat suffers from:

• Poor discoverability
  • It’s hard to figure out what you can do, how well it will work, and how safely it will perform it.
• Poor representation of current status
  • In large workflows, it can be difficult to understand where things stand.
• Poor at repeated structured input
  • If I’m constantly prompted to fill out fields one by one, that’s much more tedious than seeing them all and filling them in at once.

Examples of when to use chat

Chat can function for many workflows, but should be emphasized for:

• Recommendations
  • Context-aware CTAs that give the user an entry point into fixing an issue. This can trigger a structured UI (a drift remediation run) or further discovery within chat (“how do I fix this issue?”).
• Explanations
  • I’m a new user trying to figure out what a page or concept does — I can open chat and ask it.
  • I can look at a workspace that has drifted and ask it to “explain how this drift happened.”
  • I can use it to understand existing infrastructure.

A decision framework

Every situation needs to be evaluated on a case-by-case basis, and we can strengthen our prescription as we get more comfortable with the types of UI that come out of it. But a useful starting point is to run each workflow stage across these dimensions: