GenUI docs missing requirement to call provideFinalOutput (causes repeated POSTs, TPM spikes)

[ulusoyca]

Summary

The GenUI documentation does not explicitly state that, under forced tool calling, the model must always call provideFinalOutput to terminate a turn. Without this, a single sendRequest() can trigger repeated POSTs (tool-loop retries) and rapidly exhaust token quotas. This behavior was only discoverable by inspecting the sample app's system instructions (link).

Symptoms

• A single sendRequest call kept sending POSTs roughly every 4–6 seconds.
• The model repeatedly re-sent the same surfaceUpdate + beginRendering calls.
• Token usage (TPM) spiked dramatically, exceeding quota from a single developer session.
• The app became non-functional once the quota was exceeded and the my users couldn't use the app because the model returned for every user:

Root Cause

genui_firebase_ai uses forced tool calling with a safety loop. If the model never calls the special tool provideFinalOutput, the loop continues until a max cycle limit (default 40 - link) is hit, reissuing requests and resending full context. The documentation does not mention this termination requirement.

Evidence

From the GenUI SDK code path:

• FirebaseAiContentGenerator.sendRequest() calls _generate() in a tool loop.
• The loop continues until a final tool call is returned.
• The sample app's system instructions explicitly say to call provideFinalOutput, but this requirement is not stated in the public docs.

Impact

• Excessive and repeated POSTs per user action.
• Inflated prompt size each iteration due to full history/tool payload.
• Quota exhaustion (TPM) from a single developer session.

• Risk of catastrophic costs/outage if released to production.

Fix Implemented in This Project

I added explicit instructions to both refinement and generation prompts:

• After the last beginRendering, the model must call provideFinalOutput once with:
  {"response":"ok"}.

This terminates the tool loop and stops the repeated POSTs.

Recommendation to GenUI Docs

Add a clear section in the GenUI documentation that states:

• When using genui_firebase_ai (forced tool calling), the model must call
  provideFinalOutput to end the turn.
• Omitting this will cause repeated POSTs and high TPM usage.
• Provide a short example snippet for both refinement and generation steps.

Suggested Doc Addition (Proposed Wording)

"When using GenUI with forced tool calling, always call provideFinalOutput after
sending all surfaceUpdate/beginRendering calls. If omitted, the tool loop
will retry and can cause repeated POSTs and quota exhaustion."

[jacobsimionato]

@ulusoyca thank you very much for this issue!

This makes total sense - we have seen this issue and added instructions to example system prompts to fix it, e.g.

genui/examples/travel_app/lib/src/travel_planner_page.dart

Line 365 in 1342f08

conversation turn is complete, and you should call the provideFinalOutput tool.

It's somewhat awkward right now that the LLM has to perform so many tool calls correctly to successfully output UI. We have the sense that this pattern can be confusing for the LLM.

In v0.9 of the A2UI specification, we're planning to migrate to a different approach that does not use tool calling which will make this issue disappear. That's one of the reasons we haven't focused on this mechanism in the Gen UI SDK right now, because we will likely migrate to A2UI v0.9 in a few weeks. See https://github.com/google/A2UI/blob/main/specification/0.9/docs/evolution_guide.md for more info.

That said, I will leave this issue open because we can improve docs for v0.8 as you request.