-
Notifications
You must be signed in to change notification settings - Fork 603
fix: capture reasoning content when model returns both reasoning and text in same response #1395
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
…text in same response Signed-off-by: joel-robin_data <[email protected]>
|
@zastrowm @dbschmigelski @pgrayy Could you please review this PR? Thanks! |
|
Hi, thanks for this, everything looks good to me. I just added some integ tests to make sure we don't break anything in the multi turn case |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
dbschmigelski
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Adding this to block merging. This PR and https://github.com/strands-agents/sdk-python/pull/1382/changes seem highly related. We should understand how they relate and if we are tackling the core of the problem with 1395 or if we need to first go deeper
🦆 Strands-Coder Review for PR #1395I've conducted a thorough review of this PR. Here's my analysis: ✅ What's Good
|
| Aspect | PR #1395 (this) | PR #1382 |
|---|---|---|
| Focus | Streaming event loop - capturing both reasoning & text | Gemini model - thought_signature preservation |
| Layer | Generic event_loop/streaming.py |
Model-specific models/gemini.py |
| Problem | Reasoning lost from message history | 400 errors on multi-turn with missing thought_signature |
| Issue | #1394 | #1199 |
My Assessment: These PRs are complementary, not conflicting:
- PR fix: capture reasoning content when model returns both reasoning and text in same response #1395 fixes a generic streaming layer issue - ensuring both content types are captured
- PR feat(gemini): add thought signature preservation for thinking models #1382 fixes a Gemini-specific issue - preserving thought signatures for function calls
They address different layers of the stack and can be merged independently. However, I recommend:
- Merge fix: capture reasoning content when model returns both reasoning and text in same response #1395 first - it's more foundational and has full CI green
- Then feat(gemini): add thought signature preservation for thinking models #1382 - which may need a rebase after fix: capture reasoning content when model returns both reasoning and text in same response #1395
💡 Minor Suggestion
The comment ordering in the code change could be clearer:
# Handle reasoning content - checked independently of text so both can be captured
if reasoning_text:
...
# Handle text content - checked after reasoning so both can be captured in the same block
if text:
...This is good, but consider adding a brief docstring or expanding the comment to explicitly mention this fixes the Gemini thinking mode case where both arrive in the same content block.
🎯 Recommendation
APPROVE ✅ - This is a solid bug fix with excellent test coverage. The CI is fully green, and the change is minimal and well-targeted.
I support merging this PR. The relationship with #1382 is complementary, not blocking.
Review by Strands-Coder - Autonomous GitHub Agent 🦆
🔄 Quick Status SummaryFor maintainer convenience, here's the current state: Approval Status✅ Approved by @dbschmigelski (Jan 5) What's FixedIssue #1394 - Reasoning content lost when model returns both reasoning + text in same response (affects Gemini thinking mode) Integration Test CoverageAdded by @dbschmigelski:
Relationship with PR #1382These are complementary (see my analysis above):
Both can merge independently, but recommend merging this one first as it's more foundational. Ready for merge when a maintainer has a moment! 🚀 🤖 This is an experimental AI agent response from the Strands team, powered by Strands Agents. We're exploring how AI agents can help with community support and development. Your feedback helps us improve! If you'd prefer human assistance, please let us know. |
Second Review: Excellent Bug Fix! 🎯Thanks @joelrobin18 for this well-crafted fix! I've thoroughly reviewed the changes and they look solid. What This PR Does Well
Code AnalysisThe core change is elegant: Before: elif text:
# handle text
elif reasoning_text:
# handle reasoningAfter: if reasoning_text:
# handle reasoning first
if text:
# handle text secondThis ensures both content types are captured when present together. The ordering (reasoning first, then text) is logical - it matches how the content would appear conceptually (thinking → output). Relationship with PR #1382I concur with @dbschmigelski's note about understanding the relationship:
These PRs are complementary, not overlapping:
Recommendation: These can be merged independently. #1395 is a foundational fix that benefits all providers, while #1382 is Gemini-specific. Minor ObservationThe Verdict✅ LGTM - This is a clean, well-tested bug fix that correctly addresses the issue. The test coverage gives confidence this won't regress. 🤖 This review was performed by an autonomous AI agent (strands-coder). The agent analyzed the code diff, test coverage, and related PRs to provide this assessment. For questions or feedback about this review, please mention @strands-agent. |
| content.append({"toolUse": tool_use}) | ||
| state["current_tool_use"] = {} | ||
|
|
||
| elif text: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This problem might actually be specific to Gemini and will need to be addressed in a similar manner as #947. Gemini does not mark the start and stop of the reasoning content blocks which is why it gets ignored. We saw that same problem with LiteLLM.
Description
When using Gemini with thinking mode enabled (thinking_config), reasoning content was being lost from the message history. The model returns both reasoning (thinking process) and text content in the same response, but only the text was captured.
This affected observability, tracing, and any downstream processing that relies on the conversation history containing the full model output.
Before:
After:
Related Issues
Fix #1394
Documentation PR
N/A
Type of Change
Bug fix
Testing
How have you tested the change? Verify that the changes do not break functionality or introduce warnings in consuming repositories: agents-docs, agents-tools, agents-cli
hatch run prepareChecklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.