Notebooks · Provider API features · 66

Interleaved thinking & context editing

Name: SHIPIT Agent
Author: SHIPIT

Two Anthropic beta passthroughs exposed on AnthropicChatLLM:

Open in Colab View on GitHub 7 code cells · 6 with saved output

Its imports and arguments are checked against shipit-agent 3.0. The outputs were saved when it was written, for an earlier release.

Provider support

Interleaved thinking (interleaved-thinking-2025-05-14 beta) and context editing (context-management-2025-06-27 beta + the context_management request param) are Anthropic API shapes. They work with the Anthropic API directly and with Anthropic models via Bedrock / LiteLLM where the beta is forwarded.
They do not apply to OpenAI / Gemini / Groq / Ollama. (Those providers expose their own reasoning controls — e.g. OpenAI reasoning_effort — which shipit-agent surfaces separately.)

In [1]

python

from pathlib import Path
import sys

ROOT = (
    Path.cwd().resolve().parent
    if Path.cwd().name == "notebooks"
    else Path.cwd().resolve()
)
if str(ROOT) not in sys.path:
    sys.path.insert(0, str(ROOT))

Enabling interleaved thinking adds the beta header

Important: the beta header is only attached when interleaved thinking is on and extended thinking is actually enabled (thinking_budget_tokens is set). The two go together — interleaved thinking is meaningless without a thinking budget. So construct the adapter with both.

In [2]

python

import json
from shipit_agent.llms import AnthropicChatLLM
from shipit_agent.models import Message

llm = AnthropicChatLLM(
    model="claude-opus-4-1",
    api_key="offline-demo-key",
    interleaved_thinking=True,
    thinking_budget_tokens=2048,
)

req = llm._build_request_kwargs(
    messages=[Message(role="user", content="Plan a 3-step task.")],
    tools=None,
    system_prompt="You are a planner.",
)
print("thinking config:", json.dumps(req.get("thinking")))
print("betas:", req.get("betas"))

Out

thinking config: {"type": "enabled", "budget_tokens": 2048}
betas: ['interleaved-thinking-2025-05-14']

Without a thinking budget the beta is not added — the header would be meaningless, so the adapter leaves the request on the GA path. (This AND-condition is intentional.)

In [3]

python

llm_no_budget = AnthropicChatLLM(
    model="claude-opus-4-1", api_key="x", interleaved_thinking=True,
)
req_nb = llm_no_budget._build_request_kwargs(
    messages=[Message(role="user", content="hi")], tools=None, system_prompt=None,
)
print("thinking config:", req_nb.get("thinking"))
print("betas (None — no budget, so no interleaved beta):", req_nb.get("betas"))

Out

thinking config: None
betas (None — no budget, so no interleaved beta): None

Context editing passthrough

context_management is forwarded verbatim as a request param, and its beta header is added automatically. A typical config clears old tool-use results once they pile up.

In [4]

python

llm_ctx = AnthropicChatLLM(
    model="claude-opus-4-1",
    api_key="x",
    context_management={"edits": [{"type": "clear_tool_uses_20250919"}]},
)
req_ctx = llm_ctx._build_request_kwargs(
    messages=[Message(role="user", content="Do a long multi-tool task.")],
    tools=None,
    system_prompt=None,
)
print("context_management forwarded:", json.dumps(req_ctx.get("context_management")))
print("betas:", req_ctx.get("betas"))

Out

context_management forwarded: {"edits": [{"type": "clear_tool_uses_20250919"}]}
betas: ['context-management-2025-06-27']

Both features can be combined — the betas list accumulates both headers (order-stable, de-duplicated).

In [5]

python

llm_both = AnthropicChatLLM(
    model="claude-opus-4-1",
    api_key="x",
    interleaved_thinking=True,
    thinking_budget_tokens=4096,
    context_management={"edits": [{"type": "clear_tool_uses_20250919"}]},
)
req_both = llm_both._build_request_kwargs(
    messages=[Message(role="user", content="Long task.")], tools=None, system_prompt=None,
)
print("betas:", req_both.get("betas"))

Out

betas: ['interleaved-thinking-2025-05-14', 'context-management-2025-06-27']

Thinking blocks surface in metadata (offline, fake client)

On the response side, thinking blocks (with their signature) are preserved in metadata['thinking_blocks'] when interleaved thinking is on — so they can be round-tripped on a later turn. The parsing is inline in complete(), so we inject a fake anthropic client returning a thinking block + a text block.

In [6]

python

import sys, types
from types import SimpleNamespace as NS

fake_content = [
    NS(type="thinking", thinking="Let me reason step by step about the plan...",
       signature="sig_abc123"),
    NS(type="text", text="Here is the 3-step plan: 1) scope 2) build 3) ship."),
]
fake_response = NS(
    content=fake_content,
    usage=NS(input_tokens=50, output_tokens=40,
             cache_read_input_tokens=0, cache_creation_input_tokens=0),
)

fake_anthropic = types.ModuleType("anthropic")
class _Messages:
    def create(self, **_kwargs):
        return fake_response
class _Beta:
    def __init__(self):
        self.messages = _Messages()
class _Client:
    def __init__(self, **_kwargs):
        self.messages = _Messages()
        self.beta = _Beta()
fake_anthropic.Anthropic = _Client
sys.modules["anthropic"] = fake_anthropic

out = llm.complete(
    messages=[Message(role="user", content="Plan a 3-step task.")],
)
print("answer:", out.content)
print("reasoning_content:", out.reasoning_content)
print("\nmetadata['thinking_blocks'] (preserved with signature for round-trip):")
print(json.dumps(out.metadata.get("thinking_blocks"), indent=2))

Out

answer: Here is the 3-step plan: 1) scope 2) build 3) ship.
reasoning_content: Let me reason step by step about the plan...

metadata['thinking_blocks'] (preserved with signature for round-trip):
[
  {
    "type": "thinking",
    "thinking": "Let me reason step by step about the plan...",
    "signature": "sig_abc123"
  }
]

Multi-turn round-trip status (honest note)

The adapter side of the interleaved-thinking round-trip is wired: response thinking blocks (with signatures) are surfaced in metadata['thinking_blocks'], and _convert_messages will re-emit those blocks first on the next assistant message if its metadata carries them. Quoting the adapter source directly:

NOTE: completing the round-trip across a multi-turn tool loop also requires the runtime to copy that metadata onto the next assistant Message — runtime.py/agent.py are owned elsewhere, so this is a degrade-gracefully passthrough until they propagate thinking_blocks.

In other words: single-call surfacing works today; the full multi-turn tool-loop round-trip is a documented passthrough pending runtime propagation of the preserved blocks. The cell below shows the re-emit path the adapter implements when the blocks are present on an assistant message.

In [7]

python

# Show that _convert_messages re-emits preserved thinking blocks first on an
# assistant turn that carries tool_calls + thinking_blocks metadata.
assistant_msg = Message(
    role="assistant",
    content="Calling a tool.",
    metadata={
        "tool_calls": [{"id": "tc1", "name": "search", "arguments": {"q": "x"}}],
        "thinking_blocks": [
            {"type": "thinking", "thinking": "I should search first.", "signature": "sig_xyz"},
        ],
    },
)
converted = llm._convert_messages([assistant_msg])
print("block types on the converted assistant message:")
print([b["type"] for b in converted[0]["content"]])
print("\n-> thinking block is re-emitted BEFORE text / tool_use, as the API requires.")

Out

block types on the converted assistant message:
['thinking', 'text', 'tool_use']

-> thinking block is re-emitted BEFORE text / tool_use, as the API requires.

Recap

AnthropicChatLLM(interleaved_thinking=True, thinking_budget_tokens=N) attaches the interleaved-thinking-2025-05-14 beta — both flags are required (verify offline with _build_request_kwargs(...)).
context_management=... is forwarded as a request param with the context-management-2025-06-27 beta.
Response thinking blocks (with signatures) surface in metadata['thinking_blocks']; the adapter re-emits them on assistant turns that carry them.
The full multi-turn tool-loop round-trip is a documented passthrough until the runtime propagates thinking_blocks onto subsequent messages.

Notebooks · Provider API features · 66

Interleaved thinking & context editing

Two Anthropic beta passthroughs exposed on AnthropicChatLLM:

Open in Colab View on GitHub 7 code cells · 6 with saved output

Its imports and arguments are checked against shipit-agent 3.0. The outputs were saved when it was written, for an earlier release.

Provider support

Interleaved thinking (interleaved-thinking-2025-05-14 beta) and context editing (context-management-2025-06-27 beta + the context_management request param) are Anthropic API shapes. They work with the Anthropic API directly and with Anthropic models via Bedrock / LiteLLM where the beta is forwarded.
They do not apply to OpenAI / Gemini / Groq / Ollama. (Those providers expose their own reasoning controls — e.g. OpenAI reasoning_effort — which shipit-agent surfaces separately.)

In [1]

python

from pathlib import Path
import sys

ROOT = (
    Path.cwd().resolve().parent
    if Path.cwd().name == "notebooks"
    else Path.cwd().resolve()
)
if str(ROOT) not in sys.path:
    sys.path.insert(0, str(ROOT))

Enabling interleaved thinking adds the beta header

In [2]

python

import json
from shipit_agent.llms import AnthropicChatLLM
from shipit_agent.models import Message

llm = AnthropicChatLLM(
    model="claude-opus-4-1",
    api_key="offline-demo-key",
    interleaved_thinking=True,
    thinking_budget_tokens=2048,
)

req = llm._build_request_kwargs(
    messages=[Message(role="user", content="Plan a 3-step task.")],
    tools=None,
    system_prompt="You are a planner.",
)
print("thinking config:", json.dumps(req.get("thinking")))
print("betas:", req.get("betas"))

Out

thinking config: {"type": "enabled", "budget_tokens": 2048}
betas: ['interleaved-thinking-2025-05-14']

Without a thinking budget the beta is not added — the header would be meaningless, so the adapter leaves the request on the GA path. (This AND-condition is intentional.)

In [3]

python

llm_no_budget = AnthropicChatLLM(
    model="claude-opus-4-1", api_key="x", interleaved_thinking=True,
)
req_nb = llm_no_budget._build_request_kwargs(
    messages=[Message(role="user", content="hi")], tools=None, system_prompt=None,
)
print("thinking config:", req_nb.get("thinking"))
print("betas (None — no budget, so no interleaved beta):", req_nb.get("betas"))

Out

thinking config: None
betas (None — no budget, so no interleaved beta): None

Context editing passthrough

context_management is forwarded verbatim as a request param, and its beta header is added automatically. A typical config clears old tool-use results once they pile up.

In [4]

python

llm_ctx = AnthropicChatLLM(
    model="claude-opus-4-1",
    api_key="x",
    context_management={"edits": [{"type": "clear_tool_uses_20250919"}]},
)
req_ctx = llm_ctx._build_request_kwargs(
    messages=[Message(role="user", content="Do a long multi-tool task.")],
    tools=None,
    system_prompt=None,
)
print("context_management forwarded:", json.dumps(req_ctx.get("context_management")))
print("betas:", req_ctx.get("betas"))

Out

context_management forwarded: {"edits": [{"type": "clear_tool_uses_20250919"}]}
betas: ['context-management-2025-06-27']

Both features can be combined — the betas list accumulates both headers (order-stable, de-duplicated).

In [5]

python

llm_both = AnthropicChatLLM(
    model="claude-opus-4-1",
    api_key="x",
    interleaved_thinking=True,
    thinking_budget_tokens=4096,
    context_management={"edits": [{"type": "clear_tool_uses_20250919"}]},
)
req_both = llm_both._build_request_kwargs(
    messages=[Message(role="user", content="Long task.")], tools=None, system_prompt=None,
)
print("betas:", req_both.get("betas"))

Out

betas: ['interleaved-thinking-2025-05-14', 'context-management-2025-06-27']

Thinking blocks surface in metadata (offline, fake client)

In [6]

python

import sys, types
from types import SimpleNamespace as NS

fake_content = [
    NS(type="thinking", thinking="Let me reason step by step about the plan...",
       signature="sig_abc123"),
    NS(type="text", text="Here is the 3-step plan: 1) scope 2) build 3) ship."),
]
fake_response = NS(
    content=fake_content,
    usage=NS(input_tokens=50, output_tokens=40,
             cache_read_input_tokens=0, cache_creation_input_tokens=0),
)

fake_anthropic = types.ModuleType("anthropic")
class _Messages:
    def create(self, **_kwargs):
        return fake_response
class _Beta:
    def __init__(self):
        self.messages = _Messages()
class _Client:
    def __init__(self, **_kwargs):
        self.messages = _Messages()
        self.beta = _Beta()
fake_anthropic.Anthropic = _Client
sys.modules["anthropic"] = fake_anthropic

out = llm.complete(
    messages=[Message(role="user", content="Plan a 3-step task.")],
)
print("answer:", out.content)
print("reasoning_content:", out.reasoning_content)
print("\nmetadata['thinking_blocks'] (preserved with signature for round-trip):")
print(json.dumps(out.metadata.get("thinking_blocks"), indent=2))

Out

answer: Here is the 3-step plan: 1) scope 2) build 3) ship.
reasoning_content: Let me reason step by step about the plan...

metadata['thinking_blocks'] (preserved with signature for round-trip):
[
  {
    "type": "thinking",
    "thinking": "Let me reason step by step about the plan...",
    "signature": "sig_abc123"
  }
]

Multi-turn round-trip status (honest note)

NOTE: completing the round-trip across a multi-turn tool loop also requires the runtime to copy that metadata onto the next assistant Message — runtime.py/agent.py are owned elsewhere, so this is a degrade-gracefully passthrough until they propagate thinking_blocks.

In [7]

python

# Show that _convert_messages re-emits preserved thinking blocks first on an
# assistant turn that carries tool_calls + thinking_blocks metadata.
assistant_msg = Message(
    role="assistant",
    content="Calling a tool.",
    metadata={
        "tool_calls": [{"id": "tc1", "name": "search", "arguments": {"q": "x"}}],
        "thinking_blocks": [
            {"type": "thinking", "thinking": "I should search first.", "signature": "sig_xyz"},
        ],
    },
)
converted = llm._convert_messages([assistant_msg])
print("block types on the converted assistant message:")
print([b["type"] for b in converted[0]["content"]])
print("\n-> thinking block is re-emitted BEFORE text / tool_use, as the API requires.")

Out

block types on the converted assistant message:
['thinking', 'text', 'tool_use']

-> thinking block is re-emitted BEFORE text / tool_use, as the API requires.

Recap

AnthropicChatLLM(interleaved_thinking=True, thinking_budget_tokens=N) attaches the interleaved-thinking-2025-05-14 beta — both flags are required (verify offline with _build_request_kwargs(...)).
context_management=... is forwarded as a request param with the context-management-2025-06-27 beta.
Response thinking blocks (with signatures) surface in metadata['thinking_blocks']; the adapter re-emits them on assistant turns that carry them.
The full multi-turn tool-loop round-trip is a documented passthrough until the runtime propagates thinking_blocks onto subsequent messages.