shipitagent

Tools · Media

vision

Read-only

Send an image (file path, URL, data URL, or base64) to a vision-capable LLM and return a textual analysis.

When to use: Use vision to read screenshots, diagrams, UI mocks, error dialogs, whiteboard photos, and charts. Pair with computer_use for the capture -> analyse loop.

Example

Run for real by scripts/tool-examples.py in a small demo project, with Haiku 5.5 where the tool calls a model. The output is exactly what the tool returned, in 7.7s.

The call

tool call
{
  "tool": "vision",
  "arguments": {
    "image": "screenshot.png",
    "prompt": "What does this image show? One sentence."
  }
}

Real output

This image is a promotional banner for "shipit agent," an open-source Python runtime for AI agents with tools, MCP support, an OS sandbox, and compatibility with any model, tagged with the headline "Agents that finish the job," version v3.0.0, Python 3.11+, and MIT license.

Parameters

NameTypeDescription
image*stringImage to analyse. One of: filesystem path, http(s) URL, data URL (`data:image/png;base64,...`), or raw base64.
promptstringThe question to ask about the image.
detailstringVision detail hint — 'high' for OCR / dense UI, 'low' for quick layout, 'auto' lets the provider decide.lowhighauto
max_tokensintegerUpper bound on response tokens.

* required

Look at images, and make images, speech and video.