Images and Files

Content parts as input

Every run method (run, arun, start, stream, astream, stream_events, astream_events, run_with_response, arun_with_response) takes text or a list of content parts in the OpenAI chat format. LiteLLM turns the parts into each provider’s own format.

from cyclops import Agent, AgentConfig

agent = Agent(AgentConfig(model="anthropic/claude-sonnet-4-5"))

answer = agent.run(
    [
        {"type": "text", "text": "What is in this picture?"},
        {"type": "image_url", "image_url": {"url": "https://example.com/cat.png"}},
    ]
)

Local files go in as base64 data URLs:

import base64
from pathlib import Path

def data_url(path: str, media_type: str) -> str:
    data = base64.b64encode(Path(path).read_bytes()).decode()
    return f"data:{media_type};base64,{data}"

answer = agent.run(
    [
        {"type": "text", "text": "Summarize the report and describe the chart."},
        {"type": "file", "file": {"file_data": data_url("report.pdf", "application/pdf"), "filename": "report.pdf"}},
        {"type": "image_url", "image_url": {"url": data_url("chart.png", "image/png")}},
    ]
)

Part	Shape
Text	{"type": "text", "text": "..."}
Image	{"type": "image_url", "image_url": {"url": "https://... or data:image/png;base64,..."}}; OpenAI also takes "detail"
File, such as a PDF	{"type": "file", "file": {"file_data": "data:application/pdf;base64,...", "filename": "..."}}, or {"file_id": "..."} for a file uploaded to the provider
Audio	{"type": "input_audio", "input_audio": {"data": "<base64>", "format": "wav"}}

The types are LiteLLM’s: cyclops.ContentPart is one part and cyclops.UserContent is text or a list of parts. A part of any other shape, such as Anthropic’s own {"type": "image", "source": ...} or the OpenAI Responses API’s input_image, raises ValueError before the run starts.

As LiteLLM 1.105 sends them: Anthropic gets image and document blocks, with a URL image as a URL source; Gemini gets inline_data; Bedrock gets image and document bytes. For Gemini and Bedrock, LiteLLM downloads an image URL and sends its bytes.

Models that can’t take the input

Before the run starts, Cyclops reads LiteLLM’s model info (litellm.get_model_info, or for Ollama, Lemonade and Hugging Face only its model map, so no provider is asked over the network) for the model, each of its deployments in a router (not the router’s fallback groups) and each of its fallback_models, since they all read the same history. If the entry sets supports_vision to false and an image is sent, supports_pdf_input to false and a PDF is sent, or supports_audio_input to false and audio is sent, the run raises ValueError, so nothing is sent and no part is dropped:

o3-mini, one of its deployments or one of its fallback models can't take image input: LiteLLM's model map sets supports_vision to false for one of them; if that is wrong, correct its entry with litellm.register_model at startup

Only an explicit false counts. Most entries leave these flags out, and a model with no entry at all, such as one on a custom OpenAI compatible endpoint, gets the parts and its provider decides, as in Pydantic AI, the OpenAI Agents SDK and the Vercel AI SDK, which don’t check. The check reads only the map LiteLLM ships plus any entries registered in this process, never a provider over the network. Capability checks are cached per process, so call litellm.register_model at startup, before the first run. A file whose media type is unknown (a file_id without format) is not checked.

Steering, sessions and guards

RunHandle.steer() takes the same input and checks it the same way:

handle = agent.start("Describe this diagram.", stream=False)
handle.steer([{"type": "text", "text": "Compare it with this one."}, {"type": "image_url", "image_url": {"url": url}}])
result = await handle

The parts are the user message’s content in the transcript, so a session keeps them, stores save them as JSON, and every later completion sends them again. Base64 data makes each saved session and each request that large: for big or many files, prefer URLs or provider file ids. GuardContext.messages, AgentHooks.on_run_start and Message.content see the list as it is.

Compaction counts the parts with litellm.token_counter, except input_audio, which it can’t count: an audio part is estimated from its size, at one token per 500 bytes (Gemini’s 32 tokens a second at 128 kbit/s).

TelemetryHooks records the length of the input’s text only, never the parts’ data.

Size limits

Cyclops sends images as given: it doesn’t resize, re-encode or drop them, and neither do other agent libraries such as Pydantic AI, the OpenAI Agents SDK or the Vercel AI SDK. LiteLLM’s model map has no image limits to check against. Anthropic and OpenAI downscale large images themselves; what providers reject is a request over their count or size limits, which at the time of writing are:

Provider	Images per request	Size
Anthropic	600 (100 on 200k context models); over 20, each at most 2000 px a side	8000 px a side; 10 MB per image (5 MB on Bedrock and Vertex AI); 32 MB per request
OpenAI	1,500	512 MB per request
Gemini	3,600	20 MB of inline data per request; larger files go through the Files API

A long session that keeps adding images reaches these counts across turns. To shrink an image before sending it, as with Pillow:

import io
from PIL import Image

def downscaled(path: str, longest_side: int = 1568) -> bytes:
    image = Image.open(path)
    image.thumbnail((longest_side, longest_side))
    buffer = io.BytesIO()
    image.convert("RGB").save(buffer, format="JPEG", quality=85)
    return buffer.getvalue()