Introducing Perceptron Multilook API

Today we're releasing an endpoint that prefills a shared media context once and reuses it across up to 16 prompts in a single call. You send video and image context once, attach a list of questions, and get back one result per question.

Multilook costs less, returns sooner, and moves more work through the same fleet. The cost savings holds on any fleet at any load. The throughput and latency savings appear when the fleet is busy, which is when they matter.

Input cost, 4 prompts

46%

of 4 separate requests

2.2× cheaper · 32% at 16 prompts

Throughput under load

3.9×

vs separate requests, at c=8

1.6× to 3.9× across shapes

Median latency under load

4.9× lower

vs separate requests, at c=16

2× to 4.9× across shapes

Prompts per call

16max

up to 64 completions

Comparisons are against the same workload sent as individual /v1/chat/completions requests, one request per prompt per sample. That is the base route referred to throughout this post, and it is the strongest client pattern that route supports.

Reducing the cost of re-sending context

The default way to ask several questions about one video is several requests. The Perceptron Files API took the byte transfer out of that loop: upload once, reference file-abc123 from then on. The prefill challenge remained unsolved:

The model re-reads the media for every question. A file id saves the upload. It does not save the forward pass over the frames. Ten questions about a ten minute clip means ten prefills over the same frames.
Cost tracks question count, not media volume. A 60 frame clip carries roughly 8,600 tokens of context. The question you attach to it is a sentence. Adding a question adds a full context, not a sentence.

Built for two specific workloads

Two patterns account for much of the media reuse we see in production. In both, the media is fixed and the questions are not.

A fixed query set against one visual asset

Query sets are often fixed. Every teleop episode gets the same twelve questions. Every inspection frame gets the same eight. Every social media clip gets a predefined moderation pass.

Figure 1. A fixed query set against one clip, shown at four prompts. The base route on the left prefills the same video once per query. Multilook on the right prefills it once and serves the other three from that prefill. The 9,740 tokens billed at the input rate are the context once plus the four query texts, which sit outside the shared prefix and never cache.

With Multilook, the first query pays for the context at the input rate, and the remaining P − 1 prompts pay for it at the cache-read rate. Nothing about the answers changes.

A fixed query set against one visual asset

Perceptron models support in-context learning. They can learn an image detection class (e.g., a niche brand logo, a car model) or a video detection class (e.g., OSHA violations, pre-recorded industrial SOPs) from visual examples, rather than from a text label. These visual examples are token-rich but remain byte-identical on every call.

Reuse is scoped to one call, so the saving comes from batching your targets into a single Multilook request. An identical call a second later prefills from scratch. Put the exemplar set in context and the targets in prompts, and the exemplars, which are the token-rich half, are computed once for the whole batch.

Figure 2. Four queries against one exemplar set, four samples each. The base route on the left is sixteen requests, and each one carries the full exemplar set again, which is the sixteen orange bars. Multilook on the right sends the set once. Resampling adds completions without re-billing prompt tokens, so the second effect stacks on the first.

Put the exemplars in context and they are prefilled once for the whole batch. For example, drawing eight samples to vote over costs one prefill rather than eight, because n completions per prompt do not re-bill the prompt tokens.

Four prompts at four samples each bills about 11% of the input tokens the same sixteen requests would. The pricing follows how the call is processed: the shared context is billed at the full input rate once, each additional prompt reads it at the 25% cache rate, and extra samples cost only their output tokens.

How it works

One shared prefix, prefilled once. context takes the same message format as a chat request. It is prefilled a single time per call, and every entry in prompts extends it.
Prompts are isolated. Each entry in prompts is the content of an implicit final user turn, either a bare string or a list of content parts. No prompt observes another prompt's text or completions.
Reuse is scoped to the call. A per-request salt keeps the prefill private to the call that built it. An identical call a second later reports cached_tokens: 0. Nothing is shared across requests, across keys, or across organizations.
Reuse is reported, not inferred. usage.prompt_tokens_details.cached_tokens is the subset of prompt_tokens served from the in-request prefill, and it is exactly what the cache-read rate is applied to.

# one video, three questions, one call
curl https://api.perceptron.inc/v1/chat/completions/multilook \
  -H "Authorization: Bearer $PERCEPTRON_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "perceptron-mk1",
    "context": [
      { "role": "user", "content": [
        { "type": "video_file_id", "video_file_id": { "file_id": "file-abc123" } }
      ] }
    ],
    "prompts": [
      "How many people are visible?",
      "Describe the setting in one sentence.",
      "Does anyone pick up an object? If so, what?"
    ],
    "n": 1,
    "max_completion_tokens": 128
  }'

Results come back in request order, one entry per prompt, with the reuse reported on the envelope:

"usage": {
  "prompt_tokens": 35612,
  "completion_tokens": 94,
  "total_tokens": 35706,
  "prompt_tokens_details": { "cached_tokens": 25872 }
}

Performance

If your workload asks several questions of the same image or video, or wants multiple samples per prompt, Multilook does in one API call what would otherwise take many parallel /v1/chat/completions requests, while delivering better latency and cost.

Per-unit wall time at concurrency=16

Shapep50 to p95, seconds

P=4, n=1

P=8, n=1

P=4, n=4

Base routeMultilookfilled marker p50, hollow marker p95, shared axis 0 to 44 s

Median and 95th percentile per-unit wall time at the highest offered load. Multilook's median is roughly 2x lower at P=8 and 4.9x lower at P=4 n=4.

Single Multilook call latency with number of prompts (P)

SamplesP=2P=4P=8P=16

n=1

4.0 s

2.00 s / prompt

P=2, n=1
4.0 s for 2 completions
2.00 s per completion

3.7 s

0.93 s / prompt

P=4, n=1
3.7 s for 4 completions
0.93 s per completion

4.5 s

0.56 s / prompt

P=8, n=1
4.5 s for 8 completions
0.56 s per completion

8.6 s

0.54 s / prompt

P=16, n=1
8.6 s for 16 completions
0.54 s per completion

n=4

3.5 s

1.75 s / prompt

P=2, n=4
3.5 s for 8 completions
0.44 s per completion

6.1 s

1.52 s / prompt

P=4, n=4
6.1 s for 16 completions
0.38 s per completion

10.0 s

1.25 s / prompt

P=8, n=4
10.0 s for 32 completions
0.31 s per completion

18.3 s

1.14 s / prompt

P=16, n=4
18.3 s for 64 completions
0.29 s per completion

bars share one scale, 0 to 20 s · per-prompt time falls from 2.00 s at P=2 to 0.54 s at P=16

Latency grows sub-linearly with prompt count. Sixteen prompts return in 8.6 seconds, about 0.54 seconds per prompt, against 2.0 seconds per prompt at two prompts. The largest supported call, sixteen prompts at four samples, returns sixty-four completions in about eighteen seconds, or 0.29 seconds per completion.

On latency, adding prompts to a call costs far less than adding calls: the shared media is processed once, so per-prompt time falls as calls get bigger, multi-sample requests return in a fraction of the time the per-request pattern needs, and response times stay flat under load where the parallel pattern queues against its own duplicated work.

Input cost by prompts per call

PromptsAt input rateAt cache-read rateInput cost vs separate requests

P=29,1828,624

64% · 1.6× cheaper

P=49,74025,872

46% · 2.2× cheaper

P=810,85660,368

36% · 2.7× cheaper

P=1613,088129,360

32% · 3.1× cheaper

Separate requestsMultilookdashed line marks the 25% floor · input cost only

Input cost of one Multilook call against the same work sent as separate requests, at one sample per prompt. Cached tokens bill at $0.0375 per million against the $0.15 input rate, so the saving deepens as more prompts share the context, toward a floor of 25%. Completions bill identically on both routes, so a workload with long answers sees a smaller saving on the full bill.

On cost, the shared context is billed at full rate exactly once and every additional prompt reads it from cache at a fraction of the price, and each prompt is billed once no matter how many samples you draw. This creates savings that grow with the number of prompts and samples. In terms of implementation, it replaces a client-side orchestration problem. Now, with a single request, the media is sent once, and one response is grouped per prompt.

Built for production

Partial failure is per prompt. A prompt that fails returns an error object in place of its completions, and the others are unaffected. The call returns 200 if at least one prompt succeeded. If all of them fail, the request returns the first error's status.
A separate rate bucket. 150 requests per minute, independent of /v1/chat/completions, so a Multilook batch job does not eat your interactive budget.
Bounded by design. Up to 16 prompts, 64 completions, and 256 image or video inputs per call, inside a 20 MB body and a 300 second budget. On timeout, retry with fewer prompts, a lower n, or a smaller max_completion_tokens .

Applications

Robotics and automation. Teleop annotation against a standing annotation set: subtask boundaries, grasp attempts, success or failure, and an episode quality score for filtering.

Manufacturing and industrial. Continuous procedure and safety monitoring against an SOP, leveraging in-context visual examples, rather than fine-tuning.

Media and content. Content moderation where every upload requires a full policy pass (e.g., brand safety, age rating, caption).

Geospatial and critical infrastructure. One aerial view of a corridor queried for serveral characteristics, for example vegetation encroachment, hardware condition, and access road state.

Security and surveillance. Event monitoring and triage where every detection requires detailed analysis: what happened, is this actionable, and does it match exemplars of past false alarms.

Retail. Operational monitoring where a single shelf capture requires analysis of multiple attributes, for example stockouts and price tag legibility.

Getting startedMultilook is available today.

EndpointPOST api.perceptron.inc/v1/chat/completions/multilook

Model idperceptron-mk1

Input$0.15 / M tokens

Cache read$0.0375 / M tokens

Output$1.50 / M tokens

Prompts per call1 to 16

Completions per call64 (prompts x n)

Media inputs per call256, each video frame counts as one

Rate limit150 / min, separate bucket

Request budget300 s

Questions or higher limits: sales@perceptron.inc.

Get Started

You can start using Perceptron Multilook API today.

Create API Key

Read Docs

Get Started

We believe in building in the open and empowering the community. You can start building with Isaac today.

Create API Key

Read Docs