Today we're releasing Perceptron Agentic Detection, a specialized API built on top of our latest model, Perceptron Mk1 ("Mark One"). The Agentic Detection API enables precise localization of anything you can describe in natural language, with no fine-tuning required.
Mk1 already delivers step-change gains in visual understanding and embodied reasoning. Perceptron Agentic Detection takes those further, wrapping the model in an agentic harness that coordinates multiple Mk1 calls to resolve complex detection tasks.
What is Agentic Detection?
Traditional detectors are brittle. They’re built by fine-tuning a specialized model, which forces a fixed set of detection classes, or hand-engineering the perfect detection prompt. Agentic Detection addresses this.
With our Agentic Detection API, you can describe what you want in plain language, not only long-tail object classes, but also open-vocabulary labels and detections that require physical reasoning. Ask for "the source of the smoke" and the detection agent reasons about the scene to localize the origin of a forest fire; ask for “utility pole needing repair” and the detection agent identifies deformed energy assets; ask for "empty parking spots" and it finds them (a task many models struggle with).
Two things make this work:
A structured interface, no prompt engineering. Describe your detection categories in natural language and pass them through typed request fields. The API builds the optimal prompt for you, so there's no prompt tuning and no fine-tuning.
An agentic harness that queries and requeries. A built-in agent issues multiple Mk1 calls per request instead of a single forward pass, taking more than one look to resolve the small, dense, and ambiguous details that one-shot detectors miss at full-image resolution.
On top of that interface, Agentic Detection exposes three query types for the most common detection tasks:
Detect everything. If you do not specify detection labels, the API returns an exhaustive, structured inventory of the scene (people, objects, and artifacts), so you can catalog what's there before looking deeper.
Open-vocabulary categories. Hand it a list of detection classes, specified in natural language, and get back boxes or points for every instance.
Visual exemplars. Teach the agent a class by showing rather than naming it: pass one or more example crops (and optional negatives), and the detection agent finds more like them. This is how you find hyper-specific classes and one-off objects that have no good name.
In short, Perceptron Agentic Detection replaces brittle, single-purpose detectors with one model that’s steerable via plain language and visual examples.
Performance
Perceptron Agentic Detection exceeds frontier model detection performance, including Perceptron Mk1, Google’s Gemini, and Alibaba’s Qwen models.
Image Benchmark Results
Agentic Detection is built for production environments that depend on reliable visual understanding. It generalizes across radically different industries, environments, camera viewpoints, and even sensor modalities, all with the same API:
Geospatial intelligence. Wildfire detection from orbital imagery, locating the origin and extent of a fire. The same satellite and fixed-camera analysis can track activity at airports and other high-traffic zones, like construction sites.
Drones and aerial inspection. Detecting vehicles, freeway congestion, and road hazards from the air.
Utilities and energy. Detecting and analyzing power-pole components such as insulators and arrestors, and flagging vegetation encroachment on lines or hazards on rigs.
Robotics and automation. Embodied reasoning across human and robot viewpoints in cluttered, multi-instance, many-class scenes: pointing for grasps and multi-view understanding across cameras. Offline, curate teleoperation episodes into structured training data without human annotators.
Retail. Inventory detection and counting for shelf and stock analysis, as well as throughput measurements outside retail locations.
Automotive and autonomous vehicles. Identifying objects directly in LiDAR imagery for autonomous driving, driver assistance, and data annotation pipelines.
Agriculture. Ripe-fruit detection for robotic picking, identifying harvest-ready produce or weeds and invasive species in dense foliage.
Wearables and devices. Egocentric scene understanding from a head-mounted camera, extending to real-time perception on glasses or headsets and enhanced vision for text-first agents.
Getting Started
Perceptron Agentic Detection is available today via the Perceptron Platform.
API Endpoint:
api.perceptron.inc/v1/detectPricing: $0.15/M input tokens | $1.50/M output tokens
Safety and Responsible AI
While developing our agentic detection API, we have observed a larger risk profile for these detection systems. These systems need to be built carefully. As a result, we are experimenting with safety filters that screen detection requests for targeting-adjacent use: queries that localize individuals for harm, weapons systems guidance, and surveillance of protected categories.
With a select set of partners who have full access to our API, we're exploring appropriate guardrails. If that's of interest, reach out to support@perceptron.inc.
The Road Ahead
Agentic Detection is the first of a series of agentic APIs we're releasing to simplify how you interact with the Perceptron Mk1 model series. Each release in this series is intended to make multimodal intelligence more performant and more accessible.