> ## AI Agent Context > Get the complete documentation index at: https://aistudio.google.com/llms.txt > Use this file to discover all available pages. > For the full documentation in a single file, fetch: https://aistudio.google.com/llms-full.txt # Gemini Robotics ER {% callout type="note" %} The Gemini Robotics ER model is in preview. {% /callout %} Gemini Robotics ER (embodied reasoning) models are vision-language models (VLMs) that let robots perceive and interact with the physical world. They interpret visual data, perform spatial and temporal reasoning, plan multi-step tasks, and orchestrate robots and tools. ## Models {% id="models" %} The Gemini Robotics ER 2 model is the latest model in Gemini Robotics. It is our updated reasoning model that enables robots to understand their environments precisely. It specializes in embodied reasoning capabilities, such as agentic orchestration of robots (e.g. using VLAs), robot video understanding including progress understanding and success detection, instrument reading, pointing, and spatial reasoning. The Gemini Robotics ER 2 model introduces two model endpoints: * **`gemini-robotics-er-2-preview`**: The standard ER 2 model. Builds on Gemini 3.5 Flash with improved spatial reasoning, video moment finding, video progress classification, multi-robot orchestration, and multi-step tool use. * **`gemini-robotics-er-2-streaming-preview`**: Optimized for real-time streaming using the [Live API](/docs/live-api/get-started-sdk). Use this model for low-latency robot agents that process continuous audio and video input. Gemini Robotics ER 1.6 was [deprecated on August 31, 2026](/docs/deprecations#robotics-models). If you are still using Gemini Robotics ER 1.6, upgrade to Gemini Robotics ER 2 by replacing `model="gemini-robotics-er-1.6-preview"` with `model="gemini-robotics-er-2-preview"` or `model="gemini-robotics-er-2-streaming-preview"` in your API calls. {% button href="https://aistudio.google.com/prompts/new_chat?model=gemini-robotics-er-2-preview" variant="ais" %}Try Gemini Robotics ER 2 in Google AI Studio{% /button %} ## Robotics capabilities {% id="capabilities" %} Gemini Robotics ER supports a range of embodied reasoning capabilities. Select a capability to learn more: {% table %} * Capability * Description * Guide --- * Spatial reasoning * Point to objects, track them in video, detect with bounding boxes, plan trajectories. * [Spatial reasoning](/docs/robotics-spatial) --- * Agentic vision * Use code execution to enhance other capabilities by leveraging image manipulation tools. * [Agentic vision](/docs/robotics-agentic) --- * Task orchestration * Combine spatial reasoning with custom robot APIs to complete long-horizon tasks. * [Task orchestration](/docs/robotics-orchestration) --- * Streaming (Gemini Robotics ER 2 Streaming endpoint only) * Bidirectional streaming for real-time robot agents with low-latency function calling. * [Streaming for robotics](/docs/robotics-streaming) --- * Video progress (Gemini Robotics ER 2 only) * Moment finding and progress classification from continuous video feeds. * [Video understanding](/docs/robotics-video-progress) {% /table %} ## Before you begin {% id="before-you-begin" %} The Gemini Robotics ER models are available through the **Gemini Developer API**. To get started, create a [Gemini API key](/docs/api-key) from [Google AI Studio](https://aistudio.google.com/apikey) or the [Google Cloud console](/docs/api-key#cloud-console), and [set up your environment](/docs/api-key#setup-environment). For details on key types, security, and how to configure your key in Python, JavaScript, and other languages, see [Using Gemini API keys](/docs/api-key). {% callout type="note" %} Cloud Storage URIs (`gs://`) are not directly accessible through the Gemini Developer API. Load your images locally or use the [Files API](/docs/files). {% /callout %} ## Getting started {% id="getting-started" %} The following example finds objects in an image and returns their normalized 2D coordinates and labels. You can pass this output directly to a robotics API or a VLA model to generate robot actions. {% tabs %} {% tab filename="robotics_overview.py" label="Python" %} ```python from google import genai PROMPT = """ Point to no more than 10 items in the image. The label returned should be an identifying name for the object detected. The answer should follow the json format: [{"point": , "label": }, ...]. The points are in [y, x] format normalized to 0-1000. """ client = genai.Client() uploaded_file = client.files.upload(file="my-image.png") image_response = client.interactions.create( model="gemini-robotics-er-2-preview", input=[ { "type": "image", "uri": uploaded_file.uri, "mime_type": uploaded_file.mime_type }, {"type": "text", "text": PROMPT} ], generation_config={"thinking_level": "high"}, ) print(image_response.output_text) ``` {% /tab %} {% tab label="JavaScript" %} ```javascript import { GoogleGenAI } from "@google/genai"; const PROMPT = ` Point to no more than 10 items in the image. The label returned should be an identifying name for the object detected. The answer should follow the json format: [{"point": , "label": }, ...]. The points are in [y, x] format normalized to 0-1000. `; const client = new GoogleGenAI(); const uploadedFile = await client.files.upload({ file: "my-image.png" }); const imageResponse = await client.interactions.create({ model: "gemini-robotics-er-2-preview", input: [ { type: "image", uri: uploadedFile.uri, mime_type: uploadedFile.mimeType, }, { type: "text", text: PROMPT }, ], generation_config: { thinking_level: "high" }, }); console.log(imageResponse.output_text); ``` {% /tab %} {% tab label="Java" %} ```java import com.google.genai.Client; import com.google.genai.gaos.models.interactions.Content; import com.google.genai.gaos.models.interactions.CreateModelInteraction; import com.google.genai.gaos.models.interactions.GenerationConfig; import com.google.genai.gaos.models.interactions.ImageContent; import com.google.genai.gaos.models.interactions.ImageContentMimeType; import com.google.genai.gaos.models.interactions.Interaction; import com.google.genai.gaos.models.interactions.InteractionsInput; import com.google.genai.gaos.models.interactions.Model; import com.google.genai.gaos.models.interactions.TextContent; import com.google.genai.gaos.models.interactions.ThinkingLevel; import com.google.genai.gaos.models.operations.CreateInteractionRequestBody; import com.google.genai.types.File; import com.google.genai.types.UploadFileConfig; import java.util.List; Client client = new Client(); String prompt = "Point to no more than 10 items in the image. The label returned " + "should be an identifying name for the object detected. " + "The answer should follow the json format: [{\"point\": , " + "\"label\": }, ...]. The points are in [y, x] format " + "normalized to 0-1000."; File uploadedFile = client.files.upload( new java.io.File("my-image.png"), UploadFileConfig.builder().mimeType("image/png").build()); Content imageContent = ImageContent.builder() .uri(uploadedFile.uri().orElse("")) .mimeType(ImageContentMimeType.of(uploadedFile.mimeType().orElse("image/png"))) .build(); Content textContent = TextContent.builder().text(prompt).build(); CreateModelInteraction params = CreateModelInteraction.builder() .model(Model.of("gemini-robotics-er-2-preview")) .input(InteractionsInput.ofContent(List.of(imageContent, textContent))) .generationConfig( GenerationConfig.builder().thinkingLevel(ThinkingLevel.HIGH).build()) .build(); Interaction imageResponse = client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get(); System.out.println(imageResponse.outputText().orElse("")); ``` {% /tab %} {% tab label="Go" %} ```go package main import ( "context" "fmt" "log" "google.golang.org/genai" "google.golang.org/genai/interactions/models/interactions" "google.golang.org/genai/interactions/models/operations" ) func main() { ctx := context.Background() client, err := genai.NewClient(ctx, nil) if err != nil { log.Fatal(err) } prompt := `Point to no more than 10 items in the image. The label returned should be an identifying name for the object detected. The answer should follow the json format: [{"point": , "label": }, ...]. The points are in [y, x] format normalized to 0-1000.` uploadedFile, err := client.Files.UploadFromPath(ctx, "my-image.png", &genai.UploadFileConfig{ MIMEType: "image/png", }) if err != nil { log.Fatal(err) } res, err := client.Interactions.Create(ctx, operations.CreateInteractionRequest{ Body: operations.NewCreateInteractionRequestBody(interactions.CreateModelInteraction{ Model: interactions.Model("gemini-robotics-er-2-preview"), Input: interactions.NewInteractionsInput([]interactions.Content{ interactions.NewContent(interactions.ImageContent{ URI: genai.Ptr(uploadedFile.URI), MimeType: genai.Ptr(interactions.ImageMimeTypeImagePng), }), interactions.NewContent(interactions.TextContent{ Text: prompt, }), }), GenerationConfig: &interactions.GenerationConfig{ ThinkingLevel: interactions.ThinkingLevelHigh.ToPointer(), }, }), }) if err != nil { log.Fatal(err) } if res.Interaction.OutputText != nil { fmt.Println(*res.Interaction.OutputText) } } ``` {% /tab %} {% tab filename="robotics_overview.sh" label="REST" %} ```bash # First, ensure you have the image file locally. # Encode the image to base64 IMAGE_BASE64=$(base64 -w 0 my-image.png) curl -X POST \ "https://generativelanguage.googleapis.com/v1beta/interactions" \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "gemini-robotics-er-2-preview", "input": { "parts": [ { "inlineData": { "mimeType": "image/png", "data": "'"${IMAGE_BASE64}"'" } }, { "text": "Point to no more than 10 items in the image. The label returned should be an identifying name for the object detected. The answer should follow the json format: [{\"point\": [y, x], \"label\": }, ...]. The points are in [y, x] format normalized to 0-1000." } ] }, "generation_config": { "thinking_config": { "thinking_level": "high" } } }' ``` {% /tab %} {% /tabs %} The output will be a JSON array containing objects, each with a `point` (normalized `[y, x]` coordinates) and a `label` identifying the object. {% tabs %} {% tab filename="output.json" label="JSON" %} ```json [ {"point": [376, 508], "label": "small banana"}, {"point": [287, 609], "label": "larger banana"}, {"point": [223, 303], "label": "pink starfruit"}, {"point": [435, 172], "label": "paper bag"}, {"point": [270, 786], "label": "green plastic bowl"}, {"point": [488, 775], "label": "metal measuring cup"}, {"point": [673, 580], "label": "dark blue bowl"}, {"point": [471, 353], "label": "light blue bowl"}, {"point": [492, 497], "label": "bread"}, {"point": [525, 429], "label": "lime"} ] ``` {% /tab %} {% /tabs %} The following image is an example of how these points can be displayed: ![](https://www.gstatic.com/cloud-cms/9520553bf3464dfdca8d4bf4b4da2951.png) ## How it works {% id="how-it-works" %} Gemini Robotics ER takes image, video, or audio input with natural language prompts. It identifies objects, reasons about scene context and spatial relationships, and returns structured output like coordinates or bounding boxes. Gemini Robotics ER is also agentic: it breaks complex tasks into sub-tasks and executes them by calling your robot functions or running generated code. For example, "put the apple in the bowl" becomes a sequence of locate, grasp, and place steps. See [Function calling](/docs/function-calling?example=meeting#how-it-works) for details on how Gemini executes tool calls. ## Safety {% id="safety" %} While Gemini Robotics ER was built with safety in mind, it is your responsibility to maintain a safe environment around the robot. Generative AI models can make mistakes, and physical robots can cause damage. To learn more, visit the [Gemini Robotics safety page](https://deepmind.google/models/gemini-robotics/safety). ## Best practices {% id="best-practices" %} 1. Use plain, natural language. Describe what you want the robot to do as you would to a person. If a term isn't working, try a common synonym. 2. Optimize visual input. Crop or zoom into small or unclear objects before sending the image. Lighting and low color contrast can affect detection. 3. Break complex tasks into steps. Send each step as a separate prompt to keep the model focused and improve accuracy. 4. Query multiple times and average results for high-precision tasks. This consensus approach reduces variance on spatial outputs. ## Limitations {% id="limitations" %} Consider the following limitations when developing with Gemini Robotics ER: * **API key restrictions:** The Gemini API does not accept requests from unrestricted API keys and returns a `403 Forbidden` error. Secure your API key by adding restrictions in [AI Studio](https://aistudio.google.com/api-keys). See [Secure unrestricted API keys](/docs/api-key#secure-unrestricted-keys) for details. * **Latency vs performance:** Complex queries, high-resolution inputs or high thinking levels can lead to increased processing times. For thinking level use medium for a good balance between latency and performance. * **Hallucinations:** Like all large language models, Gemini Robotics ER models can occasionally "hallucinate" or provide incorrect information, especially for ambiguous prompts or out-of-distribution inputs. * **Dependence on prompt quality:** Output quality depends on the clarity of the input prompt. Use specific, well-structured prompts. * **Computational cost:** Running the model, especially with video inputs or high `thinking_budget`, consumes computational resources and incurs costs. See the [Thinking](/docs/thinking) page for more details. * **Input types:** See the following topics for details on limitations for each mode. * [Image inputs](/docs/image-understanding#technical-details-image) * [Video inputs](/docs/video-understanding#supported-formats) * [Audio inputs](/docs/audio#supported-formats) ## Privacy Notice {% id="privacy-notice" %} You acknowledge that the models referenced in this document (the "Robotics Models") leverage video and audio data in order to operate and move your hardware in accordance with your instructions. You therefore may operate the Robotics Models such that data from identifiable persons, such as voice, imagery, and likeness data ("Personal Data"), will be collected by the Robotics Models. If you elect to operate the Robotics Models in a manner that collects Personal Data, you agree that you will not permit any identifiable persons to interact with, or be present in the area surrounding, the Robotics Models, unless and until such identifiable persons have been sufficiently notified of and consented to the fact that their Personal Data may be provided to and used by Google as outlined in the Gemini API Additional Terms of Service found at [https://ai.google.dev/gemini-api/terms](https://ai.google.dev/gemini-api/terms) (the "Terms"), including in accordance with the section entitled "How Google Uses Your Data". You will ensure that such notice permits the collection and use of Personal Data as outlined in the Terms, and you will use commercially reasonable efforts to minimize the collection and distribution of Personal Data by using techniques such as face blurring and operating the Robotics Models in areas not containing identifiable persons to the extent practicable. ## Pricing {% id="pricing" %} For detailed information on pricing and available regions, refer to the [pricing](/docs/pricing) page. ## Model endpoints {% id="model-versions" %} {% tabs %} {% tab title="Gemini Robotics ER 2 Preview" %} {% table %} * Property * Description --- * {% section class="model-property" %} {% icon name="id_card" /%}Model code {% /section %} * `gemini-robotics-er-2-preview` --- * {% section class="model-property" %} {% icon name="save" /%}Supported data types {% /section %} * **Inputs** Text, images, video, audio **Output** Text --- * {% section class="model-property" %} {% icon name="token_auto" /%}Token limits[[*]](/docs/tokens) {% /section %} * **Input token limit** 131,072 **Output token limit** 65,536 --- * {% section class="model-property" %} {% icon name="handyman" /%}Capabilities {% /section %} * {% section class="three-column-grid" %} {% section %} **[Audio generation](/docs/speech-generation)** Not supported {% /section %} {% section %} **[Caching](/docs/caching)** Supported {% /section %} {% section %} **[Code execution](/docs/code-execution)** Supported {% /section %} {% section %} **[Computer use](/docs/computer-use)** Supported {% /section %} {% section %} **[File search](/docs/file-search)** Supported {% /section %} {% section %} **[Function calling](/docs/function-calling)** Supported {% /section %} {% section %} **[Grounding with Google Maps](/docs/maps-grounding)** Supported {% /section %} {% section %} **[Image generation](/docs/image-generation)** Not supported {% /section %} {% section %} **[Live API](/docs/live-api)** Not supported {% /section %} {% section %} **[Search grounding](/docs/google-search)** Supported {% /section %} {% section %} **[Structured outputs](/docs/structured-output)** Supported {% /section %} {% section %} **[Thinking](/docs/thinking)** Supported {% /section %} {% section %} **[URL context](/docs/url-context)** Supported {% /section %} {% /section %} --- * {% section class="model-property" %} {% icon name="speed" /%}Consumption options {% /section %} * {% section class="three-column-grid" %} {% section %} **[Batch API](/docs/batch-api)** Supported {% /section %} {% section %} **[Flex inference](/docs/flex-inference)** Not supported {% /section %} {% section %} **[Priority inference](/docs/priority-inference)** Not supported {% /section %} {% /section %} --- * {% section class="model-property" %} {% icon name="123" /%}Versions {% /section %} * Read the [model version patterns](/docs/models#model-versions) for more details. Preview: `gemini-robotics-er-2-preview` --- * {% section class="model-property" %} {% icon name="calendar_month" /%}Latest update {% /section %} * July 2026 --- * {% section class="model-property" %} {% icon name="id_card" /%}Model card {% /section %} * [Model card](https://deepmind.google/models/model-cards/gemini-robotics-er-2/) --- * {% section class="model-property" %} {% icon name="paid" /%}Pricing {% /section %} * [Pricing](/docs/pricing#gemini-robotics-er-2) {% /table %} {% /tab %} {% tab title="Gemini Robotics ER 2 Streaming Preview" %} {% table %} * Property * Description --- * {% section class="model-property" %} {% icon name="id_card" /%}Model code {% /section %} * `gemini-robotics-er-2-streaming-preview` --- * {% section class="model-property" %} {% icon name="save" /%}Supported data types {% /section %} * **Inputs** Text, images, video, audio **Output** Text --- * {% section class="model-property" %} {% icon name="token_auto" /%}Token limits[[*]](/docs/tokens) {% /section %} * **Input token limit** 131,072 **Output token limit** 65,536 --- * {% section class="model-property" %} {% icon name="handyman" /%}Capabilities {% /section %} * {% section class="three-column-grid" %} {% section %} **[Audio generation](/docs/speech-generation)** Not supported {% /section %} {% section %} **[Caching](/docs/caching)** Not supported {% /section %} {% section %} **[Code execution](/docs/code-execution)** Not supported {% /section %} {% section %} **[Computer use](/docs/computer-use)** Not supported {% /section %} {% section %} **[File search](/docs/file-search)** Not supported {% /section %} {% section %} **[Function calling](/docs/function-calling)** Supported {% /section %} {% section %} **[Grounding with Google Maps](/docs/maps-grounding)** Not supported {% /section %} {% section %} **[Image generation](/docs/image-generation)** Not supported {% /section %} {% section %} **[Live API](/docs/live-api)** Supported {% /section %} {% section %} **[Search grounding](/docs/google-search)** Supported {% /section %} {% section %} **[Structured outputs](/docs/structured-output)** Not supported {% /section %} {% section %} **[Thinking](/docs/thinking)** Supported {% /section %} {% section %} **[URL context](/docs/url-context)** Not supported {% /section %} {% /section %} --- * {% section class="model-property" %} {% icon name="speed" /%}Consumption options {% /section %} * {% section class="three-column-grid" %} {% section %} **[Batch API](/docs/batch-api)** Not supported {% /section %} {% section %} **[Flex inference](/docs/flex-inference)** Not supported {% /section %} {% section %} **[Priority inference](/docs/priority-inference)** Not supported {% /section %} {% /section %} --- * {% section class="model-property" %} {% icon name="123" /%}Versions {% /section %} * Read the [model version patterns](/docs/models#model-versions) for more details. Preview: `gemini-robotics-er-2-streaming-preview` --- * {% section class="model-property" %} {% icon name="calendar_month" /%}Latest update {% /section %} * July 2026 --- * {% section class="model-property" %} {% icon name="id_card" /%}Model card {% /section %} * [Model card](https://deepmind.google/models/model-cards/gemini-robotics-er-2/) --- * {% section class="model-property" %} {% icon name="paid" /%}Pricing {% /section %} * [Pricing](/docs/pricing#gemini-robotics-er-2-streaming) {% /table %} {% /tab %} {% tab title="Gemini Robotics ER 1.6 Preview" %} {% table %} * Property * Description --- * {% section class="model-property" %} {% icon name="id_card" /%}Model code {% /section %} * `gemini-robotics-er-1.6-preview` --- * {% section class="model-property" %} {% icon name="save" /%}Supported data types {% /section %} * **Inputs** Text, images, video, audio **Output** Text --- * {% section class="model-property" %} {% icon name="token_auto" /%}Token limits[[*]](/docs/tokens) {% /section %} * **Input token limit** 131,072 **Output token limit** 65,536 --- * {% section class="model-property" %} {% icon name="handyman" /%}Capabilities {% /section %} * {% section class="three-column-grid" %} {% section %} **Audio generation** Not supported {% /section %} {% section %} **Batch API** Supported {% /section %} {% section %} **Caching** Supported {% /section %} {% section %} **Code execution** Supported {% /section %} {% section %} **Computer use** Supported {% /section %} {% section %} **File search** Supported {% /section %} {% section %} **Flex inference** Supported {% /section %} {% section %} **Function calling** Supported {% /section %} {% section %} **Grounding with Google Maps** Supported {% /section %} {% section %} **Image generation** Not supported {% /section %} {% section %} **Live API** Not supported {% /section %} {% section %} **Priority inference** Supported {% /section %} {% section %} **Search grounding** Supported {% /section %} {% section %} **Structured outputs** Supported {% /section %} {% section %} **Thinking** Supported {% /section %} {% section %} **URL context** Supported {% /section %} {% /section %} --- * {% section class="model-property" %} {% icon name="123" /%}Versions {% /section %} * Read the [model version patterns](/docs/models#model-versions) for more details. Shut down: `gemini-robotics-er-1.6-preview` --- * {% section class="model-property" %} {% icon name="calendar_month" /%}Latest update {% /section %} * December 2025 --- * {% section class="model-property" %} {% icon name="cognition_2" /%}Knowledge cutoff {% /section %} * January 2025 {% /table %} {% /tab %} {% /tabs %} ## What's next {% id="whats-next" %} * [Spatial reasoning](/docs/robotics-spatial) — pointing, tracking, bounding boxes, trajectories. * [Agentic capabilities](/docs/robotics-agentic) — code execution, instrument reading, image annotation. * [Task orchestration](/docs/robotics-orchestration) — long-horizon tasks with custom robot APIs. * [Robotics with streaming](/docs/robotics-streaming) — real-time bidirectional streaming (Gemini Robotics ER 2 only). * [Video understanding](/docs/robotics-video-progress) — moment finding and progress classification (Gemini Robotics ER 2 only). * [Gemini Robotics safety](https://deepmind.google/models/gemini-robotics/safety) — safety research behind the model family.