> ## AI Agent Context
> - Full documentation index: https://aistudio.google.com/llms.txt
> - Gemini Docs MCP server (consistent live info): https://gemini-api-docs-mcp.dev
> - Coding agent skills & setup guide: https://aistudio.google.com/docs/coding-agents.md

# Audio understanding

Gemini can analyze audio input and generate text responses.

{% tabs %}
  {% tab label="Python" %}
  ```python
    from google import genai
  
    client = genai.Client()
  
    myfile = client.files.upload(file="path/to/sample.mp3")
  
    response = client.models.generate_content(
        model="gemini-4-hydrogen", contents=["Describe this audio clip", myfile]
    )
  
    print(response.text)
  ```

  {% /tab %}

  {% tab label="JavaScript" %}
  ```javascript
    import {
      GoogleGenAI,
      createUserContent,
      createPartFromUri,
    } from "@google/genai";
  
    const ai = new GoogleGenAI({});
  
    async function main() {
      const myfile = await ai.files.upload({
        file: "path/to/sample.mp3",
        config: { mimeType: "audio/mp3" },
      });
  
      const response = await ai.models.generateContent({
        model: "gemini-4-hydrogen",
        contents: createUserContent([
          createPartFromUri(myfile.uri, myfile.mimeType),
          "Describe this audio clip",
        ]),
      });
      console.log(response.text);
    }
  
    await main();
  ```

  {% /tab %}

  {% tab label="Go" %}
  ```go
    package main
  
    import (
        "context"
        "fmt"
        "log"
        "google.golang.org/genai"
    )
  
    func main() {
        ctx := context.Background()
        client, err := genai.NewClient(ctx, nil)
        if err != nil {
            log.Fatal(err)
        }
  
        localAudioPath := "/path/to/sample.mp3"
        uploadedFile, _ := client.Files.UploadFromPath(
            ctx,
            localAudioPath,
            nil,
        )
  
        parts := []*genai.Part{
            genai.NewPartFromText("Describe this audio clip"),
            genai.NewPartFromURI(uploadedFile.URI, uploadedFile.MIMEType),
        }
        contents := []*genai.Content{
            genai.NewContentFromParts(parts, genai.RoleUser),
        }
  
        result, _ := client.Models.GenerateContent(
            ctx,
            "gemini-4-hydrogen",
            contents,
            nil,
        )
  
        fmt.Println(result.Text())
    }
  ```

  {% /tab %}

  {% tab label="REST" %}
  ```shell
    AUDIO_PATH="path/to/sample.mp3"
    MIME_TYPE=$(file -b --mime-type "${AUDIO_PATH}")
    NUM_BYTES=$(wc -c < "${AUDIO_PATH}")
    DISPLAY_NAME=AUDIO
  
    tmp_header_file=upload-header.tmp
  
    # Initial resumable request defining metadata.
    # The upload url is in the response headers dump them to a file.
    curl "https://generativelanguage.googleapis.com/upload/v1beta/files" \
      -H "x-goog-api-key: $GEMINI_API_KEY" \
      -D upload-header.tmp \
      -H "X-Goog-Upload-Protocol: resumable" \
      -H "X-Goog-Upload-Command: start" \
      -H "X-Goog-Upload-Header-Content-Length: ${NUM_BYTES}" \
      -H "X-Goog-Upload-Header-Content-Type: ${MIME_TYPE}" \
      -H "Content-Type: application/json" \
      -d "{'file': {'display_name': '${DISPLAY_NAME}'}}" 2> /dev/null
  
    upload_url=$(grep -i "x-goog-upload-url: " "${tmp_header_file}" | cut -d" " -f2 | tr -d "\r")
    rm "${tmp_header_file}"
  
    # Upload the actual bytes.
    curl "${upload_url}" \
      -H "Content-Length: ${NUM_BYTES}" \
      -H "X-Goog-Upload-Offset: 0" \
      -H "X-Goog-Upload-Command: upload, finalize" \
      --data-binary "@${AUDIO_PATH}" 2> /dev/null > file_info.json
  
    file_uri=$(jq ".file.uri" file_info.json)
    echo file_uri=$file_uri
  
    # Now generate content using that file
    curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-4-hydrogen:generateContent" \
        -H "x-goog-api-key: $GEMINI_API_KEY" \
        -H 'Content-Type: application/json' \
        -X POST \
        -d '{
          "contents": [{
            "parts":[
              {"text": "Describe this audio clip"},
              {"file_data":{"mime_type": "'"${MIME_TYPE}"'", "file_uri": '$file_uri'}}]
            }]
          }' 2> /dev/null > response.json
  
    cat response.json
    echo
  
    jq ".candidates[].content.parts[].text" response.json
  ```

  {% /tab %}

{% /tabs %}

## Overview  {% id="overview" %}

Gemini can analyze and understand audio input and generate text responses to it,
enabling use cases like the following:

* Describe, summarize, or answer questions about audio content.
* Provide a transcription and translation of the audio (speech to text).
* Detect emotion in speech and music.
* Analyze specific segments of the audio, and provide timestamps.

As of now the Gemini API doesn't support real-time transcription use cases.
For real-time voice and video interactions refer to the [Live API](/docs/live-api).
For dedicated speech to text models with support for real-time transcription,
use the [Google Cloud Speech-to-Text API](https://cloud.google.com/speech-to-text).

## Transcribe speech to text  {% id="speech-to-text" %}

This example application shows how to prompt the Gemini API to transcribe,
translate, and summarize speech, including timestamps and emotion detection
using [structured outputs](/docs/structured-output).

{% tabs %}
  {% tab label="Python" %}
  ```python
    from google import genai
    from google.genai import types
  
    client = genai.Client()
  
    YOUTUBE_URL = "https://www.youtube.com/watch?v=ku-N-eS1lgM"
  
    def main():
      prompt = """
        Process the audio file and generate a detailed transcription.
  
        Requirements:
  
        1. Provide accurate timestamps for each segment (Format: MM:SS).
        2. Detect the primary language of each segment.
        3. If the segment is in a language different than English, also provide the English translation.
        4. Identify the primary emotion of the speaker in this segment. You MUST choose exactly one of the following: Happy, Sad, Angry, Neutral.
        5. Provide a brief summary of the entire audio at the beginning.
      """
  
      response = client.models.generate_content(
        model="gemini-4-hydrogen",
        contents=[
          types.Content(
            parts=[
              types.Part(
                file_data=types.FileData(
                  file_uri=YOUTUBE_URL
                )
              ),
              types.Part(
                text=prompt
              )
            ]
          )
        ],
        config=types.GenerateContentConfig(
          response_mime_type="application/json",
          response_schema=types.Schema(
            type=types.Type.OBJECT,
            properties={
              "summary": types.Schema(
                type=types.Type.STRING,
                description="A concise summary of the audio content.",
              ),
              "segments": types.Schema(
                type=types.Type.ARRAY,
                description="List of transcribed segments with timestamp.",
                items=types.Schema(
                  type=types.Type.OBJECT,
                  properties={
                    "timestamp": types.Schema(type=types.Type.STRING),
                    "content": types.Schema(type=types.Type.STRING),
                    "language": types.Schema(type=types.Type.STRING),
                    "language_code": types.Schema(type=types.Type.STRING),
                    "translation": types.Schema(type=types.Type.STRING),
                    "emotion": types.Schema(
                      type=types.Type.STRING,
                      enum=["happy", "sad", "angry", "neutral"]
                    ),
                  },
                  required=["timestamp", "content", "language", "language_code", "emotion"],
                ),
              ),
            },
            required=["summary", "segments"],
          ),
        ),
      )
  
      print(response.text)
  
    if __name__ == "__main__":
      main()
  ```

  {% /tab %}

  {% tab label="JavaScript" %}
  ```javascript
    import {
      GoogleGenAI,
      Type
    } from "@google/genai";
  
    const ai = new GoogleGenAI({});
  
    const YOUTUBE_URL = "https://www.youtube.com/watch?v=ku-N-eS1lgM";
  
    async function main() {
      const prompt = `
          Process the audio file and generate a detailed transcription.
  
          Requirements:
  
          1. Provide accurate timestamps for each segment (Format: MM:SS).
          2. Detect the primary language of each segment.
          3. If the segment is in a language different than English, also provide the English translation.
          4. Identify the primary emotion of the speaker in this segment. You MUST choose exactly one of the following: Happy, Sad, Angry, Neutral.
          5. Provide a brief summary of the entire audio at the beginning.
        `;
  
      const Emotion = {
        Happy: 'happy',
        Sad: 'sad',
        Angry: 'angry',
        Neutral: 'neutral'
      };
  
      const response = await ai.models.generateContent({
        model: "gemini-4-hydrogen",
        contents: {
          parts: [
            {
              fileData: {
                fileUri: YOUTUBE_URL,
              },
            },
            {
              text: prompt,
            },
          ],
        },
        config: {
          responseMimeType: "application/json",
          responseSchema: {
            type: Type.OBJECT,
            properties: {
              summary: {
                type: Type.STRING,
                description: "A concise summary of the audio content.",
              },
              segments: {
                type: Type.ARRAY,
                description: "List of transcribed segments with timestamp.",
                items: {
                  type: Type.OBJECT,
                  properties: {
                    timestamp: { type: Type.STRING },
                    content: { type: Type.STRING },
                    language: { type: Type.STRING },
                    language_code: { type: Type.STRING },
                    translation: { type: Type.STRING },
                    emotion: {
                      type: Type.STRING,
                      enum: Object.values(Emotion)
                    },
                  },
                  required: ["timestamp", "content", "language", "language_code", "emotion"],
                },
              },
            },
            required: ["summary", "segments"],
          },
        },
      });
      const json = JSON.parse(response.text);
      console.log(json);
    }
  
    await main();
  ```

  {% /tab %}

  {% tab label="REST" %}
  ```shell
    curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-4-hydrogen:generateContent" \
        -H "x-goog-api-key: $GEMINI_API_KEY" \
        -H 'Content-Type: application/json' \
        -X POST \
        -d '{
          "contents": [
            {
              "parts": [
                {
                  "file_data": {
                    "file_uri": "https://www.youtube.com/watch?v=ku-N-eS1lgM",
                    "mime_type": "video/mp4"
                  }
                },
                {
                  "text": "Process the audio file and generate a detailed transcription.\n\nRequirements:\n1. Provide accurate timestamps for each segment (Format: MM:SS).\n2. Detect the primary language of each segment.\n3. If the segment is in a language different than English, also provide the English translation.\n4. Identify the primary emotion of the speaker in this segment. You MUST choose exactly one of the following: Happy, Sad, Angry, Neutral.\n5. Provide a brief summary of the entire audio at the beginning."
                }
              ]
            }
          ],
          "generation_config": {
            "response_mime_type": "application/json",
            "response_schema": {
              "type": "OBJECT",
              "properties": {
                "summary": {
                  "type": "STRING",
                  "description": "A concise summary of the audio content."
                },
                "segments": {
                  "type": "ARRAY",
                  "description": "List of transcribed segments with timestamp.",
                  "items": {
                    "type": "OBJECT",
                    "properties": {
                      "timestamp": { "type": "STRING" },
                      "content": { "type": "STRING" },
                      "language": { "type": "STRING" },
                      "language_code": { "type": "STRING" },
                      "translation": { "type": "STRING" },
                      "emotion": {
                        "type": "STRING",
                        "enum": ["happy", "sad", "angry", "neutral"]
                      }
                    },
                    "required": ["timestamp", "content", "language", "language_code", "emotion"]
                  }
                }
              },
              "required": ["summary", "segments"]
            }
          }
        }' 2> /dev/null > response.json
  
    cat response.json
    echo
  
    jq ".candidates[].content.parts[].text" response.json
  ```

  {% /tab %}

{% /tabs %}

You can prompt [AI Studio Build](https://aistudio.google.com/apps?e=0) to create
an app just like
[this example transcription app](https://aistudio.google.com/apps/bundled/echoscript)
, with the click of a button.

![](https://www.gstatic.com/cloud-cms/33400c9cca6dbde2006c54e54d5e19a5.gif)

## Input audio  {% id="input-audio" %}

You can provide audio data to Gemini in the following ways:

* [Upload an audio file](/docs/audio#upload-audio) before making a request to
  `generateContent`.

* [Pass inline audio data](/docs/audio#inline-audio) with the request to
  `generateContent`.


To learn about other file input methods, see the
[File input methods](/docs/file-input-methods) guide.

### Upload an audio file  {% id="upload-audio" %}

You can use the [Files API](/docs/files) to upload an audio file.
Always use the Files API when the total request size (including the files, text
prompt, system instructions, etc.) is larger than 20 MB.

The following code uploads an audio file and then uses the file in a call to
`generateContent`.

{% tabs %}
  {% tab label="Python" %}
  ```python
    from google import genai
  
    client = genai.Client()
  
    myfile = client.files.upload(file="path/to/sample.mp3")
  
    response = client.models.generate_content(
        model="gemini-4-hydrogen", contents=["Describe this audio clip", myfile]
    )
  
    print(response.text)
  ```

  {% /tab %}

  {% tab label="JavaScript" %}
  ```javascript
    import {
      GoogleGenAI,
      createUserContent,
      createPartFromUri,
    } from "@google/genai";
  
    const ai = new GoogleGenAI({});
  
    async function main() {
      const myfile = await ai.files.upload({
        file: "path/to/sample.mp3",
        config: { mimeType: "audio/mp3" },
      });
  
      const response = await ai.models.generateContent({
        model: "gemini-4-hydrogen",
        contents: createUserContent([
          createPartFromUri(myfile.uri, myfile.mimeType),
          "Describe this audio clip",
        ]),
      });
      console.log(response.text);
    }
  
    await main();
  ```

  {% /tab %}

  {% tab label="Go" %}
  ```go
  package main
  
  import (
      "context"
      "fmt"
      "log"
      "google.golang.org/genai"
  )
  
  func main() {
      ctx := context.Background()
      client, err := genai.NewClient(ctx, nil)
      if err != nil {
          log.Fatal(err)
      }
  
      localAudioPath := "/path/to/sample.mp3"
      uploadedFile, _ := client.Files.UploadFromPath(
          ctx,
          localAudioPath,
          nil,
      )
  
      parts := []*genai.Part{
          genai.NewPartFromText("Describe this audio clip"),
          genai.NewPartFromURI(uploadedFile.URI, uploadedFile.MIMEType),
      }
      contents := []*genai.Content{
          genai.NewContentFromParts(parts, genai.RoleUser),
      }
  
      result, _ := client.Models.GenerateContent(
          ctx,
          "gemini-4-hydrogen",
          contents,
          nil,
      )
  
      fmt.Println(result.Text())
  }
  ```

  {% /tab %}

  {% tab label="REST" %}
  ```bash
    AUDIO_PATH="path/to/sample.mp3"
    MIME_TYPE=$(file -b --mime-type "${AUDIO_PATH}")
    NUM_BYTES=$(wc -c < "${AUDIO_PATH}")
    DISPLAY_NAME=AUDIO
  
    tmp_header_file=upload-header.tmp
  
    # Initial resumable request defining metadata.
    # The upload url is in the response headers dump them to a file.
    curl "https://generativelanguage.googleapis.com/upload/v1beta/files" \
      -H "x-goog-api-key: $GEMINI_API_KEY" \
      -D upload-header.tmp \
      -H "X-Goog-Upload-Protocol: resumable" \
      -H "X-Goog-Upload-Command: start" \
      -H "X-Goog-Upload-Header-Content-Length: ${NUM_BYTES}" \
      -H "X-Goog-Upload-Header-Content-Type: ${MIME_TYPE}" \
      -H "Content-Type: application/json" \
      -d "{'file': {'display_name': '${DISPLAY_NAME}'}}" 2> /dev/null
  
    upload_url=$(grep -i "x-goog-upload-url: " "${tmp_header_file}" | cut -d" " -f2 | tr -d "\r")
    rm "${tmp_header_file}"
  
    # Upload the actual bytes.
    curl "${upload_url}" \
      -H "Content-Length: ${NUM_BYTES}" \
      -H "X-Goog-Upload-Offset: 0" \
      -H "X-Goog-Upload-Command: upload, finalize" \
      --data-binary "@${AUDIO_PATH}" 2> /dev/null > file_info.json
  
    file_uri=$(jq ".file.uri" file_info.json)
    echo file_uri=$file_uri
  
    # Now generate content using that file
    curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-4-hydrogen:generateContent" \
        -H "x-goog-api-key: $GEMINI_API_KEY" \
        -H 'Content-Type: application/json' \
        -X POST \
        -d '{
          "contents": [{
            "parts":[
              {"text": "Describe this audio clip"},
              {"file_data":{"mime_type": "'"${MIME_TYPE}"'", "file_uri": '$file_uri'}}]
            }]
          }' 2> /dev/null > response.json
  
    cat response.json
    echo
  
    jq ".candidates[].content.parts[].text" response.json
  ```

  {% /tab %}

{% /tabs %}

To learn more about working with media files, see
[Files API](/docs/files).

### Pass audio data inline  {% id="inline-audio" %}

Instead of uploading an audio file, you can pass inline audio data in the
request to `generateContent`:

{% tabs %}
  {% tab label="Python" %}
  ```python
    from google import genai
    from google.genai import types
  
    with open('path/to/small-sample.mp3', 'rb') as f:
        audio_bytes = f.read()
  
    client = genai.Client()
    response = client.models.generate_content(
      model='gemini-4-hydrogen',
      contents=[
        'Describe this audio clip',
        types.Part.from_bytes(
          data=audio_bytes,
          mime_type='audio/mp3',
        )
      ]
    )
  
    print(response.text)
  ```

  {% /tab %}

  {% tab label="JavaScript" %}
  ```javascript
    import { GoogleGenAI } from "@google/genai";
    import * as fs from "node:fs";
  
    const ai = new GoogleGenAI({});
    const base64AudioFile = fs.readFileSync("path/to/small-sample.mp3", {
      encoding: "base64",
    });
  
    const contents = [
      { text: "Please summarize the audio." },
      {
        inlineData: {
          mimeType: "audio/mp3",
          data: base64AudioFile,
        },
      },
    ];
  
    const response = await ai.models.generateContent({
      model: "gemini-4-hydrogen",
      contents: contents,
    });
    console.log(response.text);
  ```

  {% /tab %}

  {% tab label="Go" %}
  ```go
  package main
  
  import (
      "context"
      "fmt"
      "log"
      "os"
      "google.golang.org/genai"
  )
  
  func main() {
      ctx := context.Background()
      client, err := genai.NewClient(ctx, nil)
      if err != nil {
          log.Fatal(err)
      }
  
      audioBytes, _ := os.ReadFile("/path/to/small-sample.mp3")
  
      parts := []*genai.Part{
          genai.NewPartFromText("Describe this audio clip"),
        &genai.Part{
          InlineData: &genai.Blob{
            MIMEType: "audio/mp3",
            Data:     audioBytes,
          },
        },
      }
      contents := []*genai.Content{
          genai.NewContentFromParts(parts, genai.RoleUser),
      }
  
      result, _ := client.Models.GenerateContent(
          ctx,
          "gemini-4-hydrogen",
          contents,
          nil,
      )
  
      fmt.Println(result.Text())
  }
  ```

  {% /tab %}

{% /tabs %}

A few things to keep in mind about inline audio data:

* The maximum request size is 20 MB, which includes text prompts,
  system instructions, and files provided inline. If your file's
  size will make the *total request size* exceed 20 MB, then
  use the Files API to [upload an audio file](/docs/audio#upload-audio) for use in
  the request.

* If you're using an audio sample multiple times, it's more efficient
  to [upload an audio file](/docs/audio#upload-audio).


## Get a transcript  {% id="transcript" %}

To get a transcript of audio data, just ask for it in the prompt:

{% tabs %}
  {% tab label="Python" %}
  ```python
    from google import genai
  
    client = genai.Client()
    myfile = client.files.upload(file='path/to/sample.mp3')
    prompt = 'Generate a transcript of the speech.'
  
    response = client.models.generate_content(
      model='gemini-4-hydrogen',
      contents=[prompt, myfile]
    )
  
    print(response.text)
  ```

  {% /tab %}

  {% tab label="JavaScript" %}
  ```javascript
    import {
      GoogleGenAI,
      createUserContent,
      createPartFromUri,
    } from "@google/genai";
  
    const ai = new GoogleGenAI({});
    const myfile = await ai.files.upload({
      file: "path/to/sample.mp3",
      config: { mimeType: "audio/mpeg" },
    });
  
    const result = await ai.models.generateContent({
      model: "gemini-4-hydrogen",
      contents: createUserContent([
        createPartFromUri(myfile.uri, myfile.mimeType),
        "Generate a transcript of the speech.",
      ]),
    });
    console.log("result.text=", result.text);
  ```

  {% /tab %}

  {% tab label="Go" %}
  ```go
  package main
  
  import (
      "context"
      "fmt"
      "log"
      "google.golang.org/genai"
  )
  
  func main() {
      ctx := context.Background()
      client, err := genai.NewClient(ctx, nil)
      if err != nil {
          log.Fatal(err)
      }
  
      localAudioPath := "/path/to/sample.mp3"
      uploadedFile, _ := client.Files.UploadFromPath(
          ctx,
          localAudioPath,
          nil,
      )
  
      parts := []*genai.Part{
          genai.NewPartFromText("Generate a transcript of the speech."),
          genai.NewPartFromURI(uploadedFile.URI, uploadedFile.MIMEType),
      }
      contents := []*genai.Content{
          genai.NewContentFromParts(parts, genai.RoleUser),
      }
  
      result, _ := client.Models.GenerateContent(
          ctx,
          "gemini-4-hydrogen",
          contents,
          nil,
      )
  
      fmt.Println(result.Text())
  }
  ```

  {% /tab %}

{% /tabs %}

## Refer to timestamps  {% id="timestamps" %}

You can refer to specific sections of an audio file using timestamps of the form
`MM:SS`. For example, the following prompt requests a transcript that

* Starts at 2 minutes 30 seconds from the beginning of the file.
* Ends at 3 minutes 29 seconds from the beginning of the file.

{% tabs %}
  {% tab label="Python" %}
  ```python
    # Create a prompt containing timestamps.
    prompt = "Provide a transcript of the speech from 02:30 to 03:29."
  ```

  {% /tab %}

  {% tab label="JavaScript" %}
  ```javascript
    // Create a prompt containing timestamps.
    const prompt = "Provide a transcript of the speech from 02:30 to 03:29."
  ```

  {% /tab %}

  {% tab label="Go" %}
  ```go
  package main
  
  import (
      "context"
      "fmt"
      "log"
      "google.golang.org/genai"
  )
  
  func main() {
      ctx := context.Background()
      client, err := genai.NewClient(ctx, nil)
      if err != nil {
          log.Fatal(err)
      }
  
      localAudioPath := "/path/to/sample.mp3"
      uploadedFile, _ := client.Files.UploadFromPath(
          ctx,
          localAudioPath,
          nil,
      )
  
      parts := []*genai.Part{
          genai.NewPartFromText("Provide a transcript of the speech " +
                                "between the timestamps 02:30 and 03:29."),
          genai.NewPartFromURI(uploadedFile.URI, uploadedFile.MIMEType),
      }
      contents := []*genai.Content{
          genai.NewContentFromParts(parts, genai.RoleUser),
      }
  
      result, _ := client.Models.GenerateContent(
          ctx,
          "gemini-4-hydrogen",
          contents,
          nil,
      )
  
      fmt.Println(result.Text())
  }
  ```

  {% /tab %}

{% /tabs %}

## Clip audio intervals  {% id="clipping-intervals" %}

Pass `audio_metadata` (`types.AudioMetadata`) on a `Part` or
`FunctionResponsePart` to inspect only a specific time range. Gemini counts
(`client.models.count_tokens`) and bills tokens only for the clipped duration:

* `start_offset` (string, optional): Start timestamp in seconds (such as
  `"30s"` or `"12.5s"`). Defaults to `"0s"`. Values past the audio length return
  `400 INVALID_ARGUMENT`.
* `end_offset` (string, optional): End timestamp in seconds. Defaults to the end
  of the file and clamps to the audio length if it exceeds the audio duration.
  Must be greater than `start_offset`.

{% tabs %}
  {% tab filename="audio_clip.py" label="Python" %}
  ```python
  from google import genai
  from google.genai import types
  
  client = genai.Client()
  
  podcast_audio = client.files.upload(file="interview.mp3")
  
  response = client.models.generate_content(
      model="gemini-4-hydrogen",
      contents=[
          types.Part(
              file_data=types.FileData(
                  file_uri=podcast_audio.uri,
                  mime_type="audio/mpeg",
              ),
              audio_metadata=types.AudioMetadata(
                  start_offset="60s",
                  end_offset="150s",
              ),
          ),
          "Summarize the speaker's main points in this segment.",
      ],
  )
  
  print(response.text)
  ```

  {% /tab %}

  {% tab filename="audio_clip.js" label="JavaScript" %}
  ```javascript
  import { GoogleGenAI } from "@google/genai";
  
  const ai = new GoogleGenAI({});
  
  const podcastAudio = await ai.files.upload({
    file: "interview.mp3",
    config: { mimeType: "audio/mpeg" },
  });
  
  const response = await ai.models.generateContent({
    model: "gemini-4-hydrogen",
    contents: [
      {
        fileData: {
          fileUri: podcastAudio.uri,
          mimeType: "audio/mpeg",
        },
        audioMetadata: {
          startOffset: "60s",
          endOffset: "150s",
        },
      },
      "Summarize the speaker's main points in this segment.",
    ],
  });
  
  console.log(response.text);
  ```

  {% /tab %}

  {% tab filename="audio_clip.sh" label="REST" %}
  ```shell
  curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-4-hydrogen:generateContent" \
    -H "Content-Type: application/json" \
    -H "x-goog-api-key: $GEMINI_API_KEY" \
    -X POST \
    -d '{
      "contents": [{
        "parts": [
          {
            "fileData": {
              "fileUri": "https://generativelanguage.googleapis.com/v1beta/files/AUDIO_FILE_ID",
              "mimeType": "audio/mpeg"
            },
            "audioMetadata": {
              "startOffset": "60s",
              "endOffset": "150s"
            }
          },
          {"text": "Summarize the speaker'\''s main points in this segment."}
        ]
      }]
    }'
  ```

  {% /tab %}

{% /tabs %}

## Count tokens  {% id="count-tokens" %}

Call the `countTokens` method to get a count of the number of tokens in an
audio file. For example:

{% tabs %}
  {% tab label="Python" %}
  ```python
  from google import genai
  
  client = genai.Client()
  
  myfile = client.files.upload(file="path/to/sample.mp3")
  
  response = client.models.count_tokens(
      model="gemini-4-hydrogen",
      contents=[myfile],
  )
  
  print(response.total_tokens)
  ```

  {% /tab %}

  {% tab label="JavaScript" %}
  ```javascript
  import {
    GoogleGenAI,
    createUserContent,
    createPartFromUri,
  } from "@google/genai";
  
  const ai = new GoogleGenAI({});
  const myfile = await ai.files.upload({
    file: "path/to/sample.mp3",
    config: { mimeType: "audio/mpeg" },
  });
  
  const countTokensResponse = await ai.models.countTokens({
    model: "gemini-4-hydrogen",
    contents: createUserContent([
      createPartFromUri(myfile.uri, myfile.mimeType),
    ]),
  });
  console.log(countTokensResponse.totalTokens);
  ```

  {% /tab %}

  {% tab label="Go" %}
  ```go
  package main
  
  import (
      "context"
      "fmt"
      "log"
      "google.golang.org/genai"
  )
  
  func main() {
      ctx := context.Background()
      client, err := genai.NewClient(ctx, nil)
      if err != nil {
          log.Fatal(err)
      }
  
      uploadedFile, err := client.Files.UploadFromPath(
          ctx,
          "path/to/sample.mp3",
          nil,
      )
      if err != nil {
          log.Fatal(err)
      }
  
      parts := []*genai.Part{
          genai.NewPartFromURI(uploadedFile.URI, uploadedFile.MIMEType),
      }
      contents := []*genai.Content{
          genai.NewContentFromParts(parts, genai.RoleUser),
      }
  
      result, err := client.Models.CountTokens(
          ctx,
          "gemini-4-hydrogen",
          contents,
          nil,
      )
      if err != nil {
          log.Fatal(err)
      }
  
      fmt.Println(result.TotalTokens)
  }
  ```

  {% /tab %}

{% /tabs %}

## Supported audio formats  {% id="supported-formats" %}

Gemini supports the following audio format MIME types:

* WAV - `audio/wav`
* MP3 - `audio/mp3`
* AIFF - `audio/aiff`
* AAC - `audio/aac`
* OGG - `audio/ogg`
* FLAC - `audio/flac`
* MPEG - `audio/mpeg`
* M4A - `audio/m4a`
* L16 - `audio/l16`
* Opus - `audio/opus`
* ALAW - `audio/alaw`
* MULAW - `audio/mulaw`
* WebM - `audio/webm`

For the complete list of supported MIME types and parameter schemas, see the [Interactions API reference](/api-reference/interactions-api#Resource:Content).

## Technical details about audio  {% id="technical-details" %}

* Gemini represents each second of audio as 32 tokens; for example,
  one minute of audio is represented as 1,920 tokens.

* Gemini can "understand" non-speech components, such as birdsong or sirens.

* The maximum supported length of audio data in a single prompt is 9.5 hours.
  Gemini doesn't limit the *number* of audio files in a single prompt; however,
  the total combined length of all audio files in a single prompt can't exceed
  9.5 hours.

* Gemini downsamples audio files to a 16 Kbps data resolution.

* If the audio source contains multiple channels, Gemini combines those channels
  into a single channel.


## What's next  {% id="whats-next" %}

This guide shows how to generate text in response to audio data. To learn more,
see the following resources:

* [File prompting strategies](/docs/files#prompt-guide): The
  Gemini API supports prompting with text, image, audio, and video data, also
  known as multimodal prompting.

* [System instructions](/docs/text-generation#system-instructions):
  System instructions let you steer the behavior of the model based on your
  specific needs and use cases.

* [Safety guidance](/docs/safety-guidance): Sometimes generative AI
  models produce unexpected outputs, such as outputs that are inaccurate,
  biased, or offensive. Post-processing and human evaluation are essential to
  limit the risk of harm from such outputs.


