Skip to main content

OpenAI API: Batch Processing Guide

Batch processing allows you to submit multiple requests to the OpenAI API asynchronously and process them more efficiently, especially when working with large-scale tasks.

Why Use Batch Processing?​

Batch processing is ideal when you want to:

  • Submit large volumes of requests at once.
  • Avoid rate limits per second (batch runs under different limits).
  • Run longer or non-interactive jobs without needing immediate results.

Core Concepts​

Batch Job Lifecycle​

File Format​

  • Use JSONL (JSON Lines): Each line is a single request object.
  • Each line should contain the same structure as a regular POST /v1/chat/completions or other supported endpoint.

Example Line:

{"messages":[{"role":"user","content":"Tell me a joke."}],"model":"gpt-4.1-nano"}

Step-by-Step Guide​

1. Prepare Your Input File​

  • Create a .jsonl file containing all your individual requests.

2. Upload Your File​

Use the OpenAI file upload endpoint:

curl https://api.openai.com/v1/files \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -F "file=@my_requests.jsonl" \
  -F "purpose=batch"

3. Create the Batch Job​

curl https://api.openai.com/v1/batches \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file-abc123",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h",
    "metadata": {"project": "my-batch-project"}
  }'

4. Monitor Your Batch​

Check the status with:

curl https://api.openai.com/v1/batches/batch-abc123 \
  -H "Authorization: Bearer $OPENAI_API_KEY"

Status may be:

  • validating
  • in_progress
  • finalizing
  • completed
  • failed

5. Download the Results​

Once completed, download the output file via the provided URL in the batch object.

Python Example: Creating a Batch Job with OpenAI​

Here's how you can create and submit a batch job using the OpenAI Python SDK:

import openai

# Set your API key
openai.api_key = "your-api-key-here"

# Step 1: Upload the input JSONL file
upload_response = openai.File.create(
    file=open("my_requests.jsonl", "rb"),
    purpose="batch"
)

file_id = upload_response.id
print("Uploaded File ID:", file_id)

# Step 2: Create the batch job
batch_response = openai.Batch.create(
    input_file_id=file_id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
    metadata={"project": "my-python-batch"}
)

batch_id = batch_response.id
print("Batch ID:", batch_id)

# Step 3: Check the status
status = openai.Batch.retrieve(batch_id)
print("Batch Status:", status.status)

Note:

  • You must have a valid .jsonl input file named my_requests.jsonl.
  • Make sure to install the openai SDK using pip install openai before running the script.
  • You can periodically poll the batch status using openai.Batch.retrieve(batch_id).

This approach provides a cleaner and more maintainable workflow for developers using Python.

Key Considerations​

  • Maximum file size: 100 MB
  • Maximum requests: 50,000
  • Completion window: Options include 24h, 48h, etc.
  • Batches are asynchronous and not designed for real-time interactivity.

Use Cases​

  • Large-scale content generation (e.g., summaries, blogs)
  • Data labeling and classification
  • Backfilling datasets with new model outputs

Conclusion​

Batch processing with the OpenAI API is a powerful tool for handling large-scale or offline workloads efficiently. It optimizes throughput while simplifying the management of massive volumes of tasks.

For more, visit the official OpenAI Batch Guide.