# Replicate Rate Limits: Concurrency Caps, HTTP 429 Errors, and Agent Storage

Replicate rate limits govern the number of simultaneous model predictions and API requests an account can execute before returning HTTP 429 Too Many Requests errors. Production AI agents face two bottlenecks when scaling inference: request throttling from polling prediction endpoints and temporary file storage where outputs vanish after one hour. Using asynchronous webhooks, client-side semaphores, and persistent workspaces prevents pipeline failures.

Source: https://fast.io/resources/replicate-rate-limits/
Author: [Derek Labian](https://fast.io/authors/derek-labian/)
Last reviewed: 2026-10-24

## What Are Replicate Rate Limits and Default Concurrency Caps?

Replicate caps prediction creation at 600 requests per minute and general API endpoints at 3,000 requests per minute, while unverified accounts with granted credits operate under a strict ceiling of 1 request per second and 6 requests per minute, according to [Replicate's official rate limit documentation](https://replicate.com/docs/topics/predictions/rate-limits). In high-throughput AI agent architectures, rate limits and concurrency caps govern how many parallel inference tasks can execute before model requests fail.

Replicate rate limits govern the number of simultaneous model predictions and API requests an account can execute before returning HTTP 429 Too Many Requests errors.

Understanding these boundaries requires distinguishing between two distinct throttling mechanisms enforced by Replicate: request throughput limits and execution concurrency caps. Request throughput limits restrict the frequency of HTTP calls sent to the Replicate REST API, including prediction creation (`POST /v1/predictions`) and polling endpoints (`GET /v1/predictions/{id}`). Execution concurrency caps limit the number of machine learning models running simultaneously on Replicate's GPU clusters.

When developers deploy autonomous agents that generate images, transcribe audio, or run open-source language models, operations fail if the agent confuses API request capacity with hardware execution capacity. A script can submit 10 HTTP requests within a single second without exceeding the 600 requests per minute API limit. However, if that account holds an execution cap of two concurrent predictions, the third prediction either queues behind the running models or gets rejected with an HTTP 429 status code once queue thresholds are crossed.

### Replicate Account Tiers and Concurrency Boundaries

New accounts and accounts relying on trial credits start with strict concurrency allocations. As account spending increases and a verified payment method remains on file, Replicate raises concurrency thresholds to support production workloads.

| Account Status or Endpoint | Concurrency Cap | Request Rate Limit | Primary Error Code | Status and Reset Behavior |
|---|---|---|---|---|
| Granted Credit (No Card on File) | 1 to 2 concurrent jobs | 1 request per second (max 6/min) | HTTP 429 Too Many Requests | Throttled until credit reload or payment method added |
| Standard Paid Account | 10 concurrent predictions | 600 requests per minute | HTTP 429 Too Many Requests | Bursts tolerated; resets in approximately 30 seconds |
| High-Volume Organization Tier | 10 to 50+ concurrent predictions | Custom negotiated volume | HTTP 429 Too Many Requests | Custom hardware queue; higher sustained throughput |
| Query Endpoints (GET /v1/predictions) | No execution concurrency limit | 3,000 requests per minute | HTTP 429 Too Many Requests | Fixed sliding window reset |
| Payload Uploads (Binary in Request) | Not applicable | Standard gateway boundaries | HTTP 413 Payload Too Large | Immediate rejection; requires hosted URL input |

While short bursts above 600 requests per minute are temporarily accommodated by Replicate's gateway, sustained prediction creation exceeding 600 requests per minute triggers API rate limit throttling. Accounts with unpaid balances or depleting credit balances face dynamic throttling designed to prevent accidental overspending.

## Why Does Replicate Return HTTP 429 and HTTP 413 Errors?

Automated pipelines and multi-step agents encounter two recurring HTTP status codes when interacting with Replicate: HTTP 429 Too Many Requests and HTTP 413 Request Entity Too Large. Each status code signals a distinct failure mode in how client code interacts with Replicate's gateway and worker pools.

### The Mechanics of HTTP 429 Throttling

When an API client crosses a rate threshold or exceeds account concurrency boundaries, Replicate returns an HTTP 429 response containing a JSON body:

```json
{
  "detail": "Request was throttled. Your rate limit resets in ~30s."
}
```

Three primary operational conditions trigger this response:

1. Bursting Beyond Request Thresholds: Sending rapid bursts of requests to `POST /v1/predictions` faster than Replicate's token bucket algorithm allows. While brief bursts of 10 requests per second are tolerated, sustained prediction creation exceeding 600 requests per minute triggers API rate limit throttling.
2. Concurrency Pool Saturation: Replicate models run on physical hardware (such as Nvidia A100 or H100 GPUs). If an account is capped at 10 concurrent predictions and attempts to dispatch 15 parallel video generation jobs, the excess requests saturate the account queue. When shared hardware pools face high demand, Replicate sheds load by rejecting excess predictions with HTTP 429 errors.
3. Low Credit Balance Defensive Throttling: Replicate automatically enforces stricter rate limits as an account balance approaches zero. This mechanism prevents accounts from sliding into negative balances during high-concurrency tasks. Replicate recommends maintaining a credit balance above `$20` with auto-reload configured to prevent defensive throttling.

### The Mechanics of HTTP 413 Payload Failures

While HTTP 429 errors stem from timing and volume, HTTP 413 Request Entity Too Large errors stem from payload construction.

When developers use client libraries to run models on audio files, high-resolution source images, or video clips, passing a local file object directly into the input payload causes the client to serialize large binary streams into the HTTP request body:

```python
"""Problematic pattern: triggers HTTP 413 when audio exceeds body limits"""
output = client.run(
    "openai/whisper:cdd978ddf443c64bde530ddab65c28a501bbdbda16d1a9f4116db7a114e975a0",
    input={"audio": open("interview_recording_120mb.mp3", "rb")}
)
```

Replicate's API gateway rejects large binary payloads before the data ever reaches a model container. The documented architecture pattern requires hosting the input file on external cloud storage and passing a publicly accessible HTTPS URL to the input dictionary:

```python
"""Recommended pattern: passing a public HTTPS URL avoids HTTP 413"""
output = client.run(
    "openai/whisper:cdd978ddf443c64bde530ddab65c28a501bbdbda16d1a9f4116db7a114e975a0",
    input={"audio": "https://workspaces.example.com/audio/interview_recording.mp3"}
)
```

Staging source assets in cloud workspaces before initiating inference eliminates HTTP 413 errors and keeps API request bodies lightweight.

## How to Handle Throttling with Exponential Backoff and Concurrency Control

Autonomous agents executing multi-step workflows must manage outbound requests client-side rather than spraying unthrottled API calls. Combining an in-memory concurrency semaphore with exponential backoff and jitter ensures predictable execution under tight platform caps.

### Enforcing Client-Side Concurrency with Semaphores

If an account has an execution cap of 10 concurrent predictions, the client application should never dispatch 20 simultaneous requests. Using an asynchronous semaphore restricts active workers to match the platform concurrency tier:

```python
import asyncio
import os
import random
import time
import httpx

REPLICATE_API_TOKEN = os.environ.get("REPLICATE_API_TOKEN")
CONCURRENCY_LIMIT = 5
semaphore = asyncio.Semaphore(CONCURRENCY_LIMIT)

async def create_prediction_with_retry(
    client: httpx.AsyncClient,
    model_version: str,
    model_input: dict,
    max_retries: int = 5
) -> dict:
    url = "https://api.replicate.com/v1/predictions"
    headers = {
        "Authorization": f"Bearer {REPLICATE_API_TOKEN}",
        "Content-Type": "application/json"
    }
    payload = {
        "version": model_version,
        "input": model_input
    }
    async with semaphore:
        base_delay = 2.0
        for attempt in range(max_retries):
            response = await client.post(url, json=payload, headers=headers)
            if response.status_code == 201:
                return response.json()
            if response.status_code == 429:
                """Add randomized jitter to avoid thundering herd problem"""
                jitter = random.uniform(0.5, 1.5)
                delay = (base_delay * (2 ** attempt)) + jitter
                print(f"Throttled by Replicate (429). Retrying in {delay:.2f} seconds...")
                await asyncio.sleep(delay)
                continue
            """Non-retryable error"""
            response.raise_for_status()
        raise RuntimeError("Exceeded maximum retries for Replicate prediction creation.")
```

### Avoiding Synchronous Connection Exhaustion

Replicate provides a `Prefer: wait` HTTP header that tells the gateway to hold an HTTP connection open for 60 seconds while the model generates its result. While this synchronous mode is convenient for quick scripts, it creates severe bottlenecks in multi-agent systems:

- Holding connections open ties up local HTTP socket pools and client memory.
- If model generation takes 65 seconds, the HTTP connection times out even though the model is still executing on Replicate's cluster.
- The client re-issues the prediction, duplicating compute costs and consuming a second concurrency slot.

For production agents running parallel tasks, asynchronous prediction creation is the superior pattern. The client initiates the job, receives a prediction identifier, and releases the connection.

## Eliminating Polling Overhead with Asynchronous Webhooks

A common architecture flaw in AI agent pipelines is polling `GET /v1/predictions/{id}` in a tight loop to inspect completion status. While Replicate allows up to 3,000 requests per minute on non-creation endpoints, polling creates hidden operational traps as workloads scale.

### The Hidden Costs of Polling

Consider an autonomous pipeline processing a batch of 30 image upscaling tasks:

1. Each upscaling prediction takes 15 seconds to run on Replicate hardware.
2. An agent polling every two seconds per prediction generates roughly 8 calls per job.
3. Thirty concurrent jobs generate 240 polling requests in 15 seconds, creating high network chatter.
4. When multiple agents or workflow runs execute simultaneously, polling volume rapidly consumes the 3,000 requests per minute budget, starving other critical API calls.

Polling also introduces latency. If a prediction finishes 50 milliseconds after a poll check, the client waits nearly two seconds before discovering the result on the next polling cycle.

### Configuring Asynchronous Webhooks

Replicate provides first-class webhook delivery. When submitting `POST /v1/predictions`, developers can specify a public HTTPS webhook endpoint and filter the exact events that trigger callbacks:

```json
{
  "version": "50d859923762e880bf4b386192a73e01aa86cfa51fbe913e7737b1993c2c27bc",
  "input": {
    "prompt": "Architectural elevation drawing of an intelligent workspace library"
  },
  "webhook": "https://agent-gateway.example.com/api/webhooks/replicate",
  "webhook_events_filter": ["start", "completed"]
}
```

Filtering events with `["start", "completed"]` suppresses intermediate logging events (which Replicate throttles to once every 500 milliseconds) and ensures the webhook fires only when the job initiates and reaches a terminal state (`succeeded`, `failed`, or `canceled`).

### Implementing an Idempotent Webhook Receiver

Network partitions can cause Replicate to retry webhook delivery. A production webhook handler must be idempotent, acknowledging incoming events immediately and deduplicating events by prediction identifier:

```python
from fastapi import FastAPI, Request, status, Response

app = FastAPI()
processed_prediction_ids = set()

@app.post("/api/webhooks/replicate")
async def handle_replicate_webhook(request: Request):
    payload = await request.json()
    prediction_id = payload.get("id")
    status_val = payload.get("status")
    """Return 200 OK immediately to satisfy Replicate delivery timeout"""
    if not prediction_id:
        return Response(status_code=status.HTTP_400_BAD_REQUEST)
    if prediction_id in processed_prediction_ids and status_val == "succeeded":
        return Response(status_code=status.HTTP_200_OK)
    if status_val == "succeeded":
        processed_prediction_ids.add(prediction_id)
        output_urls = payload.get("output")
        """Dispatch background persistence task for generated media"""
        print(f"Prediction {prediction_id} succeeded. Output assets: {output_urls}")
    return Response(status_code=status.HTTP_200_OK)
```

Transitioning from active polling to event-driven webhooks frees up the 3,000 requests per minute API ceiling and provides sub-second notification when generation finishes.

## Solving the One-Hour Ephemeral Storage Ceiling for Generated Assets

Overcoming Replicate rate limits and 429 errors only solves the ingestion half of the pipeline. The second major architectural pitfall for autonomous agents is data retention.

For predictions created through the API, Replicate automatically deletes all input parameters, output values, output files, and logs after one hour, according to [Replicate's official data retention documentation](https://replicate.com/docs/topics/predictions/data-retention).

Output files generated by models (such as PNG images, MP4 videos, WAV audio files, or JSON text outputs) are hosted on temporary URLs under the `replicate.delivery` domain. If an AI agent saves the raw URL string into a database or context memory, that link becomes a dead 404 error sixty minutes later.

### Evaluating Storage Layers for AI Media Pipelines

Autonomous agents require persistent storage where generated media assets remain durable, auditable, and accessible to human team members:

1. Local Disk Storage: Saving generated files to a local server or container filesystem works for single-node scripts. However, local files cannot be easily shared with other team members, fail when container instances recycle in serverless cloud environments, and offer no search capabilities.
2. Object Storage (AWS S3 or Google Cloud Storage): Object storage provides durable file persistence. However, teams must configure custom IAM credentials, manage expiring pre-signed URLs for human review, and build separate search indexes to locate assets later.
3. Fast.io Intelligent Workspaces: Fast.io provides shared organization-owned workspaces designed for agentic teams. Rather than treating storage as an isolated bucket, a Fast.io workspace combines persistent cloud storage with built-in indexing and multi-agent coordination.

### Persisting Model Outputs into Fast.io Workspaces

When an autonomous agent receives an incoming webhook from Replicate, the payload delivers generated media URLs hosted on `replicate.delivery`. The agent can immediately ingest these assets into a Fast.io workspace before the temporary links expire.

Agents can pull files using URL import directly from external sources without routing heavy binary payloads through local machine memory, or upload files directly through Fast.io's remote Model Context Protocol (MCP) server.

Agents connect over Streamable HTTP to the Fast.io MCP endpoint:
- Coding agents (such as Claude Code, Cursor, Cline, or Devin) connect to `https://mcp.fast.io/mcp/code`.
- General AI assistants and autonomous orchestrators connect to `https://mcp.fast.io/mcp/tools`.

When an autonomous agent writes model outputs into a Fast.io workspace, the assets gain several operational capabilities:

- Built-in Intelligence and Search: When Intelligence is enabled on a workspace, files are indexed for hybrid search (full-text, semantic meaning, and metadata values). An agent or human can query the workspace using natural language to retrieve generated media, transcription notes, or research documents with exact source citations.
- Per-File Version History: If an agent re-runs an image refinement or upscaling model on an existing graphic, the workspace keeps every version. Team members can inspect the evolution of assets or restore previous versions at any time.
- Detailed Activity Logging: Fast.io maintains a detailed activity log recording what changed in an organization and who changed it, covering uploads, edits, moves, and deletions across people and agents alike.
- Scoped Team Sharing: Files and folders can be shared securely through branded shares (Send, Receive, or Exchange) with per-recipient access controls and download restrictions, without granting external parties access to the broader workspace.

Monthly plans start with a 14-day free trial that requires a credit card and ends early if trial credits run out. Plans are Starter at `$9.99/mo` (`3` seats, `250 GB` storage, `5` workspaces, `100,000` credits a month), Business at `$49.99/mo` (`10` seats, `5 TB` storage, `50` workspaces, `600,000` credits a month), and Enterprise at `$199.99/mo` (`30` seats included, `25 TB` storage, `200` workspaces, `3,000,000` credits a month). Credits meter AI operations, with overage at `$10` per `100,000` credits. Plan storage and seats are fixed allocations included with each subscription tier.

Pairing client-side rate limiters with durable workspace persistence allows engineering teams to run high-concurrency Replicate pipelines without hitting HTTP 429 bottlenecks or losing ephemeral model outputs.

## Frequently asked questions

### What is the rate limit for Replicate API?

Replicate limits prediction creation (POST /v1/predictions) to 600 requests per minute with bursts up to 10 requests per second. All other endpoints, such as checking prediction status or listing models, are limited to 3,000 requests per minute. Accounts operating on granted credits without an active payment method are restricted to 1 request per second and a maximum of 6 requests per minute.

### How many concurrent predictions can I run on Replicate?

Concurrency allocations depend on account billing status and account spend. Free and new accounts typically start with 1 to 2 concurrent predictions, scaling to 10 concurrent predictions on standard paid accounts and 50 or more concurrent predictions for high-volume enterprise accounts. Predictions return HTTP 429 status codes when account concurrency or hardware pool capacities are exceeded.

### How do I fix Replicate 429 Too Many Requests error?

Fix Replicate 429 errors by enforcing client-side concurrency caps with semaphores, implementing exponential backoff with jitter on retries, and switching from polling to asynchronous webhooks. In addition, ensure your account credit balance stays above `$20` with credit auto-reload enabled, because Replicate automatically throttles accounts that approach zero credits.

### How long do output URLs on replicate.delivery last?

Output files and delivery URLs generated through the Replicate API expire and are automatically deleted after approximately one hour. To prevent broken links and data loss in production pipelines, autonomous agents must download or transfer output assets to persistent cloud storage before the 60-minute window closes.

### What causes an HTTP 413 error on Replicate and how do I avoid it?

HTTP 413 Request Entity Too Large occurs when client code sends large binary files directly inside the API request payload. Avoid this error by uploading source files to an external cloud storage provider or workspace first, and then passing the public HTTPS URL in the prediction input dictionary.

### Why should autonomous agents use webhooks instead of polling Replicate?

Polling Replicate prediction endpoints generates hundreds of unnecessary HTTP requests that consume the 3,000 requests per minute limit and introduce polling latency. Asynchronous webhooks notify agent runtimes the instant model execution finishes, freeing API headroom and enabling immediate ingestion of generated files into persistent storage.

## Sources

- [Replicate Documentation: Rate limits](https://replicate.com/docs/topics/predictions/rate-limits): Replicate limits prediction creation to 600 requests per minute and other API endpoints to 3,000 requests per minute.
- [Replicate Documentation: Data retention](https://replicate.com/docs/topics/predictions/data-retention): Replicate automatically deletes prediction inputs, outputs, output files, and logs created through the API after one hour.

## About Fast.io

Fast.io provides shared workspaces where people and AI agents work on the same files, with built-in semantic search and citation-backed chat over what they hold. Agents reach it through a remote MCP server, a REST API at https://api.fast.io/current/, and a command line client published on npm as @vividengine/fastio-cli. MCP setup is at https://mcp.fast.io/docs: Claude and most MCP clients connect to https://mcp.fast.io/mcp/tools, ChatGPT to https://mcp.fast.io/mcp/operations, and coding agents to https://mcp.fast.io/mcp/code.