Rate Limiting

Rate Limiting

Rate limit communication and handling

---

Overview

Rate limiting protects services from overload. Mesh provides standardized mechanisms for:

1. Proactive visibility — Clients see usage before hitting limits
2. Clear errors — Structured error when limits are exceeded
3. Recovery guidance — Explicit retry timing

---

Rate Limit Metadata

Servers SHOULD include rate limit information in response meta for every request:

{
  "meta": {
    "duration": { "value": 127, "unit": "millisecond" },
    "rate_limit": {
      "limit": 1000,
      "used": 153,
      "remaining": 847,
      "window": { "value": 1, "unit": "minute" },
      "resets_in": { "value": 45, "unit": "second" }
    }
  }
}

Fields

Field	Type	Required	Description
limit	integer	Yes	Maximum requests allowed in the window
used	integer	Yes	Requests used in current window
remaining	integer	Yes	Requests remaining in current window
window	object	Yes	Time window duration (value/unit)
resets_in	object	Yes	Time until window resets (value/unit)

---

Rate Limit Error

When a client exceeds the rate limit:

{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_123",
  "result": null,
  "errors": [{
    "code": "RATE_LIMITED",
    "message": "Rate limit exceeded",
    "retryable": true,
    "details": {
      "limit": 1000,
      "used": 1000,
      "window": { "value": 1, "unit": "minute" },
      "retry_after": { "value": 45, "unit": "second" }
    }
  }],
  "meta": {
    "rate_limit": {
      "limit": 1000,
      "used": 1000,
      "remaining": 0,
      "window": { "value": 1, "unit": "minute" },
      "resets_in": { "value": 45, "unit": "second" }
    }
  }
}

Error Details

Field	Type	Description
limit	integer	Maximum requests allowed
used	integer	Requests used (equal to limit when exceeded)
window	object	Time window duration
retry_after	object	Recommended wait before retrying

---

Rate Limit Scopes

Rate limits MAY apply at different scopes:

Scope	Description
global	Across all clients
service	Per calling service
function	Per function
user	Per authenticated user

When multiple scopes apply, include all in metadata:

{
  "meta": {
    "rate_limits": {
      "global": {
        "limit": 10000,
        "used": 4523,
        "remaining": 5477,
        "window": { "value": 1, "unit": "minute" },
        "resets_in": { "value": 32, "unit": "second" }
      },
      "service": {
        "limit": 1000,
        "used": 847,
        "remaining": 153,
        "window": { "value": 1, "unit": "minute" },
        "resets_in": { "value": 32, "unit": "second" }
      }
    }
  }
}

---

Client Behavior

Proactive Throttling

Clients SHOULD monitor meta.rate_limit.remaining and throttle requests before hitting limits:

if (remaining < threshold) {
    delay = calculate_backoff(remaining, resets_in)
    wait(delay)
}

Handling RateLimited

When receiving RATE_LIMITED error:

1. Extract retry_after from error details
2. Wait the specified duration
3. Retry with exponential backoff if still limited
4. Set maximum retry attempts

Clients MUST NOT retry immediately without waiting.

Backoff Strategy

Recommended exponential backoff:

wait_time = min(retry_after * (2 ^ attempt), max_wait)

Where:

• retry_after — From error details
• attempt — Retry attempt number (0, 1, 2, …)
• max_wait — Maximum wait time (e.g., 5 minutes)

---

Server Implementation

Rate Limit Algorithms

Common algorithms:

Algorithm	Description
Fixed Window	Reset counter at fixed intervals
Sliding Window	Rolling time window
Token Bucket	Tokens replenish over time
Leaky Bucket	Requests drain at constant rate

Response Requirements

Servers implementing rate limiting MUST:

1. Return RATE_LIMITED error when limit exceeded
2. Include retry_after in error details
3. Set retryable: true on the error

Servers SHOULD:

1. Include rate_limit in meta on every response
2. Use consistent window boundaries across requests
3. Document rate limit policies

Graceful Degradation

Servers MAY implement soft limits:

{
  "meta": {
    "rate_limit": {
      "limit": 1000,
      "used": 950,
      "remaining": 50,
      "window": { "value": 1, "unit": "minute" },
      "resets_in": { "value": 15, "unit": "second" },
      "warning": "Approaching rate limit"
    }
  }
}

---

Discovery

Clients can discover rate limit policies via system functions:

// Request
{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_limits",
  "call": {
    "function": "mesh.capabilities",
    "version": "1",
    "arguments": {}
  }
}

// Response
{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_limits",
  "result": {
    "service": "orders-api",
    "limits": {
      "rate_limits": [
        {
          "scope": "service",
          "limit": 1000,
          "window": { "value": 1, "unit": "minute" }
        },
        {
          "scope": "function",
          "function": "orders.create",
          "limit": 100,
          "window": { "value": 1, "unit": "minute" }
        }
      ]
    }
  }
}

---

Examples

Normal Response with Rate Limit Info

{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_123",
  "result": {
    "order_id": 456,
    "status": "created"
  },
  "meta": {
    "duration": { "value": 89, "unit": "millisecond" },
    "rate_limit": {
      "limit": 1000,
      "used": 42,
      "remaining": 958,
      "window": { "value": 1, "unit": "minute" },
      "resets_in": { "value": 47, "unit": "second" }
    }
  }
}

Approaching Limit

{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_456",
  "result": { "success": true },
  "meta": {
    "duration": { "value": 34, "unit": "millisecond" },
    "rate_limit": {
      "limit": 1000,
      "used": 985,
      "remaining": 15,
      "window": { "value": 1, "unit": "minute" },
      "resets_in": { "value": 12, "unit": "second" },
      "warning": "Rate limit nearly exhausted"
    }
  }
}

Rate Limited

{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_789",
  "result": null,
  "errors": [{
    "code": "RATE_LIMITED",
    "message": "Rate limit exceeded for orders.create",
    "retryable": true,
    "details": {
      "limit": 100,
      "used": 100,
      "window": { "value": 1, "unit": "minute" },
      "retry_after": { "value": 23, "unit": "second" },
      "scope": "function",
      "function": "orders.create"
    }
  }],
  "meta": {
    "rate_limit": {
      "limit": 100,
      "used": 100,
      "remaining": 0,
      "window": { "value": 1, "unit": "minute" },
      "resets_in": { "value": 23, "unit": "second" }
    }
  }
}