Async Operations

Asynchronous operation patterns for long-running tasks

Extension URN: urn:mesh:ext:async

Overview

Async operations allow functions to return immediately while processing continues in the background. Clients poll for completion or receive callbacks.

When to Use

Async operations SHOULD be used for:

Report generation
Bulk data processing
External API calls with long latency
Operations exceeding typical deadline

Async operations SHOULD NOT be used for:

Simple CRUD operations
Low-latency requirements
Operations without observable progress

Request Format

Clients MAY request async handling via the async extension:

{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_123",
  "call": {
    "function": "reports.generate",
    "version": "1",
    "arguments": {
      "type": "annual",
      "year": 2024
    }
  },
  "extensions": [
    {
      "urn": "urn:mesh:ext:async",
      "options": {
        "preferred": true
      }
    }
  ]
}

Extension Options

Field	Type	Description
`preferred`	boolean	Client prefers async if operation is long-running
`callback_url`	string	URL to POST result when complete (optional)

Response Format

Immediate Async Response

When the server accepts the request for async processing:

{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_123",
  "result": null,
  "extensions": [
    {
      "urn": "urn:mesh:ext:async",
      "data": {
        "operation_id": "op_xyz789",
        "status": "processing",
        "poll": {
          "function": "mesh.operation.status",
          "version": "1",
          "arguments": { "operation_id": "op_xyz789" }
        },
        "retry_after": { "value": 5, "unit": "second" }
      }
    }
  ]
}

Extension Response Data

Field	Type	Required	Description
`operation_id`	string	Yes	Unique identifier for the operation
`status`	string	Yes	Current status (see below)
`poll`	object	Yes	Function call to check status
`retry_after`	object	No	Suggested wait before next poll
`progress`	number	No	Completion percentage (0.0 to 1.0)
`started_at`	string	No	ISO 8601 timestamp when processing started

Status Values

Status	Description
`pending`	Accepted but not yet started
`processing`	Currently being processed
`completed`	Finished successfully
`failed`	Finished with error
`cancelled`	Cancelled by client or system

Polling for Status

Poll Request

{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_poll_1",
  "call": {
    "function": "mesh.operation.status",
    "version": "1",
    "arguments": {
      "operation_id": "op_xyz789"
    }
  }
}

Still Processing Response

{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_poll_1",
  "result": {
    "operation_id": "op_xyz789",
    "status": "processing",
    "progress": 0.45,
    "started_at": "2024-01-15T10:30:00Z",
    "estimated_completion": "2024-01-15T10:31:00Z"
  }
}

Completed Response

{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_poll_2",
  "result": {
    "operation_id": "op_xyz789",
    "status": "completed",
    "output": {
      "report_url": "https://storage.example.com/reports/annual_2024.pdf",
      "generated_at": "2024-01-15T10:31:23Z",
      "page_count": 47
    }
  }
}

Failed Response

{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_poll_3",
  "result": null,
  "errors": [{
    "code": "ASYNC_OPERATION_FAILED",
    "message": "Report generation failed: data source unavailable",
    "retryable": true,
    "details": {
      "operation_id": "op_xyz789",
      "failed_at": "2024-01-15T10:30:45Z",
      "reason": "database_connection_timeout"
    }
  }]
}

Callbacks

Instead of polling, clients MAY provide a callback URL:

Request with Callback

{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_callback",
  "call": {
    "function": "reports.generate",
    "version": "1",
    "arguments": { "type": "annual", "year": 2024 }
  },
  "extensions": [
    {
      "urn": "urn:mesh:ext:async",
      "options": {
        "preferred": true,
        "callback_url": "https://my-service.example.com/webhooks/mesh"
      }
    }
  ]
}

Callback Payload

When the operation completes, the server MUST POST to the callback URL:

{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "callback": {
    "operation_id": "op_xyz789",
    "original_request_id": "req_callback",
    "status": "completed",
    "result": {
      "report_url": "https://storage.example.com/reports/annual_2024.pdf"
    },
    "completed_at": "2024-01-15T10:31:23Z"
  }
}

Callback Security

Servers SHOULD:

Sign callbacks with HMAC or similar
Include signature in header: X-Mesh-Signature: sha256=...
Support callback URL allowlists

Clients SHOULD:

Verify callback signatures
Validate callback source
Respond with 200 to acknowledge

Cancellation

Clients MAY cancel pending operations:

Cancel Request

{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_cancel",
  "call": {
    "function": "mesh.operation.cancel",
    "version": "1",
    "arguments": {
      "operation_id": "op_xyz789"
    }
  }
}

Cancel Response

{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_cancel",
  "result": {
    "operation_id": "op_xyz789",
    "status": "cancelled",
    "cancelled_at": "2024-01-15T10:30:30Z"
  }
}

Cannot Cancel

If the operation has already completed or cannot be cancelled:

{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_cancel",
  "result": null,
  "errors": [{
    "code": "ASYNC_CANNOT_CANCEL",
    "message": "Operation already completed",
    "retryable": false,
    "details": {
      "operation_id": "op_xyz789",
      "status": "completed"
    }
  }]
}

Server Behavior

When to Use Async

Servers SHOULD respond asynchronously when:

Client specified options.preferred: true
Operation would exceed reasonable time (e.g., > 30s)
Operation involves external dependencies with high latency

Servers MAY respond synchronously even with preferred: true if:

Operation completes quickly
Operation is cached

Operation Lifecycle

┌─────────┐     ┌────────────┐     ┌───────────┐
│ pending │────▶│ processing │────▶│ completed │
└─────────┘     └────────────┘     └───────────┘
                      │
                      ▼
                ┌──────────┐
                │  failed  │
                └──────────┘

Operation Storage

Servers SHOULD:

Store operation state durably
Set TTL on completed operations (e.g., 24 hours)
Return NOT_FOUND for expired operation IDs

Idempotency

Async operations work with the Idempotency extension:

{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_123",
  "call": {
    "function": "reports.generate",
    "version": "1",
    "arguments": { "type": "annual" }
  },
  "extensions": [
    {
      "urn": "urn:mesh:ext:async",
      "options": {
        "preferred": true
      }
    },
    {
      "urn": "urn:mesh:ext:idempotency",
      "options": {
        "key": "report_annual_2024"
      }
    }
  ]
}

If the same idempotency key is used:

Servers MUST return existing operation ID if still processing
Servers MUST return completed result if finished

Examples

Full Async Flow

1. Initial Request

{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_report",
  "call": {
    "function": "reports.generate",
    "version": "1",
    "arguments": {
      "type": "sales",
      "date_range": {
        "start": "2024-01-01",
        "end": "2024-12-31"
      }
    }
  },
  "context": {
    "trace_id": "tr_report123",
    "span_id": "sp_init"
  },
  "extensions": [
    {
      "urn": "urn:mesh:ext:async",
      "options": {
        "preferred": true
      }
    }
  ]
}

2. Immediate Response

{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_report",
  "result": null,
  "extensions": [
    {
      "urn": "urn:mesh:ext:async",
      "data": {
        "operation_id": "op_sales_report_456",
        "status": "processing",
        "poll": {
          "function": "mesh.operation.status",
          "version": "1",
          "arguments": { "operation_id": "op_sales_report_456" }
        },
        "retry_after": { "value": 10, "unit": "second" }
      }
    }
  ]
}

3. Poll (Still Processing)

// Request
{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_poll_1",
  "call": {
    "function": "mesh.operation.status",
    "version": "1",
    "arguments": { "operation_id": "op_sales_report_456" }
  }
}

// Response
{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_poll_1",
  "result": {
    "operation_id": "op_sales_report_456",
    "status": "processing",
    "progress": 0.67,
    "message": "Processing Q3 data..."
  }
}

4. Poll (Complete)

{
  "protocol": { "name": "mesh", "version": "0.1.0" },
  "id": "req_poll_2",
  "result": {
    "operation_id": "op_sales_report_456",
    "status": "completed",
    "output": {
      "report_id": "rpt_789",
      "download_url": "https://...",
      "expires_at": "2024-01-16T10:30:00Z",
      "summary": {
        "total_sales": 1250000,
        "record_count": 45000
      }
    }
  }
}