Request Cancellation

Stop long-running agent requests and free resources

Ask ChatGPT Ask Claude.ai Open in Cursor

Cancel in-flight requests to stop processing, free up resources, and prevent unnecessary LLM costs.

Quick Start

import asyncio
from muxi import AsyncFormationClient

formation = AsyncFormationClient(
    server_url="http://localhost:7890",
    formation_id="my-assistant",
    client_key="<your-client-key>",
)

# Start a request
request_id = "my-request-123"
task = asyncio.create_task(
    formation.chat(
        {"message": "Write a very long story...", "request_id": request_id},
        user_id="user123",
    )
)

# Cancel it
await formation.cancel_request(request_id, "user123")

import { FormationClient } from '@muxi/sdk';

const formation = new FormationClient({
  serverUrl: 'http://localhost:7890',
  formationId: 'my-assistant',
  clientKey: '<your-client-key>'
});

// Start a request
const requestId = 'my-request-123';
const task = formation.chat({
  message: 'Write a very long story...',
  request_id: requestId
}, 'user123');

// Cancel it
await formation.cancelRequest(requestId, 'user123');

formation := muxi.NewFormationClient(&muxi.FormationConfig{
    ServerURL:   "http://localhost:7890",
    FormationID: "my-assistant",
    ClientKey:   "<your-client-key>",
})

// Start a request in another goroutine
requestID := "my-request-123"
go formation.Chat(ctx, &muxi.ChatRequest{
    Message: "Write a very long story...",
    RequestID: requestID,
    UserID: "user123",
})

// Cancel it
formation.CancelRequest(ctx, requestID, "user123")

curl -X DELETE 'http://localhost:7890/api/my-assistant/v1/requests/my-request-123' \
  -H 'X-Muxi-Client-Key: YOUR_CLIENT_KEY' \
  -H 'X-Muxi-User-Id: user123'

How It Works

Graceful Termination

1. User calls cancelRequest()
         ↓
2. Request marked as "cancelled"
         ↓
3. Processing continues until next checkpoint
         ↓
4. Checkpoint detects cancellation
         ↓
5. Processing stops, resources freed

Key point: Cancellation is graceful, not immediate. Processing stops at the next safe checkpoint.

Checkpoints

Cancellation is checked after every long-running operation:

Operation	Typical Duration	Checkpoint
LLM calls	1-30+ seconds	After each call
MCP tool invocations	1-60+ seconds	After call returns
A2A agent requests	5-60+ seconds	After call returns
Task decomposition	2-10 seconds	After LLM call

Use Cases

Cancel After Timeout

import asyncio
import uuid

from muxi import AsyncFormationClient

formation = AsyncFormationClient(
    server_url="http://localhost:7890",
    formation_id="my-assistant",
    client_key="<your-client-key>",
)

async def chat_with_timeout(message: str, timeout_seconds: int = 30):
    request_id = f"req_{uuid.uuid4()}"

    try:
        response = await asyncio.wait_for(
            formation.chat(
                {"message": message, "request_id": request_id},
                user_id="user123",
            ),
            timeout=timeout_seconds
        )
        return response
    except asyncio.TimeoutError:
        # Timeout - cancel the request
        await formation.cancel_request(request_id, "user123")
        raise TimeoutError(f"Request timed out after {timeout_seconds}s")

import { FormationClient } from '@muxi/sdk';

const formation = new FormationClient({
  serverUrl: 'http://localhost:7890',
  formationId: 'my-assistant',
  clientKey: '<your-client-key>'
});

async function chatWithTimeout(message: string, timeoutMs = 30000) {
  const requestId = req_${Date.now()};

  const timeoutPromise = new Promise((_, reject) => {
    setTimeout(async () => {
      await formation.cancelRequest(requestId, 'user123');
      reject(new Error(Request timed out after ${timeoutMs}ms));
    }, timeoutMs);
  });

  return Promise.race([
    formation.chat({ message, request_id: requestId }, 'user123'),
    timeoutPromise
  ]);
}

func chatWithTimeout(ctx context.Context, message string, timeout time.Duration) (*muxi.Response, error) {
    requestID := fmt.Sprintf("req_%d", time.Now().UnixNano())

    ctx, cancel := context.WithTimeout(ctx, timeout)
    defer cancel()

    response, err := formation.Chat(ctx, &muxi.ChatRequest{
        Message:   message,
        RequestID: requestID,
        UserID:    userID,
    })

    if ctx.Err() == context.DeadlineExceeded {
        formation.CancelRequest(context.Background(), requestID, userID)
        return nil, fmt.Errorf("request timed out after %v", timeout)
    }

    return response, err
}

Cancel Streaming Request

from muxi import FormationClient

formation = FormationClient(
    server_url="http://localhost:7890",
    formation_id="my-assistant",
    client_key="<your-client-key>",
)

request_id = "streaming-123"
chunks_received = 0

for chunk in formation.chat_stream(
    {"message": "Write a long essay...", "request_id": request_id},
    user_id="user123",
):
    print(chunk.get("text", ""), end="")
    chunks_received += 1

    # Cancel after receiving 10 chunks
    if chunks_received >= 10:
        formation.cancel_request(request_id, "user123")
        print("
[Cancelled]")
        break

import { FormationClient } from '@muxi/sdk';

const formation = new FormationClient({
  serverUrl: 'http://localhost:7890',
  formationId: 'my-assistant',
  clientKey: '<your-client-key>'
});

const requestId = 'streaming-123';
let chunksReceived = 0;

for await (const chunk of formation.chatStream(
  { message: 'Write a long essay...', request_id: requestId },
  'user123'
)) {
  if (typeof chunk.token === 'string') {
    process.stdout.write(chunk.token);
    chunksReceived++;
  }

  // Cancel after receiving 10 chunks
  if (chunksReceived >= 10) {
    await formation.cancelRequest(requestId, 'user123');
    console.log('
[Cancelled]');
    break;
  }
}

Cancel Button in UI

import { useState } from 'react';
import { FormationClient } from '@muxi/sdk';

const formation = new FormationClient({
  serverUrl: 'http://localhost:7890',
  formationId: 'my-assistant',
  clientKey: '<your-client-key>'
});

function ChatInterface() {
  const [requestId, setRequestId] = useState<string | null>(null);
  const [isProcessing, setIsProcessing] = useState(false);

  const sendMessage = async (message: string) => {
    const id = req_${Date.now()};
    setRequestId(id);
    setIsProcessing(true);

    try {
      const response = await formation.chat(
        { message, request_id: id },
        'user123'
      );
      // Handle response...
    } finally {
      setIsProcessing(false);
      setRequestId(null);
    }
  };

  const cancelRequest = async () => {
    if (requestId) {
      await formation.cancelRequest(requestId, 'user123');
      setIsProcessing(false);
      setRequestId(null);
    }
  };

  return (
    <div>
      {/* Chat UI */}
      {isProcessing && (
        <button onClick={cancelRequest}>Cancel</button>
      )}
    </div>
  );
}

Response Behavior

Cancelled Response

DELETE /v1/requests/{request_id} returns the standard API envelope:

{
  "object": "request_status",
  "timestamp": 1785843861099,
  "type": "request.cancelled",
  "request": { "id": "req_KmmLwVssTNetTpTm8IqnS", "idempotency_key": null },
  "success": true,
  "error": null,
  "data": {
    "request_id": "req_123",
    "status": "cancelled",
    "cancellation": "cooperative",
    "message": "Request marked for cancellation; it will stop at the next checkpoint"
  }
}

How strong is the guarantee?

data.cancellation tells you which of two things just happened:

Value	Meaning
`cooperative`	The cancellation flag is set; the request stops at its next checkpoint. This is what an ordinary chat turn gets.
`immediate`	The underlying task was cancelled outright and is already stopped. Requests tracked with a live background task - such as background workflow executions - get this.

In both cases status is "cancelled" and the call succeeds. The difference is whether the work has already stopped or is about to.

Status codes

Situation	Status	Body
Request in flight (either mode)	200	Success envelope above
Request already cancelled	200	Success envelope, `cancellation: "cooperative"`
Unknown `request_id`	404	`error.code: "NOT_FOUND"`
Request already completed or failed	400	`error.code: "OPERATION_FAILED"`, message names the final status
Request belongs to another user	403	`error.code: "FORBIDDEN"`

A request that finishes in the moment between your check and your cancel returns 400 rather than pretending to have cancelled it - and leaves no stale cancellation flag behind.

Cancelling the same request multiple times is safe - subsequent calls return the same 200 success envelope. Only completed and failed are uncancellable; cancelled is deliberately not, so a retry of your own cancel is a no-op rather than an error.

Cannot cancel during	Can cancel between
Active LLM call	LLM calls
MCP tool execution	Tool invocations
Database transaction	Agent decisions

Once an LLM call starts, it runs to completion. Cancellation takes effect at the next checkpoint.

Streaming Partial Results

Already-sent chunks cannot be recalled:

Chunk 1: "Hello"  ✓ Sent
Chunk 2: "there"  ✓ Sent
[User cancels]
Chunk 3: "friend" ✗ Not sent

User received: "Hello there" (cannot unsend)

Cost Savings

Cancellation saves money by preventing subsequent LLM calls:

Long task started: 10,000 tokens expected
User cancels after first LLM call: 1,500 tokens used
         ↓
Remaining 8,500 tokens NOT consumed
         ↓
Remaining model and tool work is avoided

Best Practices

Always show cancel button for long operations
Use timeouts - don't let requests run forever
Track request IDs - keep a map of active requests
Handle gracefully - request may complete before cancel arrives

Learn More

Request Status - poll a request for its state and result
Async Retry Escalation - what cancelling a retry chain does
Async Operations - Background task management
Streaming - Cancel streaming responses
Observability - Monitor cancellations