Rate Limiting

Control throughput for steps that hit rate-limited APIs or limited resources.

Rate limiter

Token bucket algorithm for requests-per-second limits:

import { createRateLimiter, rateLimiterPresets } from 'awaitly/ratelimit';

// Custom config
const rateLimiter = createRateLimiter('api-calls', {
  maxPerSecond: 10,        // Maximum operations per second
  burstCapacity: 20,       // Allow brief spikes (default: maxPerSecond * 2)
  strategy: 'wait',        // 'wait' (default) or 'reject'
});

// Wrap operations
const data = await rateLimiter.execute(async () => {
  return await callExternalApi();
});

// Or use presets
const apiLimiter = createRateLimiter('external-api', rateLimiterPresets.api);
// { maxPerSecond: 10, burstCapacity: 20, strategy: 'wait' }

const externalLimiter = createRateLimiter('partner-api', rateLimiterPresets.external);
// { maxPerSecond: 5, burstCapacity: 10, strategy: 'wait' }

Concurrency limiter

Limit parallel operations (for database connections, file handles, etc.):

import { createConcurrencyLimiter, rateLimiterPresets } from 'awaitly/ratelimit';

const concurrencyLimiter = createConcurrencyLimiter('db-pool', {
  maxConcurrent: 5,        // Max 5 concurrent operations
  maxQueueSize: 100,       // Queue up to 100 waiting requests
  strategy: 'queue',       // 'queue' (default) or 'reject'
});

const data = await concurrencyLimiter.execute(async () => {
  return await db.query('SELECT * FROM users');
});

// Or use preset
const dbLimiter = createConcurrencyLimiter('database', rateLimiterPresets.database);
// { maxConcurrent: 10, strategy: 'queue', maxQueueSize: 100 }

Fixed window limiter

Simpler alternative to token bucket. Resets at fixed intervals:

import { createFixedWindowLimiter } from 'awaitly/ratelimit';

const limiter = createFixedWindowLimiter('api', {
  limit: 100,          // 100 requests
  windowMs: 60000,     // per minute
  strategy: 'wait',    // 'wait' (default) or 'reject'
});

const data = await limiter.execute(async () => callApi());

// Check status
const stats = limiter.getStats();
console.log(stats.requestCount);  // Requests in current window
console.log(stats.remainingMs);   // Time until window reset

Cost-based rate limiter

Different operations can have different costs:

import { createCostBasedRateLimiter } from 'awaitly/ratelimit';

const limiter = createCostBasedRateLimiter('api', {
  tokensPerSecond: 100,  // 100 tokens/second refill
  maxTokens: 200,        // Can burst up to 200 tokens
  strategy: 'wait',
});

// Simple query costs 1 token (default)
await limiter.execute(() => simpleQuery());

// Batch operation costs 10 tokens
await limiter.execute(() => batchOperation(), 10);

// Heavy export costs 50 tokens
await limiter.execute(() => exportData(), 50);

Combined limiter

Apply both rate and concurrency limits:

import { createCombinedLimiter } from 'awaitly/ratelimit';

const limiter = createCombinedLimiter('api', {
  rate: { maxPerSecond: 10 },
  concurrency: { maxConcurrent: 3 },
});

const data = await limiter.execute(async () => callApi());

Result-returning operations

const result = await rateLimiter.executeResult(async () => {
  return ok(await callExternalApi());
});

if (!result.ok) {
  // Handle error (could be rate limit rejection or operation error)
}

Batch operations

Process many items with bounded concurrency:

const results = await concurrencyLimiter.executeAll(
  ids.map(id => async () => fetchItem(id))
);

// results: Array of outcomes

Monitoring

const stats = rateLimiter.getStats();
console.log(stats.availableTokens);  // Current available tokens
console.log(stats.waitingCount);     // Requests waiting for tokens

Strategies

Wait strategy (default)

Requests wait until capacity is available:

const limiter = createRateLimiter('api', {
  maxPerSecond: 10,
  strategy: 'wait', // Queues requests
});

Reject strategy

Requests fail immediately when at capacity:

const limiter = createRateLimiter('api', {
  maxPerSecond: 10,
  strategy: 'reject', // Throws RateLimitExceededError
});

try {
  await limiter.execute(() => callApi());
} catch (error) {
  if (isRateLimitExceededError(error)) {
    console.log('Rate limit exceeded, try later');
  }
}

In workflows

const apiLimiter = createRateLimiter('partner-api', rateLimiterPresets.external);

const result = await workflow.run(async ({ step }) => {
  // Rate-limited API calls
  const users = await step('fetchUsers', async () => {
    const ids = ['1', '2', '3', '4', '5'];
    return await apiLimiter.executeAll(
      ids.map(id => async () => fetchUser(id))
    );
  });

  return users;
});

Configuration reference

Rate limiter

{
  maxPerSecond: number;      // Operations per second (required)
  burstCapacity?: number;    // Max burst (default: maxPerSecond * 2)
  strategy?: 'wait' | 'reject';
}

Concurrency limiter

{
  maxConcurrent: number;     // Max parallel operations (required)
  maxQueueSize?: number;     // Max waiting requests (default: Infinity)
  strategy?: 'queue' | 'reject';
}

Presets

Preset	Type	Config
rateLimiterPresets.api	Rate	10/s, burst 20
rateLimiterPresets.external	Rate	5/s, burst 10
rateLimiterPresets.database	Concurrency	10 concurrent, queue 100