Redis & queues

Monitor and clear worker BullMQ queues in Redis.

3 min read

Connect your agent today

Draft from chat, review in your calendar, and publish only what you approve.

Overview

Workers and the API share REDIS_* (and optional REDIS_BULLMQ_DB). Configure connection strings on Redis cache. Read Cache design for source of truth and connection limits. This page focuses on BullMQ queue keys and safe redis-cli usage.

Pick the correct DB

If you set REDIS_BULLMQ_DB, use that DB index when running redis-cli (with -n). Do not run destructive commands against the wrong DB.

Local development

Connect

# Local dev Redis (no password)
redis-cli -h localhost -p 6379 -n 0 ping

Find BullMQ keys for a queue

Replace <QUEUE_NAME> with your queue name (for example scheduled-social-post).

redis-cli -h localhost -p 6379 -n 0 --scan --pattern "bull:<QUEUE_NAME>:*"

Inspect queue sizes (common keys)

BullMQ typically uses:

bull:<QUEUE_NAME>:wait (list)
bull:<QUEUE_NAME>:active (list)
bull:<QUEUE_NAME>:paused (list)
bull:<QUEUE_NAME>:delayed (zset)
bull:<QUEUE_NAME>:completed (zset)
bull:<QUEUE_NAME>:failed (zset)

redis-cli -h localhost -p 6379 -n 0 LLEN "bull:<QUEUE_NAME>:wait"
redis-cli -h localhost -p 6379 -n 0 LLEN "bull:<QUEUE_NAME>:active"
redis-cli -h localhost -p 6379 -n 0 ZCARD "bull:<QUEUE_NAME>:delayed"
redis-cli -h localhost -p 6379 -n 0 ZCARD "bull:<QUEUE_NAME>:failed"
redis-cli -h localhost -p 6379 -n 0 ZCARD "bull:<QUEUE_NAME>:completed"

Clear a local queue (debugging)

If you are iterating on worker behavior and want to remove stuck jobs in a local Redis DB, it’s safer to clear by queue prefix in that local DB.

Do not do this against production Redis

Only run bulk delete/clear steps against your local dev Redis. In production, prefer stopping workers and letting BullMQ retries settle, or use a targeted cleanup plan.

# List the keys first
redis-cli -h localhost -p 6379 -n 0 --scan --pattern "bull:<QUEUE_NAME>:*"

# Delete all BullMQ keys for the queue
redis-cli -h localhost -p 6379 -n 0 --scan --pattern "bull:<QUEUE_NAME>:*" | xargs -n 50 redis-cli -h localhost -p 6379 -n 0 DEL

The safest pattern is to use SCAN (not KEYS) when exploring key sets.

Production

Monitor queues

Be careful with production Redis

Prefer read-only commands. Use SCAN (not KEYS) and double-check the DB index (REDIS_BULLMQ_DB).

Export REDIS_HOST, REDIS_PORT, REDIS_PASSWORD, and the logical database index (for BullMQ, usually REDIS_BULLMQ_DB; fall back to REDIS_DB if unset). If your host requires TLS (for example Redis Cloud), add —tls to every redis-cli invocation, or use a rediss:// URL (see Redis cache and your provider’s connection string).

redis-cli -h "$REDIS_HOST" -p "$REDIS_PORT" -a "$REDIS_PASSWORD" -n "$REDIS_BULLMQ_DB" ping

With TLS (typical for managed endpoints):

redis-cli --tls -h "$REDIS_HOST" -p "$REDIS_PORT" -a "$REDIS_PASSWORD" -n "$REDIS_BULLMQ_DB" ping

Inspect memory and key counts (bloat check)

Read-only: confirm how much memory the instance uses, whether a maxmemory cap is set, and the eviction policy (BullMQ expects noeviction in production; see BullMQ → Redis OOM).

redis-cli -h "$REDIS_HOST" -p "$REDIS_PORT" -a "$REDIS_PASSWORD" -n "$REDIS_BULLMQ_DB" INFO memory

Clearing Redis keys (last resort)

If you have to do destructive cleanup, stop the worker service(s) first (see Railway → Stopping a worker service) so no process races with your deletes. Keep this as a last resort; prefer fixing retention in code and a short, targeted plan.

Be careful with FLUSHDB

FLUSHDB deletes everything in the selected logical Redis DB (-n): BullMQ queues, Flowcraft run state, notification digests, and (if you use it) API cache. Only use it if you are sure the DB is safe to wipe.

Recommended: RedPanda cleanup script (prefix deletes)

This removes RedPanda-owned prefixes with SCAN + UNLINK (not FLUSHDB), using the same Redis connection settings as the app:

pnpm --filter openquok-orchestrator script:free-openquok-redis-memory -- --targets=bull,digest,flowcraft

To also clear cache keys (under REDIS_PREFIX, typically app:cache:):

pnpm --filter openquok-orchestrator script:free-openquok-redis-memory -- --targets=bull,digest,flowcraft,cache

Manual (redis-cli): clear BullMQ queues by prefix

redis-cli -h "$REDIS_HOST" -p "$REDIS_PORT" -a "$REDIS_PASSWORD" -n "$REDIS_BULLMQ_DB" --scan --pattern "bull:notification-email:*" | xargs -n 200 redis-cli -h "$REDIS_HOST" -p "$REDIS_PORT" -a "$REDIS_PASSWORD" -n "$REDIS_BULLMQ_DB" UNLINK
redis-cli -h "$REDIS_HOST" -p "$REDIS_PORT" -a "$REDIS_PASSWORD" -n "$REDIS_BULLMQ_DB" --scan --pattern "bull:scheduled-social-post:*" | xargs -n 200 redis-cli -h "$REDIS_HOST" -p "$REDIS_PORT" -a "$REDIS_PASSWORD" -n "$REDIS_BULLMQ_DB" UNLINK
redis-cli -h "$REDIS_HOST" -p "$REDIS_PORT" -a "$REDIS_PASSWORD" -n "$REDIS_BULLMQ_DB" --scan --pattern "bull:integration-refresh:*" | xargs -n 200 redis-cli -h "$REDIS_HOST" -p "$REDIS_PORT" -a "$REDIS_PASSWORD" -n "$REDIS_BULLMQ_DB" UNLINK

Nuclear option: wipe the whole DB (FLUSHDB)

redis-cli -h "$REDIS_HOST" -p "$REDIS_PORT" -a "$REDIS_PASSWORD" -n "$REDIS_DB" FLUSHDB

Re-check DBSIZE / INFO memory after cleanup, then re-deploy or start the worker(s) when finished.

Next steps

Docker (local Redis)

Compose Redis and local worker scripts

Railway (workers)

Deploy and operate always-on worker services

Redis cache

REDIS_* variables, TLS, and provider setup