Upstash

Upstash learning module

Upstash is a serverless data platform that provides Redis, vector, search, messaging (QStash) and durable workflow services over HTTP, with pay-per-request pricing and no servers to manage.

• HTTP-based Redis fits serverless: no connection pools, works on the edge.
• QStash is at-least-once: verify signatures and make handlers idempotent.
• Workflow persists each step, so a failure retries only that step.
• Pick the rate-limit algorithm and identifier for the traffic you expect.

Stateful infrastructure that does not fit serverless: Serverless functions spin up by the thousand and die within seconds; opening a TCP connection to a self-hosted Redis from each one exhausts connection limits, cold-start latency grows, and an always-on cluster is paid for even at zero traffic. There is also nowhere to run retries, delays or cron jobs. Upstash replaces the connection with a stateless HTTP request, bills per request with scale to zero, and moves background delivery and durable multi-step execution into managed services the function simply calls.

QStash callbacks

Point a message at a callback URL so the result arrives as a normal HTTP request.

• Configure a callback URL for QStash delivery results.
• Verify callback authenticity before trusting its status payload.
• Correlate the result with the original message identifier.

QStash failure callbacks

Receive a request when every retry has been exhausted, instead of finding it in a dashboard.

• Send exhausted-delivery notifications to an authorized failure endpoint.
• Record the last destination error and original message reference.
• Make recovery actions idempotent if callbacks are retried.

Workflow failure functions

Handle a workflow run that could not complete, in code, next to the workflow itself.

• Use the Workflow failure hook for terminal run failures.
• Include run and step identifiers in diagnosis.
• Keep compensation safe to repeat and separately observable.

Workflow steps

context.run records each step's result, which is what makes a resumed run skip finished work.

• Wrap durable units of work in named Workflow steps.
• Return serializable results required by later steps.
• Keep external side effects safe across execution retries.

Ratelimit results

The limit call returns remaining and reset, which is what a response header should carry.

• Inspect success, remaining allowance and reset information.
• Return a clear rate-limit response when permission is denied.
• Complete pending analytics work using the runtime's supported lifetime mechanism.

blockUntilReady

Wait for capacity up to a deadline rather than rejecting a request immediately.

• Wait only up to the configured timeout for capacity.
• Handle a denied result when the waiting budget expires.
• Avoid holding latency-sensitive requests indefinitely.

Client retry configuration

The REST client retries with backoff; set it deliberately for a serverless caller.

• Set retry counts and delay behavior deliberately.
• Distinguish a failed request from an uncertain write outcome.
• Bound retries to the enclosing operation's deadline.

REST and HTTP clients

Call Redis without a persistent connection from an edge or serverless runtime.

• Use the REST client in runtimes where persistent TCP is unsuitable.
• Keep database URL and authorization token in server configuration.
• Reuse the client and account for per-request network latency.

Core data types

Use strings, hashes, lists, sets and sorted sets for the right access pattern.

• Choose strings, hashes, sets or sorted sets by access pattern.
• Use atomic commands for counters and membership changes.
• Check supported commands before adopting a Redis-specific extension.

Expiry and eviction

Set TTLs deliberately and know what happens when memory fills.

• Set explicit TTLs for temporary data.
• Distinguish expired keys from entries removed under memory pressure.
• Keep durable business records outside an eviction-dependent cache.

Pipelines

Batch commands into one round trip instead of paying per call.

• Batch independent Redis commands to reduce HTTP overhead.
• Inspect command results in their submitted order.
• Do not rely on a pipeline for transaction isolation.

Transactions

Group commands so they apply together.

• Use the client's transactional API for grouped atomic operations.
• Keep related keys compatible with deployment restrictions.
• Handle individual command errors without assuming automatic rollback.

Global replication

Read from a nearby replica and understand the consistency that implies.

• Choose regions around actual readers and writers.
• Account for replication delay when reading remote copies.
• Verify consistency requirements before routing critical reads.

Rate limiting

Apply fixed, sliding or token-bucket limits with the ratelimit library.

• Identify users or tenants with trusted limiter keys.
• Select a fixed, sliding or token-based algorithm for the workload.
• Distinguish quota exhaustion from limiter-service failure.

Caching

Cache responses under versioned keys with an explicit invalidation rule.

• Cache a bounded projection of expensive source data.
• Include tenant and relevant query parameters in the key.
• Invalidate or expire values when authoritative data changes.

Session storage

Store sessions with an expiry that matches the cookie's lifetime.

• Use unpredictable session IDs and bounded expiry.
• Store only server-controlled session attributes.
• Revoke sessions deliberately rather than waiting only for expiration.

Idempotency keys

Record processed request ids so a retry cannot duplicate an effect.

• Create a unique operation key before applying a side effect.
• Store operation status long enough to cover client retries.
• Avoid treating an expired key as proof the operation never occurred.

Locks

Serialize work across invocations with a lock that expires on failure.

• Use token-owned leases with bounded validity.
• Release a lock only if its ownership token still matches.
• Protect external writes from owners whose lease has already expired.

Counters and leaderboards

Use atomic increments and sorted sets instead of read-modify-write.

• Increment counters atomically instead of reading and rewriting values.
• Use sorted-set scores for ordered rankings.
• Define retention and tie behavior for rolling leaderboards.

Kafka over HTTP

Understand the former HTTP Kafka integration when migrating a legacy Upstash workload.

• Upstash Kafka is a discontinued legacy product.
• Treat HTTP Kafka examples as migration context, not new-service setup.
• Select a supported Kafka provider or suitable QStash workflow.

Topics and consumer groups

Understand Kafka partitioning and consumer offsets when migrating legacy streaming workloads.

• Kafka topics retain partitioned records for consumers.
• Consumer groups divide partitions and track processing offsets.
• Migrate legacy Upstash Kafka workloads to a supported platform.

QStash

Schedule and deliver HTTP messages with retries and delays.

• Publish HTTP work for asynchronous destination delivery.
• Verify QStash signatures in the receiving endpoint.
• Configure bounded retries and idempotent destination processing.

Dead letter handling

Capture messages that exhaust their retries for inspection.

• Inspect messages whose delivery retry budget is exhausted.
• Fix the destination or payload before replaying.
• Alert on dead-letter age so failed work is not forgotten.

Observability

Watch command latency, request counts and cost per endpoint.

• Track request latency, error rate, usage and quota pressure.
• Correlate QStash message IDs with destination processing logs.
• Exclude database tokens and sensitive values from diagnostics.

Security

Scope tokens per environment and rotate them without downtime.

• Keep write-capable service tokens off the client.
• Use narrowly scoped credentials and rotate exposed secrets.
• Validate webhook signatures and restrict accessible destinations.