Relay is a middleware SDK that sits between an application and third party LLM APIs from vendors such as OpenAI and Anthropic. Instead of hand rolling retry and fallback code, developers point their client at Relay and get a reliability layer built in.
Core features
- Provider failover to a backup vendor when the primary is degraded.
- Retry logic with decorrelated jitter to avoid retry storms.
- Response cache: exact match on the free tier, semantic cache on paid plans.
- Telemetry dashboard for latency, cost, cache hit rate and failover events.
Deployment and plans
The proxy runs on Cloudflare Workers at the edge. Relay uses a bring your own keys model, so provider keys are encrypted and the user keeps the billing relationship with each LLM vendor. The SDK is on npm as @relay-api/sdk, with Python support noted on the site. A free tier covers roughly 1,000 requests per month. Paid plans Hobby, Pro and Scale are listed at 19, 49 and 199 USD per month for larger quotas. Confirm the current numbers on the site.










Comments
No comments yet
Be the first to comment