#122
B+
Rank #59
LiteLLM
A unified LLM gateway/proxy that exposes an OpenAI-compatible API in front of 100+ model providers, with virtual keys, budgets, and routing logic. Standout trait is its genuinely stateless proxy design, making it one of the more horizontally-scalable apps in this batch.
75.8
/ 100
⚙ Full Stack
A Python (FastAPI) proxy server that translates OpenAI-format requests into provider-specific calls, with optional Postgres for spend-tracking, virtual-key management, and Redis for rate-limit/cache state. Core routing logic itself needs no database at all.
📈 Scaling Analysis
One of the most cloud-native-friendly apps here — the proxy layer is stateless per-request, so adding replicas behind a load balancer scales throughput linearly with no session-affinity concerns. The only stateful pieces (spend tracking, key management) are cleanly isolated in an optional Postgres.
🚀 Running on Nexlayer
Single app pod is enough for basic routing; add a Postgres pod (`postgres.pod:5432`) if using virtual keys/budget tracking. Model provider API keys should be injected as env vars on the pod, never baked into the image or exposed client-side.