Manifest

Manifest is an open source LLM gateway and router that connects AI agents and harnesses to multiple LLM providers through a single endpoint. It is designed to help developers and teams build reliable AI products by automatically repairing broken requests on the fly and providing observability tools. Users can choose between a cloud version or self-hosting the software on their own infrastructure using Docker.

• Autofix retries, reroutes, and repairs broken provider calls on the fly
• Model fallbacks that automatically retry on another provider when one goes down
• Full body logs to inspect complete request and response bodies for every call
• Support for local models including Ollama, LM Studio, llama.cpp, or custom local servers
• Bring your own key support for popular providers and OAuth tokens for subscriptions
• Cost visualization to track spending broken down by agent, key, and provider
• Visual setup for setting AI model parameters per route without touching code
• Budget alerts and email notifications

Pros

• Provides self-healing capabilities to prevent avoidable LLM API call failures in production workflows
• Offers flexible deployment choices including an MIT-licensed self-hosted Docker option and a cloud-hosted version
• Consolidates multiple LLM providers, subscription keys, and local models behind a single unified endpoint
• Includes detailed request and token tracking alongside cost visualization to monitor inference consumption

Cons

• The Free cloud plan enforces a strict limit of 10,000 routed requests per month and a 7-day dashboard data retention window
• The Pro plan costs $19 per month but is explicitly noted as not suited for teams
• Advanced team controls like SSO, SAML, audit logs, and custom BAAs are restricted to the custom-priced Enterprise tier

Manifest fits individual hackers, AI builders, and engineering teams shipping AI agents who need to control inference consumption, eliminate downtime, and route requests across multiple providers. It is a poor fit for organizations requiring enterprise governance features like SSO, audit logs, and contractual SLAs without moving to a custom-priced enterprise agreement or managing the infrastructure themselves.

Manifest offers three main plans. The Free plan is $0 per month for prototypes and small projects, supporting unlimited agents, all providers, community Discord support, and 10,000 routed requests per month on Manifest Cloud or unlimited requests when self-hosted with Docker under an MIT licence. The Pro plan costs $19 per month for production projects, providing unlimited routed requests on Manifest Cloud, 365 days of dashboard retention, and basic support, though it is not suited for teams. The Enterprise plan requires contacting sales and is designed for teams needing on-premise or private cloud deployment, multiple seats, team management, SSO, SAML, audit logs, custom retention, contractual uptime SLAs, HIPAA and custom BAAs, custom guardrails, and dedicated Slack and email support.

Unlike direct API connections or basic proxy setups that fail when a provider goes down or returns a malformed payload, Manifest acts as a dedicated LLM router equipped with an autofix agent that actively repairs broken requests before the host application sees the error. It bridges the gap between cloud-hosted providers, local models running on Ollama or llama.cpp, and subscription OAuth tokens under a single configurable endpoint. Additionally, its dual delivery model gives teams the freedom to use a managed cloud version or run the entire MIT-licensed stack locally via Docker with no request limitations.

OpenAI, Claude, ChatGPT, Gemini, Ollama, LM Studio, llama.cpp, and custom providers accessed via API keys or subscription OAuth tokens.

Manifest is worth trying for developers and teams wanting to eliminate LLM API errors and unify their provider routing under a single endpoint. The free self-hosted option and the cloud tier offer flexible paths to get started without immediate financial commitment. Teams requiring audit logs, SSO, or custom contracts will need to evaluate the Enterprise tier. Those with minimal routing needs can rely on the free tier, while production solo projects may consider the $19 per month Pro plan.