Stop paying twice for identical LLM queries.
EchoCache intercepts your LLM calls, maps prompt semantics in a local vector space, and returns cached responses in under 10ms. Save 80% on API costs without blocking runtime execution.
Designed for distinct performance
Traditional exact-match caches fail on LLMs due to slight wording adjustments. EchoCache maps inputs into vector spaces to capture semantic equivalence.
int8 Quantized Payloads
Compresses vector dimensions using 8-bit scalar quantization. Fits 4x more items inside local cache indexes, minimizing memory usage with zero latency degradation.
Vector Similarity Search
Sentence embeddings calculate cosine distance to match prompts even when syntax is completely rephrased.
Configurable Threshold
Fine-tune confidence requirements (e.g. 0.92) per endpoint to balance accuracy vs. cache hit ratios.
Drop-In Middleware
Seamlessly wrap existing OpenAI, Anthropic, or custom REST client instances without infrastructure overhauls.
Provider-Agnostic Engine
Works universally across OpenAI GPT-5.6 Sol, Anthropic Claude Opus 5, Gemini 3.6 Flash, DeepSeek-R1, or open-source models hosted anywhere.
Parallel Background Threading
EchoCache pushes completions asynchronously in background threads. Clients receive answers instantly; writes never block.
Three steps to your first cache hit
No new infrastructure, no migration. EchoCache sits in front of the LLM call you already have.
Install the Node.js package
Add EchoCache to your Node.js or TypeScript project with npm.
Get your API key
Create a free account, grab your API key from the dashboard, and drop it into your environment variables.
Wrap your LLM call
Pass your prompt and a fallback function. EchoCache checks for a semantic match first — and only calls your model on a miss.
1npm install echocache-node
1import { EchoCache } from "echocache-node";23const echoCache = new EchoCache(process.env.ECHOCACHE_API_KEY);45// Wrap any LLM call — EchoCache checks6// semantic similarity before it runs.7const response = await echoCache.ask(userQuery, () =>8 openai.chat.completions.create({9 model: "gpt-5.6-sol",10 messages: [{ role: "user", content: userQuery }],11 })12);
Get your API key and ship it today
Create a free account, generate a key from your dashboard, and drop it into the snippet above. Most teams have their first cache hit in under five minutes.
Simple, usage-based pricing
Start free, scale as your cache traffic grows. Only pay for what you use.
Free
For side projects and exploring cache hit rates.
- 10,000 cache lookups / mo
- 50 MB cache storage
- 1 Project max
- 5 requests / sec rate limit
- 1 API key
- 3-day analytics retention
- Community support
Pro
For production apps with steady LLM traffic.
- 100,000 cache lookups / mo
- 500 MB cache storage
- 5 Projects max
- 50 requests / sec rate limit
- Unlimited API keys
- 10-day analytics retention
- Configurable similarity threshold
- Delete cached entries
- Email support
No surprise charges · Cancel anytime