Enterprise-Grade Caching SDK

Stop paying twice for identical LLM queries.

EchoCache intercepts your LLM calls, maps prompt semantics in a local vector space, and returns cached responses in under 10ms. Save 80% on API costs without blocking runtime execution.

Setup Guide
Simulation Environment — echocache-client
// EchoCache Initialized. Ready for prompt execution.
// Select a suggested query in the sidebar to simulate Cache MISS/HIT states.
Vector Embedding Space
Vector Similarity Match
0.00 (No Match)
Status
—
Latency
—
Estimated Savings
$0.00
Step 1: Click to test query
Step 2: Test rephrased query
Node.jsPythonGPT-5.6 SolClaude Opus 5Gemini 3.6 FlashDeepSeek-R1LangChainVercel AI SDK
Mechanics

Designed for distinct performance

Traditional exact-match caches fail on LLMs due to slight wording adjustments. EchoCache maps inputs into vector spaces to capture semantic equivalence.

int8 Quantized Payloads

Compresses vector dimensions using 8-bit scalar quantization. Fits 4x more items inside local cache indexes, minimizing memory usage with zero latency degradation.

Vector Similarity Search

Sentence embeddings calculate cosine distance to match prompts even when syntax is completely rephrased.

Configurable Threshold

Fine-tune confidence requirements (e.g. 0.92) per endpoint to balance accuracy vs. cache hit ratios.

Drop-In Middleware

Seamlessly wrap existing OpenAI, Anthropic, or custom REST client instances without infrastructure overhauls.

Provider-Agnostic Engine

Works universally across OpenAI GPT-5.6 Sol, Anthropic Claude Opus 5, Gemini 3.6 Flash, DeepSeek-R1, or open-source models hosted anywhere.

Parallel Background Threading

EchoCache pushes completions asynchronously in background threads. Clients receive answers instantly; writes never block.

SDK Integration

Three steps to your first cache hit

No new infrastructure, no migration. EchoCache sits in front of the LLM call you already have.

01

Install the Node.js package

Add EchoCache to your Node.js or TypeScript project with npm.

02

Get your API key

Create a free account, grab your API key from the dashboard, and drop it into your environment variables.

03

Wrap your LLM call

Pass your prompt and a fallback function. EchoCache checks for a semantic match first — and only calls your model on a miss.

terminal
1npm install echocache-node
app.js
1import { EchoCache } from "echocache-node";
2
3const echoCache = new EchoCache(process.env.ECHOCACHE_API_KEY);
4
5// Wrap any LLM call — EchoCache checks
6// semantic similarity before it runs.
7const response = await echoCache.ask(userQuery, () =>
8  openai.chat.completions.create({
9    model: "gpt-5.6-sol",
10    messages: [{ role: "user", content: userQuery }],
11  })
12);
Get started

Get your API key and ship it today

Create a free account, generate a key from your dashboard, and drop it into the snippet above. Most teams have their first cache hit in under five minutes.

Read the docs
Pricing

Simple, usage-based pricing

Start free, scale as your cache traffic grows. Only pay for what you use.

Free

For side projects and exploring cache hit rates.

₹0
  • 10,000 cache lookups / mo
  • 50 MB cache storage
  • 1 Project max
  • 5 requests / sec rate limit
  • 1 API key
  • 3-day analytics retention
  • Community support
Most popular

Pro

For production apps with steady LLM traffic.

₹699/mo
  • 100,000 cache lookups / mo
  • 500 MB cache storage
  • 5 Projects max
  • 50 requests / sec rate limit
  • Unlimited API keys
  • 10-day analytics retention
  • Configurable similarity threshold
  • Delete cached entries
  • Email support

Business

For high-volume apps and growing teams.

₹1,999/mo
  • 1,000,000 cache lookups / mo
  • 5 GB cache storage
  • 25 Projects max
  • 200 requests / sec rate limit
  • Unlimited API keys
  • 30-day analytics retention
  • Configurable similarity threshold
  • Delete cached entries
  • Priority email support

Enterprise

For teams with high-volume traffic or self-hosting needs.

Custom
  • Unlimited lookups
  • Custom cache storage
  • Unlimited Projects
  • No rate limits
  • SSO & audit logs
  • Dedicated infrastructure
  • SLA guarantee
  • Priority support

No surprise charges · Cancel anytime

Knowledge Base

Frequently Asked Questions

Everything you need to know about semantic vector caching, int8 scalar quantization, and reducing LLM API bills.

Traditional caches (like standard Redis key-value stores) require an exact byte-for-byte match on prompt strings. If a user asks 'How to reset password?' and another asks 'Where can I change my password?', a traditional cache misses both. Semantic caching converts prompts into dense vector embeddings and computes cosine similarity in high-dimensional vector space. If two prompts mean the same thing, EchoCache returns the cached answer in under 10ms without hitting your LLM provider.