Documentation

Get started in 30 seconds

EchoCache wraps any LLM call and returns a cached response when it detects a semantically similar prompt — no new infra required.

Install

npm install echocache-node

Init

new EchoCache(apiKey)

Use

echoCache.ask(prompt, fn)

Install the Node.js package

Works in any Node.js or edge runtime.

terminal

npm install echocache-node

Get your API key

Dashboard → API Keys → Copy key → paste into environment

Keys follow the pattern ec_prod_••••••••. Store it in your env:

.env

ECHOCACHE_API_KEY=ec_prod_your_key_here

Initialize the client

Import once. Key is validated on startup automatically.

lib/cache.js

import { EchoCache } from "echocache-node";

const echoCache = new EchoCache(process.env.ECHOCACHE_API_KEY);
// Done — key is validated on startup automatically.

Constructor options

Param	Type		Description
apiKey	string	required	Your ec_prod_… key from the dashboard.
options.baseUrl	string	optional	Override the server URL. Defaults to http://localhost:5000.

Wrap your LLM call with .ask()

One method. That's the entire API surface.

app.js

const response = await echoCache.ask(
  userQuery,           // the prompt string
  () => openai.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: userQuery }],
  })
);

// Cache HIT  → returns in ~40 ms, LLM is never called
// Cache MISS → calls your function, saves result in background

echoCache.ask(prompt, fallbackFn) → Promise<string>

Param	Type		Description
prompt	string	required	The user input to check against the cache.
fallbackFn	() => Promise<string>	required	Called only on a cache MISS. Must return the LLM response string.

Cache HIT

Similar prompt found. Returns in ~40 ms. fallbackFn is never called.

Cache MISS

No match found. Calls fallbackFn, then saves the result to cache in the background.

[3]

Full example — Next.js API Route

A complete, production-ready endpoint.

app/api/chat/route.js

import { EchoCache } from "echocache-node";
import OpenAI from "openai";

const openai = new OpenAI();
const echoCache = new EchoCache(process.env.ECHOCACHE_API_KEY);

export async function POST(req) {
  const { question } = await req.json();

  const answer = await echoCache.ask(question, () =>
    openai.chat.completions.create({
      model: "gpt-4o",
      messages: [{ role: "user", content: question }],
    }).then(r => r.choices[0].message.content)
  );

  return Response.json({ answer });
}

Graceful fallback

If the server is unreachable or the key is invalid, echoCache.ask() automatically calls your fallbackFn directly — your app never breaks.

Ready to ship?

Grab a free key and get your first cache hit in under five minutes.