Grounding the Ask Box: An 8 KB Fact Blob, a Cite-Slugs Rule, and the Guards Around It

The ask box on this site talks to Groq’s llama-3.3-70b-versatile, and large language models invent facts the way compilers invent warnings: confidently, and at the worst moment. The fix here is deliberately boring — a build-time JSON blob of verified frontmatter, inlined into the system prompt, plus a rule that says cite the slug or say you don’t know. This post is the full mechanism: the generator, the budget, the cite rule, the guards, and the holes I am knowingly leaving open.

What the blob is

scripts/generate-chat-facts.mjs reads the frontmatter of three content collections (src/content/work, src/content/blog, src/content/uses) and writes one compact JSON array to src/generated/chat-facts.json. Titles, slugs, summaries, tags, dates, and (for work entries) up to three metrics. No body text, no env vars, no secrets; the script’s own header says it plainly: “frontmatter is public content; env is never read here.”

Each entry uses single-letter keys to stay small (k for collection (w|b|u), t title, s summary, d date, g tags, m metrics), and per-field caps enforced by a str() helper that cuts at the last space so summaries never end mid-word:

// Entry shape (single-letter keys keep the blob under budget):
// k: collection (w|b|u): the route is derived (`/work|/blog|/uses` + slug),
// slug, t: title (≤70), s: summary|note (≤110), d: date, g: tags (≤60),
// m: up to 3 metrics (≤32 each, work only).
const summary = str(flat.summary ?? flat.note, 110);
if (summary) entry.s = summary;
if (flat.publishedAt) entry.d = str(flat.publishedAt, 10);
if (flat.tags) entry.g = str(flat.tags.replace(/[[+\]'"]/g, ''), 60);
if (metrics.length) entry.m = metrics.map((x) => str(x, 32)).slice(0, 3);

Two details worth stealing. First, drafts are skipped (“the blob must never ground claims about unpublished pages”): via a draft: true regex check before parsing. Second, the frontmatter parser is a ~35-line hand-rolled reader for flat key: value lines plus metrics: value/label pairs, not a YAML dependency. Fewer deps, zero new attack surface, and the only shapes it accepts are the ones these collections actually use.

The 8 KB budget that fails loudly

The blob is capped at FACT_BUDGET_BYTES = 8 * 1024, and the enforcement philosophy is the interesting part: exceeding it fails the build with a non-zero exit instead of silently truncating into a misleading subset. A truncated blob wouldn’t just be shorter. It would be a biased sample of the site that the model would still cite with full confidence. Truncation would lie quietly.

const json = JSON.stringify(entries);
const bytes = Buffer.byteLength(json, 'utf8');
if (bytes > FACT_BUDGET_BYTES) {
  fail(
    `fact blob is ${bytes} bytes over the ${FACT_BUDGET_BYTES}-byte budget ` +
      `(${entries.length} entries). Tighten per-field caps in this script: ` +
      `do NOT raise the budget without checking GROQ_MAX_TOKENS headroom.`,
  );
}

Note the last line: the budget is coupled to GROQ_MAX_TOKENS (1,536, in src/lib/api-security.ts). Every byte of grounding context eats output headroom, so raising one without checking the other is a silent quality regression. The generator runs on prebuild, predev, and pretest alongside generate-build-info.mjs, so the blob is never stale relative to the content it describes.

The cite-slugs rule

src/pages/api/chat.ts maps the blob into prompt text with a route table and one line per page:

const FACT_ROUTES = { w: '/work', b: '/blog', u: '/uses' } as const;

const FACT_BLOB = (chatFacts as Array<Record<string, unknown>>)
  .map((e) => {
    const route = FACT_ROUTES[(e.k as keyof typeof FACT_ROUTES) ?? 'w'] ?? '/work';
    const parts = [`${route}/${e.slug}`, String(e.t)];
    if (e.s) parts.push(String(e.s));
    if (e.d) parts.push(String(e.d));
    if (e.g) parts.push(`tags: ${e.g}`);
    if (Array.isArray(e.m)) parts.push(`metrics: ${e.m.join('; ')}`);
    return parts.join(' | ');
  })
  .join('\n');

And the system prompt’s rule 4 is the actual grounding contract:

For any factual claim about Deepak’s work, posts, or tooling, ground it in the list above and cite the page as route/slug. Never invent slugs, metrics, or dates — if it is not listed above, say so.

This is an allowlist by construction: the model can only cite slugs that exist in the blob, because those are the only slugs it has ever seen. There is no post-hoc citation validator. Nothing in the route checks the model’s output against the blob, so the enforcement is prompt-level, not code-level. It works as well as system prompts work, which brings us to the guards that are code.

What the endpoint enforces in code

The POST handler in src/pages/api/chat.ts layers defenses in order, each cheap and each doing one job:

1. Origin allowlist: isAllowedOrigin(origin, referer) against ALLOWED_ORIGINS (deepakdinesh.com, www, localhost, 127.0.0.1) returns 403 otherwise. CSRF-grade, not auth-grade.
2. Per-minute burst limit: RATE_LIMIT_CHAT_MAX = 100 per IP per minute, in-memory sliding window.
3. Daily budget: CHAT_DAILY_BUDGET_MAX = 50 requests per IP per day. Each request is already token-bounded (MESSAGE_MAX_LENGTH in, GROQ_MAX_TOKENS out), so capping requests caps daily token spend, which is the only thing standing between this endpoint and the Groq free-tier ceiling.
4. Body limits, twice: BODY_LIMIT_CHAT = 10 KB, checked first against Content-Length (fast path) and again on the bytes actually read, because chunked bodies carry no Content-Length.
5. Message cap: 2,000 characters max, defense in depth on top of the body limit, plus strict application/json content-type (415 otherwise) and non-empty-string validation.

Client IP comes from x-real-ip first, falling back to the last entry of x-forwarded-for, which the route’s comment justifies in one line: Vercel appends the real client IP last, so leftmost entries are client-controlled and trivially spoofable. Keying a rate limiter on [0] would let anyone rotate identities per request.

Honest limitations

• No prompt-injection filter. The user message is passed to Groq verbatim. Rules 1–3 of the system prompt (professional topics only, salary deflection, concise tone) are instructions, not enforcement. A determined prompt can steer tone and topic, and there is no input sanitizer or output citation checker. The blast radius is bounded (read-only facts, 1,536 output tokens, no tool calls), which is the actual mitigation.
• In-memory limits are per-instance. Both rate limiters lose state on cold start and don’t share across instances; the code comments admit this. Same tradeoff as the KV teardown: fine at this traffic, wrong at ten times it.
• The blob is lossy by design. 110-character summaries and 3 metrics per entry mean nuanced questions get shallow grounding — the model says “see /work/tweakdeals” rather than answering from depth. That deflection is intentional, but it’s worth naming: grounding here means pointing, not knowing.
• Stale-blob window. The blob regenerates on prebuild/predev/pretest, so a content edit without a rebuild serves yesterday’s facts. Same class of staleness discussed in the slim CI rebuild: the pipeline shape decides freshness, not the code.
• No measured hallucination rate. I have not A/B tested grounded vs. ungrounded answers or logged citation accuracy. The mechanism is reasoned, not measured. I learned that gap from the CSP outage: fixes without probes are stories, not evidence. A chat.test.ts exists for the route’s validation paths, but nothing asserts grounding quality.