How accurate is LLM-based company enrichment?

LLM-based company enrichment is accurate for well-known companies and slow-changing facts: name, website, broad industry, and headquarters for a company with a real public footprint. It gets unreliable on the long tail of small companies, on numbers that change (headcount, funding), and on domains that could belong to several different companies. Treat each profile as a draft with a confidence score attached, and put checks between the model and anything that acts on its output.

That's the honest version. Below is why it fails where it does, what those failures look like in a real schema, and the guardrails we'd put in front of any AI enrichment source, ours included.

Why the model is right about Stripe and wrong about a 12-person agency

A model answering from its own weights knows what it saw during training. Stripe appears in an enormous number of documents, consistently described. A small agency's domain might appear a handful of times, or never.

Kalai et al.'s 2025 paper Why Language Models Hallucinate gives a useful way to think about this. The authors argue models hallucinate partly because training and evaluation reward guessing over admitting uncertainty, and they tie error rates on arbitrary facts to how often those facts appear exactly once in the training data. Their example: if 20% of birthday facts appear only once in pretraining data, expect base models to hallucinate on at least 20% of birthday facts. A small company's founding year or funding total is that kind of fact. Nothing in the domain name lets the model work it out; it either memorized it or it is guessing.

So accuracy is not one number. It is a function of how famous the company is and how stable the field is. Any vendor quoting a single accuracy percentage for "AI enrichment" is averaging over a mix you don't know.

The failure modes you will actually see

The wrong-company case is the nastiest because the output is internally consistent. Every field describes a real company with real funding and a real HQ. It just isn't the company that owns the domain. Parked domains, domains that changed hands, and regional sites of unrelated firms with the same name all produce this.

The calibration problem is documented too. Xiong et al. (ICLR 2024) evaluated how well LLMs express uncertainty and found that, when verbalizing confidence, they tend to be overconfident. A score is still useful for throwing out the worst rows. It is not a probability you can multiply by.

What pingutil does about it, and what it can't

Here is exactly how our own enrichment behaves.

• The model reads the company's own homepage, and the prompt forbids guessing. The default provider (gpt-oss-20b on Cloudflare Workers AI) has no web search. Before calling it, the Worker fetches the domain's homepage and passes the title, meta description, any schema.org Organization data and the start of the page text as evidence. The system prompt tells it to return null for every field with confidence 0 unless it is certain a company owns that exact domain, and never to fabricate URLs, years, funding amounts or customer names. Temperature is 0.1. Prompts reduce guessing. They don't eliminate it.
• Output is schema-validated. The response must parse as JSON and pass a Zod schema. Loose formats are coerced ("$25M" becomes 25000000, "Series E" becomes series_d_plus, "340 employees" becomes "201-500"); anything unparseable fails the whole enrichment. Validation catches malformed answers. It does nothing about well-formed wrong ones.
• No partial rows. If enrichment fails, the API returns the previous cached profile (meta.served_stale: true) or a category-only response with meta.enrichment_error. It never stores half a profile.
• Low confidence is flagged. Rows where the model's confidence is below 0.5 come back with meta.low_confidence: true.
• Grounding is shallow by default. meta.grounded is true when the homepage excerpt was used, and meta.grounding_sources lists that page so a human can check it. A homepage rarely states funding or headcount, so those fields still lean on the model's memory. Gemini with Google Search grounding is a supported alternative provider that consults more sources.
• Profiles are cached until refreshed. That keeps lookups fast and free, and it also means a profile can be months old. last_refreshed_at tells you when it was produced.

What we can't do is make a model know things it doesn't know. Assume our company block has the same failure modes as the table above.

Guardrails that work

1. Validate structure and internal consistency

Schema validation is table stakes. The cheaper win is consistency rules, because hallucinated rows often contradict themselves or the input.

import { getDomain } from 'tldts';

type Company = {
  name: string | null; website: string | null; founded_year: number | null;
  funding_stage: string | null; total_funding_usd: number | null;
  is_public: boolean | null; ticker: string | null;
  linkedin_url: string | null; confidence: number | null;
};

export function problems(domain: string, c: Company): string[] {
  const out: string[] = [];
  const year = new Date().getUTCFullYear();

  if (c.website && getDomain(new URL(withScheme(c.website)).hostname) !== domain)
    out.push('website_domain_mismatch');      // strong signal of the wrong company
  if (c.founded_year && (c.founded_year < 1800 || c.founded_year > year))
    out.push('founded_year_out_of_range');
  if (c.is_public && !c.ticker) out.push('public_without_ticker');
  if (c.ticker && c.is_public === false) out.push('ticker_but_private');
  if (c.funding_stage === 'bootstrapped' && (c.total_funding_usd ?? 0) > 0)
    out.push('bootstrapped_with_funding');
  if (c.linkedin_url && !/^https:\/\/(www\.)?linkedin\.com\/company\/[^/]+\/?$/.test(c.linkedin_url))
    out.push('linkedin_url_malformed');
  return out;
}

function withScheme(u: string) { return /^https?:\/\//.test(u) ? u : `https://${u}`; }

The website_domain_mismatch check is the most useful single rule here: if the profile's website isn't the domain you asked about, the model was describing someone else. Acquisitions and redirects cause false positives, which is fine; those go to review, not to the bin.

2. Trust fields by tier, not rows

Different fields deserve different thresholds. A rough split:

• Tier 1, usable at moderate confidence: name, website, industry.
• Tier 2, check before routing on them: employee_size_estimate, headquarters, founded_year.
• Tier 3, never act on without a source: total_funding_usd, funding_stage, notable_customers.

export function usableFields(res: { domain: string; company: Company | null; meta: any }) {
  const c = res.company;
  if (!c || c.confidence == null || c.confidence < 0.5 || res.meta.low_confidence) return {};
  if (problems(res.domain, c).length) return {};          // send to review instead

  const fields: Partial<Company> = { name: c.name, website: c.website };
  if (c.confidence >= 0.7) Object.assign(fields, { founded_year: c.founded_year });
  // Funding only when a search-grounded source can back it up (a homepage excerpt rarely states it).
  const searchGrounded = res.meta.grounded && (res.meta.grounding_sources?.length ?? 0) > 1;
  if (searchGrounded && c.confidence >= 0.8) {
    Object.assign(fields, { funding_stage: c.funding_stage, total_funding_usd: c.total_funding_usd });
  }
  return fields;
}

The thresholds are starting points. Tune them against your own sample (step 4), not against our numbers or anyone else's.

3. Put humans where the money is

You don't need to review every row. Review the rows that will drive a decision: accounts above your deal-size line, leads about to be routed to an enterprise rep by company size, anything shown to a customer. A reviewer with the company's own website open can confirm or fix most fields quickly. Store who confirmed it and when, so a later refresh doesn't silently overwrite a verified value.

4. Measure your own error rate

Once a month, pull a random sample of 50 enriched domains from your actual traffic and check each tier-2 and tier-3 field by hand. Record per-field accuracy. This is the only accuracy number that matters, because it reflects your mix of famous and obscure companies. If your inbound is mostly small agencies, you'll see much worse numbers than someone enriching Fortune 500 domains, and that should change which fields you automate.

5. Refresh on a schedule that matches the field

Stale is a separate failure from wrong. Headcount and funding drift; names and websites rarely do. Re-enrich active pipeline accounts every quarter or so (on pingutil, refresh: true), diff the result, and flag changes in tier-3 fields for review rather than accepting them. Caching domain lookups covers refresh triggers and debouncing.

Where not to use it at all

Don't use model-generated firmographics for credit decisions, compliance checks, sanctions screening, or anything legal. Don't paste notable_customers into sales emails. Don't show total_funding_usd to a prospect. These are places where a confident wrong answer costs far more than an empty field.

For lead scoring, routing hints, enriching an inbound form so it can be shorter, and giving a rep context before a call, LLM enrichment is useful today, provided the guardrails above sit in between. If you're weighing it against other sources, free company enrichment APIs compared covers what each kind of source gets you, and enriching leads from an email domain shows the end-to-end flow. To see how our model handles a domain you know well, including how it scores itself, try it in the live demo.