Who’s who in your data: why entity resolution matters

Recognising that different records are the same person, company or place is what lets people get their data back, hospitals open the right file and companies trust their numbers. What entity resolution is, why it’s hard, and where LLMs help.

Ask an organisation for all the data it holds about you, and it first has to find every record that’s yours, under every spelling of your name. The same goes for a hospital opening a patient’s file, or a supplier working out which company it’s really selling to. Recognising that different records belong to the same real person, company or place is called entity resolution.

One company, many names

Ahold Delhaize is a Dutch company based in Zaandam, but you’ll rarely see its name on a shop. You shop at Albert Heijn, Etos or Gall & Gall in the Netherlands, Delhaize in Belgium, Albert in the Czech Republic or Food Lion in the United States (Ahold Delhaize). Two of its American chains are called Giant: Giant Food, based near Washington, and The GIANT Company, based in Pennsylvania. Giant Eagle, from the Pittsburgh area, is a third grocer with a similar name and no link to Ahold Delhaize.

Five of these supermarket names belong to one group, Ahold Delhaize. Giant Eagle sounds like it fits, but it's a different company.

It works the other way round too. Since November 2024 every Delhaize supermarket in Belgium has been run by an independent business owner, under the same Delhaize name (RetailDetail).

A supplier that sells to Albert Heijn, Delhaize and Food Lion has one customer group, and its credit exposure adds up across all three. Search a customer list for “Giant” and you’ll find two Ahold Delhaize chains and one that isn’t.

One person, many systems

People have the same problem, and here entity resolution is about serving them. Under GDPR you can ask an organisation for a copy of the personal data it holds about you, and in certain cases have it deleted (GDPR, Articles 15 and 17). In a large organisation those records sit in many systems, sometimes in different countries.

A made up customer. When she asks for a copy of her data, the company has to find all five records first, in systems spread over three countries.

In 2024, 30 European data protection authorities checked how 1,185 organisations handle these requests, and searching too narrowly was one of the problems they found (EDPB). In the Netherlands, parents caught up in the childcare benefits scandal asked the tax authority for their files, and in January 2020 the then finance minister told Parliament the files were incomplete in parts (NOS). In September 2026 the Raad van State, the highest Dutch administrative court, ruled on a request from two people who wanted to know if they were on tax authority blacklists. The minister had only searched the main fraud register; the court said every system that holds fraud signals has to be searched, including a shielded data vault (Raad van State). The same law also limits what an organisation may do with the records it links: data collected for one purpose can’t be reused for an incompatible one (Article 5).

When a Dutch bank fails, the central bank needs one record per customer that adds up all their accounts, so it can pay out the deposit guarantee within seven working days (DNB). In hospitals, ECRI reported a case where a patient in cardiac arrest wasn’t resuscitated because staff pulled up another patient’s record and followed that patient’s do not resuscitate order (ECRI). At the Arolsen Archives, a filing system built to absorb the many spellings of the same name links the records of people persecuted by the Nazis; in 2025 the archive received 25,500 individual inquiries (Arolsen Archives). It matters for fraud victims too. At the US Postal Service, cases of online change of address fraud and attempted identity theft rose from 8,857 in 2020 to 23,606 in 2021; in this fraud, someone files a change of address in another person’s name so their mail goes somewhere else (USPS OIG via Nextgov).

Companies, compliance and supply chains

Companies need it to stay on the right side of the law. In 2019 Apple agreed to pay the US Treasury $466,912 to settle apparent sanctions violations, after its screening tool failed to match “SIS DOO” in its own records with “SIS d.o.o.” on the sanctions list (OFAC).

Supply chains are full of it. Trade data names the same exporter in many ways; even a US Census Bureau technical note says the exporter code on US import records “is not always reliable, leading to multiple MIDs referring to the same exporter” (Heise and Dam, 2023). In the last mile, a 2021 study by Loqate, an address validation company, found that 39 percent of deliveries with an inaccurate or incomplete address failed (Loqate).

Two sources, one truth

There’s a quieter version too: two systems that overlap, like a CRM and an ERP, two companies after a merger, or an old platform and its replacement. Add two overlapping systems together and everyone in both counts twice.

Two systems, one customer base, with illustrative numbers. Adding them up counts the overlap twice. Matching finds the overlap, so each real customer counts once.

When Commonwealth Bank bought BankWest, Roy Morgan estimated from survey data that BankWest had about 606,000 retail customers, of whom 155,000 already banked with Commonwealth Bank, so the real gain would be only about 451,000 (Banking Day). The US Census Bureau estimated that the 2010 census held about 8.5 million duplicate records and missed about 16 million people. Because it measured errors in both directions, it could show they largely cancelled out, leaving a net overcount of just 0.01 percent (Census Bureau).

Why there’s no shortcut

The obvious fix is a global company ID. The Legal Entity Identifier comes closest, but there are just over 3.1 million active LEIs (GLEIF), while the World Bank counted more than 322 million formal micro, small and medium businesses in 2019, in only 77 economies (World Bank). That’s about one company in a hundred, at best. National IDs help, until they change: a Dutch business that gets a new owner or legal form usually gets a new Chamber of Commerce number (KvK). Business customers rarely carry the same ID in two systems, so names, addresses and websites end up doing the work.

Is it underrated?

In research, no. Halbert Dunn wrote the first known paper on “record linkage” in 1946, and Fellegi and Sunter gave it a statistical footing in 1969.

In practice, I think entity resolution is underrated. Before GeoBirds, I spent years in data roles at large organisations, reconciling sources with billions of records. Double counting is what makes analytics unreliable, and it always took a lot of domain knowledge before anyone could trust the numbers.

How it’s done

You can’t compare every record with every other one. A million records make about 500 billion pairs.

Why nobody compares every pair, with illustrative numbers. The dollar amounts are what it would cost to ask GPT-4o about each pair, at the 2024 price per pair that Peeters, Steiner and Bizer measured.

The basic steps are decades old. Clean the records, find likely candidates for each one (blocking: only comparing records that share something cheap, like a postcode or the start of a name), compare them, decide, and group the matches. A few things decide whether it works:

• Shared identifiers settle most matches; similar names only suggest them. A shared LEI is close to proof. The exact same website domain is strong evidence too, but some subsidiaries use their parent’s domain, so it can point to the group, not the company.
• Blocking sets your ceiling. A pair that never becomes a candidate is never compared, however good the model.
• Grouping can run away. If A matches B and B matches C, A and C end up together, and one bad link can chain many records into one entity. Keep merges reversible.
• Small test sets flatter you. If you check 20 matched pairs by hand and all 20 are right, the true precision (the share of matches that are correct) could still be as low as 84 percent, at 95 percent confidence.

Others have shared a lot of good work in the open. ING released its Entity Matching Model for company names; its documentation says matching 30 million names against 10 million one pair at a time “would take almost 1000 years”, and ING’s cluster does it in about an hour. The UK Ministry of Justice built Splink, which links a million records on a laptop in about a minute. AnHai Doan’s research group at the University of Wisconsin, whose work now continues at the startup MadMatcher, and OpenSanctions, which screens names against sanctions lists, have published their matchers too.

Where LLMs help

LLMs match well even without training examples. Peeters, Steiner and Bizer found that GPT-4, given none, beat a fine tuned matching model on three of six benchmarks (arXiv).

It gets expensive at scale. At the price per pair Peeters and colleagues measured for GPT-4o, asking about every pair in a million records would cost around $120 million. Use an LLM only on the pairs that identifiers and cheap scores can’t settle. It can also label training data for a small, fast model. On seven of eight benchmarks, Steiner and Bizer report, such models do about as well as ones trained on hand labelled data, and run 34 to 459 times faster than asking the LLM directly (arXiv).

Entity resolution at GeoBirds

At GeoBirds we build a digital map of the supply chain, so knowing which company sits behind a site is the core of the job. We’ve brought together the lessons from all of this work, from the research and from teams like ING’s, and built on the best practices of the leading researchers in the field. It works very well: our customers are impressed with the results, and the quality of the data helps them get a return on their investment faster.

It’s also one of the most interesting problems we work on, because there’s no answer key and every improvement has to be proven. If you’ve built a matcher, or you’d like to, I’d like to hear from you.