Skip to content

Entity resolution

A useful knowledge graph needs one graph identity for each real-world concept. Text extraction works with surface forms, so the same model may enter the graph as “DDM,” “drift diffusion model,” and a citation-qualified variant. Left unresolved, those nodes split edges and make graph traversal incomplete.

Deduplication is not one decision

Cortex divides resolution by confidence:

  1. Exact matching groups case-insensitive identical names. This is the most mechanical pass.
  2. Fuzzy matching proposes parenthetical, acronym, citation, dash, and author variants. A human edits the YAML proposal before mutation.
  3. Semantic matching blocks candidates by embedding similarity and can use a model to classify pairs. A human still owns the final proposal.
  4. Alias registration preserves approved surface forms so later ingestion can map them to a canonical name.

Separating the phases makes risk visible. A substring can describe a related but distinct concept, and high embedding similarity does not prove identity.

Canonical-node choice

Exact matching keeps the node with the most graph connections, breaking ties in favor of a longer name. Reviewed proposals name an explicit canonical node. When a merge is applied, Cortex redirects incoming and outgoing edges, drops self-loops created by the merge, removes duplicate edges, and deletes the duplicate node.

The desired invariant is not “fewest nodes.” It is one node per justified identity while preserving all valid relationships.

Human approval and recovery

Fuzzy and semantic commands generate proposal files separately from their --apply mode. Those files are review artifacts: removing a questionable pair is expected, not an exceptional failure.

Applied phases create a timestamped Kuzu backup by default. The backup protects the local graph, while the reviewed proposal explains the intended identity decision. Skipping either weakens auditability.

Prevention after cleanup

data/alias_registry.yaml combines canonical graph nodes with aliases retained in the fuzzy and semantic proposal files. Ingestion consults the registry after building graph state to identify new nodes that match known aliases. This reduces repeat fragmentation, but new terminology can still require review.

Follow Resolve duplicate entities to operate the phases safely.