Skip to main content

Consolidate duplicates

Run resolution over every entity in the graph. Review the matches it finds. Then apply them.

When to use this guide​

Graph.add already compares new mentions with the entities that the graph stores. Run Graph.consolidate in two cases. You added data before you configured an LLM, so some pairs stayed unmatched. Or you want to review the whole graph again.

Before you start​

  • A running Neo4j and the NEO4J_* variables. The Quickstart shows how.
  • The extras for local extraction:
uv pip install "agentic-graphrag[neo4j,extract]"
  • Optional: an LLM for the last tier. Install the llm extra, and set LLM_BASE_URL, LLM_API_KEY, and LLM_MODEL_ID. Without an LLM, that tier returns no match for every pair and raises no error.

1. Add documents that name one person in three ways​

import asyncio

from agrag.common.data_models import GENERIC
from agrag.embedding import build_embedder
from agrag.graphdb import build_graph_store
from agrag.ingestion import GlinerExtractor, Graph

DOCUMENTS = [
    "Charles Babbage designed the Analytical Engine in London.",
    "Babbage Charles taught mathematics at Cambridge.",
    "C. Babbage proposed the Difference Engine.",
]


async def main() -> None:
    graph_store = build_graph_store("neo4j")
    graph = await Graph.open(
        schema=GENERIC,
        graph_store=graph_store,
        embedder=build_embedder("ibm-granite/granite-embedding-small-english-r2"),
        extractor=GlinerExtractor(),
    )
    try:
        for text in DOCUMENTS:
            result = await graph.add(text=text)
            print(result.resolution)
    finally:
        await graph_store.close()


asyncio.run(main())

Each add prints a resolution summary. ambiguous_count counts the pairs that an LLM called uncertain. Without an LLM, "C. Babbage" stays a separate entity.

2. Preview and apply the matches​

Graph.consolidate() is a dry run. It writes nothing. Graph.consolidate(apply=True) writes the matches. This script previews, applies, and then reads the groups back.

import asyncio

from agrag.common.data_models import GENERIC
from agrag.embedding import build_embedder
from agrag.graphdb import build_graph_store
from agrag.ingestion import GlinerExtractor, Graph


async def main() -> None:
    graph_store = build_graph_store("neo4j")
    graph = await Graph.open(
        schema=GENERIC,
        graph_store=graph_store,
        embedder=build_embedder("ibm-granite/granite-embedding-small-english-r2"),
        extractor=GlinerExtractor(),
    )
    try:
        report = await graph.consolidate()
        ids = [
            str(entity_id)
            for match in report.would_match
            for entity_id in (match.entity_a_id, match.entity_b_id)
        ]
        rows = await graph_store.execute_read(
            "MATCH (e:_AgragNode) WHERE e.id IN $ids RETURN e.id AS id, e.name AS name",
            {"ids": ids},
        )
        names = {row["id"]: row["name"] for row in rows}
        for match in report.would_match:
            first = names[str(match.entity_a_id)]
            second = names[str(match.entity_b_id)]
            print(f"{first} = {second} ({match.comparator}, score {match.score})")
        print("uncertain pairs:", report.ambiguous_count)

        applied = await graph.consolidate(apply=True)
        print("applied:", applied.applied, "failures:", len(applied.failures))
        groups = await graph_store.execute_read(
            "MATCH (r:ResolvedEntity)<-[:RESOLVED_AS]-(e) "
            "RETURN r.name AS resolved, collect(e.name) AS members"
        )
        for row in groups:
            print(row["resolved"], "<-", sorted(row["members"]))
    finally:
        await graph_store.close()


asyncio.run(main())

Without an LLM, the output shows the one match that the fuzzy tier found:

Charles Babbage = Babbage Charles (fuzzy_fast_path, score 1.0)
uncertain pairs: 0
applied: True failures: 0
Charles Babbage <- ['Babbage Charles', 'Charles Babbage']

With an LLM configured, the last tier also matches "C. Babbage", and the three names form one group:

Babbage Charles = Charles Babbage (fuzzy_fast_path, score 1.0)
Charles Babbage = C. Babbage (llm, score None)
Babbage Charles = C. Babbage (llm, score None)
uncertain pairs: 0
applied: True failures: 0
Babbage Charles <- ['Babbage Charles', 'C. Babbage', 'Charles Babbage']

The order of the names, and the name of the group, can differ in your run.

3. Read the report​

  • would_match lists every confirmed match. Each item has the two entity ids, the tier, a score when the tier has one, and the reason when an LLM decided. The tier is fuzzy_fast_path, embedding, or llm.
  • ambiguous_count is the number of pairs that an LLM called uncertain. These pairs never merge.
  • applied is True when the matches were written. failures lists any match that Agentic GraphRAG did not write, and any entity whose candidates it did not read. Consolidate does not compare that entity in this call. Run it again after you fix the cause.

The report lists matches that the graph already stores. Applying the same matches again is safe.