Back to Blog
Douglas Adams has no English name

Douglas Adams has no English name

Wikidata is already a knowledge graph, so feeding it to an LLM memory should be easy. I built the cognee connector that does it, and the live API returned nameless items, failed whole batches over one bad id, and buried 80 statements under 274 identifiers.

9 min read
  • wikidata
  • knowledge-graph
  • cognee
  • python
  • open-source
  • rag
  • sparql

Wikidata is already a knowledge graph. Every fact is a statement: an item, a property, a value. Douglas Adams is Q42, "notable work" is P800, The Hitchhiker's Guide to the Galaxy is another item. It's the graph a lot of other graphs are built from.

So turning it into a cognee knowledge graph should be the easiest connector of the three I wrote during the same hackathon. The MediaWiki one had to deal with a wiki that never stops changing, and the WordPress one with an API that has no change feed at all. Wikidata has clean JSON, revision ids on every item, and a SPARQL endpoint.

Then I asked the API for Douglas Adams's English name, and it said he doesn't have one.

Item — a Wikidata entity with an id like Q42. It has a label (its name), a description, aliases, and a list of statements.

Statement — one fact about an item: a property (P569, date of birth) and a value (1952-03-11), optionally with qualifiers (start time, end time) and references.

Why bother, if it's already a graph? Because cognee puts it in the same graph as your other documents. A Wikidata item for a company, next to that company's wiki pages and blog posts, is more useful than either on its own.

Ask Wikidata

The scenario: start from Douglas Adams, follow his notable works one hop, and ask a question that needs both.

import cognee
from cognee_community_connector_wikidata import wikidata_source

await cognee.remember(
    wikidata_source(
        items=["Q42"],                  # Douglas Adams
        expand_properties=["P800"],     # ... and his notable works
        user_agent="my-app/1.0 (me@example.com)",
    ),
    dataset_name="wikidata",
    primary_key="id",
    write_disposition="merge",
    max_rows_per_table=0,
)

results = await cognee.recall(
    "Which works is Douglas Adams known for, and when was he born?",
    datasets=["wikidata"],
)

The answer, from 4 ingested items (Q42 plus 3 works):

Douglas Adams is known for The Hitchhiker's Guide to the Galaxy, the Dirk Gently series, and The Private Life of Genghis Khan. He was born on March 11, 1952.

Seeds can be ids, entity URIs or item page URLs. Instead of following properties you can hand it a SPARQL query (?item wdt:P50 wd:Q42 LIMIT 5, books authored by Adams, synced 5 items in 6 requests). Either way max_items (default 500) caps the selection, so seeding with Q5 ("human") can't pull in millions of people.

The nameless man

The naive version fetched Q42 with languages=en. Here's what came back:

GET wbgetentities?ids=Q42&props=labels&languages=en

"labels": {}

Empty. No English label for one of the most famous items on Wikidata.

What happened is that Wikidata now stores many labels only once, under the language code mul ("multiple languages"), so a name that's spelled the same across languages isn't copied hundreds of times. Ask for en and you get exactly en, which is nothing. Add languagefallback=1 and you get:

{ "value": "Douglas Adams", "language": "mul", "for-language": "en" }

That would have been bad enough for Q42 alone. But every property, every item a statement points at, and every unit gets its label the same way. Without the fallback, a document comes out as a list of bare ids: P106 instead of "occupation", a Q-number instead of "novelist". The connector always sends languagefallback=1, and there's a test for the mul case so nobody removes it later.

One bad id fails the batch

wbgetentities takes 50 ids per request, so the connector batches. I seeded it with a few awkward ids on purpose: a deleted item, a merged one, and one that was never assigned. They come back three different ways:

Id What it is What the API returns
Q141623501 deleted {"id": "Q141623501", "missing": ""}
Q59431323 merged into Q42 the Q42 entity, keyed by the requested id, plus "redirects": {"from": "Q59431323", "to": "Q42"}
Q99999999999 never assigned {"error": {"code": "no-such-entity", ...}} for the whole request

The last row is the dangerous one. One typo in a seed list, and the other 49 items in that batch fail with it. The client drops the bad id and retries the rest. As a seed, a never-assigned id raises WikidataScopeError the first time it's configured, so a typo shows up as an error instead of a sync that quietly has less in it.

The first row is a small trap too: "missing" is an empty string, not true. A check like if item.get("missing"): is false for it, and the deleted item looks alive.

And the merged id is handled as what it is: the old id gets a tombstone, and the item it was merged into is synced in its place.

Turning statements into something an LLM can read

cognee extracts entities from text, so each item becomes a document. The interesting decisions were all about what to leave out. Q42 has 354 statements, and 274 of them (77%) are external identifiers: VIAF, ISNI, library catalogue ids, social media handles. Useful for linking databases. Useless for answering "what did he write?"

So external ids are off by default. Deprecated statements (ones known to be wrong) are dropped, and preferred ones are listed first. Here's the start of Q42's document:

# Douglas Adams (Q42)
British science fiction writer and humorist (1952–2001)

## Statements
- instance of: human
- occupation: novelist (start time: 1979)
- notable work: The Hitchhiker's Guide to the Galaxy
- date of birth: 1952-03-11
- place of burial: Highgate Cemetery (coordinate location: 51.566, -0.145)
- country of citizenship: United Kingdom
...

## Sitelinks
- enwiki: Douglas Adams (https://en.wikipedia.org/wiki/Douglas_Adams)

Three smaller things made that readable:

Dates have a precision. "Novelist since 1979" is stored as +1979-00-00T00:00:00Z with precision 9, meaning year. Printed naively it's 1979-00-00, which isn't a date. The connector renders it at its precision: 1979. Decades, centuries, BCE and Julian-calendar dates are spelled out too, so cognee's temporal search can use them.

Some texts come in every language. The qualifier for the photo of Adams's grave had a caption in 11 languages. All 11 went into the document. Keeping only the configured language took the document from 5,505 characters to 4,558 characters (80 lines), about 17% smaller, and what it lost was translations.

Some things look like bugs and aren't. While checking labels I hit an item whose label was literally "Q42". I spent a while hunting a rendering bug before I opened it: Q42395533 is an item about the item Q42. Its label is correct.

Sync by revision

Every Wikidata item carries a lastrevid, which changes on every edit. That makes incremental sync almost boring, in a good way:

  1. Resolve the selection (seeds, followed properties, SPARQL results).
  2. Ask for every item's lastrevid, 50 ids per request, with no content.
  3. Fetch and re-render only the items whose revision moved.

The arithmetic for the default cap:

500 items / 50 ids per request = 10 requests to check everything
items changed                  = 0  -> nothing fetched, nothing rendered

On the real run:

Run Requests Time Re-rendered
first sync, connector only (4 items) 9 ~7 s 4
unchanged re-sync, connector only 3 ~1.4 s 0
full example with OpenAI, run 1 not logged 1m12s 4 ingested
full example with OpenAI, run 2 not logged 21.5 s 0 ingested, 4 unchanged

The last row is the one that matters. Nothing changed upstream, so cognee did no extraction at all. That's 50 seconds and every LLM token saved on four items. On 500 items it would decide whether you can run the sync every hour.

Items that are deleted, merged, or leave the selection (a seed removed, a P800 link removed upstream, a SPARQL result gone) become tombstones, and cognee removes them from the graph, vector and relational stores.

The empty-selection rule

The tombstone logic has one dangerous edge. If a SPARQL query times out, or the query service is down, the selection comes back empty, and a naive diff concludes that every item was deleted.

So an empty selection after a non-empty one deletes nothing the first time (failed_empty_scope). It's believed only when the API itself reports every synced item gone, when you just changed the selection, or on the second empty run in a row. Being wrong once is cheap. Wiping a dataset because a query timed out is not.

What the real run actually did

The connector worked. The demo, the first time, didn't. My network was flaky, and the run hit an OpenAI embedding timeout (Graph edge indexing error) and then a DNS failure. Neither was the connector's fault, and the connector's own retry did recover a Wikidata ConnectError in the same run. I mention it because the post would be a lie without it: a real first run hits real network problems, and I'd rather the failures be someone else's and retried than silent.

One known limitation is worth stating plainly. If a referenced item is renamed (say an occupation gets a better label), the items pointing at it don't get a new revision, so their documents keep the old label until they're next edited or you run with full_sync=True. Lexemes and statement references aren't rendered yet.

Three connectors, one graph

The best reason to have this connector is the other two. Each item keeps its sitelinks, the links to its Wikipedia articles. So Q42 in the Wikidata dataset and the "Douglas Adams" page from the MediaWiki connector point at the same thing. Wikidata gives you the structured facts (born 1952-03-11, buried at Highgate), and the article gives you the prose. Each one has its own node set (wikidata:www.wikidata.org, mediawiki:en.wikipedia.org), so you can query them together or apart.

If you keep one thing

Even when the source is already a graph, the work is in what you leave out: the right label, the right precision, one language, and the 77% of statements no one will ever ask about.

Try it

Sources

  • My own runs against live Wikidata on 9 October 2026: the connector alone, and examples/example.py with OpenAI. Every request count, timing and statement count in this post comes from those.
  • The mul label, missing: "", redirect and no-such-entity responses were each reproduced against the live API, and the package's 55 tests replay those recorded shapes.
  • Wikibase API: wbgetentities.
  • Wikimedia User-Agent policy, which is why the connector sends a descriptive one.
  • cognee and cognee-community.

Comments

Sign in with GitHub to leave one — every thread is a public issue on the site's repo.

Loading comments…