building a knowledge graph with 1.8 million entities

when i started building dropthe, the initial idea was straightforward: a media site that covers movies, crypto, and tech with actual data behind it. not opinions dressed as analysis. not listicles generated from thin air. real entities, real connections, real numbers.

that simple idea led to building a knowledge graph. not because i planned to + but because the content demanded it. you cannot write meaningfully about a film director without understanding which studios they worked with, which actors they collaborated with repeatedly, and how their box office numbers compare across decades.

the entity model

the core of dropthe.org is a single postgres table called entities. every movie, every person, every cryptocurrency, every company lives in the same table with a type column and a jsonb data field. this sounds reductive, but it turned out to be the right call. a unified schema means every entity gets the same treatment: slug, name, type, structured data, and a web of links to other entities.

the entity_links table is where the graph lives. each row is a directional relationship: entity a relates to entity b with a specific link type. christopher nolan directed the dark knight. warner bros produced it. christian bale acted in it. these are not metadata fields on a movie record. they are first-class connections in a graph.

enrichment as a discipline

the hardest part was not building the schema. it was filling it with real, verified data. i run enrichment pipelines that pull from public apis, cross-reference sources, and validate before writing. every enrichment run happens on a local postgres clone first + never directly on production. i validate row counts, check for nulls, and only then push changes upstream to the live dropthe data hub.

this discipline came from a painful lesson early on. a bad enrichment run once overwrote clean data with incomplete records. since then, i maintain three databases: production (supabase), a local work copy for daily experiments, and a stable snapshot that serves as a rollback point. paranoid, maybe. but i have never lost data since.

what the graph enables

the real payoff is what you can do once the graph exists. on dropthe, every entity page automatically shows related entities. a director's page lists their films. a company page shows its people. a cryptocurrency page links to its founders and the companies behind it. none of this is manually curated. it all flows from the graph.

the auto-linker runs bidirectional link inference: if christopher nolan directed inception, then inception was directed by christopher nolan. both directions are stored, both queryable. this sounds obvious, but getting it right at scale across 2.9 million links required careful deduplication and type validation.

the architecture in practice

the stack is deliberately simple. postgres for storage, python for enrichment pipelines, wordpress for the frontend with direct supabase queries. no graphql layer, no neo4j, no fancy graph database. a well-indexed relational database with a links table handles the workload. queries that join entities through links return in under 50 milliseconds on production.

i chose this approach because the graph is a means to an end, not the product itself. the product is dropthe.org + a site where every piece of content is backed by structured, interconnected data. the knowledge graph is the engine. the articles, the entity pages, the data visualizations + those are what people see.

building this system taught me that data infrastructure is not about the technology you choose. it is about the discipline you bring to maintaining it. a knowledge graph is only as good as its last enrichment run.