OpenStreetMap Is a Brilliant Wiki and a Bad Data Warehouse

81,455,423 places and 4,688,229 administrative divisions, counted in 0.37 seconds on a laptop on 2026-10-11, without unzipping a single file. The counts answer a query against Overture Maps release 2026-09-23.1, and the striking part is not their size. It is the manner of the counting: DuckDB read the Parquet metadata on S3 and answered the way a database answers a query, not the way a bulk download answers one.

OpenStreetMap is the most consequential mapping project people have ever built, and that success is exactly the problem when your pipeline treats it as data infrastructure. A wiki optimizes for consensus among its mappers. A warehouse optimizes for typed columns and pinned, queryable truth. The two goals collide, and the collision is why Overture Maps, a consortium steered by Amazon, Meta, Microsoft, and TomTom, publishes a consolidated, schema'd copy of the world's base-map data every month. This post argues for moving your base-map layer onto that copy, and names what Overture will not do for you.

A wiki is the wrong shape for a data pipeline

Start with the concrete question. Your team wants to know which restaurants in a delivery zone have outdoor seating. Posing it to OpenStreetMap becomes a tag hunt: amenity=restaurant crossed with outdoor_seating=yes, and the answer is silently incomplete because nothing in the data model has to mention outdoor seating at all. Tags are free keys. The map-feature pages on the wiki document thousands of them, and any mapper can invent another one this afternoon. No schema forces those keys to exist, to carry types, or to mean the same thing in two neighboring departments.

Identity is the second snag. OSM's atomic object is the node, so a building traced by two mappers whose edits meet along a shared wall arrives as two ways with no stable link between them. A mapper who splits a street mints new node IDs as a side effect of being careful. Your Tuesday extract and your Friday extract differ, and there is no release name to pin any of it in a repo.

The obvious self-help fix is a crosswalk table of your own, tags on one side and your columns on the other. It holds until a key gets retired on the wiki or renamed by regional convention, and the crosswalk quietly becomes a data product without an owner. We know this part of the cost from our own work: the trail graph we build from OpenStreetMap extracts carries a tag-interpretation layer that grew longer than the graph code underneath it, and it is never done.

Overture is consolidation with a release calendar

The fix, it turns out, is somebody else's job, done in public. Overture takes the crowd sources, OSM among them, plus Meta's places work, Microsoft's buildings, TomTom's roads, pushes the mixture through a validation and consolidation pipeline, and publishes a typed snapshot every month stamped twice: a release date and a schema version. Release 2026-09-23.1 carries schema v2.0.0, and its notes record the removal of the places categories field in favor of taxonomy plus basic_category, along with a new shelter building class.

Every release lives at the same pattern of paths on S3:

s3://overturemaps-us-west-2/release/2026-09-23.1/theme=<theme>/type=<type>/*.parquet

The theme= and type= segments are hive partitioning, which is a dull phrase until a query engine uses it. Point DuckDB at the bucket with hive_partitioning=1 and it skips whole directories untouched: a question about buildings starts reading at theme=buildings and never opens a file belonging to the 81 million places. Your extract now has a name a reviewer can pin, one string in one repo, and everyone who reads it later gets the same bytes.

My reading of why four companies that each profit from map data fund this: each one was paying to consolidate the world privately, and the shared substrate frees all of them to compete above the data layer. Whatever the internal reasons, the artifact is public, GeoParquet, and published under an open database license.

Stable GERS identifiers and file pruning are the two changes you feel first

The identifier first, because it changes what a join is. Every consolidated feature carries a stable GERS identifier that survives source churn, which turns "attach your records to the map" from a re-trace into a join. A café from the Tahoe sample I pulled from release 2026-09-23.1 sits in the places theme as id e13e7209-a09a-4cf7-9d45-0532936fcc4c, taxonomy primary furniture_store, confidence 1.0, with a version counter on the row. When a later consolidation re-merges its sources, the identifier stays put and the version moves, so your downstream join notices instead of breaking. In OSM the same attachment is a node-ID trace, and one mapper's split can invalidate it overnight.

The second change is pruning. Because bbox is a plain struct column, a predicate on it prunes Parquet files and rows alike, without touching a single geometry. I ran four questions against release 2026-09-23.1 with DuckDB 1.5.6 on 2026-10-11; every wall time below is that run:

Question against release 2026-09-23.1	Answer	Wall time
Count of places, full release	81,455,423	0.37 s
Places in a Tahoe bounding box (-121.0 to -119.6, 38.7 to 39.5)	28,166	2.77 s
Buildings in the same box	228,935	37.1 s
Transportation segments in the same box	106,242	18.0 s

The two-second row is the one worth staring at. The query is:

SELECT count(*)
FROM read_parquet(
  's3://overturemaps-us-west-2/release/2026-09-23.1/theme=places/type=place/*.parquet',
  hive_partitioning = 1
)
WHERE bbox.xmin >= -121.0 AND bbox.xmax <= -119.6
  AND bbox.ymin >= 38.7 AND bbox.ymax <= 39.5;

Nothing here downloads. The engine prunes files by partition, prunes rows by bbox, and reads only what the predicate admits. The same question against OSM is either a server-side Overpass query whose latency is a community donation or a regional extract followed by an XML parse. This is the practical meaning behind the phrase I use on calls: geodata that behaves like normal data. Your count this Tuesday and your count next Tuesday differ only if you asked against different release IDs, which is a fact you can print in the report.

Two limits to plan around, and the workarounds

Limit one is addresses. Release 2026-09-23.1 ships 474 million address features, the figure Overture publishes for the theme, and the docs mark that theme alpha: no GERS identifiers yet, and a schema that can still move. Keep anything customer-facing off it. Geocode against places and divisions today, hold addresses in a side experiment with its own environment, and revisit the theme when it matures and the identifiers land.

Limit two is consolidation lag, with a shadow. A mapper answers a café's opening hours in OSM at breakfast; the edit reaches Overture no sooner than next month's release. The workaround keeps both tools honest: run OSM as the live venue layer where last week matters, then re-join it to the pinned release's stable identifiers once the release lands. And be as skeptical of the new layer as the old one. In my Tahoe sample, 896 of 28,166 places had no taxonomy.primary, so any logic built on that field needs a fallback path. The release notes of 2026-09-23.1 also record identifier churn in base and building_part after a UUID generation change. Pin the release, read the notes, and treat GERS as stable where the data model is mature.

Where this leaves your pipeline

If your base-map layer is grep over tags, then for every question you have already answered you maintain a hand-rolled schema, and for every question arriving next month you maintain a plan instead. The decision fits in one line: pin release 2026-09-23.1, load theme=places and theme=divisions, query them in place with DuckDB, keep OSM where freshness wins, and re-pin when the next release notes clear.

The wiki stays. The warehouse stops being your job, because somebody else funds it, ships it on a calendar, and names every release so you can pin it.

AngoraTek builds geospatial data pipelines for teams that ship map products. Talk to us if your base-map layer is still a pile of tag greps.