Your Tables Can Change Shape Without a Maintenance Window
Tuesday 9:00 on the deploy calendar, and the trips table has readers its writer never met: the enrichment job, and a nightly export a partner parses by column name. Adding one column should be a ten-minute commit. In most production warehouses it is a two-quarter negotiation, because each of those readers broke the last time the shape moved.
Here is our position, in plain terms: the maintenance window is a symptom, and the disease is a compatibility contract nobody ever wrote down. Each reader is coupled to a version of the table's shape, the readers deploy on calendars that have nothing to do with the writer's, and a table that serves both versions at once is the normal state of production. Evolve by expanding first, migrating second, contracting last, and the window can close permanently. The order is the entire method. Skipping the last step is how a warehouse ends up with columns from 2019 that nothing reads anymore.
The coupling is to shape, so make the first move additive
The expand step has one rule and one consequence. The rule: anything you add must be invisible to whoever is reading the table today, which means new columns arrive nullable, and readers keep requiring only what existed when they shipped. The consequence: the writer and the readers deploy on their own schedules, without a merge window between them, because the table's new shape is a superset of the old one and nothing has to be aware of anything.
The databases have met this contract halfway, and the platform notes are worth carrying into the design doc. PostgreSQL 11 made adding a column with a constant default a metadata-only change, with no table rewrite, per the release notes from 2018. Apache Iceberg tracks columns by an internal identifier rather than by position, so a rename is a dictionary edit and widening a type from int to bigint is supported while narrowing is not, all documented under schema evolution. Those notes settle one question for good: a table format can absorb additive change cheaply, which means the rest of the cost of a schema change is being paid somewhere other than the storage engine.
That somewhere is NOT NULL. A required column is a contract change, however the migration tool frames it, because every writer in the house must now satisfy it before the constraint lands. PostgreSQL 18 ships the safe sequence in its own ALTER TABLE syntax: add the constraint NOT NULL as NOT VALID first, which binds only new writes, then run VALIDATE CONSTRAINT, which checks existing rows under a weaker lock without blocking ordinary reads and writes. On earlier versions, the same two steps are a NOT VALID CHECK on the column, a validation, and then SET NOT NULL. The same shape repeats in every store we have changed: bind the new obligation to new writes and verify the old rows separately, so the two waits never land behind one deploy.
Compatibility has names because the registry already grades it
For event streams the vocabulary already exists, and it is worth borrowing for tables. Confluent's Schema Registry grades compatibility modes explicitly: a backward-compatible change lets the new reader consume data the previous writer produced, a forward-compatible one lets the old reader consume new data, and full compatibility is what a team gets to promise when both must hold at once during a rollout. A table with fourteen readers is, in this reading, a schema whose every version has to bridge in both directions for however long the fleet lags.
The one rule the event world keeps sharpening is for protocol buffers, and the language guide states it flatly: never reuse a removed field number, and never repurpose one, because message data produced years ago carries that number's old meaning in a decoder that has since forgotten. A rename that reuses an old tag, a field flipped from optional to required: each is the specific mechanism by which a reader and a writer stop sharing a language mid-deploy. The table version of that rule is unglamorous and it carries the whole section: every field is read by name and coupled by assumption, so the assumptions, not the bytes, are what compatibility has to preserve.
One rename, worked all the way through
The method earns its keep on the case the naive migration breaks: a rename. Ours is vendor_ref, a column from the first weeks of the trips table, becoming external_case_id in a push to make naming provider-agnostic. A rename touches no data and breaks every reader at once, which is exactly why it is the best worked example.
Expand first. The new column is added, nullable, and the writer fills both: external_case_id and the old vendor_ref get the same value on every insert and every update, on every path in the code, on the paths the docs never mention as well. This dual-write period is the part teams try to skip, and the skip is what converts a rename from boring to an incident, because the moment the writer stops writing both, the old name starts holding facts that are older than the table's newest rows.
While both names fill, the oldest readers get a bridge. A view over the table serves the shape the stragglers still expect, and it can even serve rows written before the rename existed:
CREATE VIEW trips_compat AS
SELECT
trip_id,
started_at,
external_case_id,
COALESCE(external_case_id, vendor_ref) AS vendor_ref
FROM trips;
Migrate second, and the unit of migration is a reader, not a table. Backfilling history proceeds in batches, idempotent by primary key following exactly the write discipline from the post on resuming mid-flight, so an interrupted backfill costs nothing but time. Each reader ships its switch from vendor_ref to external_case_id on its own release train. The compat view stays up until its last consumer is off it, which is knowable per reader, not guessed per warehouse.
Contract last. The old column is dropped only when a reader census shows zero references over a full release cycle. The drop itself repairs easily: the column can be re-added under its old name and repopulated from the landing zone, at the cost of a backfill. The contract step is where the two quarters of negotiation used to happen; spread over three deploys on three calendars, the same work contains no step anyone has to stay up for.
Contract steps lack a forcing function, so build one
Nothing in any system makes step three happen, and this is the honest center of the method. Expand is urgent, because the feature ships. Contract is gratitude, owed to a future team, and gratitude does not hold a release calendar. Left alone, the compatibility view and the dual-write outlive every person who remembered why.
The census is what we build first, precisely because it is the part with no natural owner. Gather every reader that names the column: the saved queries inside the BI tool that nobody admits to owning, and the analyst's local notebook that reads prod because its copy went stale. A column is only safe to drop when that list is provably empty, and the proof needs a mechanism: query logs over the window, or a deprecation log table the compat view writes to. Postgres will show connection counts and statement counts, but column-level usage is not in its default statistics, so the mechanism is yours to assemble, and "we checked the other day" is not a mechanism.
Then give the drop a date and an owner, the way a rollback date gets one. A column nobody owns never leaves, and a drop without an owner is the same non-event wearing a ticket number.
What this method cannot do
It cannot save a rename for readers outside the warehouse. An export a partner parses by position, an event payload a client never upgraded still consumes: those couplings are physical, not logical, and compatibility for them means a version, in a filename or a field, that both sides agree on before the shape moves. We have shipped that version column to a partner exactly once, and it was cheaper than the second quarter it replaced.
Default values are the second tax, and it compounds. A default that backfills thousands of rows turns "missing" into "zero", and the zero is a fact from then on, one every reader is entitled to believe. Iceberg's docs allow the default, and our own regret is concrete: a status column we left riding on an "unknown" sentinel instead of backfilled history, and a year on there are three dashboards quietly doing arithmetic with the sentinel. Fill history with backfills, not with plausible placeholders, or write the placeholder down in the code comments so the next reader knows it is a lie.
And full compatibility has a price, the same one the registry's FULL mode charges: fields must stay optional, which a domain with genuine required fields refuses. Where the domain wins, the resolution is the fleet's convergence speed, because full compatibility is only required for the duration of a rollout, and it stops being required once every reader has moved. The window closes by shrinking to exactly the lag of your slowest reader.
The method is the order. Expand first, migrate at the pace of your readers, contract with proof in hand, and it is the third step that pays for the first two. A shape stops belonging to its writer the moment a second reader touches it.
AngoraTek runs these evolutions on real pipelines for small teams. If the next shape change on your calendar still carries a window, bring us the table and we will expand it together, in that order.Thanks for reading this far. If this sounds like the problem you're wrestling with, tell us about it. The team reads every message.
Start a conversation