Living Document Notice
Published 2026-09-12. The evolving architecture, revisions, and connected notes for this dispatch live in the Stax Digital Garden.

Additive Data Contracts - Evolving Note Schemas Without Breaking Legacy Vaults

Summary

As personal knowledge bases age, frontmatter schemas evolve to incorporate new workflow requirements like review stages, task states, and geographic coordinates. Introducing mandatory schema validation often breaks older markdown files, forcing users into risky bulk migration scripts that pollute Git histories.

The Outrigger Protocol specifies an additive schema contract for frontmatter definitions. By enforcing append-only field semantics, default fallbacks, and unknown property passthrough, engines can introduce new structural capabilities while maintaining backward compatibility with notes written years earlier.

The Hazard of Destructive In-Place Migrations

Automated migration scripts that rewrite every markdown file in a vault introduce major risks. Batch rewriting touches file modification times (mtime), invalidating sync caches across secondary devices and creating noisy version control diffs.

If a migration script encounters an unexpected edge case halfway through execution, the vault is left in a partially migrated state. Restoring consistency requires manual conflict resolution across thousands of plain text files.

External tools that rely on specific frontmatter keys will fail if an automated script renames or restructures properties without coordinated updates across all integrated applications.

Additive Schema Evolution Rules

Outrigger defines three strict invariants for schema evolution across all processing layers:

1. Never Make New Fields Required: Any newly added metadata key must be declared optional or paired with an immutable default value.
2. Preserve Unknown Fields: Parsers must never discard unmapped properties during deserialization and serialization loops.
3. Never Rename Keys in Place: Deprecated keys remain supported as read-only aliases rather than undergoing automated find-and-replace routines.

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct EvolvingDocumentMetadata {
    pub title: String,
    pub date: chrono::NaiveDate,
    // V2 additive field with explicit default
    #[serde(default)]
    pub review_status: ReviewState,
    // V3 additive field with optional resolution
    #[serde(default, skip_serializing_if = "Option::is_none")]
    pub coordinates: Option<GeoPoint>,
    // Retain all unrecognized legacy or custom keys
    #[serde(flatten)]
    pub passthrough: BTreeMap<String, serde_yaml::Value>,
}

By flattening unmapped values into the passthrough map, older notes retain custom extensions without failing schema gates.

Contract Verification in CI Pipelines

Schema changes are validated in CI against historical test fixture vaults spanning multiple release years. The verification test suite parses older test collections through the new schema parser, verifying zero validation errors and zero byte mutation upon re-serialization.

This automated check guarantees that legacy notes written under older schema iterations remain completely readable and unchanged under modern parser releases. Any proposed schema modification that fails to parse historical fixtures without warnings is automatically rejected at the pull request boundary before reaching staging branches.

In addition, synthetic vaults containing randomized legacy key orderings are generated during stress testing to verify that property order variations do not cause parser panics.

Schema Compatibility Verification Results

The table below outlines test results across historical schema versions evaluated against current Outrigger parsers.