Metadata Lifecycle & HMR

This page documents the metadata data path introduced by ADR-0008 and refined by ADR-0005. It is the canonical event stream that powers Studio Hot Module Replacement (HMR), REST writes, and future cloud editing.

Architecture (one diagram)

   ┌─────────────────────────────────────────────┐
   │          Console UI (incl. Studio)          │
   │            (MetadataHmrReloader)            │
   └────────────────────┬────────────────────────┘
                        │ SSE  /api/v1/dev/metadata-events
                        ▼
   ┌─────────────────────────────────────────────┐
   │           MetadataManager (bridge)          │
   │   forwards Repository events → SSE channel  │
   └────────────────────┬────────────────────────┘
                        │ ChangeEvent{kind, metadataType, name, seq?}
                        ▼
   ┌─────────────────────────────────────────────┐
   │       Overlay precedence (protocol.ts)      │
   │   ┌─────────────────────────────────────┐   │
   │   │  Top: SysMetadataRepository (org)   │   │ ← writable overlay
   │   │  Bottom: FS / InMemory (artifact)   │   │ ← read-only baseline
   │   └─────────────────────────────────────┘   │
   └────────────────────┬────────────────────────┘
                        │ append(change log) + emit watch event
                        ▼
                   sys_metadata table
                  (or filesystem artifact)

Reads walk top-to-bottom: the first non-null layer wins. Writes always route to the topmost writable layer (the overlay). Deletes from the layered API only remove the overlay row — the artifact baseline survives.

The four primitives

MetaRef = (type, name, org). As of ADR-0008 §0 amendment (2026-04-13), project and branch are removed from the runtime tuple. Project survives only as an artifact-packaging concept on the objectstack.json envelope; branching is left to Git.

Write path: Studio → SSE → repository

When a user edits a view in Studio:

1. Studio calls PUT /api/v1/meta/view/case_grid (REST).
2. protocol.ts:saveMetaItem() runs the two-tier gate (see overlay whitelist): an item that already exists as a packaged artifact requires MetadataTypeRegistryEntry.allowOrgOverride; a brand-new item requires allowRuntimeCreate or allowOrgOverride.
3. If allowed, the call lands on MetadataRepository.put(ref, body, { parentVersion, actor }).
4. The repository:
   • Verifies parentVersion matches the current head (ConflictError on mismatch).
   • Writes the new row + appends a change-log entry with the next seq.
   • Emits a MetadataEvent { op, ref, hash, parentHash, actor, seq, ts, source }, where op is one of create | update | delete | rename | publish | revert.
5. The MetadataManager bridge forwards the event over the SSE channel /api/v1/dev/metadata-events. The wire payload is not the internal MetadataEvent — it is a ChangeEvent { kind: 'metadata-change', type, metadataType, name, path?, timestamp, seq? } (emitted as event: metadata-change). seq is the canonical repository sequence and is absent for FS-watcher (chokidar) dev events.
6. In dev, the console's MetadataHmrReloader receives the metadata-change event and schedules a debounced location.reload() (default debounceMs: 400, plus a 150 ms toast delay).

The seq is the single source of truth for ordering on the server. Note that the shipped console client does not read seq — it reloads the whole page rather than invalidating per-item caches. There is no granular-refetch hook or seq status badge in the current UI.

Package-first authoring (ADR-0070)

Every runtime-authored item lives inside a writable package — there are no orphans. Code-defined and installed packages are read-only at runtime, so the first step of any Studio/API authoring action is to target a writable package (a "base"): a create/update aimed at a read-only package is rejected, and the author is asked to pick or create a writable base first. New objects, fields, views, and flows are namespaced into that package — which is exactly what os package publish later ships. See ADR-0070.

Note: "package" here is the runtime authoring base (a package_id-bearing container of metadata), distinct from the npm @objectstack/* packages listed in the repo README.

Overlay whitelist (shared-DB tenancy invariant)

In shared-database multi-tenancy, most metadata types must not be per-org customizable — overriding them would break the physical schema. The whitelist lives in one place: MetadataTypeRegistryEntry.allowOrgOverride in packages/spec/src/kernel/metadata-plugin.zod.ts.

Those five are the complete allowOrgOverride: true set: of the 27 types in DEFAULT_METADATA_TYPE_REGISTRY, every other one is false. The ❌ rows above are the false types whose second tier (allowRuntimeCreate) is worth calling out; any type not listed is allowOrgOverride: false.

There is no workflow metadata type (per ADR-0020, record state machines are a state_machine validation). Nor is there a standalone validation type any more — it was retired in protocol 17 under ADR-0088 because ValidationRuleSchema carries no object-binding key, so a rule authored through that door could never say what it protected; author rules in the object's own validations[] instead. The runtime gate is implemented in OVERLAY_ALLOWED_TYPES (derived from the registry) and enforced by SysMetadataRepository.put().

The gate is two-tier — allowOrgOverride: false is not the same as "no runtime writes":

• Overwriting an item that ships from a code package requires allowOrgOverride. Denied → 403 not_overridable.
• Creating a brand-new item that has no artifact backing requires allowRuntimeCreate (or allowOrgOverride). Denied → 403 not_creatable.

See ADR-0005 for the full design and amendments.

Conflict handling

Every put() requires a parentVersion (or null for fresh creates). If the head has moved since the caller read it, the repository throws ConflictError(ref, expectedParent, actualHead) with code METADATA_CONFLICT. protocol.ts catches it at every write entry point (save, publish, rollback, overlay-delete) and re-throws an API error with code metadata_conflict, HTTP 409 Conflict, and the expectedParent / actualHead pair attached. Clients are expected to re-read the item and retry; the console does not yet surface a dedicated conflict-resolution UI.

The hash is sha256: + 64-hex of a canonical (sorted-keys, no-undefined) JSON serialization of the body. Identical bodies produce identical hashes — put() short-circuits on no-op writes.

HMR end-to-end (latency & ordering)

• Latency. The server half (REST PUT → DB write → SSE flush) is fast on localhost, but the client half dominates: MetadataHmrReloader debounces for 400 ms, waits another 150 ms so its toast can paint, and then does a full page reload. Budget roughly half a second plus a cold React mount, not a sub-100 ms in-place re-render.
• Ordering. The seq is monotonic per repository. The repository-level watch() API supports replay from a since cursor, but the dev HMR SSE endpoint does not replay on reconnect — it registers a fresh listener, emits a ready event, and then streams only live events. A tab that disconnects and reconnects will miss any events that occurred while it was offline; it should refetch the affected metadata on reconnect. (The stream also emits : ping heartbeat comments every 15 s.)
• Multi-tab. Every open tab registers its own listener on the same in-process broadcast hub, so all tabs receive the same events in the same order. No tab tracks seq client-side — the shipped reloader ignores it entirely.

Cross-references

• ADR-0005: Metadata customization overlay — overlay semantics, whitelist, conflict rules.
• ADR-0008 §0 amendment (2026-04): project/branch removal — why MetaRef.project and MetaRef.branch are dead.
• ADR-0008: Metadata Repository & Change Log — the four-primitive architecture.
• IMetadataService Contract — the runtime API plugins call.
• North Star — the product shape these primitives serve.

Status (as of 2026-04)

Cross-replica sync happens above the repository, not inside it. SysMetadataRepository.watch() is deliberately scoped to the local instance — it has no LISTEN/NOTIFY or pub/sub of its own, and never will. Cross-node propagation is instead a cluster concern: @objectstack/runtime registers ClusterServicePlugin and MetadataClusterBridgePlugin by default (opt out with cluster: false), and at kernel:ready the bridge calls MetadataManager.attachClusterPubSub(), fanning every local watch event out over the metadata.changed channel (loopback-suppressed by originNode) so peer nodes invalidate their caches. Note this replays the invalidation event, not the overlay row — peers re-read from the shared database. The default cluster driver is memory, which is in-process, so real cross-replica fan-out still requires configuring a distributed driver; and if no cluster (or metadata) service is registered the bridge logs and skips, leaving each replica seeing only its own writes.

FAQ — When does code-defined metadata enter the database?

Short answer: never. This is a deliberate design choice; the table below is the authoritative reference and supersedes any other description you may find.

Why artifact never enters the database

1. Single source of truth. Code → artifact → memory. If artifact also wrote to sys_metadata, every deploy would need a reconciliation step ("the row says v3.0.0 but the code says v3.0.1") that is impossible to get right under concurrent edits.
2. Overlay stays small and auditable. Per-org overlay = "what this organization deliberately changed at runtime". A clean table that's safe to dump, diff, or reset to factory.
3. Immutable infrastructure. Rolling back a deploy = switching image tags. No DB migration, no reverse put, no history replay required.
4. Per-org multi-tenant economics. If artifacts were materialised into sys_metadata, every org would carry tens of thousands of baseline rows. Overlay-only keeps each org's row count proportional to its actual customisation.

What if I want auditable history of deployed artifact changes?

That's a deployment log, not a metadata change log. Track the envelope hash of each deployed objectstack.json against the org/environment in a deployment-history table. The internal items inside the artifact need no per-item history because their canonical history is the Git repo that produced them.