I added task metadata. The next field still got a column.

Plantronic’s task page needed information that did not belong to Plantronic.

The task itself had a deliberate model: requested dates and effort, planning dates, process state, team assignment, and identifiers used to exchange updates with the project-management system. Planners also needed context from the surrounding project, such as production details and organizational contacts. That information lived in another part of the database, followed a different lifecycle, and was still evolving.

I could have copied every useful source field into the task table. That would have made the task schema an incomplete replica of another system and required a migration whenever its display needs changed. I could also have exposed the source tables directly to the LiveView templates, coupling the interface to a schema Plantronic did not own.

Instead, I added metadata. This sounds like the moment where a carefully modeled application begins hiding everything in JSON, but that was exactly what I wanted to avoid. The metadata had one narrow purpose: carry additional information across a system boundary and place it on the task page. It was not allowed to replace the fields that controlled Plantronic’s behavior.

The distinction became concrete almost immediately. Even with task metadata available, I still added an ordinary column for a new routing attribute. Flexibility had become available, but it did not make explicit modeling obsolete.

The feature was larger than a JSONB column

The final feature commit added more than a thousand lines. Most of those lines were not the JSONB field itself. They described how the data entered Plantronic, how project information was read safely, how the two sources were combined, how metadata appeared in the interface, and how all of this could be tested without relying on a production database.

There were two sources of metadata.

The first came with a task through the existing external API. These values belonged to that task and were persisted with it. The XML input could contain one metadata element or several, so the API boundary normalized both forms into a list before Ecto cast them.

The second source was project data. Plantronic already knew the project’s reference, but the descriptive details remained in the client-owned public schema. A SQL view selected the relevant project facts and represented them in the same metadata shape. When Plantronic loaded a task with metadata, a database function joined that current project information to the metadata stored on the task.

The result looked simple to the caller:

current project context + task-owned metadata → task page

Diagram: Project-owned context is translated before it crosses the schema boundary, then merged with task-owned metadata into one validated display shape.

The simplicity was created at the boundary rather than assumed throughout the application.

This resembles the Content Enricher pattern. A message arrives without all the information its consumer needs, so a transformer uses an identifier from the message to retrieve additional data. Plantronic was not forwarding a message through an integration pipeline, but the structural move was the same: use the project reference to enrich the task for one consumer without pretending that the source system had sent everything.

Flexible names still had a schema

I did not store one arbitrary JSON object. Metadata was an ordered list of records with four fields:

%{
  field_name: "some_source_field",
  field_value: "Some value",
  display: :top,
  order: 2
}

field_name and field_value carried the source detail. display selected one of two areas on the task page. order made placement deterministic within that area.

That shape was an Ecto embedded schema. Its changeset restricted display to known values, and a later validation required field names to use a predictable lowercase format. Ecto’s documentation describes embedded schemas as data mappers that can cast and validate inputs even when the data does not have its own queryable table. That was a good fit here: each metadata item needed validation and serialization, but not its own identity or lifecycle.

PostgreSQL stored the list in a jsonb column. Its JSON documentation notes that jsonb is stored in a decomposed binary form and supports efficient processing and indexing. It also does not preserve the order of keys inside an object. I did not make object-key order carry interface meaning; ordering was an explicit integer in each record.

A list of records also allowed repeated field names. A project can have several people in one organizational role or several related production details. Turning the data into one JSON object keyed by field name would either discard repetitions or invent arrays for some fields and scalar values for others. The uniform record shape avoided that special case.

The schema was flexible about which display fields could arrive, but not about what a metadata item was. That is a modest constraint, yet it kept malformed presentation instructions out of the rest of the application.

The source schema stayed behind a boundary

The project data lived in a public PostgreSQL schema beside Plantronic’s own schema. I added a second Ecto repository configuration for migrations in that namespace. Both repositories connected to the same database, but they had different schema prefixes and migration histories.

This mattered in two ways. It gave the source-facing views an explicit owner in the codebase, and it allowed tests to create a representative public schema instead of quietly depending on tables that existed only in the client environment.

Plantronic did not query a collection of source tables from every screen. One view translated the relevant fields into the metadata record shape. A function in Plantronic’s schema then merged the view result with the task’s persisted metadata.

The function also contained a failure boundary. If retrieving or combining project metadata failed, it returned the task’s own metadata with a fallback marker instead of making the entire task unavailable. Planners could continue working with the information Plantronic owned even when optional context from the other schema could not be loaded.

Catching every database exception is a broad instrument, and I would not use it to hide failures in core task state. Here the asymmetry was intentional. Dates, effort, process state, and team assignment were necessary to operate the task. Project metadata enriched the page. Losing the latter should be observable, but it should not erase the former from the planner’s screen.

This boundary also preserved ownership. Project facts could remain current in their source schema rather than being copied into every task and becoming stale. Task-specific metadata could remain attached to the task. They shared a display shape without being forced into the same storage lifecycle.

Metadata reached the interface, not the process model

The LiveView did not know every field that the source view might return. It iterated over metadata marked for the top section, then rendered the bottom section in the footer. An overflow component handled the case where the source supplied more context than fit comfortably on one line.

That was where generic rendering helped. Adding another descriptive project fact did not require a new task column, another template branch, or a deployment solely to place a label and value on the page. The source view determined the name, value, placement, and order within the constrained envelope.

Plantronic’s own process concepts remained ordinary Ecto fields and associations. Requested and planned dates still had typed date validation. Effort still had numeric constraints. Process steps still had explicit transitions. Team assignment remained a relation. The metadata renderer could show supporting information, but no process transition depended on looking up a string key in a JSON array.

The new routing attribute is the clearest evidence of that rule. It could have been sent as one more metadata record. Instead, it became a nullable string column on the task, was accepted by the external changeset, and was deliberately excluded from later external updates along with other stable routing dimensions. The routing editor then offered only a short whitelist of explicit task attributes as match criteria.

Routing did not ask a generic question such as “does metadata contain this key and value?” It matched properties that Plantronic had chosen to understand.

That choice cost a migration and several changeset updates. It also provided a stable name, a type, update rules, direct queryability, and a clear place in routing tests. Once a value influences behavior, those are not incidental benefits. They are the model.

The representation still leaked

The original design contained a seam that caused trouble later.

Metadata entering through Ecto could appear as embedded structs with atom keys. Metadata assembled by PostgreSQL arrived as JSON-shaped maps with string keys. XML added another irregularity: one element was decoded as a map, while repeated elements became a list.

I normalized the XML cardinality at the API boundary with List.wrap, but the distinction between embedded structs and database maps survived further into the application than it should have. Later fixes taught rendering helpers to accept both atom and string keys, sort either representation, trim values consistently, and return sensible fallbacks.

Those changes are a useful correction to the clean architectural story. A flexible storage format does not remove a schema; it can create several implicit versions of one. Martin Fowler makes this point in Schemaless Data Structures: so-called schemaless data still has an implicit schema, only one that may be hidden in the code that reads it.

In Plantronic, expressions such as metadata["field_name"] || metadata[:field_name] revealed where that implicit schema had leaked. The eventual helper functions centralized the knowledge, but the better boundary would have normalized every source into one internal representation as soon as it entered the application.

This is the tax on flexibility. It appears in parsers, tests, sorting, fallback behavior, translations, and every consumer that starts to rely on a particular key. JSONB makes adding data cheap; it does not make interpreting data free.

I now use behavior as the promotion test

The useful question is not whether a fact is important. Many descriptive facts are important to the person reading a task without being part of Plantronic’s behavior. The question is what the application must know about the fact.

I would keep a value in this metadata channel when Plantronic only needs to preserve or display it, the source system owns its meaning, and generic placement is sufficient. I would promote it into the explicit model when any of the following becomes true:

The value controls a transition or calculation.
Routing, authorization, or filtering depends on the value.
The value needs a database constraint or a non-string type.
Updates require domain-specific rules.
The value participates in core reporting.
Several features repeatedly reach into metadata for the same key.

Promotion does not mean every metadata field must begin as an experiment and later receive a migration. Often the behavior is known from the start, as it was for the routing attribute. The point is that the presence of a flexible column must not become an excuse to avoid deciding.

The reverse is also important. Adding a first-class field merely because one source happens to provide it gives Plantronic responsibility it may not need. The application then owns naming, migration, synchronization, constraints, and removal for a value it only wanted to show beside a task.

Flexibility worked because it had a job description

The metadata feature survived and later expanded into project and reporting views. The implementation evolved: source views changed, metadata access helpers became more robust, and query composition was refined. The distinction at its center remained useful.

Plantronic owns the task’s process. It models the facts required to plan, route, authorize, and report that work. The source system owns a wider set of project details. Metadata lets selected details cross the boundary without importing the source model wholesale.

Looking back, I would normalize the in-memory representation earlier and make degradation from the source schema easier to observe. I would still keep the basic split. A constrained JSONB channel is a practical way to carry facts whose only required behavior is display. It becomes dangerous when convenience allows it to absorb facts the application actually reasons about.

That is why the next field still got a column. Metadata gave Plantronic room to accept unfamiliar context; it did not release me from deciding what Plantronic needed to understand.