How an Elixir umbrella let KMS become its own project

KMS exists because a client project needed encryption to be part of its storage design from the beginning.

That client project was Formfix. It handled sensitive form submissions, so persisting plaintext first and adding encryption later was never an acceptable implementation plan. The application needed an encrypt-first path while its business logic was still taking shape.

Elixir and Phoenix made the technical response unusually straightforward. I created KMS as its own OTP application inside the Formfix umbrella. During development it behaved like part of a monolith: one repository, one dependency graph, one command for tests, and direct function calls in the same BEAM. At the same time, key management had its own application boundary, supervision tree, storage, migrations, tests, and public API.

The boundary that began as a practical way to keep cryptography out of client business logic became the path by which KMS moved into separate releases, separate pods, and finally a separate repository and open-source project.

I am grateful to Formfix for proving the encrypt-first concepts that became KMS. This is the origin story of that project.

Privacy by design starts before the first row

In my Privacy by Design talks, I make a simple argument: privacy decisions belong in system design, not in a compliance pass after production data has accumulated.

Encryption is only one of those decisions. Purpose, access, retention, deletion, logging, and data minimization matter just as much. Still, the storage boundary is a good place to see whether a project means what it says. If sensitive values enter an ordinary plaintext column, every backup, replica, debugging session, and ad hoc query inherits that exposure. Retrofitting encryption then becomes a migration project with old and new formats, uncertain copies, and a difficult question about whether plaintext has really disappeared.

Encrypt-first changes the default. The application accepts plaintext at an input boundary, turns it into ciphertext before persistence, and makes decryption an explicit operation. A database reader should encounter encrypted payloads rather than the original value. The application still needs authorization and careful runtime handling, but storage no longer assumes that every database consumer should see everything.

Formfix gave this idea a real workload. It was not a cryptography demo built around one hard-coded string. A client application had to write, load, and process encrypted values while its normal Phoenix and Ecto code continued to evolve. That pressure exposed what the KMS API needed to own and what should remain application policy.

The first useful boundary was narrow. Formfix supplied plaintext, a key alias, and optional context. KMS returned an encrypted payload. To decrypt, Formfix supplied the payload and the same context. Formfix decided when its business rules allowed either operation. KMS decided how keys and ciphertext were handled.

That division still shapes the public project.

Wrapped envelope encryption became the core

KMS uses AES-GCM through Erlang’s :crypto. AES-GCM is an authenticated-encryption mode standardized in NIST SP 800-38D. With correct key and nonce handling, it encrypts plaintext and produces an authentication tag that detects a modified payload.

The primitive is deliberately boring. I do not want application code inventing ciphers or implementing authenticated encryption itself. Erlang already provides the operation. KMS adds the key lifecycle and application-facing contract around it.

For each encrypted value, a data encryption key encrypts the plaintext. A key encryption key then wraps that data key. The application stores the ciphertext together with the wrapped data key, while KMS manages the key encryption key by alias. This is envelope encryption, a common pattern for keeping the key that encrypts bulk data separate from the longer-lived key used to protect data keys.

This envelope encryption design stores a wrapped data key beside each ciphertext. KMS can change how it protects key encryption keys without changing every application’s business schema. Current KMS versions can protect persisted KMS secrets through a root master key backed by a local file or an external provider. The application continues to call the same alias-oriented encryption interface.

KMS also accepts additional authenticated data, or AAD. AAD is not secret and is not encrypted. It is authenticated along with the ciphertext. An application can bind a payload to a stable context such as a tenant, record, and field. Moving valid ciphertext to a different context then causes authentication to fail.

AAD does not replace authorization, and it does not stop replay within the same accepted context. It must also remain reproducible for as long as the ciphertext needs to be decrypted. Those constraints are why KMS treats AAD as an application policy that travels through its API rather than trying to infer it.

The original implementation was much smaller than the current system. It already had AES-GCM, aliases, wrapped data keys, and a dedicated KMS store, but root-key handling and authorization were still early. Formfix was valuable precisely because it proved the basic data flow without pretending that the first version had solved every security concern.

Umbrellas let me build a monolith without mixing everything

I like umbrella applications because they separate code before they force separate operations.

The Elixir guide to umbrella projects describes child applications that share one repository, build directory, configuration, dependency directory, and lock file. Mix can compile and test them together, while dependencies between applications remain explicit. The guide also calls out the limit: shared configuration and dependencies mean umbrella children are not fully independent projects.

That balance suited Formfix and KMS.

During early development, both applications ran in the same BEAM and shared the same memory space. A call from Formfix to KMS was an ordinary Elixir function call. There was no serialization, network timeout, API token, service discovery, or second deployment to manage. I could change both sides of the interface in one branch and run the whole suite from the umbrella root.

This felt like writing a monolith because it was one operational unit. It did not feel like one undifferentiated application. KMS had its own Mix project, application callback, schemas, repository, migrations, tests, and dependency direction. Formfix consumed a KMS API instead of reaching into cryptographic helpers scattered among client modules.

An umbrella does not enforce that discipline by itself. One child can still call another child’s internals. Shared runtime configuration can create accidental coupling. A direct in-memory call also means both applications share one trust and failure boundary. The package structure gives developers a place to draw a line; reviews and tests still have to defend it.

For this project, that was enough to begin. I could keep the development speed of a monolith while giving a security-sensitive requirement room to become a subsystem.

The first small step was a separate OTP application

The KMS child started with its own supervision and persistence. This was more than a namespace. OTP applications have startup behavior, dependencies, configuration, and a lifecycle. KMS could be tested and reasoned about as one unit even while Formfix started it in the same release.

This boundary answered a design question before it became an operational question. Key management changed for reasons unrelated to the client project’s business rules. It needed to evolve its payload format, root-key handling, authorization, and recovery. Formfix needed stable encrypt and decrypt behavior. A public module between two OTP applications gave both sides a contract.

The setup was still cheap. No team had to operate another service. No user request could fail because a second pod was temporarily unavailable. There was one deployment and one place to inspect logs. Starting KMS alongside Formfix was the correct trade-off while both shared an operator and trust boundary.

The second small step was a separate release

An umbrella can produce more than one release. mix release accepts named release configurations and an explicit list of applications. The Formfix umbrella eventually defined a Formfix release, a KMS release, and a combined umbrella release.

That meant the same repository could support two deployment shapes. For a compact installation, Formfix and KMS could run together. Where operations or security required more isolation, the applications could be packaged into separate releases and run in separate pods or on separate servers.

This step is small in code organization because the OTP boundary already exists. It is not free in operations. Two BEAM instances no longer share memory, so direct calls need a network-facing client and server. Configuration and credentials must be separated. The remote path needs TLS or trusted private networking, authentication, timeouts, retries, monitoring, and a decision about what the application should do when KMS is unavailable.

KMS now makes that choice explicit. Embedded mode uses a local client in the same BEAM. Remote mode runs a KMS authority process with its HTTP API enabled, while application hosts use a remote client. The public functions remain recognizably similar, but the trust and availability boundaries change.

This gradual move matters more to me than claiming that distributed systems are easy. The umbrella did not remove network complexity. It delayed that complexity until there was a reason to pay for it, while preserving the application boundary needed to introduce it cleanly.

The third small step was a project of its own

Once KMS had its own concepts, release, documentation needs, and uses beyond one client, keeping it inside the Formfix repository stopped helping.

Extracting it was mostly an exercise in removing assumptions. KMS needed its own Mix project and lock file, independent configuration, public documentation, CI, examples, and a release process. Formfix-specific dependencies and shared helpers had to disappear. The client project then consumed KMS like any other dependency rather than through in_umbrella: true.

Elixir’s own umbrella guide describes this path directly: move the application outside apps/, remove umbrella-specific project paths, and depend on it through a path, Git repository, or package. The framework does not make API design automatic, but it avoids treating extraction as a rewrite.

That is how parlant-co/kms came to be. It is now one OTP application with an optional HTTP API, its own persistence choices, root master key providers, authorization features, operational documentation, and a Phoenix example. The repository history still carries its umbrella origin, but the project no longer requires Formfix to make sense.

I like the progression because each stage remained useful:

KMS began as a separate OTP application in the same umbrella and same BEAM.
Named releases allowed KMS and the client application to run together or in separate pods.
Local and remote clients turned the process boundary into a configuration choice.
Extraction gave KMS a separate repository, release cycle, and public audience.

There was no weekend in which a monolith was replaced by microservices. Each boundary became operational only after the code had already learned to respect it.

KMS now supports several useful stopping points

Not every application needs the final stage. The current KMS use-case guide treats deployment as a choice rather than a maturity ladder.

Crypto-only mode exposes AES-GCM helpers when an application needs sound primitives but does not need KMS storage, supervision, sessions, or authorization. The application owns its keys and may decrypt whenever its own policy allows.

Application-wide and field-level encryption fit applications that want ciphertext in a database while keeping ordinary Ecto schemas and changesets. A virtual plaintext field can be encrypted into a persisted ciphertext field before insertion. Aliases group key policy, and AAD binds ciphertext to its row or field context.

Embedded KMS starts the KMS OTP application with a Phoenix or Elixir application. This is closest to the original Formfix arrangement. It offers managed key-encryption keys and root-key wrapping without adding a network dependency. The application and KMS still share one host and trust boundary.

Standalone or remote KMS runs the authority separately. Application hosts no longer need direct access to the KMS database or local root-key file. This can support several applications, centralized operations, and a stronger credential boundary. It also adds a critical network dependency and must be operated accordingly. The local versus remote guide makes those trade-offs explicit.

Multi-user encryption adds principals, sessions, factors, and bindings when KMS itself must decide who may use a key. This differs from an application that authorizes a request and then calls a trusted embedded KMS.

KMS also includes patterns for caller-supplied high-entropy secrets, external root master key providers, and fallback recovery. These options solve different threat models. They are not features every installation should enable.

The smallest fitting mode is usually the best one. If the running application is authorized to decrypt everything, moving KMS to another pod does not automatically protect data from a compromised application process. If database-only disclosure is the concern, embedded encryption with a root key outside that database may be enough. If application hosts must not possess KMS storage or root-key credentials, a remote authority creates a meaningful boundary.

Formfix proved the boundary before KMS became a product

KMS was not born from a plan to publish a generic security platform. It was born because Formfix had an important technical requirement that did not belong among the client project’s other business logic.

The Phoenix umbrella made the responsible first step cheap. I could isolate key management without slowing the product down with another service. The same-memory implementation then proved the encryption API against a real application. Separate releases proved that the OTP boundary could become a deployment boundary. The final extraction proved that KMS could stand on its own.

That sequence is why I remain fond of umbrellas. They let developers work with monolith ergonomics while preserving an honest path toward stronger separation. The path is gradual, and every step asks for more operational discipline. None requires throwing away the previous architecture.

Formfix gave KMS its first real reason to exist. It tested the encrypt-first assumptions, exposed missing pieces, and kept the design grounded in application use rather than cryptographic abstraction. KMS could become an open-source project because the client work had already shown where its boundary belonged.