# Lukas Rieder — Full Site Context

> Notes on software, systems, and building small things well.

This file is generated during every site build from Lukas Rieder's public pages, writing, and talks. Parlant GmbH remains the primary source for company capabilities, projects, and contracting information.

## Parlant GmbH — Start Here

Primary company and commercial context.

- [Parlant GmbH](https://www.parlant.co/llms-full.txt): Software development and IT consulting company founded by Lukas Rieder.

## About and Work

### [Lukas Rieder](https://www.lukasrieder.com/)

Notes on software, systems, and building small things well.

#### Lukas Rieder

I write about software, freelancing, and the useful details discovered while building things.

#### Recent writing

#### Work

### [About](https://www.lukasrieder.com/about/)

Personal notes on building software, running systems, and work.

#### About

I am Lukas Rieder: a self-taught software engineer, consultant, and founder. Since 2008, I have worked on web and data systems, first as an independent developer and later with teams and companies of my own. I founded [Parlant GmbH](https://www.parlant.co/en/) in 2022.

Most of my work lives where application code meets the systems around it: data, infrastructure, deployment, operations, and the small decisions that keep a product understandable after its first release. I like tools with visible seams, plain interfaces, and a path from local development to a system someone else can run.

##### Why this site exists

This is not a company brochure or a complete CV. It is my working notebook: a place to turn decisions, experiments, and hard-won lessons into notes I can revisit. The [writing](https://www.lukasrieder.com/blog/) follows work closely enough to explain not only what I built, but why a particular constraint, tool, or design choice mattered.

Expect posts about software design, developer tools, infrastructure, freelancing, and the maintenance work behind long-lived systems. Some begin with a small local annoyance; others with a production deployment or a client project. Both deserve a record when they lead to a reusable idea.

For a compact map of companies, ventures, and selected projects, see [Work](https://www.lukasrieder.com/work/).

### [Work](https://www.lukasrieder.com/work/)

Companies, ventures, and selected projects.

## Writing

### [I found a new home at Goatmire](https://www.lukasrieder.com/blog/i-found-a-new-home-at-goatmire/)

Goatmire reminded me why unfinished experiments and the people around them matter as much as shipping software.

I went to [Goatmire](https://goatmire.com/) expecting a Nerves and Elixir conference. I came home feeling that I had found another community I wanted to belong to.

The timing was good. Coding agents and large language models have made my normal programming work slightly mad. An agent can read a repository, change several parts of it, and run the test suite before I have finished thinking through the first change. I love what this makes possible. I also notice how easily a day becomes a chain of tasks completed at machine speed.

At Goatmire, I spent part of a workshop making a native desktop counter. Elsewhere, I turned a crow sample into a musical instrument. Neither needed a customer or a roadmap. I had forgotten how much I missed that kind of programming.

A reassuringly large part of the [Elixir Berlin](https://elixir.berlin/) user group had made the trip too. I got to spend time with familiar people away from our usual meetup and met many bright people I had not known before. People listened to unfinished ideas. Conversations picked up again in the next hallway or over the next drink without anyone having to call it networking.

I do not mean "home" in a grand sense. Goatmire felt like another place where I could bring something half built, admit what I did not know, and expect curiosity rather than posturing. That is rare enough to notice.

##### Programming without a ticket

Most of my code starts with a reason. A client needs something, a service is too slow, or a tool is missing. Even my side projects tend to acquire issue trackers and release plans before they deserve them.

At Goatmire, I let curiosity choose the next step. I was not trying to finish a client task. I wanted to see a BEAM application open its own desktop window. Then I wanted to know whether a crow could carry a melody.

This was not a rejection of agentic programming. I am still excited by it. Goatmire simply reminded me that faster completion leaves more time to explore only if I actually use some of that time for exploration.

##### A native Elixir window

The workshop that stuck with me was Damir Batinović's introduction to [Emerge](https://github.com/emerge-elixir/emerge) and [Solve](https://github.com/emerge-elixir/solve). Emerge draws native interfaces from Elixir. Solve handles state without tying it to a particular interface.

My workshop project was a counter, about as glamorous as counters usually are. Its state fit in one controller:

```elixir
defmodule Example.Counter do
  use Solve.Controller, events: [:increment, :decrement]

  def init(_params, _deps), do: %{count: 0}

  def increment(_payload, state),
    do: %{state | count: state.count + 1}

  def decrement(_payload, state),
    do: %{state | count: state.count - 1}
end
```

The viewport looked up that controller and connected its events to buttons:

```elixir
credits = solve(Example.App, :credits)

Input.button(
  [Event.on_press(event(credits, :increment))],
  text("+")
)
```

Solve updates the controller, then the viewport asks Emerge to render again. The counter and viewport run under a normal OTP supervisor. The application code has no browser tab and no JavaScript bridge.

The dependencies are equally plain:

```elixir
defp deps do
  [
    {:emerge, "~> 0.4"},
    {:solve, "~> 0.3"}
  ]
end
```

Before this workshop, I associated an Elixir interface with Phoenix, LiveView, or a terminal. Now a native window is on that list.

Back home, the project also caught one bit of workshop sloppiness. It requires Elixir `~> 1.20`, but my default shell runs `1.19.5`, so `mix test` refuses to start. I had not committed a `.tool-versions` or `mise.toml` file. The code survived the trip; its toolchain did not. Next time I will record both.

##### The Murder

The music workshop went in a less sensible direction. My Goatmire folder now contains several [Sonic Pi](https://www.sonic-pi.net/) tracks, a lot of medieval techno, and a crow instrument called `The Murder`.

Sonic Pi ships with a sample named `:misc_crow`. I cut a tiny piece from the caw and sent it through an echo with a very short delay. The reciprocal of a note's frequency gives the delay needed for a tuned comb filter:

```ruby
define :murder_phase do |n|
  rt(1.0 / midi_to_hz(note(n)))
end

define :caw do |n, opts = {}|
  with_fx :echo,
          phase: murder_phase(n),
          decay: rt(opts.fetch(:decay, 1.5)),
          max_phase: rt(0.06) do
    sample :misc_crow,
           start: 0.1,
           finish: 0.16,
           amp: opts.fetch(:amp, 1) * 0.3
  end
end
```

The crow provides a noisy attack. The short echo provides the pitch. Once that worked, I kept going. `Rookery Techno` uses crows for its kick, snare, hi-hat, bass, and arpeggio. `Carol of the Crows` turns the familiar bell ostinato into a round for plucked birds.

It was completely unnecessary. I learned how the delay time in a comb filter determines pitch, and I had fun writing it. I had missed writing code that only needed to do that.

##### KMS, finally on Hex

I also brought a serious project. On the lightning track, I presented [KMS](https://gitlab.com/parlant-co/kms) and published its first package on [Hex](https://hex.pm/packages/kms).

KMS began inside another Elixir application before becoming its own OTP application. I wrote about [why I separated it](https://www.lukasrieder.com/blog/how-an-elixir-umbrella-let-kms-become-its-own-project/). It can run inside an application with SQLite, use PostgreSQL, or expose an HTTP API when the key authority needs a separate deployment.

The smallest example creates a key-encryption-key alias and encrypts a value with additional authenticated data:

```elixir
{:ok, _kek} = KMS.create_alias("owner:alice")

{:ok, encrypted} =
  KMS.encrypt_alias(
    "secret",
    "owner:alice",
    aad: "document:123/body"
  )

{:ok, "secret"} =
  KMS.decrypt_alias(
    encrypted,
    "owner:alice",
    aad: "document:123/body"
  )
```

The `aad` value is not secret. It binds the ciphertext to that document and field. If someone copies the ciphertext to another field, decryption with the new context fails.

The package installs as a normal Mix dependency:

```elixir
defp deps do
  [
    {:kms, "~> 0.1.0"}
  ]
end
```

I had already spent enough time wondering whether KMS needed another pass before release. Once I had explained it to a room full of people, keeping it off Hex felt silly. I published it.

##### Git hosting from Europe

One of the people I met was Peter Ullrich, who is building [Pushin.eu](https://www.pushin.eu/). I wholeheartedly support his attempt to build a sovereign Git service in Europe.

Pushin says it keeps repositories in the European Union under European jurisdiction. It supports ordinary Git over SSH and HTTPS, and its API follows GitHub's request and response shapes closely enough for many existing tools. The `pun import` command runs on the developer's machine, so the GitHub token stays there.

What convinced me was the boring exit path. A normal `git clone` still gives me the complete repository history. Issues and pull requests are available through the API. Moving to a European host should not require handing control to a different kind of trap.

##### What I brought home

The most honest record of Goatmire is still sitting in `~/Projects/Goatmire/`. It contains an unfinished counter that currently refuses to compile in my default shell and more crow techno than anyone requested. Both make me want to keep programming.

I am not going to use fewer agents. I am going to be more careful about what I use the time they save for. Not every experiment needs to become a product, and not every useful conference result fits into a work plan.

I found a new home because the people at Goatmire made curiosity feel normal. I want to keep exploring this field with them, including the corners that look ridiculous at first.

KMS made it onto Hex because I finally stopped preparing to release it. If encryption belongs in your Elixir application's data model, try [the package](https://hex.pm/packages/kms), read the [documentation](https://hexdocs.pm/kms), or inspect [the source](https://gitlab.com/parlant-co/kms). If the API or documentation gets in your way, open an issue and show me where.

### [How wrapped-key encryption protects data at scale](https://www.lukasrieder.com/blog/wrapped-key-encryption/)

A small encrypted key beside each ciphertext looks redundant until rotation, access control, and KMS outages expose what that extra layer buys.

Wrapped-key encryption solves a practical problem: the key that encrypts application data needs protection of its own.

Encrypting every record directly with one long-lived master key looks simpler. It also spreads one key across a large amount of data, ties bulk encryption to the system that holds the master key, and makes changes to that key expensive. Wrapped-key encryption inserts a deliberate second layer. A scoped data encryption key encrypts the data. A separately managed key encryption key encrypts, or wraps, that data key.

The result is commonly called **envelope encryption**. "Key wrapping" has a narrower cryptographic meaning, while envelope encryption describes the complete design around data and keys. The distinction matters when choosing an API, but both terms point to the same useful separation: data keys handle data; wrapping keys protect data keys.

This article explains what that extra layer buys, what it costs, and where implementations tend to go wrong.

##### The two keys have different jobs

A data encryption key, usually shortened to DEK, is a symmetric key generated for a limited unit of data. That unit might be one record, one object, one tenant, or one storage segment. The application uses the DEK with an authenticated-encryption algorithm such as AES-GCM to produce ciphertext and an authentication tag.

A key encryption key, or KEK, protects the DEK. The KEK normally lives behind a key management system or hardware security module. Rather than handing the KEK to application code, the service exposes operations that wrap and unwrap data keys under an identity and access policy.

The stored envelope usually contains:

- The encrypted application data.
- The wrapped DEK.
- A nonce or other initialization value required by the data cipher.
- An authentication tag, unless the cipher format already includes it.
- A format version and a stable reference to the wrapping key.
- Any non-secret metadata needed to reconstruct authenticated context.

The wrapped DEK does not need to be hidden from the database. Its confidentiality and integrity come from the KEK and wrapping operation. Keeping it beside the ciphertext often makes backup, replication, and retrieval easier because the pieces travel together.

{% include diagram.html post="wrapped-key-encryption" diagram="envelope-flow" caption="Diagram: The KMS protects the small data key, while bulk data encryption remains local to the application." %}

##### How an encryption operation works

A typical write follows five steps:

1. Generate a fresh, random DEK with a cryptographically secure random-number generator.
2. Encrypt the plaintext locally with that DEK and an authenticated cipher.
3. Ask the KMS to wrap the DEK with the selected KEK.
4. Store the ciphertext, wrapped DEK, nonce, tag, version, and key reference as one logical envelope.
5. Remove the plaintext DEK from application memory as soon as the runtime permits.

Reading reverses the flow. The application loads the envelope, sends the wrapped DEK to the KMS for unwrapping, and uses the returned plaintext DEK to authenticate and decrypt the ciphertext locally.

Google Cloud's [envelope-encryption guidance](https://cloud.google.com/kms/docs/envelope-encryption) recommends generating DEKs locally, storing them only in encrypted form, keeping each near the data it encrypts, and managing the smaller set of KEKs centrally. Google Tink can implement this flow with its [KMS envelope AEAD API](https://developers.google.com/tink/client-side-encryption), including generation of a fresh DEK and storage of its wrapped form in the ciphertext.

The exact serialization is part of the security design. A format should say which algorithm and version produced the envelope instead of asking future code to guess. It should also reject unknown versions rather than silently falling back to unsafe behavior.

##### Why not send all data through the KMS?

Key management services are designed to protect and control keys, not to act as high-throughput storage ciphers. They often limit request size and charge or throttle by operation. Sending a large file to a remote KMS also adds network latency and puts plaintext on another service boundary.

Envelope encryption keeps bulk work local. The application performs fast symmetric encryption close to the data and asks the KMS to process only a small DEK. A large object and a short database field create roughly the same KMS payload.

This separation is also useful when the application must encrypt while disconnected from the KMS for short periods, although such caching needs a strict threat model. Reusing or caching plaintext DEKs reduces KMS traffic but increases the amount of data exposed by one key and keeps sensitive material in memory longer. Availability cannot be improved for free.

##### The concrete benefits

###### Smaller key-management surface

A system may create millions of DEKs while managing only tens or hundreds of KEKs. The KMS can concentrate access policies, audit events, rotation schedules, and hardware protection on that smaller set. Applications do not need one centrally registered master key per record.

This is management efficiency, not fewer cryptographic keys. The many DEKs still exist, but only in wrapped form outside active operations.

###### Separation between data and key custody

OWASP recommends storing keys separately from encrypted data where possible in its [Cryptographic Storage Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Cryptographic_Storage_Cheat_Sheet.html). An attacker who obtains only a database backup should not also receive the KEK needed to unwrap its data keys.

That boundary becomes meaningful only if credentials and permissions are also separate. If the same compromised application process can read every ciphertext and ask the KMS to unwrap every DEK, the KMS has not prevented that process from decrypting data. It may still provide audit logs and block offline decryption of a stolen database, but it does not turn an authorized runtime into an untrusted one.

###### More practical rotation

Without an envelope, replacing a key can require decrypting and re-encrypting every byte of application data. With wrapped keys, the bulk ciphertext can remain unchanged while each DEK is unwrapped under an old KEK and wrapped under a new one.

That can reduce data movement substantially, but the work does not disappear. Every affected envelope still needs a safe migration. Old KEK versions must remain available until migration and validation finish. The process needs checkpoints, retries, counts, and a tested rollback or recovery plan.

Many systems use an even simpler first step: new writes use the current KEK, while readers retain access to old KEK versions. Rewrapping then happens gradually. Rotation policy should distinguish creating a new KEK version, directing new encryption to it, rewrapping old DEKs, and eventually disabling or destroying old versions. Those are separate actions.

###### Finer containment

A fresh DEK per record or object limits how much ciphertext shares one data key. A tenant-level or segment-level DEK costs less to unwrap and cache but exposes a larger set if that DEK leaks. There is no universally correct granularity.

The useful question is concrete: how much data should one recovered DEK reveal, and how many KMS operations can this workload afford? Google Cloud explicitly recommends choosing DEK granularity according to the workload rather than treating one pattern as mandatory.

###### Central policy and auditability

A KMS can decide which workload identity may use a KEK, for which operations, and under which conditions. It can also produce an audit trail for unwrap attempts. This is much easier to inspect than master keys copied into application configuration across many hosts.

Audit logs are evidence, not prevention. They need retention, alerting, and identities narrow enough to make events meaningful. A log saying that a broad production role unwrapped a key is less useful than one identifying a specific workload and purpose.

##### What wrapped keys do not solve

Envelope encryption protects a specific boundary. It does not provide complete data security.

It does not replace authorization. A caller must still be allowed to read the record and invoke decryption. It does not protect plaintext while an authorized application is processing it. It does not remove secrets from process memory, crash dumps, traces, or logs. It does not make backups recoverable after someone destroys the only usable KEK. It does not make encrypted values searchable without an additional, carefully designed index. It also does not protect passwords, which should normally be stored with a password-hashing function rather than reversible encryption.

Authenticated encryption matters too. Confidentiality alone does not detect a modified ciphertext or swapped wrapped key. AES-GCM is one established authenticated mode described by [NIST SP 800-38D](https://csrc.nist.gov/pubs/sp/800/38/d/final). Nonces must follow the chosen algorithm's rules, and additional authenticated data must be stable and reproducible.

AAD can bind an envelope to context such as a tenant ID, record ID, and field name. If an attacker copies ciphertext into another row, authentication then fails. AAD is not secret, does not stop replay in the same context, and can make data undecryptable if it depends on mutable values.

##### Key wrapping is more specific than ordinary encryption

Developers sometimes use "wrap" to mean any encryption of a key. Standards use it more precisely. [NIST SP 800-38F](https://csrc.nist.gov/pubs/sp/800/38/f/final) defines approved methods for protecting both the confidentiality and integrity of cryptographic keys, including AES Key Wrap and AES Key Wrap with Padding.

A managed KMS may expose a dedicated wrapping operation, an authenticated-encryption operation used for a DEK, or a data-key API that returns plaintext and encrypted copies together. Follow the provider's documented construction. Do not improvise by feeding raw key bytes into an unauthenticated cipher mode, and do not assume that an API named `encrypt` automatically implements a standard key-wrap format.

Interoperability depends on the exact algorithm, padding, metadata, and byte encoding. This matters during provider migration and disaster recovery. Test that a restored system can actually unwrap representative historical envelopes, not merely that it can list the KEK.

##### Operational costs worth planning for

The extra layer introduces state and failure modes:

- Each envelope is larger because it carries a wrapped DEK and metadata.
- Decryption may require a network call, adding latency, cost, throttling, and an availability dependency.
- Caching DEKs changes the exposure window and complicates eviction.
- Destroying or disabling a KEK can make every dependent envelope unreadable.
- Restores must preserve the relationship among ciphertext, wrapped DEK, key version, and authenticated context.
- Rotation and rewrapping need observable migration jobs rather than a configuration toggle.
- Access to unwrap keys is highly sensitive and should not be granted merely because a workload can encrypt.

Google Tink's documentation makes one trade-off explicit: its envelope API handles a fresh DEK automatically but performs a KMS request for every encryption and decryption. An alternative protects a longer-lived keyset with the KMS and can reduce calls, at the cost of more key handling and a wider effect if that keyset is exposed.

These costs do not negate the design. They explain why the right unit for a DEK depends on throughput, failure tolerance, and the amount of data one key may expose.

##### A review checklist

Before shipping an envelope-encryption design, I would want clear answers to these questions:

1. What exact data is sensitive, and against which plausible disclosure are we protecting it?
2. Where are DEKs generated, and how long can plaintext DEKs remain in memory?
3. What is the DEK granularity, and how much data does one leaked DEK expose?
4. Which authenticated-encryption and wrapping constructions does the implementation use?
5. How are nonces generated and prevented from repeating under one key?
6. Which stable values become AAD, and can they be reconstructed after a restore or migration?
7. Can the application encrypt without permission to decrypt where the workflow permits it?
8. What happens to reads when the KMS is slow, unavailable, throttled, or returns an unknown key error?
9. How do new KEK versions, gradual rewrapping, disabling, and destruction differ operationally?
10. Can a restore test decrypt old envelopes with restored KMS access and metadata?
11. Do logs, traces, exceptions, and crash reports exclude plaintext and key material?
12. Which metric or audit event reveals unusual unwrap volume?

If these answers live only in the encryption function, the system is not ready. Key management crosses application code, identity, storage, backups, monitoring, and incident response.

##### Use the extra layer for a reason

Wrapped-key encryption is valuable because DEKs and KEKs live different lives. DEKs can be numerous, narrow, and close to ciphertext. KEKs can be scarce, centrally controlled, audited, and protected by a KMS or hardware boundary.

That separation makes large-data encryption efficient, limits some forms of key exposure, and gives rotation a tractable path. It also adds metadata, service calls, migrations, and a dangerous dependency on the availability and recoverability of wrapping keys.

Use it when those boundaries address a stated threat model. For a small application, a well-protected local key and a sound authenticated-encryption library may be the simpler correct answer. For systems with many records, several workloads, centralized access policy, or regular key rotation, the wrapped-key pattern earns its extra moving parts.

I use this pattern in [KMS, which grew from an Elixir application into its own project](https://www.lukasrieder.com/blog/how-an-elixir-umbrella-let-kms-become-its-own-project/). The implementation details differ across providers and libraries, but the architectural question stays the same: which key should protect the data, and which boundary should protect that key?

##### Try the pattern with KMS

[KMS is available on Hex](https://hex.pm/packages/kms) as one OTP application:

```elixir
defp deps do
  [
    {:kms, "~> 0.1.0"}
  ]
end
```

Use it as an embedded library in the same BEAM, or enable its HTTP API and run it as a separate key-management service. The [KMS documentation](https://hexdocs.pm/kms) starts with the embedded setup and explains when the remote boundary is worth operating. The [source is on GitLab](https://gitlab.com/parlant-co/kms). Try the embedded example first, then move the service boundary only if your threat model requires it.

### [My terminal now has a 40-millisecond gut feeling](https://www.lukasrieder.com/blog/my-terminal-now-has-a-40-millisecond-gut-feeling/)

TypeSafe's new model answers questions without writing a single word. An open-weight counterpart now does the same on my Mac in 40 milliseconds, and one of my first tests showed where it stops being trustworthy.

[feelsneat](https://gitlab.com/parlant-co/feelsneat) answers small questions about text on my Mac and returns probabilities instead of prose:

```sh
$ feelsneat noul "Is this a phishing attempt?" \
    "Your mailbox is full. Verify your password within 24 hours at http://mail-support-login.example"
0.9177
```

It runs the open-weight [Laya](https://laya.convaiinnovations.com) decision models natively on Apple Silicon. A warm English question takes about 40 milliseconds on my M2, a multilingual one about 16. Nothing leaves the machine. I built it for the places where my scripts and coding agents need a quick judgment call and a large language model is the wrong tool.

##### A model that refuses to write

This month [TypeSafe AI](https://typesafe.ai) launched Jev, which it calls the first "System One" model. The name borrows Daniel Kahneman's fast, intuitive System 1 thinking. You send Jev a state, such as a support ticket or a chat log, and a set of typed questions. It returns typed answers. It never generates text, so there is nothing to parse, and a malformed JSON response is not a possible failure. The Register summarized the idea well in its headline, [Shut up and calculate](https://www.theregister.com/devops/2026/09/23/shut-up-and-calculate-jevs-new-ai-primitives-for-coders/5298431).

Jev has three question types. A `noul` asks whether a statement is true and returns P(true). A `choice` picks one option from a set and returns a probability for each. A `score` places the state on an ordered rubric and returns the expected level. [TypeSafe's documentation](https://docs.typesafe.ai/introduction) is excellent, and its framing matches how I already like to build software: code owns the control flow, and the model answers narrow questions at the branches.

Jev is a hosted, metered API with closed weights. On September 18, Nandakishor Mukkunnoth of ConvAI Innovations published the first release of [Laya](https://laya.convaiinnovations.com), an open-weight model family that takes the same request shape. It grew out of his earlier work on [reinforcement learning for sales conversion prediction](https://arxiv.org/abs/2503.23303) and [confidence-aware routing](https://arxiv.org/abs/2510.01237). The [three checkpoints](https://huggingface.co/convaiinnovations/laya) are ModernBERT and mmBERT encoders with a small decision head. They are Apache-2.0 licensed, and the [reference runtime](https://github.com/NandhaKishorM/laya) is Python and PyTorch.

I wanted Laya next to my other local tools. [sayneat](https://www.lukasrieder.com/blog/i-replaced-macos-say-with-local-neural-speech/) already speaks for my terminal, and looksneat reads images for it. A third single-binary tool that judges text fit the pattern, so I built feelsneat with [pi](https://pi.dev/) in one long session on September 25.

##### What I ask it

The most useful habit so far is asking every question I might need in one request and letting code decide what matters. This support ticket gets four questions:

```sh
$ feelsneat ask ticket.json
team     billing   0.9834
refund   0.9099    0.9099
churn    0.8393    0.8393
urgency  1.9086    0.8909
```

The model answers all four in one batched forward pass in about 140 milliseconds. The last column is the probability of the reported answer. That is the number I branch on: act automatically above a threshold that fits the stakes, and hand everything below it to a person or a reasoning model. TypeSafe describes this and several other patterns in its [patterns guide](https://docs.typesafe.ai/patterns).

The single-question commands print one value and set an exit code, so they fit into shell scripts and git hooks:

```sh
if feelsneat noul -t 0.7 "Is this commit message vague?" --state-file "$1" >/dev/null; then
  echo "warning: the commit message looks vague" >&2
fi
```

Most of my interest is in agent harnesses. A coding agent makes small decisions all the time. It picks a model, reads or skips a file, and decides whether a shell command deserves a confirmation. Asking a frontier model each time is slow and costs money. Asking Laya locally takes tens of milliseconds and costs nothing. feelsneat ships a `system1` tool for pi and a stdio MCP server, which is how a local LLM served by [oMLX](https://github.com/jundot/omlx) can call it.

Language routing happens before the model runs. The English checkpoint stays confidently wrong on scripts it cannot read, so feelsneat ports Laya's detector and sends other text to the multilingual checkpoint:

```sh
$ feelsneat -f json noul "Möchte der Kunde kündigen?" \
    "Hiermit kündige ich meinen Vertrag zum Monatsende." | jq .routing.reason
"Latin script but language looks like de, not English"
```

##### Where it disappoints

Laya is a fast foundation, not an oracle. Upstream reports 0.65 accuracy on task families it never trained on, and below 0.45 on sentiment and rating. The English checkpoint reads at most 512 tokens, and the question and options count against that limit. Choice questions degrade beyond about twenty options because every option shares one token budget.

My first agent-safety test made the point for me. Laya gave `git push --force origin main` a 0.51 probability of being destructive. I use it as a second opinion that asks for confirmation, never as a gate that approves something. The TypeSafe team publishes a candid list of [Jev's weak spots](https://docs.typesafe.ai/model-jaggedness/jev-1.13), such as counting, date arithmetic, and indirection. Laya shares them at least as strongly.

##### Fast enough to ask every time

feelsneat is written in Rust and runs the model through MLX on the GPU. These are warm measurements on an M2 with 24 GB, using a support ticket and a mix of question types:

| Checkpoint | 1 question | 10 questions |
| --- | ---: | ---: |
| English, ModernBERT-large | 40 ms | 245 ms |
| Multilingual, mmBERT-base | 16 ms | 90 ms |

For comparison, Laya's authors report 39.5 ms on an Nvidia T4 GPU and 580 ms on a four-core cloud CPU for one English question, so a laptop GPU now matches a data-center card. TypeSafe describes Jev's response times as around 150 ms before the network round trip. Through the feelsneat daemon, a complete CLI call takes about 50 ms, including process start and HTTP. Both checkpoints use about 1.8 GB of memory while resident.

The first working version took 103 ms for one question. The largest fix was almost invisible. The GELU activation in the Rust MLX bindings multiplies by 32-bit scalars, which silently promoted my 16-bit model to 32-bit arithmetic for every layer after the first. Replacing that function, storing weights pre-transposed, keeping them wired in GPU memory, and warming each kernel variant at startup brought the time down to 40 ms.

##### Matching upstream first

Speed would mean little if the answers drifted. I checked feelsneat against a float32 PyTorch reference, which itself matches the released Laya package within 0.00005. The token sequences are identical, and probabilities differ by at most 0.007 across all three checkpoints. The language router agrees with upstream on all 562 test inputs mined from upstream's own tests.

That comparison also surfaced one uncomfortable detail. Laya's README shows its example ticket routed to infrastructure with a churn risk of 91 percent. Upstream's own runtime returns billing and 19 percent for that input at the published revision, and so does feelsneat. I treat the README numbers as illustrative.

The daemon speaks Jev's `POST /v1/systemone`. TypeSafe's official Python SDK worked unchanged once I pointed `TYPESAFE_BASE_URL` at `http://127.0.0.1:3212`. Code written against Jev can therefore move between the hosted model and a local one with an environment variable.

##### Try it

feelsneat needs macOS on Apple Silicon and a Rust toolchain. The first build compiles MLX and takes several minutes. Setup downloads the pinned checkpoints and verifies their checksums:

```sh
cargo install --locked --git https://gitlab.com/parlant-co/feelsneat.git
feelsneat --setup --model english,multilingual
feelsneat serve
```

The [feelsneat repository](https://gitlab.com/parlant-co/feelsneat) documents use cases, integrations for pi, oMLX, Claude Code, and the Jev SDKs, and the full parity and performance methods. If you run agents from a terminal, [my tmux setup for parallel coding agents](https://www.lukasrieder.com/blog/how-i-supervise-parallel-coding-agents-from-tmux/) shows where a local judgment call fits into that workflow. The model is Nandakishor Mukkunnoth's work, and TypeSafe's documentation gave me the vocabulary and most of the patterns. feelsneat makes both useful on the machine in front of me.

### [Freelance or GmbH? I built a lifetime model before answering](https://www.lukasrieder.com/blog/freelance-or-gmbh/)

My Holding was supposed to create room for the future. The first complete comparison put a simpler Direct GmbH ahead of it.

I spent roughly fifteen years working as an independent software engineer before I founded Parlant GmbH in 2022. People now ask when they should make the same move. They usually expect a revenue threshold or a tax rate.

I wrote this article for [Techtelmechtel Talks #1](https://luma.com/z702649c), a thirty-minute account of working on my own terms followed by an open Q&A. The [talk page includes the slides and speaking notes](https://www.lukasrieder.com/talks/freelance-or-gmbh/). The event title asks "Freelance or GmbH?" I cannot answer it with one number. My own model does not give me one either.

The model starts with the same work and the same economic profit, then routes the money through an Einzelunternehmen, a qualifying Freiberufler activity, a directly owned GmbH, or a Holding with an operative GmbH. Under its current default assumptions, the Direct GmbH finishes ahead of my Holding. That result is inconvenient. It is also why I trust the model more than I would trust a calculation built to approve my decision.

The tax calculation sits beside liability, administration, access to private cash, business runway, mobility, and the possibility that I will want a different life in ten years. A legal form connects all of them. It is an interface between work, risk, consumption, saving, and the future.

##### The word freelance hides the first distinction

The event title uses "freelance" in its ordinary English sense: a person who works independently. German tax law uses a narrower category, `Freiberufler`. A software consultant may work independently without qualifying as a Freiberufler. The classification depends on the actual activity and facts, not the label on a website.

My model therefore keeps two personal-business cases apart. The Freiberufler path pays Einkommensteuer but no Gewerbesteuer. The Einzelunternehmen path includes Gewerbesteuer, its €24,500 allowance, and the §35 EStG credit against personal income tax. At a Hebesatz near 400 percent, that credit can offset much of the additional burden. It does not reduce the solidarity-surcharge assessment base under §3(2) SolzG. The two paths can be closer than the slogans suggest, but equal assumptions should not make the trade-tax path outperform the Freiberufler path merely through a lower solidarity surcharge.

A GmbH changes the flow. The company deducts a gross managing-director salary, pays corporation tax and Gewerbesteuer on the remaining profit, and can retain the rest. Salary becomes taxable personal income. Money retained by the company is not mine to spend privately.

A Holding adds a second company. The operative GmbH earns the money and can distribute after-tax profit to the Holding. The participation rules can make that transfer lightly taxed when the ownership thresholds apply, but not free. The Holding also brings another annual account, another tax return, and another set of formal duties.

None of these structures changes the value created for a client. They change who receives the cash, when tax becomes due, and what must happen before I can use it.

##### My first comparison was unfair

Most online comparisons begin with mismatched numbers. One column shows the taxable profit of a sole proprietor. Another shows GmbH revenue after salary. A third quietly assumes that money inside a company and money in a private account are interchangeable.

I set one rule for the model: every structure starts with the same economic business profit before owner remuneration and taxes.

The model then applies the choices that differ by structure:

- A sole proprietor cannot deduct a salary paid to themselves.
- A GmbH treats managing-director salary as a company cost and personal taxable income.
- Private spending means after-tax consumption, not gross salary.
- Administration reduces the money available for tax, spending, or investment.
- Corporate assets remain corporate until a distribution makes them private.

The distinction between gross salary and private spending caught one of my own mistakes. A €100,000 salary does not fund €100,000 of private consumption. Income tax comes out first. If the model asks for €100,000 of after-tax spending, the company must fund the gap with a dividend or the private portfolio must absorb it.

That sounds obvious when written down. It was easy to blur inside a large spreadsheet. Several of the model's most useful corrections came from outputs that looked wrong and forced me back to the cash flow.

##### A lifetime model needs a memory

The first workbook compared four pure careers. It assumed that a person chose one structure at the start of working life and kept it until retirement. That makes a controlled comparison, but it does not describe me.

I worked personally for years, then incorporated. Wealth accumulated in one wrapper does not jump tax-free into another because I change a dropdown. A private ETF remains private. Assets retained by a Direct GmbH remain in that company. Holding assets remain in the Holding. A real restructuring can create tax, notary, legal, and adviser costs.

I added a yearly career path so the model could preserve those separate pools. Then another problem appeared. The hypothetical history said I should have accumulated far more than I had.

A smooth model can turn an uneven career into flattering fiction. It assumes the same profit and savings discipline in every year. Real life includes weak markets, changed priorities, unplanned spending, and years when cash had another job.

The model now supports calibration to known opening balances. Calibration throws away the imaginary historical wealth and starts the forward calculation from the assets that exist. It is not a cosmetic setting. It separates a clean counterfactual from an honest personal plan.

##### Tax timing changed the result

The early model taxed the full investment return every year. That was wrong for the fully accumulating Aktienfonds in the scenario.

The current engine separates nominal appreciation, inflation, Vorabpauschale, prior taxed basis, realized gains, and ETF sales. The 2026 Basiszins is a visible planning assumption rather than a prediction. Prior Vorabpauschalen increase the modeled tax basis so a later sale does not tax the same gain twice.

The rules also differ by wrapper. Section 20 InvStG exempts 30 percent of qualifying Aktienfonds income for a private investor and 80 percent for corporation-tax purposes. Only half of the corporate exemption applies when calculating Gewerbesteuer. The model applies those layers separately.

Tax timing matters because capital that remains invested can compound. A correct tax percentage applied in the wrong year still produces a misleading lifetime result.

Retirement required another engine. Gross company wealth is not privately consumable retirement wealth. A company portfolio must cover ETF tax, company administration, and the gross dividend. The shareholder may then owe tax on that dividend.

The model solves for a constant annual real withdrawal from retirement age to age 100. It tracks market value and tax basis through each sale. Its useful output is privately spendable real money, not the largest balance printed in a depot.

##### The default result does not crown the Holding

The current frozen baseline uses these assumptions:

- Work starts at age 20 and retirement starts at 67.
- Annual economic business profit is €150,000 in real 2026 euros.
- Annual private spending is €60,000 after tax.
- Gross managing-director salary is €100,000.
- Nominal portfolio return is 6 percent and inflation is 2 percent.
- The Gewerbesteuer Hebesatz is 410 percent.
- Annual administration is €1,500 for both personal-business paths, €4,000 for a Direct GmbH, and €6,000 across the Holding and OpCo.
- Social-insurance rates for the managing-director salary are set to zero. That is a scenario input, not a general statement about managing directors.

Under model version `2026.09-payout-tax.1`, with the optional EXIT disabled, the four pure careers produce these rounded results at retirement:

| Structure | Gross wealth | Private-equivalent wealth | Sustainable real spending per year |
|---|---:|---:|---:|
| Einzelunternehmen | €3.941 million | €3.941 million | €181,540 |
| Qualifying Freiberufler | €3.992 million | €3.992 million | €183,856 |
| Direct GmbH | €4.902 million | €4.084 million | €196,160 |
| Holding and OpCo | €4.603 million | €3.866 million | €186,837 |

These are outputs from one frozen scenario, not forecasts or recommendations. The pure careers also retain every modeled saving from age 20. They must not be compared with a calibrated personal path as if both had the same history.

The Direct GmbH leads this baseline. The Holding carries another €2,000 of annual administration and a small tax leakage when profit moves from the OpCo. At €150,000 of economic profit and a €100,000 gross salary, the surplus available for corporate investment is not enormous. The second company consumes a meaningful part of it.

That does not prove my Holding was a mistake. It does prove that "the Holding always wins" is false.

##### Why I still chose a Holding

I did not create the Holding for one tax curve.

I wanted accumulated capital outside the company that signs consulting and product contracts. I wanted one place for participations and future operating companies. I wanted the option, not the promise, of different treatment if an operative company became genuinely saleable. I also wanted a hard boundary between private consumption, operating runway, and long-term capital.

That last point is personal. Money inside a Holding is less convenient to spend. Reduced access can be a defect. For me, it can also enforce a decision I made while thinking clearly.

I pay for that choice. Two entities still cost money in mediocre years. I depend on advisers and accurate records. The structure makes a move abroad, succession, and eventual wind-down more complicated. Tax law can change before the lifetime projection reaches its final column. A consulting company can remain tied to me and never attract a buyer.

A Holding can improve the tax treatment of a qualifying sale. It cannot create transferable intellectual property, recurring revenue, a team that works without the founder, or a buyer. Company design and sale taxation are different problems.

##### The threshold is retained surplus, not revenue

I have used revenue thresholds as shorthand in the past. "Above €120,000, consider a GmbH" sounds useful. It is also too imprecise to defend.

A business with €300,000 in revenue can retain nothing. Another business with lower revenue and high margins can keep substantial capital invested. Private needs can absorb a large salary. Annual company costs can erase a narrow tax advantage. A short holding period leaves little time for deferral to compound.

The better variable is annual investable surplus:

> Economic profit minus operating costs, gross owner remuneration, company taxes, all entity administration, and required business runway.

Then ask how long the surplus can stay inside the company and what it is meant to do. Capital needed privately next year has a short deferral runway. Capital intended for products, participations, or long-term investment has a different job.

This is also why a bad year belongs in the decision. The structure that wins on a smooth thirty-year curve may be the one that feels worst after a client disappears. Fixed company costs survive a revenue gap. Salary may need to fall. Private spending still needs funding. A decision should survive the stress case, not only the average.

##### What the model still cannot decide

The workbook and [interactive calculator](https://www.lukasrieder.com/freelance-or-company/) expose assumptions, but they do not turn a personal choice into a mechanical one.

The current model excludes §34a EStG retained-profit relief. That omission can bias a retain-and-invest comparison against eligible personal businesses. It does not establish Freiberufler status. It does not calculate exact health insurance, social-insurance status, church tax, spouse splitting, children, statutory pension tax, or loss carryforwards.

It also excludes formation and restructuring transactions, a full company sale, exit tax on moving abroad, succession, and exact liquidation timing. The winding-down amounts are planning allowances, not quotes. Exact ETF tax lots and company accounts must be reconciled with broker and accounting records.

A Steuerberater remains necessary. Open formulas and tests improve the conversation with an adviser. They do not replace the adviser.

The right use of the model is to discover which assumptions control the choice. If €2,000 of annual cost reverses the result, get an actual quote. If the result depends on retaining €80,000 each year, test three weak years. If moving abroad matters, price that option before company shares make it harder.

##### What self-employment taught me about trust

My career started before I had enough evidence that it would work. I learned software by reading, experimenting, shipping production systems, and working with people who knew things I did not. Each commitment arrived before certainty.

That is not an argument that everyone should become self-employed. Employment can provide income, colleagues, structure, and relief from risks a person does not want to carry. Choosing it can be clear judgment rather than fear.

Self-employment asks for a particular kind of trust. I have to make decisions with incomplete information, accept the outcome, and revise without pretending that hindsight was available at the start. Repeating that process builds confidence because confidence stops depending on certainty.

The company did not complete that process. It added obligations: accounts, payroll, resolutions, deadlines, and money I cannot treat as private. I chose those constraints because separation and runway can let me say no, keep a promise, or fund work beyond the next invoice.

A legal form is not an identity. It is not proof that someone has become a serious entrepreneur. It is a structure chosen under assumptions. Those assumptions deserve the same scrutiny I would apply to a production system.

##### Run your own numbers

I built [the Freelance or company? calculator](https://www.lukasrieder.com/freelance-or-company/#calculator-title) so that the conclusion does not have to be mine. Change the profit, private spending, salary, fees, return, inflation, and retirement horizon. Watch where the paths separate. Look for the small change that reverses the ranking.

Do not copy my Holding or somebody else's threshold. Run one honest scenario, including a bad year, then take the sensitive assumptions to a German tax adviser.

Self-employment does not remove uncertainty. It gives you practice making responsible choices before uncertainty disappears. Trust yourself enough to inspect the choice, and enough to change it when the evidence changes.

##### Sources and model version

The calculator and figures use model version `2026.09-payout-tax.1`, ported from the supplied workbook with one documented correction. The workbook reduced the solidarity-surcharge base by the §35 EStG trade-tax credit. The web model follows §3(2) SolzG and calculates the surcharge before that credit. It also contains a disabled-by-default fictional EXIT for comparing direct and Holding ownership of an eligible share sale. The public [Google Sheet](https://docs.google.com/spreadsheets/d/1j9ArcJJoSrgco7WT1BpPCjilBwOzd8jY5hZYhfKGZ2M/edit?gid=1450622860#gid=1450622860) remains available for inspecting the full selected-career model.

The material statutory inputs come from primary sources: [§32a EStG](https://www.gesetze-im-internet.de/estg/__32a.html) for the 2026 income-tax tariff, [§35 EStG](https://www.gesetze-im-internet.de/estg/__35.html) for the Gewerbesteuer credit, [§3 SolzG](https://www.gesetze-im-internet.de/solzg_1995/__3.html) for the solidarity-surcharge assessment base, [§11 GewStG](https://www.gesetze-im-internet.de/gewstg/__11.html) for the allowance and assessment rate, and [§23 KStG](https://www.gesetze-im-internet.de/kstg_1977/__23.html) for the enacted corporate-tax schedule. The illustrative direct EXIT refers to [§17 EStG](https://www.gesetze-im-internet.de/estg/__17.html). Holding participation treatment comes from [§8b KStG](https://www.gesetze-im-internet.de/kstg_1977/__8b.html) and [§9 GewStG](https://www.gesetze-im-internet.de/gewstg/__9.html). ETF treatment follows [§§18 to 20 InvStG](https://www.gesetze-im-internet.de/invstg_2018/__18.html) and the [BMF notice setting the 2026 Basiszins](https://www.bundesfinanzministerium.de/Content/DE/Downloads/BMF_Schreiben/Steuerarten/Investmentsteuer/2026-01-13-basiszins-berechnung-vorabpauschale.pdf).

### [We had the images, but not permission to use them](https://www.lukasrieder.com/blog/we-had-the-images-but-not-permission-to-use-them/)

Our first Blickwinkel content review produced Instagram drafts and no record of where their images came from.

I am working on [Blickwinkel](https://www.blickwinkel.app) and its Instagram account. During our first content review, our junior business communications manager showed me a set of possible posts. She had selected an image for each post and arranged them as a set. I liked her choices.

Then I asked, "Can we safely use the image material?"

The conversation went quiet. We had the images, but not their original sources, license terms, creator names, or model releases. We could see and save each image. Blickwinkel still lacked permission to publish it.

At first I wondered whether age explained the gap. She grew up with the "web of platforms," including Instagram, Pinterest, and Cosmos. I reached for the Generation Z label, although I find those categories rather strange.

One conversation cannot tell me what a generation believes. It showed that our review process had never required an answer about image rights. These platforms let a person save an image in seconds. Screenshots and downloaded copies often omit the source URL, the creator's name, and the license.

I liked one part of her approach. She expected people to share creative work and make something new from it. That expectation reminded me of learning Photoshop and programming online.

##### The internet I remember

I spent a lot of time in forums. Some had Photoshop battle threads. One person posted an image, and everyone else transformed it. We cut out subjects, changed backgrounds, exaggerated each other's edits, and passed the result around again.

When I needed information, somebody had usually written a tutorial on a personal site. When I needed inspiration, a designer had posted experiments on a blog. People shared snippets, brushes, textures, source files, and unfinished ideas because another person might find them useful.

People gave useful material to strangers. We were also legally naive. Within a Photoshop battle, posting an image meant that other participants would edit it. That invitation rarely came with a license. A link to the author might survive the next repost, or it might disappear. Some files we treated as common material were almost certainly not ours to use.

I would still work that way if the permission were explicit.

Once I began publishing for clients and my own companies, every external image needed a clear source and permission. Liking an image does not pay its photographer. Finding a file does not grant a license. Crediting a creator does not cure a use the creator never permitted. When Blickwinkel publishes an image to support its business, we need permission before the post goes live.

I want people to take the same care with my work. Much of my open-source software, including [sayneat](https://gitlab.com/parlant-co/sayneat) and [KMS](https://gitlab.com/parlant-co/kms), uses the MIT License. It permits use, modification, distribution, sublicensing, and sale. Copies or substantial portions must retain the copyright and permission notice.

The MIT License states the permission and the condition in a few paragraphs. Users do not need to infer what I might tolerate. The required notice preserves the author's name and the license terms in copies of the software.

For images, I now look for the same two facts. The license must permit the intended use, and it must state which notice to retain.

##### Discovery is not permission

Pinterest and the main Cosmos product help us show one another the kinds of images we mean. They do not prove that Blickwinkel may publish an image. A pin, saved element, screenshot, or repost may not identify the copyright holder. It also says nothing about permission from the people shown in an advertisement.

[Public Work by Cosmos](https://www.cosmos.so/public-work) searches material that Cosmos believes is in the public domain. Its [terms say that Cosmos does not warrant that every work is in the public domain](https://www.cosmos.so/pw-terms-of-service). Users must check the work themselves. I follow each result to the museum or library record and save that institution's rights statement.

Stock libraries publish reuse terms. That addresses the photographer's copyright, but those terms may not cover every right visible inside the photograph.

Copyright in the photograph is one layer. A commercial post may also involve a recognizable person's privacy or publicity rights, a trademark on clothing, copyright in a painting, or rules for photographing a private interior. The exact rules depend on the jurisdiction and the proposed use. A source can grant only the rights it owns or has secured.

Unsplash documents this distinction on two separate pages. Its standard images may be used for most commercial and editorial projects without permission or credit under the [Unsplash license](https://help.unsplash.com/en/articles/2612315-can-i-use-unsplash-images-for-personal-or-commercial-projects). However, Unsplash [does not guarantee the scope of model releases, property releases, trademark rights, or rights in depicted objects](https://help.unsplash.com/en/articles/2612329-releases-and-trademarks).

Unsplash states that [Unsplash+](https://help.unsplash.com/en/articles/6536959-how-can-i-use-unsplash-images) images are model and property released when they depict people or property. They are also royalty-free and carry additional legal protection. For Blickwinkel, Plus is the better default when a recognizable person or private place is the main subject.

##### A sourcing policy for Blickwinkel

We will not ask the team to compare thirty license pages for every post. I reduced the research to this order of preference.

| Priority | Sources | How we use them |
| --- | --- | --- |
| 1 | [Unsplash](https://unsplash.com/license) and Unsplash+ | We use standard images for generic scenes without a prominent person, brand, artwork, or private property. We prefer Plus when a recognizable person or property is central. |
| 2 | [Dupe](https://dupephotos.com/license-and-terms) | We use it when its contemporary phone photography matches the draft better than conventional stock. Its current license allows commercial use and adaptation without attribution, but not a competing image service. |
| 3 | [Pexels](https://www.pexels.com/license/) | We use it for lifestyle photography and video. Its license permits free commercial use and editing without required attribution, subject to its restrictions. |
| 4 | [Kaboompics](https://kaboompics.com/page/terms-of-use) | We use it for interiors, fashion, and composed lifestyle scenes under its commercial-use license. We still inspect brands, people, art, and property. |
| 5 | [StockSnap](https://stocksnap.io/license), [ISO Republic](https://isorepublic.com/license/), and [NegativeSpace](https://negativespace.co/license/) | These sites label their collections CC0, which permits commercial reuse without credit. Privacy, publicity, trademark, and other rights can remain. |
| 6 | [Wikimedia Commons](https://commons.wikimedia.org/wiki/Commons:Reusing_content_outside_Wikimedia/licenses/en) and public cultural archives | We use these for historical material, art, type, and collage. We check every object's rights statement because terms vary by file and institution. |
| 7 | [Pixabay](https://pixabay.com/service/license-summary/) | We use it as a broad fallback for photographs, illustrations, and other media under the Pixabay Content License. We inspect its prohibited uses and anything recognizable in the asset. |

We prefer a source whose license and release information we can save with the asset, even if another source imposes fewer conditions.

For a forest, an empty room, a texture, or an abstract object, a standard stock license will often answer the rights question. For a portrait, designer chair, Nike logo, painting, private apartment, or celebrity, we check the additional rights. If we cannot find that evidence, we replace the image before the team builds the rest of the campaign around it.

##### The longer source directory

The broader search produced more candidates. I grouped them by the evidence we need to retain. "No attribution required" describes the photographer's stated license. It does not erase other rights or the source's restrictions.

| Group | Sources | What to check |
| --- | --- | --- |
| Broad stock libraries | [Burst by Shopify](https://www.shopify.com/stock-photos/licenses/shopify-some-rights-reserved), [Gratisography](https://gratisography.com/license/), [picjumbo](https://picjumbo.com/faq-and-terms), and [Barnimages](https://barnimages.com/about/) | Each source permits commercial reuse without required credit under its own terms. The licenses restrict resale or redistribution of unchanged files, and some sources warn that releases may be absent. |
| CC0 libraries | [Skitterphoto](https://skitterphoto.com/license), [Picography](https://picography.co/terms/), [ShotStash](https://shotstash.com/license/), and [LibreShot](https://libreshot.com/license-information/) | The sites describe their images as CC0 or public-domain material. CC0 does not waive another person's privacy, publicity, patent, or trademark rights. |
| Representation and specialist material | [Nappy](https://nappy.co/license/), [Foodiesfeed](https://www.foodiesfeed.com/license), and [Startup Stock Photos](https://startupstockphotos.com/faq/) | Nappy focuses on Black and Brown people and prohibits degrading uses. Foodiesfeed includes both photographs and AI-generated images. Startup Stock Photos says attribution is optional, but its FAQ does not supply formal model releases. |
| Public-domain institutions | [Smithsonian Open Access](https://www.si.edu/openaccess), [New York Public Library Public Domain Collections](https://www.nypl.org/research/resources/public-domain-collections), and [Library of Congress Free to Use and Reuse](https://www.loc.gov/free-to-use/) | We stay within material marked CC0, public domain, or free to use. We preserve the institution's object page and rights statement. |
| Rights-statement aggregators | [Europeana](https://pro.europeana.eu/page/available-rights-statements), [rawpixel](https://www.rawpixel.com/services/licenses), and [Cosmos Public Work](https://www.cosmos.so/public-work) | Rights vary by object, collection, or account tier. We follow the result to its underlying record rather than treating the search page as the license. |
| Large mixed archive | [Morguefile](https://morguefile.com/) | We verify the license attached to the individual download and do not assume one search result has the same terms as another. |

I left Freerange Stock and Freepik out of the working list. During this review, Freerange's license path from the original research no longer resolved. Freepik's terms and attribution duties depended on the product and account tier. I could not turn either source into a rule that the team could apply consistently.

I checked this directory on 13 September 2026. Sites can change their licenses. We will read the current terms again when we download an asset rather than treating this article as permanent permission.

##### Record the source and license

After downloading an image, we save its source and license record. Six months later, the post should still have an answer to "Where did this image come from?"

For every external asset, we record:

- The original asset page and the downloaded filename.
- The creator's displayed name and profile URL, when available.
- The license name, version, and license URL.
- The date on which we downloaded the asset and checked the terms.
- Any statement about a model or property release.
- The edits we made and the channels on which we published the result.

When a license requires attribution, we include it in the post and follow the specified format. For Creative Commons material, the [recommended attribution practice](https://wiki.creativecommons.org/wiki/Recommended_practices_for_attribution) calls for the title, author, source, and license, plus an indication of changes. When credit is optional and the format permits it, I name the photographer. Optional credit does not replace legal compliance. It tells readers who made the photograph and where to find more of their work.

If we cannot reconstruct that record, we do not publish the asset. We can license another photograph, ask the creator, make our own, or change the idea. Once several people have edited and scheduled a post, memory is not sufficient evidence of permission.

The source record keeps the photographer's name with a reusable photograph. For a public-domain scan, it keeps the museum's object page with the image. In an open-source release, the license keeps the author's copyright notice in copies of the code.

The next time we review a Blickwinkel post, the image should arrive with its source URL, creator, license, and release notes. Then "Can we safely use this?" should not end the conversation. It should take less than a minute to answer.

### [Giving back to the community](https://www.lukasrieder.com/blog/giving-back-to-the-community/)

Communities turned a solitary interest into a shared practice. I am giving back with a camera and a hobby I take seriously.

I want to give something back to communities that taught me how much easier it is to learn with other people than alone. Video is a hobby I enjoy, and I would like to use it for more than recording my own talks.

For community events that interest me, I am offering a day of filming and another day of editing at no charge. I choose projects according to my available time and contribute the work independently.

{% include post-image.html src="/assets/images/posts/giving-back-to-the-community/community-camera.jpeg" alt="Lukas Rieder smiling while holding a caged camera rig at Serious Games for Serious Change in Berlin." width="1536" height="1536" eager=true caption="Covering Serious Games for Serious Change in Berlin, September 2026. Photo: Michael Martens." %}

##### What I received

I taught myself programming, but I did not learn how to work with other people alone. User groups gave me a place to bring unfinished ideas, hear how experienced people approached the same problems, and discover that failure was worth discussing.

At a small software company, experienced developers built an early startup with me through pair programming. Immediate feedback replaced slow trial and error. Later, speaking at companies, user groups, and conferences taught me to explain an idea clearly and let other people question it. My [talk archive](https://www.lukasrieder.com/talks/) grew out of that exchange.

The important part was not a list of appearances. Programming stopped being a solitary interest and became a shared practice.

##### Giving back with a different skill

I bought my first mirrorless camera in 2010, took the inevitable mirror selfie, and then let the interest lie dormant for years.

{% include post-image.html src="/assets/images/posts/giving-back-to-the-community/first-mirrorless-2010.jpeg" alt="A young Lukas Rieder taking a mirror selfie while holding his first Panasonic Lumix camera." width="1084" height="724" caption="My first mirrorless camera in 2010, when both the camera and my hair were new." %}

I returned to cameras in 2025 because I wanted a permanent record of my talks. A user group let me record an event, and I enjoyed the work enough to continue after my own talk had ended.

The first challenge was making reliable video in rooms that were never designed as studios. Lighting is uneven, sound systems vary, and an audience should not wait while I construct a film set. I gradually built a portable camera, lighting, and live-mixing setup that lets me adapt without making the production the center of the event.

Editing posed a different problem. A full talk needs clear sound and an unobtrusive cut. A recap needs to reduce hours of unscripted activity into a few minutes without misrepresenting the day. Learning when to preserve the complete record and when to compress it is now one of the parts I enjoy most.

That distinction guides my recent work: a [recorded and streamed user-group evening](https://www.youtube.com/watch?v=lWI99wHyGbc), a full [Open Space](https://luma.com/1myenpab) shaped into a three-minute recap, and three complete talks prepared for a new community's [YouTube channel](https://www.youtube.com/@berlinrb_org).

##### What a lasting record changes

Most community events are temporary. The people in the room benefit, then the room empties and much of the work disappears with it. A thoughtful record lets an idea travel, helps someone decide whether to attend next time, and gives a community a memory of its own development.

That is useful beyond programming and beyond Berlin. Any group built through voluntary effort faces the same question: how can a member contribute without taking on every part of organizing? Giving back does not have to resemble what you received. The useful contribution may be the skill you enjoy practising now.

##### What I am offering

Depending on the event, I can produce:

- A two-to-three-minute recap that preserves the atmosphere.
- Complete talk recordings for people who could not attend.
- Short interviews with members of the community.
- A live stream when the venue and format support one.

I have bounded the contribution so it remains sustainable and enjoyable: one event day and one editing day. I ask what matters, then handle filming and editing as autonomously as possible. I am based in Berlin and happy to consider invitations elsewhere.

This is not another agency service. It is a hobby I take seriously and genuinely enjoy. I want to offer it to communities that would otherwise finish a valuable event with no lasting record.

##### Keeping it possible

I do not charge organizers for these days, but the work still consumes time and equipment. My [Patreon page](https://patreon.com/LukasRieder) lets anyone help me cover more events in my spare time.

That support maintains the equipment and makes room for another free editing day. It lets me keep contributing without turning every invitation into client work.

### [My terminal learned my voice from one recording](https://www.lukasrieder.com/blog/my-terminal-learned-my-voice-from-one-recording/)

I gave sayneat one recording. The next terminal message came back in my own voice, generated locally without Python, a GPU, or an account.

This is now a valid terminal command:

```sh
sayneat --model pocket --clone recording.mp3 --voice my-voice
```

After it finishes, the new voice behaves like any other voice in sayneat:

```sh
sayneat --model pocket --voice my-voice \
  "The stubborn build finally passed"
```

That is almost absurdly little interface for what happened. A recording became a reusable voice. No training job appeared. There was no Python environment, GPU service, model server, web form, or account. The 98 MB local model and the recording stayed on the machine.

This is the third step in a habit that began with a shell suffix. In March, I wrote [I gave my terminal a voice worth listening to](https://www.lukasrieder.com/blog/i-gave-my-terminal-a-voice-worth-listening-to/) about `saynice`, a Bash wrapper around OpenAI speech. Last week, [Why my terminal now sounds like a person](https://www.lukasrieder.com/blog/i-replaced-macos-say-with-local-neural-speech/) introduced sayneat and compared macOS `say`, `saynice`, and four local models.

The next request was obvious: if a terminal can use a good local voice, why not let it use mine?

`--clone` was the missing toy. With no recording file, sayneat gives me a short passage, waits for Enter, and records the default microphone. Hearing it speak a sentence I never recorded, in something recognizably close to my own voice, is strange and delightful. I wanted that experience to take one command, then leave me with a voice I could use anywhere else in the CLI.

##### Cloning here means conditioning, not training

"Voice cloning" suggests that sayneat trains and saves a new model. It does not.

Pocket TTS is a zero-shot text-to-speech model. During synthesis, it derives speaker and style information from a reference recording and conditions generated speech on that information. The model weights remain unchanged. sayneat stores a prepared reference WAV and supplies it again when Pocket starts.

That distinction explains both the speed and one current limitation. Creating a clone mostly means decoding and cleaning audio, so it finishes quickly. On each fresh sayneat process, Pocket still has to derive its voice state from the reference. A longer recording can capture more of the speaker, but it also makes that first synthesis slower.

The saved object is a clean voice recording selected specifically because a model can reproduce its speaker. Keeping that reference as an ordinary WAV makes the mechanism easy to inspect, copy, and delete.

Pocket is unusually suitable for a terminal tool. [Kyutai's current Pocket TTS project](https://github.com/kyutai-labs/pocket-tts) describes a 100 million parameter model designed for CPU use, with low-latency streaming and voice cloning from a WAV reference. sayneat uses a January 2026 int8 ONNX conversion through sherpa-onnx. That package is about 98 MB and runs locally without a Python runtime.

This is what makes the short command possible. The simplicity does not come from a thin wrapper around a service. The model, audio decoder, microphone capture, sample preparation, synthesis, playback, and file output live behind one native executable.

##### Open source voice cloning is no longer one project

As of August 2026, there is no shortage of capable voice-cloning code. Comparing the projects is fun because they make noticeably different choices about size, language coverage, style control, and licensing.

[Coqui XTTS v2](https://huggingface.co/coqui/XTTS-v2) helped make short-reference multilingual cloning accessible. Its model card says it can clone from a six-second clip and speak 17 languages. The Coqui TTS code uses MPL-2.0, while XTTS weights use the separate Coqui Public Model License. Reading the repository license alone does not answer whether a product may use the model.

[OpenVoice](https://github.com/myshell-ai/OpenVoice) takes a different position. Its V1 and V2 code and models are MIT-licensed, including commercial use, and it focuses on tone-color cloning, style control, and cross-lingual output.

[F5-TTS](https://github.com/SWivid/F5-TTS) publishes its code under MIT. Its pretrained weights use CC BY-NC because the Emilia training data is non-commercial. Again, open code does not make the ready-to-run model commercially usable.

[Chatterbox](https://github.com/resemble-ai/chatterbox) is MIT-licensed and spans English, multilingual, low-latency, and smaller CPU-oriented variants. It also makes a notable product decision: generated audio includes Resemble AI's PerTh neural watermark, and the repository includes detection instructions.

Pocket brings the size down far enough for the command I wanted. Its repository code uses MIT and explicitly prohibits unauthorized, deceptive, fraudulent, and privacy-invasive cloning. The specific [sherpa-onnx conversion package](https://huggingface.co/csukuangfj2/sherpa-onnx-pocket-tts-int8-2026-01-26) bundled by sayneat contains CC BY 4.0 terms, while its package README also says the conversion is non-commercial. Those statements do not fit together neatly. sayneat therefore describes this specific Pocket package as non-commercial unless upstream clarifies its status.

That survey changed how I think about an OSS voice model. At least four separate questions matter:

- Is the inference code available under an open-source license?
- Are the model weights available, and under which terms?
- Do the training data impose additional restrictions on those weights?
- Do I have permission to use the reference speaker's identity for this purpose?

The last question does not disappear when every repository says MIT. A software license cannot grant another person's consent.

##### A cloned voice changes what a recording proves

Voice cloning has wonderful uses. Someone who is losing speech can preserve a familiar voice. A creator can correct narration without reconstructing a recording setup. An accessibility tool can speak in a voice its user identifies as their own. My terminal can report a finished task without sounding like a generic system voice.

The same mechanism can produce convincing impersonation. The [US Federal Trade Commission's Voice Cloning Challenge](https://www.ftc.gov/news-events/news/press-releases/2024/04/ftc-announces-winners-voice-cloning-challenge) explored synthetic-speech detection, source-audio protection, authentication at recording time, and real-time deepfake detection. Its useful conclusion was that no single detector solves the problem.

A familiar voice is therefore no longer proof of who made a call or recording. That fact exists independently of sayneat, and it is worth remembering as these models become ordinary software.

The command keeps its own message short:

```text
Clone only a voice you own or have explicit permission to use.
```

That matches how I want to play with the feature. I can clone my own voice, compare different rooms and microphones, and hear how much of an accent or speaking style Pocket picks up from thirty seconds.

##### Published clone audio should say what it is

The EU AI Act's Article 50 transparency rules started to apply on August 2, 2026. The [European Commission's Article 50 guidance](https://digital-strategy.ec.europa.eu/en/faqs/transparency-obligations-under-article-50-ai-act) covers machine-readable marking of synthetic output and disclosure of deepfake audio where the provision applies.

sayneat does not watermark generated audio. Pocket through sherpa-onnx does not provide the PerTh mechanism that Chatterbox ships, and ordinary conversion can remove simple file metadata. For my private terminal notifications, that distinction is academic. If I turn a clone into published audio, I will say that it is synthetic.

##### The one-line command cleans up the recording

Raw recordings vary more than the model API suggests. A stereo phone recording may start with silence, contain a loud click, run for several minutes, or have one clear section surrounded by room noise. Sending the first fixed number of seconds would make the interface simple and the results arbitrary.

sayneat decodes the source and downmixes all channels to mono. It bounds analysis to the first 45 seconds for Pocket and 180 seconds for ElevenLabs, which prevents a large podcast file from becoming an unbounded allocation.

It then measures RMS energy in 20 millisecond frames. Frames above a threshold derived from the loudest part count as active speech. sayneat trims leading and trailing silence while keeping 150 milliseconds of margin. If the active recording remains too long, a sliding energy calculation selects the clearest high-energy window rather than blindly taking the beginning.

Pocket keeps at most 30 seconds. ElevenLabs keeps at most 120 seconds. A Pocket reference needs at least 0.75 seconds of clear audio. ElevenLabs needs ten seconds, and its own guidance favors one to two minutes.

Normalization raises the peak toward 0.9 but caps gain at four times. That limit is deliberate. Turning a nearly silent, noisy recording up by twenty times would satisfy a level check while making a worse reference. Clipping, echo, background speakers, and aggressive noise remain the user's responsibility because simple amplitude code cannot repair them honestly.

Finally, sayneat resamples the prepared audio to 24 kHz and writes 16-bit mono PCM. Pocket and ElevenLabs receive the same predictable sample shape even when the input was an MP3 from a phone or a microphone using another rate.

##### A clone becomes an ordinary voice

A supplied Pocket name may contain one to 64 ASCII letters, digits, hyphens, or underscores. If no name is given, sayneat creates a non-identifying pet name such as `mossy-otter`. That avoids pretending an unverified sample establishes the identity written on its filename.

Pocket references live below the platform application-data directory, separate from downloadable model weights. Cache cleanup can remove a hydrated model without deleting a user's voice. On Unix, sayneat restricts the voice directory to mode `0700`.

Saving uses a temporary file in that directory, flushes it, and persists without replacing an existing name. If the name exists in an interactive terminal, sayneat asks for a bare Enter before replacing it atomically. A noninteractive process refuses replacement. This protects a personal recording from an unnoticed script collision.

Saved names appear through the existing voice interface:

```sh
sayneat --model pocket --voices
```

A name or numeric position can then be passed through `--voice`. The rest of sayneat does not need a second concept for clones.

The same preparation code can upload a longer sample for ElevenLabs Instant Voice Cloning when I want its remote voices. Pocket is the part I am most excited about because the complete loop stays on my laptop and needs no account.

##### My terminal can sound like me

The original goal remains modest. I want a deeply buried terminal to speak when work finishes. `saynice` made that voice pleasant. sayneat made it local. `--clone` now makes it personal.

```sh
some-long-running-task; \
  sayneat --model pocket --voice my-voice \
  "task is done, exited $?"
```

The first version of sayneat let me choose a much better voice. This release lets me make one. That changes the project from a catalog of speech models into something personal, and I have already lost time happily trying different microphones, passages, and reference lengths to hear what Pocket preserves.

I built `--clone` because it sounded like a fun feature and because voice cloning should not require assembling a Python environment before I can play with it. Now it is one flag in the same command I already use every day. Record a voice, give it a name, and let the terminal surprise you.

### [How I supervise parallel coding agents from tmux](https://www.lukasrieder.com/blog/how-i-supervise-parallel-coding-agents-from-tmux/)

Sounds and speech bring me back to the right terminal. Tests, diffs, and worktrees keep several coding sessions useful. The limit is no longer typing speed. It is review.

In early October 2025, I opened [hibase.co](https://hibase.co), an older Ruby on Rails application that we still operate, and handed Claude a bug that crossed the backend and frontend. I expected the usual result. Earlier coding agents could explain an isolated file or produce a useful snippet, then lose their footing among the awkward connections that accumulate in a long-lived application. I would take over once the real investigation began.

Claude kept digging. It traced the behavior through the Rails code, followed the corresponding frontend path, found the faulty assumption, and implemented a substantial fix. I still read every changed line, but for the first time I was reviewing a coherent solution instead of salvaging fragments. I remember sitting back and thinking that the threshold had moved. This was serious engineering work.

Anthropic had released [Claude Sonnet 4.5](https://www.anthropic.com/news/claude-sonnet-4-5) on September 29 and reported large gains in coding and long-running agent work. I did not need a benchmark to notice. The hibase session changed what I was willing to hand to a model.

I tried Google Antigravity around the same time. It never clicked. I am a terminal-oriented engineer, not because terminals are austere or fashionable, but because I have spent years making mine fit my hands. Vim is my editor. My projects, [dotfiles](https://github.com/Overbryd/dotfiles), shells, database commands, logs, and deployment tools already live there. My [`.tmux.conf`](https://github.com/Overbryd/dotfiles/blob/main/.tmux.conf) has evolved with that work, and its navigation is muscle memory. A graphical agent workspace gave the model a polished home and made the rest of my work feel like a visitor.

[`pi`](https://pi.dev/) felt native on the first day. It ran in the terminal, used the tools already installed there, and left the shape of my work alone. Pi is intentionally small and expects users to add extensions, skills, prompt templates, and project instructions. It skips built-in subagents and plan mode. I wanted exactly that. I did not need another development environment. I needed a capable model inside the one I owned.

As OpenAI's coding models improved, I put my existing Codex subscription through the same workbench. Soon nearly every active tmux window had a pi pane beside Vim or a shell. Small irritations went first. Before long, I was handing over changes that I would once have reserved for an uninterrupted afternoon.

Today my desk may have several coding agents running, but I do not pretend they form a miniature software company. There is usually no manager agent assigning work to worker agents while a reviewer agent watches them both. There are ordinary pi processes, a few Git worktrees, and tmux windows whose names I already know.

Tmux had taught me how to divide work long before models could edit code. A session holds a project. Windows separate tasks. Panes hold an editor, shell, logs, or agent. Coding agents increased the amount of work in motion, but they did not require a new map.

My problem is deciding which tasks can proceed independently, returning when judgment is needed, and proving that a plausible result is actually good.

##### The short answer

A small pi extension named [`ready-sound.ts`](https://github.com/Overbryd/dotfiles/blob/main/pi/extensions/ready-sound.ts) plays a chosen sound when a session returns control to me. I wrote it because polling silent panes was ridiculous. Later, I gave every session access to [sayneat](https://gitlab.com/parlant-co/sayneat) through my top-level [`AGENTS.md`](https://github.com/Overbryd/dotfiles/blob/main/.agent/AGENTS.md). An agent can speak a specific request when it needs a decision instead of merely sounding an alarm.

I return for an unclear requirement, an irreversible action, an unexpected failure, or a completed change ready for review. Routine tool use should not need me. Production changes, destructive commands, and scope decisions should.

I do not trust an agent's closing summary. I use it to find the evidence. I inspect the diff, read the changed tests, run the relevant checks, and look at the behavior. Passing tests are necessary, but they do not rescue the wrong requirement, a weak test, or an unnecessary abstraction.

I supervise by callback, not by watching token streams. When completed diffs arrive faster than I can understand them, I stop starting sessions. More agents would only create a larger pile of code I have not judged.

##### One process owns one context

I usually treat each pi process as a first-class work session, not as a hidden sub-session. It has one conversation, one repository view, and one task. Its context may later be compacted, but it remains attached to the work that produced it.

I can always find the state. The source tree and Git record the material result. The pi session records the investigation and decisions. The tmux window gives both a stable place. If I return after an hour, I do not ask an orchestration layer which synthetic role currently owns the task. I switch to the window whose name I already know.

Parallel work in the same repository needs a harder boundary. Two agents editing one checkout can invalidate each other's assumptions even when Git can merge the final text. I use a `git worktree` for each concurrent branch. Every session gets its own files, index, and branch, while all worktrees share the same object database.

Worktrees do not make coupled tasks independent. Two branches can still redesign the same interface, compete for a migration number, or assume incompatible states in a shared database. I split work only when I can state a clean boundary and a way to verify the pieces separately. If I cannot explain how the results will combine, I have not found parallel work. I have found a future merge meeting.

For an unusually large job, I can ask pi to start another pi process in a new tmux window and monitor it. I have had good results with that arrangement, but I use it rarely. Most work does not need recursive delegation. One or two explicit worktrees are easier to understand than a tree of sessions whose coordination becomes a separate problem.

##### What pulls me back in

A session should not demand attention merely because it has been running for ten minutes. Long tool calls, broad searches, and test suites are normal. Watching them continuously turns me into a progress indicator.

[`ready-sound.ts`](https://github.com/Overbryd/dotfiles/blob/main/pi/extensions/ready-sound.ts) was my first pi extension because it solved that mundane problem directly. When a turn ends and the session is ready for me, it plays a sound. I can move to another project, read a diff, or answer a message without checking each pane on a timer. The extension does not decide whether the work is good. It only tells me that the next exchange can happen.

Sound identifies an event, not its meaning. I later wrote a speech command called [`saynice`](https://github.com/Overbryd/dotfiles/blob/main/.bin/saynice), then replaced it with the local-first [sayneat](https://gitlab.com/parlant-co/sayneat). My global [`AGENTS.md`](https://github.com/Overbryd/dotfiles/blob/main/.agent/AGENTS.md) tells agents that sayneat is available when they need my attention. A waiting session can say that a Terraform plan is ready, that a requirement has two plausible readings, or that tests exposed an unrelated production assumption.

The result can be funny and occasionally embarrassing. During a meeting, my computer may suddenly address me in a remarkably natural voice. It is still better than maintaining a mental list of silent terminals.

There are four common reasons I return to a session.

First, the task has reached an ambiguity that code cannot settle. The agent can research existing behavior and present options, but it should not invent product policy or rewrite a business rule because one interpretation made the implementation easier.

Second, the next action crosses a risk boundary. My pi processes usually run without per-command approval inside source trees I control. Pi itself has no built-in permission sandbox and inherits the permissions of its process. That removes repetitive friction, but it is not blanket authority. My standing rules allow commands such as `terraform validate` and `terraform plan`, forbid `-auto-approve`, and require my confirmation before an apply. Untrusted repositories and stronger isolation needs belong in a container or virtual machine. A fast local loop must not become unattended production access.

Third, verification failed in a way that changes the premise. An ordinary failing test is part of the agent's job. A failure that reveals undocumented behavior, shared infrastructure, missing credentials, or a conflict with another task may need a decision from me.

Fourth, the proposed result is complete enough to review. This is the most frequent callback and the one that consumes most human time.

Speech is useful because the agent can distinguish those cases in a sentence. The completion sound remains useful because most turns need no elaborate announcement. Together they replace polling without pretending that notification solves supervision.

##### How I decide whether to trust the result

Trust starts before the agent edits anything. I try to give it a bounded outcome, the relevant constraints, and a concrete way to prove the behavior. Repository-level instructions explain where tests live, which commands are safe, which style the code uses, and what must remain untouched. Those rules live in [`AGENTS.md`](https://github.com/Overbryd/dotfiles/blob/main/.agent/AGENTS.md) because repeated expectations should be versioned, not reconstructed in every prompt.

For behavior changes and bug fixes, I prefer a red and green sequence. The agent adds or selects a focused test, runs it, and sees it fail for the expected reason. Only then does it make the smallest implementation change and rerun the same test. A test that passes before the fix does not demonstrate the gap. A test that fails for a syntax error demonstrates the wrong gap.

A captured failure tells me more than a polished summary. I can inspect the claimed failure, the change that makes it pass, and the broader checks that guard against collateral damage.

When the session calls me back, I usually review in this order:

1. I read `git status` and the diff to establish the actual scope.
2. I inspect tests as critically as implementation code.
3. I run the focused check and the relevant full suite.
4. I run language-level checks such as formatting, compilation, type checking, or linting.
5. I exercise the user-visible path when the change affects a template, interaction, browser, or deployment boundary.

The diff comes first because successful checks can coexist with an unacceptable change. An agent may modify a fixture until the test agrees with the bug, add a dependency for a five-line problem, duplicate an existing helper, or quietly broaden the requested scope. Senior engineering experience is useful here. Years spent recognizing weak boundaries, accidental coupling, and plausible but false explanations do not become less relevant because code appears faster.

I also read the agent's summary, but I do not count claims such as "all tests pass" as evidence until I have the command and result. The summary helps me navigate. The repository decides what changed, and independently executed checks decide what currently works.

For a large or sensitive patch, I may open a fresh pi session and ask it to review the diff without the implementation conversation. Fresh context can catch an assumption that the original session and I have both absorbed. It is another review input, not an approval. Two agents can agree because they inherited the same mistaken requirement.

Not every change deserves the same ceremony. A spelling correction and an authentication change have different failure costs. The constant is that responsibility remains with me. I can delegate investigation, editing, and test execution. I cannot delegate accountability by asking a model whether its own work should be trusted.

##### Parallelism stops at the review queue

I have no interest in maximizing the number of agents on my screen. My laptop can keep more sessions active than I can supervise well.

Every task has at least two human costs. I must frame it clearly enough to begin, then reconstruct enough context to judge the result. Tmux makes switching cheap because the locations are familiar. It does not make understanding free.

I spend less time writing routine code and more time selecting and reviewing work. I decide what should exist, identify independent slices, write acceptance criteria, evaluate trade-offs, and check finished work. This is a better use of my experience, but it remains work.

I stop adding parallel sessions when one of three things happens. Tasks begin touching the same conceptual boundary. Completed changes wait for review. I can no longer remember why a session exists without rereading its whole history. At that point another agent increases work in progress while reducing confidence.

I want a short feedback loop and no unattended queue of plausible code. Sometimes that means several repositories moving at once. Sometimes one difficult migration deserves the entire desk.

##### I tried building the orchestration layer

In April 2026, I tested the alternative rather thoroughly. I built a file-backed kanban system for coding agents. It launched worker and manager roles in tmux panes, assigned tickets, woke managers on a schedule, ran independent reality checks, tracked activity, accepted priority overrides, and exposed an operator role for me.

Eleven days after the first commit, I deleted it.

The system worked well enough to reveal its real cost. A worker could be alive but stuck. A periodic check could interrupt useful work. A manager needed context compaction and recovery rules. Tmux copy mode affected process control. Roles needed authority rules when priorities changed. The operator shell needed lifecycle handling. Each reasonable answer added another state transition to software whose only purpose was coordinating software that was supposed to save coordination time.

I did not delete it because the implementation failed. I deleted it because manager and reviewer agents shared the same incomplete specification as the worker. More roles produced more messages without producing independent knowledge. I had added a "reality check" role because I could no longer understand the work from the main loop. That was a damning feature for a tool I had built to improve oversight.

I kept the lessons and removed the organization chart. Tasks should be explicit. Tests and diffs should hold shared state. Fresh context is useful when review independence matters. Parallel work needs clean boundaries. Priorities remain close to the human who is responsible for the outcome.

I still do not use subagent systems by default, even though pi can support them. Recursive delegation helps with some large, separable jobs. It is not a free multiplier. Coordination consumes context, tokens, time, and attention whether the participants are people or model instances.

##### The commercial model had to change too

Hourly pricing became harder to defend. If a task that once consumed several manual days can now be investigated, implemented, and verified much faster, billing by keyboard time punishes the investment that made the work efficient. It also encourages clients to compare hours rather than outcomes while leaving me responsible for the same production risk.

I now prefer selling bounded chunks of work or a sprint at a fixed price. The client buys an agreed result. I price the scope, uncertainty, review, communication, and accountability, not the number of minutes in which code appeared.

Agents do not make engineering free. Review can dominate a short implementation. Poorly understood legacy behavior can still consume days. A production incident remains expensive even if the faulty patch took seconds to generate. With a fixed scope, finishing the same work faster helps both sides instead of turning efficiency into a smaller invoice. I still carry the estimate and the production risk.

##### The human bottleneck did not disappear

Coding agents removed a surprising amount of routine implementation work from my days. Small irritations no longer wait at the bottom of a backlog. Large refactors can begin with a capable investigation instead of a blank editor. Several independent projects can move while I concentrate on one difficult decision.

I now run out of review attention before I run out of agents. Good work still needs a coherent definition of done and an experienced person willing to reject code that merely looks finished.

On my desk, tmux holds the contexts. Pi does the work inside them. Git worktrees isolate concurrent changes. A sound tells me when a session is ready. Sayneat explains why it needs me. Tests, diffs, and direct observation decide whether the output survives.

The tools do not replace supervision. They bring me back at the right moment.

### [I replaced macOS say with local neural speech](https://www.lukasrieder.com/blog/i-replaced-macos-say-with-local-neural-speech/)

I replaced macOS say with sayneat, a local text-to-speech CLI. Six voices handled the same stubborn build very differently.

There is a macOS terminal trick I have used for years:

```sh
some-long-running-task; say "task is done, exited $?"
```

The shell starts `say` after the task exits, and the expanded `$?` tells me whether it worked. The command can run in a tmux window buried beneath several sessions. I do not have to keep checking it or trust that I will notice a terminal badge. My computer speaks when it needs me.

I use this often enough that I also gave the habit to my coding agents. If an agent reaches a point where it needs my attention, it can speak instead of waiting silently in another terminal.

Apple's `say` is old in the useful Unix sense. It joins unquoted arguments into a sentence, reads stdin when no words are supplied, accepts `-v` for a voice and `-o` for a file, and returns a meaningful exit status. It uses the macOS Speech Synthesis Manager, needs no network, and has been stable enough to become muscle memory.

The mechanism is excellent. The voice is not. Even Apple's better downloadable voices retain the cadence of an older accessibility synthesizer, especially after hearing current speech models from OpenAI or ElevenLabs. A status message does not need to pass for an audiobook, but the difference becomes irritating once you know how natural generated speech can sound.

That small irritation produced two tools. The first was a shell script called `saynice`. The second became [sayneat](https://gitlab.com/parlant-co/sayneat), a local-first text-to-speech command that has now replaced both `say` and `saynice` in my terminal.

##### The first fix was 256 lines of shell

[`saynice`](https://gitlab.com/Overbryd/dotfiles/-/blob/main/.bin/saynice) scratched the itch quickly. It wrapped OpenAI's `/v1/audio/speech` endpoint, used `gpt-4o-mini-tts` by default, and streamed the response from `curl` into `ffplay`. Its command-line shape stayed close enough to `say` that changing an existing notification was easy. I wrote about that first version at the time in [I gave my terminal a voice worth listening to](https://www.lukasrieder.com/blog/i-gave-my-terminal-a-voice-worth-listening-to/).

```sh
some-long-running-task; saynice "task is done, exited $?"
```

The result changed how the computer felt. A short spoken message no longer sounded like a system alert assembled from phonemes. It sounded like somebody inside the machine had noticed the build finish and told me about it.

For a small personal script, `saynice` was good. It accepted inline text or stdin, offered OpenAI's voices, and could save audio instead of playing it. I started using it everywhere, including the instructions I gave agents.

Its limits were equally plain. Every message left my computer. Every message needed a network connection and a paid API. Playback also depended on `curl`, `jq`, and `ffplay`. If I was offline, handling text that should remain local, or working on a machine without the right collection of command-line tools, the friendly voice disappeared.

I did not want to return to the old voice. I wanted the same quality and the same Unix-shaped interface without making a remote request the default.

Looking back, `saynice` proved two parts of the idea. Neural speech was good enough to hear throughout a working day, and preserving the shape of `say` mattered more than exposing every provider option. Its limitation was structural. OpenAI was not one engine behind the command; the HTTP request, voice list, error format, dependencies, and playback path were the command.

sayneat keeps the lesson and reverses that dependency. The command owns input, output, playback, and model selection. OpenAI is one optional engine beside ElevenLabs and four local choices. That distinction is what lets the same short invocation work offline without pretending remote providers are unnecessary.

##### sayneat starts local and asks before it goes remote

[sayneat](https://gitlab.com/parlant-co/sayneat) is a Rust program built around that decision. On its first run, it inspects available CPU threads and memory, displays every supported engine, and asks which one to install. It does not quietly choose a paid service.

The current catalog has four local engines and two remote ones:

- `kokoro` is the recommended local model. Its int8 weights take about 147 MB and provide 103 English and Chinese voices.
- `kokoro-hq` uses full-precision Kokoro weights. The download is about 365 MB, and the output skips the high-frequency filter used to soften int8 hiss.
- `kitten` is a compact English model with eight voices and a download of about 68 MB.
- `supertonic` is a 44.1 kHz English model with ten voices and a download of about 129 MB.
- `elevenlabs` uses account-owned or workspace voices and can return exact word timing.
- `openai` uses OpenAI's speech API and defaults to its `marin` voice.

After selection, sayneat downloads local weights into the operating system's application cache, verifies their SHA-256 hashes, and stores the default model and voice in its configuration. The program itself remains one native binary. Model weights stay separate, which keeps updates and model changes practical.

Remote engines remain available because they are useful. They require `ELEVENLABS_API_KEY` or `OPENAI_API_KEY` in the environment, and sayneat never writes those keys into its config. Selecting one sends the spoken text to that provider and may cost money. That boundary is visible instead of being hidden behind an apparently local command.

##### Six ways to hear the same sentence

I wanted a comparison that resembled an actual notification but gave each synthesizer more work than "done." The time and test count exercise number normalization. The first sentence is long enough to expose pacing, while the short second sentence tests whether the voice can change cadence without prompting.

> At 3:17, the stubborn build finally passed all 42 tests. Put the kettle on.

The commands below use one shell variable:

```sh
TEXT='At 3:17, the stubborn build finally passed all 42 tests. Put the kettle on.'
```

I generated every sample on the same Mac on August 24, 2026. These are complete first attempts. I changed no speed or pronunciation settings. `saynice` and sayneat wrote the MP3 files directly. macOS `say` wrote AIFF, which I converted to MP3 for consistent browser playback.

###### macOS say with Samantha

This is the baseline. `say` delegates to the speech system already installed on the Mac. It starts without a model download, an account, another binary, or a network request. I chose Samantha so the sample does not depend on whichever system voice happens to be configured on a reader's Mac.

```sh
say -v Samantha -o macos-say-samantha.aiff "$TEXT"
```

{% include audio-sample.html sample="macos-say" label="Play the macOS say Samantha sample" caption="macOS say, Samantha voice. Transcript: At 3:17, the stubborn build finally passed all 42 tests. Put the kettle on." %}

This voice does the job. It also explains why I started tinkering. The timing is regular, words do not influence one another much, and the second sentence still sounds like an operating-system announcement.

###### saynice with OpenAI Alloy

`saynice` represents my first answer to that problem. It sends the text to OpenAI's `gpt-4o-mini-tts` model and streams the returned audio through `ffplay`. Alloy was my default voice when I wrote the wrapper.

```sh
saynice --voice alloy --model gpt-4o-mini-tts \
  -o saynice-alloy.mp3 "$TEXT"
```

{% include audio-sample.html sample="saynice" label="Play the saynice OpenAI Alloy sample" caption="saynice using OpenAI gpt-4o-mini-tts, Alloy voice. Transcript: At 3:17, the stubborn build finally passed all 42 tests. Put the kettle on." %}

The jump from `say` is the reason `saynice` survived beyond one experiment. The catch is outside the recording. Producing it required an API key, an internet connection, several local programs, and sending the notification text to OpenAI.

###### sayneat with Kokoro int8

Kokoro int8 is sayneat's default recommendation on a machine with at least two CPU threads and 2 GB of memory. I used its American English `af_maple` voice.

```sh
sayneat --model kokoro --voice af_maple \
  -o kokoro-af-maple.mp3 "$TEXT"
```

{% include audio-sample.html sample="kokoro" label="Play the sayneat Kokoro int8 sample" caption="sayneat with Kokoro int8, af_maple voice. Transcript: At 3:17, the stubborn build finally passed all 42 tests. Put the kettle on." %}

The 147 MB int8 package is the compromise I expect most people to choose. sayneat applies a gentle 9.5 kHz low-pass filter because quantization can leave hiss near the model's Nyquist frequency.

###### sayneat with full-precision Kokoro

`kokoro-hq` uses the same voice family at full precision. I kept `af_maple` selected so the sample isolates the model package rather than switching speaker and weights together.

```sh
sayneat --model kokoro-hq --voice af_maple \
  -o kokoro-hq-af-maple.mp3 "$TEXT"
```

{% include audio-sample.html sample="kokoro-hq" label="Play the sayneat full-precision Kokoro sample" caption="sayneat with full-precision Kokoro, af_maple voice. Transcript: At 3:17, the stubborn build finally passed all 42 tests. Put the kettle on." %}

The download grows from about 147 MB to 365 MB. sayneat leaves this output unfiltered. Whether that difference deserves the extra disk space is easier to decide by listening than by reading model descriptions.

###### sayneat with KittenTTS Mini

Kitten is the smallest local option in the catalog. Its package is about 68 MB and contains eight English voices. I selected `expr-voice-5-m`.

```sh
sayneat --model kitten --voice expr-voice-5-m \
  -o kitten-expr-voice-5-m.mp3 "$TEXT"
```

{% include audio-sample.html sample="kitten" label="Play the sayneat KittenTTS sample" caption="sayneat with KittenTTS Mini, expr-voice-5-m. Transcript: At 3:17, the stubborn build finally passed all 42 tests. Put the kettle on." %}

Kitten takes longer over this sentence than the other local choices. That is useful information for a notification voice. Model size, synthesis speed, speaking pace, and voice preference are separate decisions.

###### sayneat with Supertonic

Supertonic differs technically as well as vocally. It produces 44.1 kHz audio while the other local engines produce 24 kHz audio. Its package is about 129 MB and offers five female and five male voices. This sample uses `F3`.

```sh
sayneat --model supertonic --voice F3 \
  -o supertonic-f3.mp3 "$TEXT"
```

{% include audio-sample.html sample="supertonic" label="Play the sayneat Supertonic sample" caption="sayneat with Supertonic 3, F3 voice. Transcript: At 3:17, the stubborn build finally passed all 42 tests. Put the kettle on." %}

Among the local models, Kokoro HQ and Supertonic are the clear winners for my taste. OpenAI and ElevenLabs still sound a little more natural to me, but the gap is no longer large enough to justify sending everyday terminal messages off the machine. With these two local options, I am happy to keep routine notifications offline.

Voice discovery stays scriptable. `sayneat --voices` prints one accepted value per line for the configured model. `sayneat --voices --model kitten` inspects another local model without downloading it. This makes a complete voice audition a normal pipeline:

```sh
sayneat --voices | xargs -I{} \
  sayneat "Hello, this is a demo for {}" --voice {}
```

A one-off `--model` or `--voice` override never changes saved configuration. I can try another voice and return to my normal one on the next command.

##### What sayneat keeps from say

I wanted a better synthesizer, not a new ritual. The common cases remain recognizable:

```sh
say "The deploy finished"
sayneat "The deploy finished"

pbpaste | say
pbpaste | sayneat

say -v Samantha -o message.aiff "Keep this"
sayneat -v af_maple -o message.wav "Keep this"
```

Both commands join positional words with spaces, fall back to stdin, use `-v` for a voice, use `-o` for output, and play speech when no output file is present. Both also have an interactive mode that follows the spoken text. The exact interface differs because sayneat can pause, replay, and display a waveform after generation.

sayneat is intentionally not a flag-for-flag clone. Apple's `-r` controls the words per minute of its synthesizer; local neural engines do not share one reliable rate control. Apple's `-a` and `-n` route audio to devices and network services; sayneat leaves that job to ordinary audio tools. Apple's `-f` reads a file, while sayneat uses the shell syntax that already works everywhere:

```sh
sayneat < release-notes.txt
```

The additions describe choices that `say` never needed: `--model`, `--models`, `--voices`, `--setup`, and `--play`. They do not affect the short form. Once setup is complete, `sayneat hello` remains the whole command.

That restraint distinguishes sayneat from `saynice` too. `saynice` imitated some `say` options, but its implementation was a chain of `curl`, `jq`, and `ffplay` around one remote provider. sayneat owns model setup, local inference, output encoding, playback, and terminal controls in one native program. It can still use OpenAI or ElevenLabs, but neither provider defines the command.

##### It still behaves like a command-line tool

Speech quality was the reason to start sayneat. Its input and output behavior is why I kept it.

Inline arguments work quoted or unquoted. Without arguments, sayneat reads stdin until EOF. Positional text wins if both exist, which avoids accidentally speaking an unrelated pipe when a command supplies an explicit message.

```sh
sayneat The deploy finished
pbpaste | sayneat
printf '%s\n' "$(git log -1 --pretty=%B)" | sayneat
```

Without `--out`, sayneat plays the result. A `.wav`, `.mp3`, or `.flac` suffix selects the encoder when saving. `-o -` writes WAV bytes to stdout while diagnostics stay on stderr, so the audio can enter another program without log lines corrupting it:

```sh
sayneat -o - "Archive this message" | \
  ffmpeg -i pipe:0 archive.m4a
```

That separation sounds minor until a command enters a pipeline. Clean stdout is the difference between a Unix command and an interactive application that happens to launch from a terminal.

Playback also accounts for terminal work. `Ctrl-T` pauses or resumes ordinary playback. The control input comes from the controlling terminal, not stdin, so it still works when the spoken text arrives through a pipe.

For longer or less disposable speech, `--interactive` generates first and opens a paused one-line player. `--play` starts as soon as generation finishes. The interface shows exact current-word highlighting when an engine returns alignment; otherwise it renders a live waveform around the playback position. Space pauses, resumes, or replays. Escape stops and rewinds. `Ctrl-C` exits.

```sh
pbpaste | sayneat --interactive
pbpaste | sayneat --interactive --play
```

I did not need a graphical application for any of this. I needed playback controls that survive a pipe and fit into one terminal line.

##### Long text cannot become one giant allocation

A task notification is tiny, but stdin invites larger input. Reading a document aloud exposed a different engineering problem: speech engines have input limits, and generated audio can consume a surprising amount of memory.

sayneat splits long input at paragraphs and complete sentences when possible. It packs chunks up to each engine's safe budget, then falls back to clauses or words for an oversized sentence. ElevenLabs requests receive adjacent text as context so prosody does not reset blindly at every boundary. Alignment offsets accumulate across chunks, which keeps highlighted words synchronized with the combined output.

Local generation shares one thread-safe sherpa-onnx session across ordered parallel batches. Up to four workers synthesize chunks, while full-precision Kokoro uses at most two. sayneat always appends completed results in source order.

It also refuses to hold an hour of finished PCM in memory. Each completed batch becomes 16-bit mono audio in a private temporary file. Playback and encoders reopen that file and stream fixed-size blocks. At 24 kHz, temporary audio uses about 173 MB per hour, so available temporary disk space becomes the practical limit instead of RAM. sayneat removes the file on normal completion or error.

This machinery is invisible for "build done," as it should be. It matters when the same small command reads an article or converts a long document to FLAC.

##### Replacing the old habit

The source is public at [gitlab.com/parlant-co/sayneat](https://gitlab.com/parlant-co/sayneat). Building currently requires Rust 1.89 or newer, CMake, and a C or C++ compiler. Linux also needs ALSA development headers.

```sh
git clone https://gitlab.com/parlant-co/sayneat.git
cd sayneat
cargo test
cargo build --release
./target/release/sayneat --setup
```

The release workflow targets Apple Silicon and Intel macOS, x86-64 Linux, and x86-64 Windows. The local model licenses differ, so the README records them: Kokoro and KittenTTS weights use Apache-2.0, while Supertonic weights use OpenRAIL-M and retain its restrictions.

My original shell suffix barely changes:

```sh
some-long-running-task; sayneat "task is done, exited $?"
```

That is the outcome I wanted. The good part of `say` was never its synthesizer. It was the ability to add speech to any shell command with a semicolon. sayneat keeps that habit, gives it voices I enjoy hearing, and lets the default path work without a network request.

It has replaced both `say` and `saynice` for me. If terminal speech is already part of your work, [give sayneat a try](https://gitlab.com/parlant-co/sayneat). Feedback and bug reports belong in the project, especially reports about machines and voices different from mine.

### [I gave my terminal a voice worth listening to](https://www.lukasrieder.com/blog/i-gave-my-terminal-a-voice-worth-listening-to/)

A 256-line Bash wrapper keeps the best part of macOS say, but replaces its voice with OpenAI speech that sounds startlingly human.

One of my favorite macOS commands is also one of its smallest:

```sh
say "The build is done"
```

It becomes much more useful after a long-running command:

```sh
some-long-running-task; say "task is done, exited $?"
```

I can start a build in a tmux window, move to something else, and wait for the computer to call me back. The shell expands `$?` after the first command exits, so the message can include its status. A semicolon announces success or failure. If I only care about success, `&&` does that too.

This is better than checking the same pane every minute. It is also better than a notification I might miss behind another screen. Sound reaches me while I am reading, sketching on paper, or looking at a different machine.

There is only one problem. I do not enjoy the voice.

The macOS synthesizer is clear, fast, local, and dependable. It also sounds unmistakably synthetic. After listening to current generated speech, I know a short status message can sound less like an accessibility system reading a dialog box and more like another person in the room.

So I wrote `saynice`, a Bash command that preserves the convenient shape of `say` and replaces its speech engine with OpenAI's text-to-speech API.

```sh
some-long-running-task; saynice "task is done, exited $?"
```

That one-word change is the whole point.

##### The interface was already right

Apple's `say` has accumulated many options, but its ordinary use follows a compact contract. Words on the command line become speech. No words means read stdin. `-v` selects a voice. `-o` saves audio instead of playing it. `-f` reads a file. `-r` controls the speaking rate.

I did not want a speech application with projects, a library, and an export screen. I wanted that contract with a better voice.

`saynice` therefore accepts the forms I already reach for:

```sh
saynice "The deploy finished"
saynice The deploy finished
printf '%s\n' "The deploy finished" | saynice
saynice -f release-notes.txt
saynice -v coral "The deploy finished"
saynice -o message.mp3 "Keep this message"
```

Unquoted positional words are joined with spaces, just as they are by `say`. If there are no positional words and no `-f`, the command reads stdin until EOF. An explicit file and positional text are rejected because choosing silently between two deliberate sources would hide a mistake.

The voice flag also keeps its familiar short form:

```sh
saynice -v alloy "Hello"
saynice --voice onyx "Hello"
saynice -v '?'
saynice --list-voices
```

The known list currently contains `alloy`, `ash`, `ballad`, `coral`, `echo`, `fable`, `onyx`, `nova`, `sage`, `shimmer`, and `verse`. OpenAI can add voices independently of this script, so an unknown name produces a warning rather than an immediate refusal. The API gets the final say.

I had once used `obsidian` when I meant `onyx`, so the wrapper maps that name and prints a note. This is exactly the kind of tiny personal accommodation that belongs in dotfiles. It costs almost nothing and prevents a repeated interruption.

The `-r` option reveals where compatibility stops. OpenAI's current speech endpoint does not accept the same words-per-minute control as the macOS synthesizer. `saynice -r 180` accepts the argument and explains that it is ignored. Existing shell snippets do not fail because of the flag, but the command does not pretend it changed the audio.

##### The whole request is visible

The default model is `gpt-4o-mini-tts`, and the default voice is `alloy`. Both can be changed for one invocation:

```sh
saynice --model gpt-4o-mini-tts --voice coral \
  "Production is healthy again"
```

They can also be changed through environment variables:

```sh
export SAYNICE_MODEL=gpt-4o-mini-tts
export SAYNICE_VOICE=coral
```

The API key comes from `OPENAI_API_KEY` or `SAYNICE_API_KEY`. `saynice` never writes it to a configuration file. It also passes `OPENAI_ORGANIZATION_KEY` and `OPENAI_PROJECT_KEY` as headers when those variables exist.

The implementation builds one JSON object with `jq`:

```json
{
  "model": "gpt-4o-mini-tts",
  "voice": "alloy",
  "input": "The build is done",
  "response_format": "mp3"
}
```

Then `curl` sends it to `/v1/audio/speech` with the bearer token and content type. There is no SDK between the command and the endpoint. If OpenAI changes the request, I have one payload and one URL to inspect.

`SAYNICE_BASE_URL` and `OPENAI_BASE_URL` can replace `https://api.openai.com`. I added that mostly because API-compatible gateways are common enough that hardcoding the host would be needless. The wrapper still assumes the `/v1/audio/speech` route and OpenAI's request shape.

This directness is why Bash was a reasonable choice. The job is argument parsing, a JSON request, one HTTP call, and either a pipe or a file move. `curl`, `jq`, and `ffplay` already do the hard parts. A separate runtime or package dependency would make this particular command harder to carry between my machines.

##### Playback should begin before the request ends

The normal path is spoken output, not a saved file. In that mode, `curl` uses `--no-buffer` and writes the response directly to `ffplay`:

```sh
curl ... | ffplay -nodisp -autoexit -loglevel error -i pipe:0
```

`ffplay` opens no video window, exits when playback finishes, and keeps its normal diagnostic chatter out of the terminal. Audio can begin as bytes arrive instead of waiting for a complete MP3 in a temporary file.

This path also gives the pipeline useful failure behavior. The script starts with `set -euo pipefail`, and `curl` uses `--fail`. A failed request or failed player makes the command fail rather than returning success because the last process happened to exit cleanly.

Saving audio needs a different route. The extension chooses the response format:

- `.mp3` requests MP3.
- `.wav` requests WAV.
- `.flac` requests FLAC.
- `.opus` requests Opus.
- `.aac` requests AAC.
- `.pcm` and `.raw` request PCM.

The default remains MP3 when the filename has another extension. I can override that with `SAYNICE_AUDIO_FORMAT` if I genuinely want an unusual filename.

For file output, `curl` writes the response to a private temporary file and records the HTTP status separately. A successful response moves into the requested destination. An error response stays temporary long enough for `jq` to extract `error.message` or `message`, then a trap removes it.

This split matters because an HTTP error body is JSON, not audio. Writing it directly to `message.mp3` would leave a plausible-looking file that fails later for a confusing reason. `saynice` instead prints the provider's explanation while it still has the response available.

##### Small commands still need boundaries

This script is personal, but it is not a loose alias around `curl`. It has to fail in ways that make sense when called from another shell command or by an automated tool.

A missing option value prints usage and exits `64`. Supplying both `-f` and positional text also uses exit `64`, the conventional command-line usage error. A missing input file exits `66`. A missing API key exits `78`. Empty text is rejected before any paid request leaves the machine.

Dependencies are checked at the point they become necessary. Every request needs `curl` and `jq`. Playback also needs `ffplay`, but file output does not. Someone saving a WAV should not be blocked because the audio player is absent.

Unknown options are errors. `--` ends option parsing, so text beginning with a hyphen remains possible:

```sh
saynice -- "--force would be a bad idea here"
```

These checks account for much of the script's 256 lines. The successful HTTP request is short. Most of the code protects the command-line boundary, reports provider errors, and cleans up after interrupted requests.

That is not wasted bulk. A tool becomes reusable when failure is as predictable as success.

##### I gave the command to my agents

A voice notification is especially useful now that coding agents can work in terminal sessions for several minutes without intervention. I can ask an agent to investigate, switch to another task, and hear when it reaches a decision that needs me.

I added `saynice` to my agent instructions on the same evening I wrote it:

```text
`saynice` is used to get the attention of the user by sending a TTS message.
Prefer `saynice` over `say`.
```

I also attached it to an important Terraform boundary. An agent may run `terraform validate` and `terraform plan`, but it must ask before applying changes and must never add `-auto-approve`. The instruction tells it to use `saynice` when that confirmation is waiting.

This is a small change in human-computer coordination. The agent does not need a notification service, desktop permission, webhook, or chat integration. It executes the same command I use. The message travels through the speakers, which is where my attention already is.

It also makes the quality difference more noticeable. Hearing the old system voice once after a thirty-minute build is tolerable. Hearing it throughout a day of agent work is grating. OpenAI's voice gives the interruption enough warmth that it feels less like an alarm.

##### The trade is explicit

`saynice` is not a private or offline command. It sends every spoken string to an API. It requires a working network connection, consumes paid service usage, and depends on OpenAI continuing to support the selected model and voice.

I will not pipe secrets, private client data, or arbitrary logs into it. A fixed sentence such as "the test suite failed" is easy to assess. Reading a whole command result aloud deserves more care because the content may contain paths, names, tokens, or customer information.

Latency is another trade. Streaming prevents an unnecessary wait after generation begins, but a remote request still has connection and synthesis time that local `say` does not. For brief notifications on a normal connection, I prefer the better voice. When offline, the original command remains installed.

This is why I kept the wrapper small. It does not hide where the speech comes from or claim to replace the operating system. It makes one remote service feel like a Unix command, with enough compatibility that my existing habit survives.

##### It lives with the rest of my small tools

The source is in [my dotfiles repository](https://gitlab.com/Overbryd/dotfiles), alongside the other commands I keep in `~/.bin`. The complete script is [`.bin/saynice`](https://gitlab.com/Overbryd/dotfiles/-/blob/main/.bin/saynice).

Setup needs the three command-line dependencies and an API key:

```sh
export OPENAI_API_KEY=...
saynice "There is now a person inside the computer"
```

I wrote and committed the command on March 3. A week later, I have not needed to change its implementation. More tellingly, I have already changed my agent instructions to prefer it. The useful threshold for a dotfile is not novelty. It is reaching for the command without thinking about how it works.

The original terminal trick remains intact:

```sh
some-long-running-task; saynice "task is done, exited $?"
```

My tmux windows can still call me back. They finally sound like someone I want to hear.

### [The Phoenix script that runs my four-slot macOS desktop](https://www.lukasrieder.com/blog/the-phoenix-script-that-runs-my-four-slot-macos-desktop/)

A 40-inch ultrawide gave me plenty of pixels and nowhere obvious to put a 1,920 by 1,080 screen-share window.

My Dell U4021QW solved one problem and created another. Its 40-inch ultrawide panel gave me enough room for several useful windows, but ordinary maximization wasted the width and manual placement made me drag the same windows into the same positions every day.

What I wanted was oddly specific. One 1,920 by 1,080 frame should sit at the top center. This is where I put the window that needs my attention and the one I usually share on calls. A second central area should fill the space below it. Two tall columns should frame the center left and right.

No window manager I tried knew that arrangement. [Phoenix](https://github.com/kasper/phoenix) did not need to know it. Phoenix is a lightweight macOS application and window manager that exposes windows, screens, Spaces, and keys to JavaScript. Kasper Hirvikoski built the application. I only had to describe my desktop in `~/.phoenix.js`.

[I wrote the first version in June 2022](https://github.com/Overbryd/dotfiles/commit/8b6ee6266df8c8add985d74de76db88f48e82390). I still use the same four-part geometry. Most adjustments since then have taught Phoenix which application windows to leave alone.

##### Four areas from one rectangle

The [Dell panel has a physical resolution of 5120×2160](https://tftcentral.co.uk/news/dell-u4021qw-announced-with-40-ips-panel-and-5120-x-2160-5k2k-resolution), a format sold as 5K2K. Its Thunderbolt 3 connection carries the display signal and supplies up to 90 watts of power, so one cable connects my computer to the display and the peripherals attached to it.

The script does not hard-code that full resolution. It asks Phoenix for the screen's [`flippedVisibleFrame()`](https://kasper.github.io/phoenix/api/screen), which excludes visible space used by the menu bar and Dock and puts the origin at the top left. It derives every area from that rectangle and one constant:

```js
const FULLHD = {
  width: 1920,
  height: 1080,
};
```

The resulting layout has four named areas:

| Area | Behavior | Frame |
| --- | --- | --- |
| `main` | one window | 1,920 by 1,080 at the top center |
| `secondary` | vertical stack | 1,920 wide, below `main` |
| `left` | vertical stack | half the width left over from `main`, full visible height |
| `right` | vertical stack | the other half, full visible height |

{% include diagram.html post="2026-02-11-the-phoenix-script-that-runs-my-four-slot-macos-desktop" diagram="window-layout" caption="Diagram: The fixed 1,920-wide center puts main above secondary, while equal side stacks use the remaining width and full visible height." %}

If Phoenix reports a 5,120-wide visible frame, each side column is 1,600 units wide. The exact available height depends on display scaling, the menu bar, and the Dock. That is why the code uses the visible frame instead of repeating a guessed screen height.

The central frame matters most during screen sharing. I can share one conventionally proportioned application window instead of exposing a 5,120-pixel-wide desktop to someone on a laptop. Video-call software may still scale the stream, but its source starts with the 16:9 shape I intended. I also know exactly which part of my own screen other people can see.

The side columns suit terminals, editors, mail, calendars, and chat. A single terminal in one column gets the full available height. Adding another window divides that same column rather than covering the first window.

##### Two container behaviors

The script implements the layout with a small container model. A `SingleFramedContainer` owns the `main` area. A `VerticalContainer` owns each of the other areas.

The single container gives its current window one fixed frame. Moving a managed window into `main` exchanges it with the existing occupant. The previous main window returns to the source area, and that area's windows reflow. If the incoming window was not managed, the previous occupant leaves management and remains at its last frame.

Vertical containers divide their height by the number of windows they contain:

```js
class VerticalContainer extends FramedContainer {
  render() {
    const windowHeight = Math.floor(this.frame.height / this.stack.length);
    for (const [i, window] of this.stack.entries()) {
      window.setFrame({
        x: this.frame.x,
        y: this.frame.y + i * windowHeight,
        width: this.frame.width,
        height: windowHeight,
      });
    }
  }
}
```

That loop is the tiling algorithm. Each window receives the container width and an equal share of its height. A new side-window goes to the top of the array, so existing windows move down and shrink to make room. Removing or minimizing one enlarges the remaining windows.

Phoenix supplies the operations underneath this code. Its [`Window` API](https://kasper.github.io/phoenix/api/window) can set a frame, inspect minimized state, and focus the closest neighbor in a direction. My configuration decides which frame and which neighbor matter.

##### Hotkeys follow the layout

I use one chord, `Control+Command`, for every operation. Arrow keys move the focused window. Vim direction keys move focus.

| Hotkey | Action |
| --- | --- |
| `Control+Command+Up` | exchange the focused window with the first window in `main` |
| `Control+Command+Down` | exchange it with the first window in `secondary` |
| `Control+Command+Left` | insert it at the top of the left stack |
| `Control+Command+Right` | insert it at the top of the right stack |
| `Control+Command+H/J/K/L` | focus the closest window west, south, north, or east |
| `Control+Command+X` | release the focused window from management |
| `Control+Command+R` | remove newly excluded windows from managed stacks and render again |
| `Control+Command+D` | log every area and the windows it contains |

The arrow keys mirror the screen. I do not need an application-specific shortcut or a command palette. Moving a window takes one chord and produces no animation.

The release key matters as much as the placement keys. `Control+Command+X` removes a window from every managed container without closing or moving it. Phoenix stops touching that window until the configuration loads again or I move the window into an area. This gives me an ordinary floating desktop when one application needs it.

##### Exceptions kept the system useful

The first version assumed that visible application windows were ordinary rectangles. macOS applications soon supplied counterexamples. 1Password opens an autofill panel near the field that requested it. Updaters create short-lived dialogs. Microsoft Teams has created invisible notification windows. Utilities may expose empty or zero-size windows that should never occupy a tile.

My current application exclusions cover 1Password, System Settings, Activity Monitor, Remote Desktop, Loopback, LaunchBar, Temp Monitor, Pixelmator Pro, BetterDisplay, and Ollama. Window checks reject empty titles, zero-size frames, the Teams notification title, and updater titles. Minimized windows live in a separate list until Phoenix reports that they have returned.

These exceptions do less than another set of placement rules would. They say that Phoenix has no responsibility for a particular application or window. The application can float a panel, choose its own position, or disappear without causing four unrelated windows to resize.

The configuration also has affinity rules. Slack, Spark, WhatsApp, and Teams go to the left. Notes, Preview, and Google Meet go to the right. Those affinities apply only to windows Phoenix finds when the configuration loads. The current file has its `appDidLaunch` and `screensDidChange` handlers commented out, and it does not register a Space-change handler. New windows do not get pulled into a column merely because an application opened them. I place those with the same directional hotkeys.

That limitation is acceptable in daily use and better than claiming automation the script does not perform. Startup is deterministic. Later choices stay under my control.

##### Keep geometry fixed and exceptions cheap

Custom automation becomes hard to trust when every unusual application forces a redesign. This configuration separates geometry from compatibility. Geometry decides where managed work appears. Compatibility decides whether Phoenix should manage a window at all.

A popup does not justify a new container. It belongs in an exclusion. A window that needs to float for a while uses `Control+Command+X`. An actual screen-layout change belongs in the functions that define the container frames. Each kind of friction has one place to go.

That boundary also limits what the script attempts. Phoenix does not infer why a window opened or whether it deserves attention. It places accepted windows into four containers and stays away from rejected ones. I make the decision when I add an affinity, an exclusion, or press a directional key.

Phoenix already moves rectangles with `setFrame()`. My configuration records the four rectangles that match my monitor, my screen-sharing habits, and the applications I use. It gives one task the central 1,920 by 1,080 frame, keeps supporting windows visible around it, and lets troublesome windows opt out.

That is enough automation for my desktop. The layout has remained useful since 2022 because exceptions can stay exceptions instead of changing the whole system.

### [Dotfiles review, September 2025](https://www.lukasrieder.com/blog/dotfiles-review-september-2025/)

I gave Neovim one column of empty space. It fixed a kind of slowness no benchmark records.

In September 2025, I chose to leave one character-wide strip of Neovim empty. The editor lost a column of usable width and became faster to work in.

That trade makes sense only if speed includes the person looking at the screen. I care about response time, but I also care about how quickly I can understand what changed and continue without finding my place again.

##### Calm tools are fast tools

I like tools with little visual movement. I turn off animations wherever I can, prefer immediate switches between states, and want layouts and text positions to remain stable. This lets my eyes rest. It also lets me develop habits, muscle memory, and a visual map of where to look.

When that map remains dependable, I can navigate my work environment almost without thinking. I know where a tmux pane begins, where a shell prompt will appear, and where the code under my cursor sits in the editor. I can act in a blink because I do not first have to work out where the interface moved something.

An animation does not have to take long enough to appear in a benchmark to slow me down. A window that slides into place still asks me to wait for its destination. A layout that changes width makes me relocate the text I was reading. The expensive part is not rendering the movement. It is reorientation.

Neovim's sign column is a small, precise example. Signs show useful information beside a line, including diagnostics from a language server. With the default `auto` behavior, [Neovim draws the column only when it has a sign to display](https://neovim.io/doc/user/options.html#'signcolumn'). The first diagnostic therefore inserts a new column at the left and shifts every line of code one character to the right. When the last sign disappears, the content shifts back.

The diagnostic is useful. The jump is not.

[My September change](https://github.com/Overbryd/dotfiles/commit/2d6f6ff58d7d5c65460b45848b15dfe40ca81192) was one line:

```vim
set signcolumn=yes
```

The setting reserves the column even when it is empty. A diagnostic now appears inside space that already belongs to it. The code, indentation, and line numbers stay where I expect them. My eye can notice the new symbol, find its line, and keep its position in the buffer.

One column is a cheap price for that stability. On a narrow screen, reclaiming every character can look efficient. In practice, the extra width was not helping if the whole editing surface moved whenever feedback arrived.

I had solved the same problem once before. My old CoC configuration contained a comment that said the sign column should remain visible because otherwise it shifted the text. After I stopped loading that plugin-specific configuration, the behavior went with it. Putting the setting in Neovim's common configuration made the layout independent of whichever completion or diagnostic plugin happened to be active.

That detail matters beyond this one setting. Stable behavior should belong to a stable layer. Diagnostics may come from different language servers over time, but their screen space should not depend on one plugin's configuration file.

The preference is much older than this September commit. The [Makefile in my dotfiles](https://github.com/Overbryd/dotfiles/blob/be49a70161c84fadc61497099021ead1339a0a04/Makefile#L377-L452) records the defaults I use when setting up macOS. It disables Finder animations, disables the Dock's launch animation, and disables automatic window animations globally. It sets Cocoa window resizing to `0.001` seconds. Keyboard repeat starts sooner and runs close to the fastest rate macOS allows.

Those defaults describe what I mean by fast more accurately than a collection of application benchmarks. Opening, resizing, switching, and repeating should happen when I ask. The computer does not need to perform the transition for me. I already know that I changed windows.

This preference also explains why I continue to feel at home in tmux, Bash, and other text-based tools. Their interfaces are dense, but density is not the opposite of calm. A terminal can show a great deal of information while keeping pane boundaries, prompts, and text in predictable places. Calm does not mean sparse or silent. It means that feedback arrives where I expect it.

Text interfaces can still violate that expectation. The sign-column jump did. Full-screen terminal programs can redraw badly, logs can erase context, and prompts can change shape for decorative reasons. I keep adjusting my own configuration when one of these details interrupts the visual map I have built.

Interfaces that do not respect this tend to push me away, even when they have useful features. The repeated cost is hard to ignore once a stable tool has shown the alternative. A calm tool lets me spend attention on the work. It does not keep asking me to watch the tool itself.

##### Other changes in September

September contained three other changes in my [public dotfiles repository](https://github.com/Overbryd/dotfiles):

- [Run Mix tests when files change](https://github.com/Overbryd/dotfiles/commit/5cb7675600f40a73ba39644cf8a9b4cde82dee0b). I added an eight-line `twix` script that watches `lib`, `test`, and matching umbrella directories with `fswatch`, then pipes change notifications into `mix test --listen-on-stdin`. Mix runs the full suite once and reruns it after each newline. This is the watcher pattern from the [Mix documentation](https://hexdocs.pm/mix/Mix.Tasks.Test.html#module-file-system-watchers), extended for umbrella projects. The script remains unchanged.
- [Migrate Alacritty from YAML to TOML](https://github.com/Overbryd/dotfiles/commit/bf54371348a100521552543774180a924ee219ad). [Alacritty 0.13 changed its configuration format to TOML](https://github.com/alacritty/alacritty/releases/tag/v0.13.0). I moved the configuration, enabled live reload, and made maximized startup explicit rather than carrying the old YAML file forward.
- [Move word navigation into Readline](https://github.com/Overbryd/dotfiles/commit/be49a70161c84fadc61497099021ead1339a0a04). I removed terminal-level mappings for `Alt` plus the arrow keys and `Alt-H` or `Alt-L`, then bound their actual input sequences to Readline's `backward-word` and `forward-word` commands in `.inputrc`. The terminal no longer had to translate a shell-editing preference.

### [How an Elixir umbrella let KMS become its own project](https://www.lukasrieder.com/blog/how-an-elixir-umbrella-let-kms-become-its-own-project/)

Formfix proved KMS's encrypt-first design while an Elixir umbrella let the key-management code move gradually from an embedded OTP application to a separate open-source project.

[KMS](https://gitlab.com/parlant-co/kms) exists because a client project needed encryption to be part of its storage design from the beginning.

That client project was Formfix. It handled sensitive form submissions, so persisting plaintext first and adding encryption later was never an acceptable implementation plan. The application needed an encrypt-first path while its business logic was still taking shape.

Elixir and Phoenix made the technical response unusually straightforward. I created KMS as its own OTP application inside the Formfix umbrella. During development it behaved like part of a monolith: one repository, one dependency graph, one command for tests, and direct function calls in the same BEAM. At the same time, key management had its own application boundary, supervision tree, storage, migrations, tests, and public API.

The boundary that began as a practical way to keep cryptography out of client business logic became the path by which KMS moved into separate releases, separate pods, and finally a separate repository and open-source project.

I am grateful to Formfix for proving the encrypt-first concepts that became KMS. This is the origin story of that project.

##### Privacy by design starts before the first row

In my [Privacy by Design talks](https://www.lukasrieder.com/talks/privacy-by-design), I make a simple argument: privacy decisions belong in system design, not in a compliance pass after production data has accumulated.

Encryption is only one of those decisions. Purpose, access, retention, deletion, logging, and data minimization matter just as much. Still, the storage boundary is a good place to see whether a project means what it says. If sensitive values enter an ordinary plaintext column, every backup, replica, debugging session, and ad hoc query inherits that exposure. Retrofitting encryption then becomes a migration project with old and new formats, uncertain copies, and a difficult question about whether plaintext has really disappeared.

Encrypt-first changes the default. The application accepts plaintext at an input boundary, turns it into ciphertext before persistence, and makes decryption an explicit operation. A database reader should encounter encrypted payloads rather than the original value. The application still needs authorization and careful runtime handling, but storage no longer assumes that every database consumer should see everything.

Formfix gave this idea a real workload. It was not a cryptography demo built around one hard-coded string. A client application had to write, load, and process encrypted values while its normal Phoenix and Ecto code continued to evolve. That pressure exposed what the KMS API needed to own and what should remain application policy.

The first useful boundary was narrow. Formfix supplied plaintext, a key alias, and optional context. KMS returned an encrypted payload. To decrypt, Formfix supplied the payload and the same context. Formfix decided when its business rules allowed either operation. KMS decided how keys and ciphertext were handled.

That division still shapes the public project.

##### Wrapped envelope encryption became the core

KMS uses AES-GCM through Erlang's [`:crypto`](https://www.erlang.org/doc/apps/crypto/crypto.html). AES-GCM is an authenticated-encryption mode standardized in [NIST SP 800-38D](https://csrc.nist.gov/pubs/sp/800/38/d/final). With correct key and nonce handling, it encrypts plaintext and produces an authentication tag that detects a modified payload.

The cryptographic operation is deliberately boring. I do not want application code inventing ciphers or implementing authenticated encryption itself. Erlang already provides the operation. KMS adds the key lifecycle and application-facing contract around it.

For each encrypted value, a data encryption key encrypts the plaintext. A key encryption key then wraps that data key. The application stores the ciphertext together with the wrapped data key, while KMS manages the key encryption key by alias. This is [envelope encryption](https://cloud.google.com/kms/docs/envelope-encryption), a common pattern for keeping the key that encrypts bulk data separate from the longer-lived key used to protect data keys.

This envelope encryption design stores a wrapped data key beside each ciphertext. KMS can change how it protects key encryption keys without changing every application's business schema. Current KMS versions can protect persisted KMS secrets through a root master key backed by a local file or an external provider. The application continues to call the same alias-oriented encryption interface.

KMS also accepts additional authenticated data, or AAD. AAD is not secret and is not encrypted. It is authenticated along with the ciphertext. An application can bind a payload to a stable context such as a tenant, record, and field. Moving valid ciphertext to a different context then causes authentication to fail.

AAD does not replace authorization, and it does not stop replay within the same accepted context. It must also remain reproducible for as long as the ciphertext needs to be decrypted. Those constraints are why KMS treats AAD as an application policy that travels through its API rather than trying to infer it.

The original implementation was much smaller than the current system. It already had AES-GCM, aliases, wrapped data keys, and a dedicated KMS store, but root-key handling and authorization were still early. Formfix was valuable precisely because it proved the basic data flow without pretending that the first version had solved every security concern.

##### Umbrellas let me build a monolith without mixing everything

I like umbrella applications because they separate code before they force separate operations.

The [Elixir guide to umbrella projects](https://hexdocs.pm/elixir/main/dependencies-and-umbrella-projects.html) describes child applications that share one repository, build directory, configuration, dependency directory, and lock file. Mix can compile and test them together, while dependencies between applications remain explicit. The guide also calls out the limit: shared configuration and dependencies mean umbrella children are not fully independent projects.

That balance suited Formfix and KMS.

During early development, both applications ran in the same BEAM and shared the same memory space. A call from Formfix to KMS was an ordinary Elixir function call. There was no serialization, network timeout, API token, service discovery, or second deployment to manage. I could change both sides of the interface in one branch and run the whole suite from the umbrella root.

This felt like writing a monolith because it was one operational unit. It did not feel like one undifferentiated application. KMS had its own Mix project, application callback, schemas, repository, migrations, tests, and dependency direction. Formfix consumed a KMS API instead of reaching into cryptographic helpers scattered among client modules.

An umbrella does not enforce that discipline by itself. One child can still call another child's internals. Shared runtime configuration can create accidental coupling. A direct in-memory call also means both applications share one trust and failure boundary. The package structure gives developers a place to draw a line; reviews and tests still have to defend it.

For this project, that was enough to begin. I could keep the development speed of a monolith while giving a security-sensitive requirement room to become a subsystem.

##### The first small step was a separate OTP application

The KMS child started with its own supervision and persistence. This was more than a namespace. OTP applications have startup behavior, dependencies, configuration, and a lifecycle. KMS could be tested and reasoned about as one unit even while Formfix started it in the same release.

This boundary answered a design question before it became an operational question. Key management changed for reasons unrelated to the client project's business rules. It needed to evolve its payload format, root-key handling, authorization, and recovery. Formfix needed stable encrypt and decrypt behavior. A public module between two OTP applications gave both sides a contract.

The setup was still cheap. No team had to operate another service. No user request could fail because a second pod was temporarily unavailable. There was one deployment and one place to inspect logs. Starting KMS alongside Formfix was the correct trade-off while both shared an operator and trust boundary.

##### The second small step was a separate release

An umbrella can produce more than one release. [`mix release`](https://hexdocs.pm/mix/Mix.Tasks.Release.html) accepts named release configurations and an explicit list of applications. The Formfix umbrella eventually defined a Formfix release, a KMS release, and a combined umbrella release.

That meant the same repository could support two deployment shapes. For a compact installation, Formfix and KMS could run together. Where operations or security required more isolation, the applications could be packaged into separate releases and run in separate pods or on separate servers.

This step is small in code organization because the OTP boundary already exists. It is not free in operations. Two BEAM instances no longer share memory, so direct calls need a network-facing client and server. Configuration and credentials must be separated. The remote path needs TLS or trusted private networking, authentication, timeouts, retries, monitoring, and a decision about what the application should do when KMS is unavailable.

KMS now makes that choice explicit. Embedded mode uses a local client in the same BEAM. Remote mode runs a KMS authority process with its HTTP API enabled, while application hosts use a remote client. The public functions remain recognizably similar, but the trust and availability boundaries change.

This gradual move matters more to me than claiming that distributed systems are easy. The umbrella did not remove network complexity. It delayed that complexity until there was a reason to pay for it, while preserving the application boundary needed to introduce it cleanly.

##### The third small step was a project of its own

Once KMS had its own concepts, release, documentation needs, and uses beyond one client, keeping it inside the Formfix repository stopped helping.

Extracting it was mostly an exercise in removing assumptions. KMS needed its own Mix project and lock file, independent configuration, public documentation, CI, examples, and a release process. Formfix-specific dependencies and shared helpers had to disappear. The client project then consumed KMS like any other dependency rather than through `in_umbrella: true`.

Elixir's own umbrella guide describes this path directly: move the application outside `apps/`, remove umbrella-specific project paths, and depend on it through a path, Git repository, or package. The framework does not make API design automatic, but it avoids treating extraction as a rewrite.

That is how [parlant-co/kms](https://gitlab.com/parlant-co/kms) came to be. It is now one OTP application with an optional HTTP API, its own persistence choices, root master key providers, authorization features, operational documentation, and a Phoenix example. The repository history still carries its umbrella origin, but the project no longer requires Formfix to make sense.

I like the progression because each stage remained useful:

1. KMS began as a separate OTP application in the same umbrella and same BEAM.
2. Named releases allowed KMS and the client application to run together or in separate pods.
3. Local and remote clients turned the process boundary into a configuration choice.
4. Extraction gave KMS a separate repository, release cycle, and public audience.

There was no weekend in which a monolith was replaced by microservices. Each boundary became operational only after the code had already learned to respect it.

##### KMS now supports several useful stopping points

Not every application needs the final stage. The current [KMS use-case guide](https://gitlab.com/parlant-co/kms/-/blob/main/docs/use_cases.md) treats deployment as a choice rather than a maturity ladder.

**Crypto-only mode** exposes AES-GCM helpers when an application needs sound primitives but does not need KMS storage, supervision, sessions, or authorization. The application owns its keys and may decrypt whenever its own policy allows.

**Application-wide and field-level encryption** fit applications that want ciphertext in a database while keeping ordinary Ecto schemas and changesets. A virtual plaintext field can be encrypted into a persisted ciphertext field before insertion. Aliases group key policy, and AAD binds ciphertext to its row or field context.

**Embedded KMS** starts the KMS OTP application with a Phoenix or Elixir application. This is closest to the original Formfix arrangement. It offers managed key-encryption keys and root-key wrapping without adding a network dependency. The application and KMS still share one host and trust boundary.

**Standalone or remote KMS** runs the authority separately. Application hosts no longer need direct access to the KMS database or local root-key file. This can support several applications, centralized operations, and a stronger credential boundary. It also adds a critical network dependency and must be operated accordingly. The [local versus remote guide](https://gitlab.com/parlant-co/kms/-/blob/main/docs/local-vs-remote.md) makes those trade-offs explicit.

**Multi-user encryption** adds principals, sessions, factors, and bindings when KMS itself must decide who may use a key. This differs from an application that authorizes a request and then calls a trusted embedded KMS.

KMS also includes patterns for caller-supplied high-entropy secrets, external root master key providers, and fallback recovery. These options solve different threat models. They are not features every installation should enable.

The smallest fitting mode is usually the best one. If the running application is authorized to decrypt everything, moving KMS to another pod does not automatically protect data from a compromised application process. If database-only disclosure is the concern, embedded encryption with a root key outside that database may be enough. If application hosts must not possess KMS storage or root-key credentials, a remote authority creates a meaningful boundary.

##### Formfix proved the boundary before KMS became a product

KMS was not born from a plan to publish a generic security platform. It was born because Formfix had an important technical requirement that did not belong among the client project's other business logic.

The Phoenix umbrella made the responsible first step cheap. I could isolate key management without slowing the product down with another service. The same-memory implementation then proved the encryption API against a real application. Separate releases proved that the OTP boundary could become a deployment boundary. The final extraction proved that KMS could stand on its own.

That sequence is why I remain fond of umbrellas. They let developers work with monolith ergonomics while preserving an honest path toward stronger separation. The path is gradual, and every step asks for more operational discipline. None requires throwing away the previous architecture.

Formfix gave KMS its first real reason to exist. It tested the encrypt-first assumptions, exposed missing pieces, and kept the design grounded in application use rather than cryptographic abstraction. KMS could become an open-source project because the client work had already shown where its boundary belonged.

##### Try KMS in your application

[KMS is now available on Hex](https://hex.pm/packages/kms):

```elixir
defp deps do
  [
    {:kms, "~> 0.1.0"}
  ]
end
```

Start with the embedded library when your application and KMS share one trust boundary. Run the optional HTTP API as a separate service when application hosts must not hold KMS storage or root-key credentials. The [KMS documentation](https://hexdocs.pm/kms) covers both setups. Install the package, run the smallest example that fits your application, and open an issue in the [GitLab repository](https://gitlab.com/parlant-co/kms) if the API or documentation gets in your way.

### [Elixir makes enterprise integration almost comically easy](https://www.lukasrieder.com/blog/elixir-makes-enterprise-integration-almost-comically-easy/)

Plantronic handled durable SAP imports with Elixir, Oban, Ecto, and its existing PostgreSQL database inside the same application that owned the planning rules.

I like Elixir for business software because the application does not develop a split personality as soon as some work takes longer than an HTTP request.

Plantronic was a Phoenix application with ordinary business concerns: people, teams, calendars, permissions, planning dates, and reports. Then it also had to import exports from SAP. The files arrived out of band, had several shapes, used real-world encodings, and encoded years of organizational rules in details that looked accidental until one of them was wrong.

A common response would be to introduce an import service, a separate queue, and perhaps an integration platform to connect both systems. Plantronic already had Elixir, PostgreSQL, and its domain model. Adding [Oban](https://hexdocs.pm/oban/Oban.html) turned that same application into a durable job processor. No new broker or cache was required. There was no second service to deploy and no API between the import code and the planning rules.

That is the part I find almost comical. Enterprise requirements stayed weird, but the operational architecture remained one application and one database.

##### Background work is part of the application

The SAP import did not run in a controller request or a detached shell script. Oban ran as a child of Plantronic's OTP application, next to the Ecto repositories, Phoenix PubSub, caches, and the web endpoint. Elixir's [Supervisor](https://hexdocs.pm/elixir/Supervisor.html) gives those processes a common lifecycle and restarts children according to an explicit policy.

Oban stored jobs in the PostgreSQL database Plantronic already needed. A deployment restart could interrupt a process without erasing the job that described its work. Operators could inspect job state in the same system that held the application data. Database backups covered both business records and pending work.

PostgreSQL is still infrastructure. The useful point is that Plantronic did not gain another kind of infrastructure solely because work had to continue in the background. There was no Redis queue, message broker, queue-specific persistence plan, or separate worker release. The same release could serve LiveView screens and process imports.

The amount of glue was small. One worker scanned a configured directory for CSV files. It scheduled itself again, sorted unseen files, and built one import job for each file. An `Ecto.Multi` inserted the Oban jobs and the matching import-log records in one transaction. Either both became visible or neither did.

In simplified form, the important part looked like this:

```elixir
Ecto.Multi.new()
|> Oban.insert_all(:jobs, fn changes -> build_jobs(changes) end)
|> Ecto.Multi.insert_all(:logs, ImportEvent, fn changes -> build_logs(changes) end)
|> Repo.transaction()
```

That transaction removed an awkward failure mode. The scanner could not record a file as seen while losing the job that was supposed to import it. The next scheduled scan provided another chance if creating the batch failed. Oban handled persistence and execution; Ecto kept the enqueue operation consistent with Plantronic's own records.

Oban also made execution policy ordinary application configuration. Queue concurrency, scheduled execution, uniqueness, cancellation, and retry limits live beside the worker. The first version deliberately allowed only one attempt while import semantics were still being established. Durable jobs do not remove the need to decide whether repeating a partially understood external operation is safe.

##### The worker is only a durable shell

A job queue does not solve integration by itself. It gives the work a reliable place to run. The valuable part was what happened inside each import job.

The worker detected the file encoding, opened a database transaction, created a temporary table, and streamed the CSV into PostgreSQL with `COPY`. Every source column entered that temporary table as text. The next statement transformed those strings into Plantronic's typed records. The temporary table disappeared when the transaction ended.

This approach kept the raw file shape local to one transaction. The rest of Plantronic never needed to know column positions or SAP date conventions. It only saw employees, availability, absences, teams, and organization-unit mappings.

The [Ecto transaction](https://hexdocs.pm/ecto/Ecto.Repo.html#c:transaction/2) also gave each file a clear outcome. If encoding detection, CSV loading, date conversion, or a domain update failed, the transaction rolled back. A half-imported personnel file would be worse than a failed job because planners could mistake incomplete data for current data.

Elixir made it comfortable to combine different tools at the point where each was strongest. File discovery and streaming used the standard library. Oban owned durable execution. Ecto owned database access and transactions. PostgreSQL handled bulk loading, date expansion, joins, and upserts. Phoenix LiveView provided an admin screen for the organization-unit mappings that affected imports.

None of these parts needed a remote call to the others. They shared one supervised application and one domain model.

##### Weird requirements became domain rules

The source files were not clean tables waiting to be copied. They represented several related feeds with different meanings. Master data described employees and organization units. Calendar data described working hours. Absence ranges had to become one record per affected day. Their filenames carried timestamps, and processing order mattered because a later export could correct an earlier one.

The importer also had to account for less obvious cases:

- Employee references appeared both padded and unpadded, so the importer normalized them before matching.
- Decimal hours used commas, while Plantronic stored numeric values.
- An end date in the year 9999 meant that employment had no known end date.
- A later master-data export could omit an email address because of a source problem, so an empty value must not erase a useful existing value.
- Organization units could suggest a team, but an existing local team assignment took precedence.
- Absence exports described date ranges, while planning needed daily availability.

Those are not transport concerns. Each one changes what a planner sees or what the scheduling model calculates. Putting them in a generic ETL service would not make them less domain-specific. It would only move Plantronic's business rules into another repository and force two systems to agree on their meaning.

Keeping the import inside Plantronic made the rules testable against the actual schemas. The worker tests created representative files, performed Oban jobs, and asserted on employees, availability, absences, and team assignments. They checked ordering across multiple files, preservation of existing values, normalized identifiers, expanded date ranges, and organization mappings.

This is what I mean by a tight integration. The import was not tightly coupled to controllers or templates. It was close to the domain operations whose correctness it determined.

##### Elixir removes the pressure to split too early

Long-running work often pushes a web application toward additional services before the domain warrants them. A request process should not spend an hour importing a file, but that does not imply the import needs another language, deployment, datastore, or team boundary.

Elixir already distinguishes processes inside one runtime. A Phoenix request, a LiveView connection, a scheduled scanner, and an Oban worker can have separate lifecycles without becoming separate applications. OTP supervision handles process lifecycle. Oban adds durable job semantics. Ecto and PostgreSQL provide transactions around the state those jobs change.

This arrangement has practical benefits for a small product team. One release contains the web interface and workers. One migration can update the domain schema and the import path together. One test suite exercises both. Logging and telemetry use the application's existing setup. A developer can follow a value from a source fixture through the worker into the same context functions and tables used by the UI.

A separate service can become useful when ownership, scaling, security, or deployment requirements truly differ. The SAP import had none of those boundaries at first. Splitting it would have created a distributed system without removing any business complexity.

##### Boundaries still matter

Keeping the import in one application does not mean letting the source format spread everywhere.

Plantronic still needed a clear inbound boundary. The received file had to remain diagnosable. Encoding and column handling belonged near ingestion. A temporary table represented the external shape. Explicit SQL transformed it into local concepts. From that point onward, routing, permissions, and reporting used Plantronic's model rather than file columns.

The outbound side needed similar care. Public database views and XML endpoints exposed selected planning data to other consumers. Their columns, ordering, filters, and treatment of missing values became contracts. "Public" inside an organization did not mean unrestricted or accidental.

The architecture mattered, but it was not the most interesting result. The interesting result was how little ceremony Elixir required to implement it. A durable background process, an admin interface, transactions, bulk data handling, telemetry, and domain tests all fit into the existing application.

SAP did not become simple. Its exports still carried odd encodings, special dates, changing organization structures, and incomplete values. Elixir let Plantronic spend its complexity budget on those actual rules instead of queue infrastructure and service coordination.

That is why I consider Elixir such a strong platform for business and web applications. It serves the web request, runs the background job, supervises both, and keeps the work close to the domain that gives it meaning.

### [Permissions belong in queries, not only menus](https://www.lukasrieder.com/blog/permissions-belong-in-queries-not-only-menus/)

Why Plantronic's planner and team rules had to constrain data access below the interface.

A hidden menu item is not authorization.

During July 2025, Plantronic's planner roles, team relationships, employee filters, task views, and public database views grew together. That combination forced a useful design question: where does permission actually live?

The interface must reflect what a person may do. But if the underlying query returns every task and the template merely hides some rows or buttons, the policy is cosmetic. Another screen, export, LiveView event, or API can expose the same data.

##### Scope data before presenting it

The safer pattern starts with a scoped query. Given an authenticated person and a purpose, return only the tasks, employees, teams, or planning rows that person is allowed to inspect.

The UI then renders an already-authorized result. Buttons can still communicate capabilities, but they are not the enforcement point. Mutations apply the policy again because visibility and permission to change are not identical.

This also improves correctness outside security. Team-scoped employee search returns relevant choices. Planner views carry less unrelated data. Tests can assert exact result sets instead of searching rendered markup for an absent link.

##### Roles are rarely enough

A global role such as "planner" is convenient, but real access often depends on relations. A person may plan for one team, observe another, and have no reason to see a third. A primary-planner relation can be more meaningful than another boolean column on the user.

Relations make policy slightly more work to express. They also make it explainable: this user can see this task because the task belongs to a team they plan for. That explanation is useful in code review, tests, support, and audits.

Public database views need the same attention. A view can become an integration contract that bypasses application-level checks. Its columns and row scope should be treated as deliberate exposure, not a convenient shortcut around the application.

##### Authorization is a property of every path

Plantronic had screens, searches, reports, imports, and external views. No single controller plug could secure all of them by itself. The durable approach was to keep policy near domain queries and operations, then exercise it through each delivery path.

A scoped query is less visible than a disabled menu item, but it is the part that blocks access.

Whenever I review authorization now, I ask two separate questions: does the interface clearly show what this person can do, and can any code path retrieve or change more than that? Good software needs both answers. Only the second one is the security boundary.

### [I made translations editable without replacing Gettext](https://www.lukasrieder.com/blog/i-made-translations-editable-without-replacing-gettext/)

Plantronic needed domain labels that could change without a release. I kept compiled Gettext catalogs for application copy, added a database fallback for operational vocabulary, and used ETS and PubSub to make edits visible.

The first translations I needed to make editable were not button labels or paragraphs. They were field names arriving from another system.

Plantronic could receive task metadata with identifiers such as `project_manager` or `machine_type`. The metadata renderer knew where to place each field, but it did not know every field at compile time. Showing those identifiers directly would expose integration vocabulary to planners. Turning each possible value into a template branch would defeat the point of accepting flexible metadata.

At the same time, Plantronic was gaining conventional interface localization. Headings, buttons, validation errors, and process labels belonged in Gettext catalogs. They changed with the code, could be extracted from source, reviewed in `.po` files, and shipped with a release.

I did not want two translation APIs: Gettext for application copy and a custom database lookup whenever a string happened to come from metadata. Call sites should be able to ask for a translation without knowing where it was stored.

The solution was smaller than replacing Gettext with a content-management system. I kept compiled catalogs as the primary source and used Gettext's missing-translation callbacks as an extension point. If a locale file knew the message, Gettext returned it normally. Only when the catalog had no entry did Plantronic look in a database-backed cache.

That ordering is the most important part of the design.

##### There were two kinds of vocabulary

Static application text and operational labels may look identical on a screen, but they have different owners and change for different reasons.

A phrase such as "Change Password" belongs to the application. Its location in source matters, translators benefit from surrounding context, and a wording change may accompany changed behavior. Keeping it in normal Gettext files makes the translation part of the same review and release as the feature.

A metadata key is different. It can originate in a database view or external API and may not appear as a literal anywhere in the Elixir source. Gettext extraction cannot discover every value that another system might send. A domain expert may also need to improve the displayed term without waiting for a software release.

Plantronic normalized these source labels into stable identifiers and passed them through the same helper used for other translations:

```elixir
t(metadata["field_name"])
```

The helper called Gettext rather than querying a `translations` table directly. That preserved one vocabulary interface for templates while leaving storage policy inside the translation backend.

The effective lookup path was:

```text
compiled Gettext catalog
  → database translation cache when the message is missing
  → original message identifier as the final fallback
```

This was not an override mechanism. An administrator could add a translation for dynamic user content, but could not silently replace a translation already compiled into the application. Correcting ordinary interface copy still went through the repository and a release. The administration screen eventually described its contents more precisely as "additional translations of user content."

I like that boundary in hindsight. Editable text is useful, but making every word in an application mutable at runtime creates a second deployment system with weaker review.

##### The database used Gettext's vocabulary

The translation table did not invent a simpler key-value convention. Its columns followed Gettext concepts: locale, domain, message context, singular message identifier, plural message identifier, translated string, plural strings, and a comment.

A unique index covered the identifying fields, expressing the intent that one row should answer a given lookup. There was a subtle hole: context and plural identifiers were nullable, and ordinary PostgreSQL uniqueness treats null values as distinct. The index therefore did not prevent every duplicate identity unless those nulls were normalized or declared `NULLS NOT DISTINCT`. The Ecto changeset required the basic identity and translation values before an administrator could save a row.

Keeping domains and context may seem excessive for translating metadata labels. It avoided narrowing the extension so far that it stopped behaving like Gettext. The missing-translation callback already received those values, and storing them preserved the lookup semantics callers expected.

The [Gettext backend documentation](https://hexdocs.pm/gettext/Gettext.Backend.html) defines callbacks for singular and plural messages that are absent from a locale. That is exactly where Plantronic connected the database layer. The backend passed the locale, domain, context, message identifiers, count, and interpolation bindings to the cache-backed fallback.

Bindings still went through Gettext's runtime interpolation after lookup. A database translation such as `Hallo %{name}!` therefore behaved like the same message from a `.po` file. The fallback to the original message identifier also remained interpolated, so a missing translation did not leave raw placeholders on the page.

This reuse mattered more than the table itself. Templates continued using `gettext`, `ngettext`, or a small dynamic helper. They did not acquire database concerns.

##### Locale selection had to work for requests and LiveView

Adding translations is only useful if every rendering process agrees on the active locale.

Plantronic configured an explicit default and a short list of allowed locales. A Plug accepted a valid locale parameter, otherwise restored a saved cookie, and finally fell back to the default. The chosen locale was written to the response cookie and session.

LiveView needed the same policy during mount. Gettext locale state is associated with the process doing the translation, so setting it in the initial HTTP request was not enough for a long-lived LiveView process. An `on_mount` hook restored the locale from parameters or session before rendering.

The user settings page exposed the allowed locales and made the persistence model clear: the choice belonged to that browser. It was not yet an account preference synchronized across devices.

This part of localization is easy to overlook because the visible feature is a language selector. The actual contract spans a request process, session, cookie, and LiveView process. Without one shared policy, the initial page and subsequent live updates can render in different languages.

##### The first cache made every read visit a process

Database lookup on every translated metadata label would have been wasteful. A task page can render many labels, and translations change rarely compared with how often they are read.

My first implementation loaded every translation into a map held by an Agent. The key contained the same five-part identity used by Gettext:

```elixir
{locale, domain, context, msgid, msgid_plural}
```

The value held the singular and plural translated forms. On a cache miss, the Agent version could query the database directly. A full refresh replaced the map with a new snapshot.

This worked, and the tests made the snapshot behavior explicit. A direct database update did not change the rendered translation until the cache was refreshed.

It was not a good final read path. `Agent.get/2` sends every lookup through one process. Translation reads are independent and overwhelmingly concurrent; serializing them through an Agent added coordination where none was needed.

I replaced the Agent with a GenServer-owned ETS table shortly afterward. The GenServer created the named table, loaded it, and handled refresh commands. Callers read ETS directly without sending a message to the owner.

The [ETS documentation](https://www.erlang.org/doc/apps/stdlib/ets.html) describes `set` lookup as constant time regardless of table size and notes that the creating process owns the table. That split suited the workload: one supervised process managed lifecycle and replacement, while request and LiveView processes performed direct reads.

A missing table or lookup failure returned the original message identifiers rather than crashing page rendering. Startup later tolerated an unavailable translation query as well. The cache made labels readable, but its failure could not prevent the application from starting or a task page from loading.

##### The administration screen exposed an invalidation gap

The first ETS implementation made reads fast, but it weakened the promise made by the administration screen.

Creating or editing a translation updated PostgreSQL. It did not automatically rebuild ETS. Tests called `update_all/0` explicitly, but the normal context operations did not. An administrator could save a correction successfully and continue seeing the old value until some other refresh occurred.

That is the familiar half-finished state of a cache: the read path is optimized before the write path has a complete invalidation story.

I later closed the gap with Phoenix PubSub. Successful translation inserts and updates broadcast an event on a `translations` topic. The cache process subscribed to that topic and rebuilt its local ETS table after receiving an event. Deletion needed its own event shape and two follow-up fixes before it followed the same path.

[Phoenix PubSub](https://hexdocs.pm/phoenix_pubsub/Phoenix.PubSub.html) provides the subscribe-and-broadcast interface and can distribute messages across connected Elixir nodes. Each Plantronic node could therefore keep a local ETS read path while hearing about edits made through another process or node.

The implementation rebuilt the complete table after every change. For a small translation set and rare administrative writes, that was easier to reason about than updating individual keys correctly across create, identity-changing update, and delete operations.

It still has a trade-off. The table is cleared before it is repopulated, so a concurrent read can briefly fall back to the message identifier. A larger or more frequently edited catalog would justify building a replacement table and swapping it, versioning snapshots, or applying precise key updates. The simple rebuild matched the actual scale, but the fallback behavior should be understood rather than treated as magic.

##### Editable translations carry more risk than their size suggests

A translation row is small, but it can affect every page that asks for that message. The administration feature therefore sits closer to code than ordinary page content.

Interpolation placeholders are part of the contract. If the source message contains `%{name}`, a translation that drops or renames the binding can produce missing-binding behavior. Long text can damage a layout designed around a short label. Terminology changes can make training material and support instructions inconsistent with the application. A mistaken domain or context can make a valid translation appear to be missing.

Restricting the screen to administrators was necessary, but permissions are only one part of ownership. For important operational vocabulary I would also want clear change history, preview in context, and validation that translated placeholders match the source message.

The compiled-catalog-first rule limited the blast radius. Runtime edits applied to additional vocabulary rather than every static sentence in the interface. If the cache was absent or a row could not be found, users saw a stable identifier instead of an exception.

##### Plural rules were where the abstraction leaked

The weakest part of the implementation was plural selection.

The database schema could store a list of plural translations, and the missing-plural callback received `n`. The fallback code selected a list element using the count and returned the first element again for larger values. That happened to satisfy the narrow tests, but it was not Gettext plural semantics.

Plural forms are selected by a locale-specific formula, not by treating the quantity as an array index. The [GNU Gettext manual](https://www.gnu.org/software/gettext/manual/html_node/Plural-forms.html) explains that languages differ both in how plurals are formed and in how many plural forms they require. Even languages with two forms generally select singular for one and plural for every other relevant count; they do not have separate array entries for two, three, four, and so on.

The compiled `.po` path already had correct plural machinery. By implementing database plural selection separately, I created exactly the kind of second translation system I had tried to avoid.

I would change that design before relying on editable plural messages. The database should store forms in a representation compatible with the locale's plural module, and Gettext's plural-rule implementation should choose the form. At minimum, tests should cover counts beyond one and two for every allowed locale and assert linguistically correct output rather than codifying the current indexing behavior.

This does not invalidate the singular operational-label use case that motivated the feature. It does show where an extension point stops being a thin adapter and starts reimplementing a mature library.

##### What I would keep

I would keep the separation between compiled application translations and editable operational vocabulary.

Gettext remains the public API and the first source of truth. Dynamic identifiers that extraction cannot see have a controlled fallback. The database uses Gettext's identity fields instead of inventing a parallel naming convention. ETS makes the common read path local and cheap. A supervised process owns cache lifecycle, and PubSub connects successful writes to refreshes on running nodes.

I would improve several details: normalize and validate placeholders, delegate plural selection, make cache replacement atomic, add stronger audit history, and test invalidation through the same context functions used by the administration screen.

"Editable translations" turned out to be a cluster of responsibilities. Vocabulary ownership and precedence determined which text could change. Locale propagation, fallback, and interpolation shaped each lookup. Cache invalidation, multi-node visibility, permissions, and plural rules appeared after the table and form were already working.

Plantronic avoided the most confusing outcome because database translations never silently displaced the catalogs shipped with the application. They filled a specific gap: operational labels that existed as data before they existed as source code.

That made translations editable where runtime editing was useful, while Gettext continued to define what translation meant.

### [Why I turned ChatGPT into a Unix command](https://www.lukasrieder.com/blog/why-i-turned-chatgpt-into-a-unix-command/)

When GPT-3.5 became useful for reshaping code and structured text, a browser chat no longer felt like the right interface. I built Q, a small streaming wrapper that still lives in my dotfiles.

I subscribed to ChatGPT not long after it became available, and I still remember realizing that GPT-3.5 could write code. The code was rarely great and sometimes impressively wrong, but it could assemble a plausible function and work with programming syntax well enough to be useful.

What interested me even more was its ability to reshape structured text. I could give it a directory listing and ask for a YAML document, hand it a Gettext catalog and ask it to draft the missing translations, or provide a set of Terraform declarations and ask it to extract candidate variable definitions. These were tasks I understood well enough to describe and verify, but which were often tedious to do by hand.

At that point the browser started to feel like the wrong interface. I work primarily in Vim and spend much of the day in a terminal. Source code, configuration, diffs, logs, SQL, JSON, YAML, and command output are already text streams there. Copying one of those streams into a chat window, waiting for a response, and copying it back broke an otherwise simple workflow.

I wanted to send text to the model in the same way I sent text to any other command. That led to `Q`, a small shell wrapper around the OpenAI API. It accepted a prompt or standard input and streamed the response back to standard output. The name was short enough that using it never felt like switching into a separate AI application, and the command has remained in my dotfiles ever since.

##### The useful surprise was transformation

Code generation attracted most of the attention around ChatGPT, but transformation was where I first found dependable everyday uses for it.

A regular expression is ideal when the input and output rules are exact. If I need the same operation repeatedly, I would rather write a small script and test it. There is another class of task, however, where the input is irregular, the desired result is easy to show, and the work may only happen once. Writing a parser for that can take longer than producing and reviewing a model-generated draft.

A directory listing is a simple example. I might want a rough service inventory from a tree that grew organically:

```sh
{
  printf '%s\n' 'Convert this directory tree into a YAML service inventory.'
  tree services
} | Q -
```

For a translation catalog, I could combine an instruction with the file and write the result somewhere temporary:

```sh
{
  printf '%s\n' 'Translate empty msgstr values into German. Preserve PO syntax.'
  cat locale/messages.po
} | Q - > /tmp/messages.de.po

msgfmt --check /tmp/messages.de.po
```

Terraform declarations worked in much the same way:

```sh
{
  printf '%s\n' 'Extract reusable inputs as Terraform variable declarations.'
  cat main.tf
} | Q - > /tmp/variables.tf
```

None of these examples ends when `Q` finishes. I still parse or format the output, compare record counts, inspect the diff, and read what changed. The model saves the mechanical first pass; the existing tools around the file tell me whether the result is structurally acceptable.

That combination made the early models much more useful than their reliability alone would suggest. They did not have to be right without supervision. They had to save enough work that reviewing their answer was cheaper than creating it manually.

##### A small interface suited the terminal

The first version of `Q` was a short Bash script built from tools I already used. It accepted command-line arguments, read standard input when called with `-`, and opened a small interactive prompt when neither was present. `curl` sent the request, while `jq` built the JSON payload and extracted text from the response.

Its interface was deliberately ordinary:

```text
arguments or stdin → Q → stdout
```

Because the answer went to standard output, I could read it directly, redirect it to a temporary file, or pass it into another command. The wrapper did not need to know whether its input was a diff, a configuration file, an error message, or a half-remembered question about `ffmpeg`.

Over time, quick questions became at least as common as transformations. I use a lot of command-line tools whose more obscure flags refuse to stay in my memory. Asking `Q` how to scale a video without losing its audio, what an unfamiliar error means, or which variation of `find` I need is often faster than leaving the terminal to search for the same fragment of documentation again.

Some questions are funny mainly because of how specific terminal problems become: turn this wall of output into a table; explain why this regular expression matches everything except the thing I need; reconstruct the `ffmpeg` command that does not rotate a portrait video into abstract art. A one-line command is a good interface for that kind of interruption because it does not ask me to create a conversation around it.

I do not pipe the resulting shell commands directly into `sh`. I read them first. There is a large difference between asking for a command and granting probabilistic output authority over the machine, even if Unix makes both equally easy to type.

##### Streaming made the wrapper pleasant to use

A wrapper like this could wait for the complete response before printing anything, but that feels unnecessarily slow in an interactive terminal. `Q` requested streamed responses from its first version. `curl --no-buffer` exposed events as they arrived, and a small `sed` and `jq --unbuffered` pipeline removed the event framing and printed each text delta.

The immediate feedback changed the feel of the tool. Text started appearing shortly after I pressed Return, in the same terminal where the prompt originated. There was no separate window holding the canonical copy of the answer and no pause followed by a large block of text.

Streaming also made a bad response easier to notice. If a request for YAML began with an essay about YAML, I could stop it without waiting for the model to finish the introduction. Most of the time, though, watching a useful transformation appear line by line was simply satisfying.

I liked that the implementation remained mundane. Bash, `curl`, `jq`, and `sed` were enough to turn a remote model into something that behaved like a local text command. Later versions added a little `grep` to select the relevant response events, but the basic approach stayed the same.

##### The implementation followed the APIs, while the command stayed familiar

The history of `Q` follows frequent model and API changes. Its first version used the Chat Completions API and selected GPT-3.5 directly in the script. I soon moved the model name into an environment variable, then did the same for the API base URL. That let me update defaults or override them for one invocation without changing callers.

The default model moved through GPT-4 previews, GPT-4 Turbo, GPT-4o, and later GPT-5 versions. Those changes improved what the command could do, but they did not require a new way to use it.

Moving from Chat Completions to the Responses API was a larger internal change. The message array became separate instructions and input, token settings changed, and streamed output arrived through different event types. I updated the request and response pipeline and explicitly disabled response storage. From the shell, `something | Q -` continued to work as before.

When reasoning effort became configurable, I exposed it as another optional environment variable. More recently, I changed input handling so piped content flows into the `jq` process that constructs the request instead of first being captured in a shell variable. That handles multiline input more cleanly and better matches the pipeline-oriented interface.

The wrapper has lasted because it keeps those implementation changes behind a command shaped around my workflow. The useful abstraction was never a particular model or endpoint. It was being able to send text and receive text without leaving the terminal.

##### The Ruby version clicked as it streamed

Alongside the shell script, I experimented with a Ruby implementation called `Qruby`. It used HTTPX for streaming, JSON for parsing, and Ruby2D for one feature the Bash version conspicuously lacked: sound effects.

I loaded a short click sample and played it whenever a response chunk was printed. A concise answer produced a pleasant mechanical rhythm. A verbose one sounded increasingly like a Geiger counter attached to a documentation generator. It was unnecessary and oddly satisfying.

The sound was only one part of the experiment. The Ruby version grew a reusable query function, configurable model and system instructions, optional conversation history, better input detection, and more careful handling of event fragments. Those pieces were easier to express as Ruby methods than as additions to a shell pipeline.

`Qruby` also required HTTPX, Ruby2D, a suitable Ruby interpreter, and a click file from my music directory. The Bash version depended on tools that were already part of my environment, so it remained the command I actually kept using. I still like that both versions live in the history: one became infrastructure, while the other records the period when making streamed model output click seemed like a perfectly reasonable evening project.

##### Plausible structure still needs scrutiny

Using the models early made their strengths obvious, but it also exposed their weaknesses quickly. They were very good at producing output that looked right, including output that was wrong in ways a parser would not catch.

A YAML document can be valid while omitting an input record. A `.po` file can pass `msgfmt --check` while using the wrong technical term. Terraform can format successfully while changing a default or dropping an important validation. A suggested shell command can use a familiar-looking flag with different behavior on my operating system.

I therefore use `Q` for work with clear review points. Depending on the task, that means preserving and counting identifiers, generating into a temporary file, parsing the result with the real parser, running formatters and validators, inspecting a diff, or executing tests. I also keep the request narrow enough that I can understand the answer.

Syntax checks are only one layer. A person still has to review the meaning of a translation, and compiling generated code does not establish that it behaves correctly. When a command matters, the manual remains more authoritative than a fluent recollection of it.

There is also a data boundary. I do not send secrets, credentials, private client material, `.env` files, or Terraform state to an external model API. A convenient pipe makes it especially important to remember where the bytes are going.

`Q` does not enforce all of this for me. Its limited role helps: it returns text and stops. I decide what becomes a file, a command, or a committed change.

##### A tool from before agents that still has a place

Coding agents later became a much larger part of my workflow. They can inspect repositories, edit files, run tests, react to failures, and maintain context across a task. `Q` does none of those things. It receives one request and returns one response without knowing whether the result worked.

That limitation is also why it remains useful. Not every question needs a repository, a tool loop, or delegated implementation. Sometimes I have text in the terminal and want help understanding or reshaping it. For that job, a small pipe is still faster and clearer than starting a larger workflow.

My early excitement about language models was always mixed with doubt about their output. `Q` found a useful place between those reactions. It helped me work with language and structure, while execution, validation, and judgment stayed with the tools and habits I already had.

I still invoke it from time to time. The models and APIs behind it have changed repeatedly, but the reason for keeping it has not: it fits the way I work with text.

### [I added task metadata. The next field still got a column.](https://www.lukasrieder.com/blog/i-added-task-metadata-the-next-field-still-got-a-column/)

Plantronic needed to display details from two systems without importing their entire schemas. JSONB made that flexible, but behavior rather than convenience decided what became part of the task model.

Plantronic's task page needed information that did not belong to Plantronic.

The task itself had a deliberate model: requested dates and effort, planning dates, process state, team assignment, and identifiers used to exchange updates with the project-management system. Planners also needed context from the surrounding project, such as production details and organizational contacts. That information lived in another part of the database, followed a different lifecycle, and was still evolving.

I could have copied every useful source field into the task table. That would have made the task schema an incomplete replica of another system and required a migration whenever its display needs changed. I could also have exposed the source tables directly to the LiveView templates, coupling the interface to a schema Plantronic did not own.

Instead, I added metadata. This sounds like the moment where a carefully modeled application begins hiding everything in JSON, but that was exactly what I wanted to avoid. The metadata had one narrow purpose: carry additional information across a system boundary and place it on the task page. It was not allowed to replace the fields that controlled Plantronic's behavior.

The distinction became concrete almost immediately. Even with task metadata available, I still added an ordinary column for a new routing attribute. Flexibility had become available, but it did not make explicit modeling obsolete.

##### The feature was larger than a JSONB column

The final feature commit added more than a thousand lines. Most of those lines were not the JSONB field itself. They described how the data entered Plantronic, how project information was read safely, how the two sources were combined, how metadata appeared in the interface, and how all of this could be tested without relying on a production database.

There were two sources of metadata.

The first came with a task through the existing external API. These values belonged to that task and were persisted with it. The XML input could contain one metadata element or several, so the API boundary normalized both forms into a list before Ecto cast them.

The second source was project data. Plantronic already knew the project's reference, but the descriptive details remained in the client-owned public schema. A SQL view selected the relevant project facts and represented them in the same metadata shape. When Plantronic loaded a task with metadata, a database function joined that current project information to the metadata stored on the task.

The result looked simple to the caller:

```text
current project context + task-owned metadata → task page
```

{% include diagram.html post="2025-06-12-i-added-task-metadata-the-next-field-still-got-a-column" diagram="task-metadata-flow" caption="Diagram: Project-owned context is translated before it crosses the schema boundary, then merged with task-owned metadata into one validated display shape." %}

The simplicity was created at the boundary rather than assumed throughout the application.

This resembles the [Content Enricher](https://www.enterpriseintegrationpatterns.com/patterns/messaging/DataEnricher.html) pattern. A message arrives without all the information its consumer needs, so a transformer uses an identifier from the message to retrieve additional data. Plantronic was not forwarding a message through an integration pipeline, but the structural move was the same: use the project reference to enrich the task for one consumer without pretending that the source system had sent everything.

##### Flexible names still had a schema

I did not store one arbitrary JSON object. Metadata was an ordered list of records with four fields:

```elixir
%{
  field_name: "some_source_field",
  field_value: "Some value",
  display: :top,
  order: 2
}
```

`field_name` and `field_value` carried the source detail. `display` selected one of two areas on the task page. `order` made placement deterministic within that area.

That shape was an Ecto embedded schema. Its changeset restricted `display` to known values, and a later validation required field names to use a predictable lowercase format. Ecto's documentation describes [embedded schemas](https://hexdocs.pm/ecto/Ecto.Schema.html) as data mappers that can cast and validate inputs even when the data does not have its own queryable table. That was a good fit here: each metadata item needed validation and serialization, but not its own identity or lifecycle.

PostgreSQL stored the list in a `jsonb` column. Its [JSON documentation](https://www.postgresql.org/docs/current/datatype-json.html) notes that `jsonb` is stored in a decomposed binary form and supports efficient processing and indexing. It also does not preserve the order of keys inside an object. I did not make object-key order carry interface meaning; ordering was an explicit integer in each record.

A list of records also allowed repeated field names. A project can have several people in one organizational role or several related production details. Turning the data into one JSON object keyed by field name would either discard repetitions or invent arrays for some fields and scalar values for others. The uniform record shape avoided that special case.

The schema was flexible about which display fields could arrive, but not about what a metadata item was. That is a modest constraint, yet it kept malformed presentation instructions out of the rest of the application.

##### The source schema stayed behind a boundary

The project data lived in a public PostgreSQL schema beside Plantronic's own schema. I added a second Ecto repository configuration for migrations in that namespace. Both repositories connected to the same database, but they had different schema prefixes and migration histories.

This mattered in two ways. It gave the source-facing views an explicit owner in the codebase, and it allowed tests to create a representative public schema instead of quietly depending on tables that existed only in the client environment.

Plantronic did not query a collection of source tables from every screen. One view translated the relevant fields into the metadata record shape. A function in Plantronic's schema then merged the view result with the task's persisted metadata.

The function also contained a failure boundary. If retrieving or combining project metadata failed, it returned the task's own metadata with a fallback marker instead of making the entire task unavailable. Planners could continue working with the information Plantronic owned even when optional context from the other schema could not be loaded.

Catching every database exception is a broad instrument, and I would not use it to hide failures in core task state. Here the asymmetry was intentional. Dates, effort, process state, and team assignment were necessary to operate the task. Project metadata enriched the page. Losing the latter should be observable, but it should not erase the former from the planner's screen.

This boundary also preserved ownership. Project facts could remain current in their source schema rather than being copied into every task and becoming stale. Task-specific metadata could remain attached to the task. They shared a display shape without being forced into the same storage lifecycle.

##### Metadata reached the interface, not the process model

The LiveView did not know every field that the source view might return. It iterated over metadata marked for the top section, then rendered the bottom section in the footer. An overflow component handled the case where the source supplied more context than fit comfortably on one line.

That was where generic rendering helped. Adding another descriptive project fact did not require a new task column, another template branch, or a deployment solely to place a label and value on the page. The source view determined the name, value, placement, and order within the constrained envelope.

Plantronic's own process concepts remained ordinary Ecto fields and associations. Requested and planned dates still had typed date validation. Effort still had numeric constraints. Process steps still had explicit transitions. Team assignment remained a relation. The metadata renderer could show supporting information, but no process transition depended on looking up a string key in a JSON array.

The new routing attribute is the clearest evidence of that rule. It could have been sent as one more metadata record. Instead, it became a nullable string column on the task, was accepted by the external changeset, and was deliberately excluded from later external updates along with other stable routing dimensions. The routing editor then offered only a short whitelist of explicit task attributes as match criteria.

Routing did not ask a generic question such as "does metadata contain this key and value?" It matched properties that Plantronic had chosen to understand.

That choice cost a migration and several changeset updates. It also provided a stable name, a type, update rules, direct queryability, and a clear place in routing tests. Once a value influences behavior, those are not incidental benefits. They are the model.

##### The representation still leaked

The original design contained a seam that caused trouble later.

Metadata entering through Ecto could appear as embedded structs with atom keys. Metadata assembled by PostgreSQL arrived as JSON-shaped maps with string keys. XML added another irregularity: one element was decoded as a map, while repeated elements became a list.

I normalized the XML cardinality at the API boundary with `List.wrap`, but the distinction between embedded structs and database maps survived further into the application than it should have. Later fixes taught rendering helpers to accept both atom and string keys, sort either representation, trim values consistently, and return sensible fallbacks.

Those changes are a useful correction to the clean architectural story. A flexible storage format does not remove a schema; it can create several implicit versions of one. Martin Fowler makes this point in [Schemaless Data Structures](https://martinfowler.com/articles/schemaless/): so-called schemaless data still has an implicit schema, only one that may be hidden in the code that reads it.

In Plantronic, expressions such as `metadata["field_name"] || metadata[:field_name]` revealed where that implicit schema had leaked. The eventual helper functions centralized the knowledge, but the better boundary would have normalized every source into one internal representation as soon as it entered the application.

This is the tax on flexibility. It appears in parsers, tests, sorting, fallback behavior, translations, and every consumer that starts to rely on a particular key. JSONB makes adding data cheap; it does not make interpreting data free.

##### I now use behavior as the promotion test

The useful question is not whether a fact is important. Many descriptive facts are important to the person reading a task without being part of Plantronic's behavior. The question is what the application must know about the fact.

I would keep a value in this metadata channel when Plantronic only needs to preserve or display it, the source system owns its meaning, and generic placement is sufficient. I would promote it into the explicit model when any of the following becomes true:

- The value controls a transition or calculation.
- Routing, authorization, or filtering depends on the value.
- The value needs a database constraint or a non-string type.
- Updates require domain-specific rules.
- The value participates in core reporting.
- Several features repeatedly reach into metadata for the same key.

Promotion does not mean every metadata field must begin as an experiment and later receive a migration. Often the behavior is known from the start, as it was for the routing attribute. The point is that the presence of a flexible column must not become an excuse to avoid deciding.

The reverse is also important. Adding a first-class field merely because one source happens to provide it gives Plantronic responsibility it may not need. The application then owns naming, migration, synchronization, constraints, and removal for a value it only wanted to show beside a task.

##### Flexibility worked because it had a job description

The metadata feature survived and later expanded into project and reporting views. The implementation evolved: source views changed, metadata access helpers became more robust, and query composition was refined. The distinction at its center remained useful.

Plantronic owns the task's process. It models the facts required to plan, route, authorize, and report that work. The source system owns a wider set of project details. Metadata lets selected details cross the boundary without importing the source model wholesale.

Looking back, I would normalize the in-memory representation earlier and make degradation from the source schema easier to observe. I would still keep the basic split. A constrained JSONB channel is a practical way to carry facts whose only required behavior is display. It becomes dangerous when convenience allows it to absorb facts the application actually reasons about.

That is why the next field still got a column. Metadata gave Plantronic room to accept unfamiliar context; it did not release me from deciding what Plantronic needed to understand.

### [I built the task routing system before we knew the rules](https://www.lukasrieder.com/blog/i-built-the-task-routing-system-before-we-knew-the-rules/)

Plantronic's routing policy was unfinished. A small rule language and manager UI let client and developer work in parallel. The same routing model still runs today.

The routing rules did not exist yet. I started building anyway.

During specification, the application manager and I knew the outcome Plantronic needed: tasks arriving from project management had to reach the right production team. What we did not have was a finished matrix of every rule. Team responsibilities were still being clarified. The relevant task attributes were becoming visible, but their final combinations were not settled.

The obvious response would have been to wait.

The faster-looking response would have been to hardcode our current assumptions, ship them, and promise to revisit the code once the organization had made every decision.

I suggested a third option: build routing as a small system whose behavior came from data, then give the application manager an interface to manage that data himself.

This was a wager. A data-driven feature needs a model, validation, an evaluator, failure behavior, persistence, and a usable management screen. A hardcoded conditional needs only a few lines. But the apparent shortcut would have coupled my implementation schedule to an unfinished organizational specification. The system let both of us continue working.

It also became one of those features whose value is easier to see in hindsight. The implementation survived while the routing data could evolve.

##### We separated the mechanism from the policy

The stable part of the requirement was already clear.

A task arrived with known attributes. Some combination of those attributes determined which team should receive it. Plantronic had to apply that decision consistently, save the assigned team, and reject tasks for which no route existed.

The unstable part was the policy: the actual combinations of attributes and teams.

Hardcoding would have placed both parts in one function. Each organizational change would then become a software change:

```elixir
cond do
  task_has_this_combination? -> first_team
  task_has_that_combination? -> second_team
  true -> raise "unroutable"
end
```

The exact syntax is not important. The coupling is. A manager changing responsibility would need a developer. The developer would need a new specification, code change, test adjustment, review, and deployment. Until the policy was complete, implementation would either stop or repeatedly chase a moving target.

Instead, I modeled a task route as data. Each route belonged to a destination team and held a bounded set of labels to match against an incoming task. A route matched only when all its configured labels matched. The first matching route selected the team. If none matched, task creation failed explicitly rather than leaving unassigned work in the system.

This created a clean division:

- I could build and test **how routing works**.
- The application manager could decide **which routes should exist**.
- Plantronic could enforce **which task attributes are valid routing inputs**.

The manager interface completed that division. It listed teams and their routes, allowed multiple routes for one team, and let an authorized manager add or remove both routes and match criteria. The behavior was configurable, but not arbitrary. Users could compose rules from concepts the application understood; they could not upload code or invent an unvalidated language.

That boundary made the design useful rather than merely "flexible."

##### Data-driven does not mean unconstrained

Configuration systems often go wrong by trying to make everything configurable. They gradually become programming languages with worse tooling: strings for types, hidden precedence, no refactoring support, and production as the test environment.

I wanted the opposite.

Plantronic's routing language was deliberately small. A rule contained a destination team and selected match labels. The application owned the allowed dimensions. Ecto changesets guarded the records. The evaluator had one deterministic meaning. Task creation and team assignment happened in one database transaction.

That last part mattered. An unknown combination did not create a half-valid task and ask somebody to repair it later. Routing either found a team and inserted the task, or returned an `unroutable` error and inserted nothing.

Persisting the selected team also separated new policy from past decisions. Changing route data affected future imports. It did not silently redistribute tasks that planners had already received.

There were still trade-offs. The first matching route won in creation order, so ordering was part of policy even though it was not represented by an explicit priority field. With a small, managed rule set this was workable. In a larger rules system I would expose precedence, detect overlapping rules, and offer a "why did this match?" explanation before adding more expressive conditions.

A useful data-driven system is not one that can express anything. It is one that can express the changes its owner genuinely needs while making invalid states and ambiguous behavior difficult.

##### The unfinished specification became parallel work

The architectural benefit was real, but the collaboration benefit was larger.

The application manager was not withholding a finished spreadsheet from me. He was doing domain work: determining how teams should be organized and how incoming tasks should be distributed. That work involved people, responsibilities, and operating details outside the codebase. It could not be accelerated by asking for a final answer more insistently.

Meanwhile, the software problem was concrete enough to solve. I knew routes needed match criteria, teams, deterministic evaluation, transactional assignment, and an administration interface. I could implement those constraints without knowing every eventual row of routing data.

So we worked in parallel.

I built the mechanism. He refined the policy. Once the interface existed, he could enter and adjust the resulting routes directly instead of translating each organizational decision into another development request.

This is an underrated form of decoupling. Software architecture usually discusses coupling between modules or services. Projects also contain coupling between people's work. If a developer cannot proceed until a domain expert finalizes volatile data, and the domain expert cannot test decisions until the developer hardcodes them, the delivery process contains a dependency cycle.

A well-bounded management interface can break that cycle.

It does not remove collaboration. We still had to agree on vocabulary, constraints, and failure behavior. It moves collaboration to the stable boundary and allows each person to continue independently on the variable details.

##### Git shows two short bursts across 37 elapsed days

I went back through Plantronic's history to see how long this actually took.

Git cannot tell me hours worked, and the main feature commits were squashed. Claiming an exact effort total would be fiction. It does preserve enough evidence to reconstruct the delivery shape.

The first routing migration is timestamped **May 18, 2025**. The core feature was committed and merged on **May 20**. That first implementation introduced the route schema and context, assigned teams during external task creation, handled unroutable tasks transactionally, and added focused tests.

The manager-facing phase began no later than a migration timestamped **June 22**. On **June 24 at 12:33**, the repository recorded the routing management interface and supporting context operations. A formatting pass followed nineteen minutes later. A final focused cleanup landed at **19:18** that evening.

From the first recorded source artifact to the last focused cleanup, **37 days elapsed**.

That does not mean 37 days of coding. The evidence points to two concentrated implementation bursts of roughly two calendar days each, separated by a month in which other Plantronic work continued and the routing policy could mature. Migration timestamps are not time sheets, so even "roughly four days" describes elapsed implementation windows, not billable effort.

This distinction supports the architectural story. The gap was not dead time spent waiting for one giant specification. The core routing mechanism already existed. Other development continued. The manager interface arrived when the policy needed to become directly manageable.

The first commit was broad because routing crossed real boundaries: schema, import transaction, team relation, UI, migrations, and tests. The administration phase added the operations and screens needed to hand policy ownership to the application manager. This was more work than a hardcoded branch, but still a small turnaround for a feature that removed future deployments from ordinary routing changes.

##### Three established patterns explain the design

I did not implement a general rules engine, and describing Plantronic that way would oversell it. The design does sit at the intersection of several established patterns.

###### Content-Based Router

Gregor Hohpe and Bobby Woolf's [Content-Based Router](https://www.enterpriseintegrationpatterns.com/patterns/messaging/ContentBasedRouter.html) sends a message to a destination based on data in that message. Their pattern also warns that routing becomes a frequent maintenance point and notes that sophisticated versions may use configurable rules.

Plantronic routed tasks to teams rather than messages to channels, but the structural problem was the same: inspect content, select one destination, and keep the sender independent of destination details.

The pattern helped clarify what the routing mechanism should own. It should evaluate task content and return a destination. It should not require project management to know Plantronic's current team structure.

###### Parameterized Specification

Eric Evans and Martin Fowler's paper on [Specifications](https://martinfowler.com/apsupp/spec.pdf) makes an even closer distinction. A specification separates criteria from the candidate being tested. Their **Hard Coded Specification** requires programming for each new rule. A **Parameterized Specification** allows users to create new specifications at runtime, but only within capabilities programmers have provided.

That is almost exactly the trade Plantronic made.

The match labels formed a parameterized specification for an incoming task. The application manager could create new combinations without a deployment. I still defined the legal vocabulary and matching semantics. We gained runtime adaptability without pretending the domain expert should become a programmer.

Evans and Fowler also describe a composite form where users combine specifications into a richer language. Plantronic did not need that power. Exact conjunctions of a few known labels covered the requirement. Choosing the least expressive model that solves the problem kept evaluation and testing small.

###### Decision tables and DMN

At the formal end of this spectrum sits the Object Management Group's [Decision Model and Notation](https://www.omg.org/dmn/). DMN separates business decisions and rules from process models and aims to make them readable by business people, analysts, and developers. Decision tables can express multi-criteria policies in an unambiguous form.

Plantronic did not need a DMN engine, graphical notation, or a general decision table editor. Referencing DMN is still useful because it names the larger architectural move: organizational decisions deserve an explicit model instead of disappearing inside process code.

The scale should match the problem. For Plantronic, a relation, a small map of criteria, a deterministic matcher, and two administration screens were enough.

##### Elixir made the safer version cheap enough

Data-driven behavior moves risk. Hardcoded code can fail because the implementation is wrong. Configurable behavior can fail because the evaluator is wrong, the stored data is invalid, two rules overlap, or no rule matches.

The answer is not to trust configuration. It is to test the constraints and semantics as first-class code.

The initial routing implementation included a focused ExUnit test module plus expanded task and controller coverage. The tests established several important properties:

- A route requires both match criteria and a team.
- Matching returns the expected team.
- Precedence is deterministic when more than one route could match.
- An unknown combination returns an explicit error.
- An unroutable import leaves the task count unchanged.
- External task creation accepts only its intended fields.

[ExUnit](https://hexdocs.pm/ex_unit/ExUnit.html) makes these examples compact and readable. Factories can create teams, tasks, and routes with only the attributes relevant to each case. Pattern matching asserts both result shape and domain value without much ceremony.

The [Ecto SQL Sandbox](https://hexdocs.pm/ecto_sql/Ecto.Adapters.SQL.Sandbox.html) gives database tests isolated transactional connections. That matters for routing because the valuable guarantee crosses the database boundary: an unroutable task must not persist. I can test the actual context and transaction instead of replacing the repository with mocks and hoping production behaves similarly.

Elixir did not make a data-driven design automatically safe. It made the feedback loop short enough that building the model, evaluator, constraints, and failure cases remained practical. The safer architecture did not require a multi-month rules-engine project.

##### When I would make this choice again

I would not turn every changing conditional into database rows and an admin screen. Hardcoded policy is often correct when rules are few, stable, developer-owned, and released with the rest of the application.

I would seriously consider data-driven behavior when several conditions hold:

- The mechanism is stable, but the policy is still evolving.
- A domain owner, not a developer, is responsible for future changes.
- Waiting for a complete policy would block implementation.
- The allowed inputs and operations can be tightly bounded.
- Wrong or missing rules have explicit, testable failure behavior.
- Changing the policy without a deployment has operational value.

I would push back when "configurable" means arbitrary expressions, invisible precedence, or an attempt to avoid understanding the domain. At that point configuration becomes a second application, usually with worse tools than the first.

Plantronic stayed on the useful side of that line. Its current code still carries the same essential model: routes hold match labels and belong to teams; the evaluator selects a team from incoming task data; manager operations create, change, and remove the route data. Later work adjusted inputs and refined the interface, but did not require replacing the routing core.

The routing model has lasted because the mechanism stayed stable while the route data had an intentional place to change.

##### Build the place where the answer will live

When a specification is unfinished, waiting can feel disciplined and hardcoding can feel agile. Sometimes both reactions miss the better option.

We knew enough to define the shape of a valid answer before we knew every answer. By turning that shape into a constrained data model and giving its owner a proper interface, I could build in parallel with the person resolving the organization itself.

The extra architecture cost time up front. It also removed a queue of future code changes, let Plantronic adapt without deployments, and kept routing ownership close to the person who understood it best.

The most interesting part is still the apparent contradiction: I built the routing system before we knew the rules.

It worked because I did not build the rules. I built the tested place where the rules could live.

### [Why I build complete, realistic seed datasets](https://www.lukasrieder.com/blog/why-i-build-complete-realistic-seed-datasets/)

Realistic seed data gave Plantronic a fast local reset, useful demos, and a safe way for colleagues to contribute without production data access.

I like opening a project and finding it already inhabited.

For Plantronic, that meant a local database with people, skills, teams, projects, tasks, assignments, working calendars, and enough variation to make the planning screens useful. The names were fictional and the records were generated, but the relationships and distributions looked plausible. I could reset the database, start Phoenix, and work on an actual planning situation instead of an empty table and one hand-made row.

An April 2025 change to work-hour patterns reminded me how much I had come to value that setup. The individual change was small. The useful retrospective is broader: building a complete seed dataset paid for itself throughout the project.

##### A usable local system changes the development loop

Many applications have a seed script that creates one user and perhaps a few lookup rows. Everything else is left to the developer. This is quick to write, but it pushes setup work into every feature. Before fixing a planning view, I would need to create a project, add tasks, assign employees, fill calendars, and arrange the exact states that make the view interesting.

I would rather do that modeling once in code.

A complete dataset gives me a known place to begin. I can inspect a busy team, a person with reduced availability, overlapping assignments, projects at different stages, and tasks with different time frames. I can move between features without rebuilding their surrounding data by hand. If an experiment leaves the database in a strange state, I throw it away and recreate it.

This is not test isolation. A seed dataset is too broad for a focused unit test and often contains some controlled randomness. Its purpose is to make the whole application useful. It supports exploratory development, visual checks, debugging, and the ordinary work of understanding how one feature affects another.

The distinction matters. Tests should create the minimum state needed for a reliable assertion. Seeds should create enough state for a human to explore the system.

##### Realistic does not mean copied from production

The strongest benefit on Plantronic had little to do with convenience. Good seeds let me separate useful development data from production data access.

Colleagues could help with the software using a local system that behaved like the real application without receiving a copy of client records. They could work on layouts, components, queries, tests, and planning behavior against fictional people and projects. The relationships were present, so the work was meaningful. The sensitive source material was not.

That separation made collaboration much cleaner. I did not have to choose between an empty development environment and sharing more data than a colleague needed. Production access remained limited, while the repository contained what was necessary to reproduce the shape of the application.

Seeds are not a complete data-security policy. They do not replace access controls, careful logs, sanitized fixtures, or a rule against copying production dumps onto laptops. They simply remove one common excuse for doing so. If the generated dataset is good enough for daily work, developers have less reason to reach for real records.

The goal was plausibility, not imitation. Fictional data should preserve useful properties such as cardinality, date ranges, optional fields, workloads, and relationships. It should not preserve names, identifiers, comments, or unusual combinations that could point back to a real person or project.

##### Demos stopped feeling staged

The same dataset made client demos unusually satisfying.

A feature shown against two records named "Test" looks unfinished even when the code is sound. Empty states are important, but an empty planning system cannot demonstrate planning. With the seeds loaded, Plantronic looked alive. Teams had work. Calendars varied. Projects and tasks formed recognizable clusters. Filters produced results that made sense.

That changed the conversation. I could demonstrate a feature in context instead of narrating what it might look like after data arrived. The client could notice whether a screen felt plausible, whether a workload was readable, or whether a label sounded wrong. Seed data helped prepare the discussion as much as it prepared the database.

There was also less demo anxiety. I did not need to assemble a fragile scenario shortly before a meeting and then avoid touching it. If I broke the local state while testing, I could recreate the same kind of useful system. The exact generated records did not have to be identical. The dataset had to retain the same coverage and character.

##### Ecto and ExMachina make this pleasantly direct

Elixir projects give me a simple way to organize this. [Ecto](https://hexdocs.pm/ecto/Ecto.Repo.html) owns schemas, constraints, associations, and persistence. [ExMachina](https://hexdocs.pm/ex_machina/ExMachina.Sequence.html) provides factories for building valid records. I use the same factories in tests and seeds.

That reuse is important. Without it, projects tend to grow two object-construction systems. Tests learn how to create a valid employee one way, while `seeds.exs` assembles one with a separate pile of attributes. As schemas evolve, one path is updated and the other quietly rots.

A shared factory keeps the defaults in one place. Tests override only the fields relevant to an assertion. Seeds compose many factory records into a complete scenario, then override values that make the dataset believable. Sequences can provide stable variation for hours, dates, names, and other repeated values. Carefully chosen random values can fill in details where exact reproducibility is not important.

The entrypoint remains a small `priv/repo/seeds.exs` script. It starts what the application needs and calls the seed module. I like that file to cover the global, idempotent setup of the application first: required roles, labels, configuration, or an initial account should be safe to create again. The larger fictional dataset can then be built for a fresh local database.

In Plantronic, the project aliases made the reset loop explicit. In simplified form, they looked like this:

```elixir
"ecto.setup": [
  "ecto.create",
  "ecto.migrate",
  "run --no-start priv/repo/seeds.exs"
],
"ecto.reset": ["ecto.drop", "ecto.setup"]
```

This custom `mix ecto.reset` command was destructive by design, so I used it only for a local database. It dropped the current database, created it again, ran migrations, and reinstantiated the seeds. One command returned me to a usable application.

That made larger refactors easier. I could change schemas and associations, reset, and immediately browse a coherent dataset. It also kept the seed code honest. A script that runs only when a developer first joins the project will decay unnoticed. A script in my regular reset loop fails while the relevant change is still fresh.

##### Complete seeds need maintenance

There is a cost. A realistic dataset touches much of the model, so schema changes often require seed changes. Generation order matters when records have associations. Random data can produce nonsense unless its ranges and combinations are constrained. A large seed run can also become slow.

I do not see those problems as reasons to keep seeds trivial. I treat them as limits to manage.

Factories handle valid defaults. Explicit scenario code handles relationships. Small sequences produce plausible variation. Bulk inserts help with dense calendar data. Required global setup stays idempotent, while local demo records can assume an empty database after `mix ecto.reset`. I also avoid trying to model every possible edge case. Seeds need enough breadth to make the application useful, not enough detail to become a second production system.

The April work-hour change was a typical maintenance pass. Generated availability had been too arbitrary, so I replaced it with more recognizable patterns. Other seeded values became more representative as our understanding improved. None of this was profound on its own. It was ordinary care for a development tool I used every day.

That is why I embrace seed data. It shortens my local feedback loop, gives colleagues a useful system without production access, and makes demos feel like the product rather than a wireframe. In an Elixir application, Ecto, shared ExMachina factories, and a plain `seeds.exs` script are enough to build that workflow. The populated database gives me a clean, repeatable place to work.

### [How much cloud fits into one server?](https://www.lukasrieder.com/blog/how-much-cloud-fits-into-one-server/)

An unfamiliar on-premise machine looked like a constraint. It became a test of whether years of infrastructure work could collapse into one repeatable deployment.

In May 2025, I had to deploy Plantronic into an on-premise environment I did not know.

The environment provided one server, corporate networking, and controlled egress. It needed to support an application, a database, ingress, migrations, backups, monitoring, and a deployment path from staging through production.

This could have become a bespoke server project: install packages, write service units, copy configuration, arrange startup order, add a reverse proxy, improvise database backups, and document which commands an operator must remember.

Instead, years of infrastructure work collapsed into one small k3s installation and a set of Terraform compositions.

That result is why I pride myself on **running the things I build**. I do not consider an application complete when it works on my laptop or when a container image exists. I want to understand how it starts, how it receives configuration, how it migrates data, how it recovers, and how another person can reproduce the deployment without borrowing my memory.

##### The deployment was unique; most pieces were not

I did not arrive with a ready-made "Plantronic platform." I arrived with a portfolio of smaller, declarative parts developed across earlier work.

My public [Terraform Modules](https://gitlab.com/parlant-co/terraform-modules) repository contains modules for web workloads, PostgreSQL, GitLab Agent, monitoring, ingress, cloud resources, secrets, jobs, and related infrastructure. One day before the Plantronic deployment milestone, I committed the initial reusable Kubernetes modules that the project composed.

Plantronic selected and connected those pieces:

- A containerized Phoenix application provided the product runtime.
- An explicit migration step managed database changes.
- A Kubernetes Deployment and Service managed the workload and its network identity.
- Ingress and TLS integration exposed the application safely.
- PostgreSQL used persistent storage.
- Scheduled logical backups applied an explicit retention policy.
- GitLab Agent provided controlled deployment access.
- Monitoring and adjacent operational services completed the deployment.

None of these modules was remarkable alone. Their value came from combining them without removing their boundaries.

That is the compounding effect I care about. Knowledge does not compound merely because I remember how I solved a problem before. It compounds when the old solution has a stable interface, remains readable, and can participate in a new system without being copied into it.

##### Why one server still became a Kubernetes cluster

A single-node Kubernetes installation sounds contradictory to some people. Kubernetes is associated with fleets, autoscaling, and highly available control planes. Plantronic had one machine. Why place a cluster API between the application and Systemd?

Because node count was not the problem I wanted Kubernetes to solve.

I wanted a consistent application runtime and a declarative workload model. I wanted an image, configuration, secrets, migrations, services, jobs, ingress, health behavior, and rollout state to be described with the same primitives in staging and production. I wanted GitLab to interact with the deployment without exposing a raw cluster endpoint through the client's firewall.

[k3s](https://docs.k3s.io/architecture) packages the Kubernetes control plane and required runtime components for smaller environments. On one server it remains one failure domain, but it gives me the Kubernetes API and controllers without pretending that I am operating a large managed cluster.

That distinction matters:

- **Supervision.** Controllers restart failed workloads and reconcile declared state.
- **Repeatability.** The same resources can be created again.
- **Host-level high availability.** One broken server is still one broken server.
- **Durability.** Storage, backups, restore procedures, and host recovery determine how durable the system is.

I chose single-node k3s for operational consistency, not for an imaginary availability claim.

##### Systemd is not the loser

The strongest counterproposal is good: build an Elixir release, run it under Systemd, run PostgreSQL as a host service, and keep the deployment small.

[Systemd is an excellent process supervisor](https://www.freedesktop.org/software/systemd/man/latest/systemd.service.html). It has restart policies, dependency ordering, timers, socket activation, credentials, resource controls, and [extensive execution sandboxing](https://www.freedesktop.org/software/systemd/man/latest/systemd.exec.html). [Podman Quadlet](https://docs.podman.io/en/latest/markdown/podman-systemd.unit.5.html) offers an appealing middle ground by declaring containers as Systemd-managed units.

[Docker Compose in production](https://docs.docker.com/compose/how-tos/production/) is another reasonable middle path when the stack is co-located and its lifecycle remains small. It preserves container images and a multi-service declaration without adding a Kubernetes control plane. I prefer admitting when that is enough instead of turning Kubernetes experience into a requirement.

For one application process and one host, I would seriously consider one of those designs. Kubernetes would add an API server, certificates, networking, storage abstractions, controllers, and upgrade work without automatically producing value.

Plantronic crossed my threshold because its operation involved more than one process:

- The application had its own release lifecycle.
- Database migrations had a separate lifecycle.
- The database required persistent storage.
- The application needed ingress.
- Backup jobs had to run on a recurring schedule.
- Secrets and runtime configuration needed controlled delivery.
- The deployment required monitoring.
- Staging and production each had their own workloads.
- Deployments had to cross a constrained network.

I could model every one of those with Systemd units, scripts, container commands, timers, and proxy configuration. I would then own a Plantronic-specific orchestration system assembled from excellent host primitives.

I already owned the equivalent Kubernetes abstractions as reusable Terraform modules. In my situation, direct Systemd would have been the bespoke choice.

There is another useful truth hiding here: Systemd still supervises k3s. I did not replace it. I moved application orchestration behind a boundary that could remain consistent across hosts and environments. Systemd owns the node service; Kubernetes owns workloads; OTP owns processes inside the Elixir release.

I prefer these nested responsibilities over asking one layer to understand every failure.

##### Containerizing Elixir without fighting OTP

Elixir already has supervision trees. Kubernetes does not improve a GenServer restart by killing the entire pod, and I do not want it trying.

I divide the responsibilities differently:

- OTP supervises application processes and internal dependencies.
- The Elixir release defines the runnable application artifact.
- The container fixes the OS-level runtime and native dependencies.
- Kubernetes supervises the container and its relationship to services.
- k3s itself remains a Systemd-supervised node service.

Phoenix documents deployment through [OTP releases](https://hexdocs.pm/phoenix/releases.html). A controlled build can produce the release, a smaller runtime image can contain it, and runtime configuration can arrive when the container starts. The same image moves through environments; the release tag and configuration identify what actually runs.

For Plantronic, containerization supplied valuable guarantees on an unfamiliar host:

- The image fixed the expected Erlang and Elixir runtimes.
- The native libraries were known.
- The listening port and process command were explicit.
- Startup behavior was reproducible.
- The deployment promoted one image instead of mutating a host in place.

Database migrations remained an explicit deployment step. I prefer that to hiding schema changes inside normal application startup. The deployment can order migration before the new workload, report failure, and avoid having every replica race to alter the database.

Containerization did not make the BEAM reliable. OTP already did much of that work. It made the boundary around the BEAM release portable.

##### Twelve-Factor, applied rather than recited

The [Twelve-Factor App](https://12factor.net/) is sometimes invoked as a cloud slogan. I find it more useful as a set of pressure tests.

###### Config

Environment-specific values should not be compiled into the release. Kubernetes configuration and secrets let the same image receive the database address, runtime identity, and deployment-specific values at startup.

###### Backing services

PostgreSQL is attached infrastructure, not a hidden directory inside the application container. That separation makes its persistence, backup, monitoring, and recovery responsibilities explicit.

###### Build, release, run

The image is the build artifact. A tagged image plus runtime configuration forms a release. Kubernetes runs that release. Terraform describes the desired resources around it.

###### Processes and disposability

The Phoenix application should start predictably, handle termination, and recover without depending on writable container-local state. Kubernetes can replace the process because durable state lives elsewhere.

###### Dev/prod parity

I do not need every developer laptop to run a production cluster. I do want staging and production to share image, configuration, migration, service, and job concepts. That parity catches deployment mistakes before production without pretending environments are identical.

None of these factors requires Kubernetes. That is important. Kubernetes was useful because my existing modules already expressed these properties coherently.

##### Terraform all the way up to the application

I want infrastructure changes to look like code changes: proposed, reviewed, planned, applied, and recorded.

The Plantronic composition referenced reusable modules rather than copying their resources. The application module owned common deployment behavior. The PostgreSQL module owned the database chart and persistence interface. The GitLab Agent module owned its Helm release. Project-level Terraform supplied the choices that were genuinely specific to Plantronic.

This creates two useful levels of change:

1. Improve a reusable module when several applications need the same capability.
2. Change the Plantronic composition when the requirement belongs only to Plantronic.

That boundary prevents two failures I see often. A giant universal platform makes every application wait for abstractions it does not need. Fully bespoke Terraform makes every project rediscover secrets, rollouts, services, and jobs.

I prefer small modules with boring inputs. Composition is where client context enters.

The initial server bootstrap still prepared the operating system, k3s, certificates, registry access, and network integration. I do not want to pretend a few documented host steps are already Infrastructure as Code. My direction is to push stable host knowledge into declarative automation while keeping credentials out of code and state exposure under control.

Above that host layer, application infrastructure belonged in Terraform. No hand-maintained Deployment YAML, no deployment recipe that existed only in shell history.

##### GitLab Agent across a controlled boundary

Traditional CI deployment often assumes that a runner can reach the Kubernetes API directly. That is uncomfortable in an on-premise network and frequently incompatible with firewall policy.

The [GitLab Agent for Kubernetes](https://docs.gitlab.com/user/clusters/agent/) changes the direction. The cluster-side agent opens a bidirectional channel to GitLab's agent server, KAS. GitLab documents this as a way to communicate with clusters behind firewall or NAT.

That fit Plantronic's egress-controlled environment. I could define a narrow outbound path instead of publishing the cluster API for inbound CI traffic. The same Terraform composition installed the Helm release and connected deployment automation to the cluster.

I use "air-gapped" carefully. A truly air-gapped network cannot talk to GitLab.com; no agent changes that fact. It needs mirrored images and artifacts, a reachable self-managed control plane, or a controlled transfer process. [k3s documents air-gap installation](https://docs.k3s.io/installation/airgap), so the host setup can support that mode. Egress-controlled and air-gapped remain different operating modes.

I value the architecture because it crosses organizational boundaries cleanly. Client networking can remain restrictive. Deployment can remain automated. The interface between both concerns is explicit.

##### A practical guide to deciding

I would not recommend single-node k3s merely because a project uses containers. I would ask these questions.

###### Choose direct Systemd or Quadlet when

- The deployment has one host and only a few processes.
- Host-specific integration is desirable.
- The team knows Systemd better than Kubernetes.
- Parity between staging and production Kubernetes environments has little value.
- Container networking, jobs, secrets, and ingress remain simple.
- No reusable Kubernetes platform already exists.

###### Consider single-node k3s when

- The application includes several independently managed workloads.
- Images and Kubernetes resources already exist elsewhere.
- Jobs, migrations, services, ingress, secrets, and health behavior need one declarative model.
- Controlled deployment access benefits from an in-cluster agent.
- Staging and production should share orchestration primitives.
- The team can operate and upgrade the cluster.
- One host failure domain is acceptable and honestly documented.

###### In either case

- Make backups and test restores.
- Keep credentials out of repositories.
- Pin and plan runtime upgrades.
- Document host recovery.
- Monitor the host, not only the application.
- Define who owns the deployment after handover.

Technology choice cannot compensate for missing operational ownership.

##### What I bring when I bring infrastructure

The Plantronic deployment did not prove that Kubernetes belongs on every server. It proved something more useful to me: a broad portfolio of composable infrastructure can make an unfamiliar boundary feel familiar without erasing what is different about it.

I brought lessons from cloud orchestration, container builds, Kubernetes resources, Terraform modules, Elixir releases, database operations, and constrained networks. I did not combine them into a private platform that only I could operate. I assembled public, declarative modules into a client-specific composition from staging to production.

That is the compounding effect I want from experience. Every project should leave behind more than memories and copied snippets. It should sharpen a module, an interface, a runbook, or a decision rule that improves the next project.

I pride myself on running what I build because deployment is where architectural claims meet reality. Configuration must arrive. State must survive. Processes must restart. Networks must permit exactly enough. Another person must be able to see what exists.

One server was enough to make all of those concerns concrete. Years of cloud work were what made one server feel small.

### [Finicky - scriptable macOS URL handler](https://www.lukasrieder.com/blog/finicky-scriptable-macos-url-handler/)

A programmable URL handler becomes much more useful when its final destination can be any small executable in my dotfiles.

The macOS default browser setting looks like a choice between applications. Safari, Firefox, Chrome, perhaps another browser.

I wanted the choice to be a program.

My eventual URL path became:

```text
macOS URL handler → Finicky → JavaScript rule → executable → destination
```

I like this chain because each arrow leaves me a seam I can understand and replace. macOS owns dispatch. Finicky owns URL parsing and routing policy. A normal executable owns the final effect. In my case, that executable forwards unknown URLs to a browser running outside my work machine.

This is exactly the kind of arrangement I want in dotfiles: visible, scriptable, composable, and replaceable one layer at a time.

##### Finicky is a programmable default browser

[Finicky](https://github.com/johnste/finicky) is an open-source macOS application that registers as the default browser. It does not need to render pages. It receives a URL, loads `~/.finicky.js`, evaluates matching and rewrite rules, then opens the selected browser or application.

A basic configuration can route work domains to one browser and personal domains to another:

```js
module.exports = {
  defaultBrowser: "Safari",
  handlers: [
    {
      match: finicky.matchHostnames(["github.com", "gitlab.com"]),
      browser: "Firefox",
    },
  ],
};
```

The JavaScript configuration is the attraction. Rules can be functions. URLs can be rewritten. Hostnames can be matched through Finicky's parsed representation instead of fragile string slicing. The file can live beside shell functions, editor configuration, and the rest of my workstation policy.

I could have used a GUI router, and that would have been easier to start with. I know myself well enough to know where that ends: sooner or later I want to review a rule in Git, generate part of it, share it between machines, or make the destination something other than an installed browser.

##### I did not want another browser destination

My unknown-link policy required a command called `SafeBrowser`. Finicky should select it just as it selected Safari, then pass the URL as one argument.

At the beginning of 2025, the version I was using modeled destinations as application names, bundle identifiers, or application paths. That was sensible for a browser picker. It stopped one layer short of the composition I wanted.

So I forked Finicky at [Overbryd/finicky](https://github.com/Overbryd/finicky) and added a `command` application type. The public implementation commit is [`e380acbf01c…`](https://github.com/Overbryd/finicky/commit/e380acbf01c20ae16b1a8f65e25cb5379c37aa52).

The configuration schema gained `appType: "command"` and a command path. The native side carried that option into its browser descriptor. When selected, Finicky constructed an argument array containing the executable path and URL.

One implementation detail mattered enough to me not to compromise on it: the URL must remain data. The implementation does not produce one shell string such as:

```sh
/Users/me/.bin/safebrowser "${url}"
```

It uses `Process`, invokes `/usr/bin/env`, and supplies the command and URL as separate arguments. Shell metacharacters inside an attacker-controlled URL do not become shell syntax. My configuration also uses an absolute executable path, avoiding a `PATH` lookup for the security boundary.

I treat a command destination as privileged configuration. Whoever can modify `.finicky.js` or the target executable can redirect every external link or run code whenever one opens. I therefore keep both files under my ownership and out of reach of less-trusted processes.

##### Default to the script

The resulting configuration reverses the usual browser-router pattern. Instead of making Safari the default and adding special cases, I make `SafeBrowser` the default and add narrow local exceptions:

```js
module.exports = {
  defaultBrowser: {
    name: "SafeBrowser",
    appType: "command",
    command: "/Users/me/.bin/safebrowser",
  },
  handlers: [
    {
      match: finicky.matchHostnames([
        /.*\.?github\.com$/,
        /.*\.?example-work-service\.com$/,
      ]),
      browser: "Safari",
    },
  ],
};
```

I like the failure direction of this policy. A new domain does not accidentally open in my work browser because I forgot to classify it. My omission produces less trust, not more: the URL follows the unknown route.

I review these rules as if the input were trying to fool them. Sometimes it is. I anchor patterns and match parsed hostnames rather than searching arbitrary strings. A rule matching `github.com` anywhere in the full URL could be fooled by a hostname such as `github.com.attacker.example` or by placing the trusted text in a query parameter.

I also keep the allowlist small. If my configuration contains half the internet, then I have not written a trust policy; I have built an unmaintainable alternate browser history.

##### The script is an adapter

`SafeBrowser` is deliberately ignorant of Finicky. It accepts one URL and arranges for another machine to open it:

```applescript
#!/usr/bin/osascript

on run argv
  set remoteMachine to first item of ¬
    (read (POSIX path of (path to home folder as string) & ¬
    ".config/vm1.local/eppc") using delimiter "\n")

  tell application "Finder" of machine remoteMachine ¬
    to open location (item 1 of argv)
end run
```

The destination address lives outside the public dotfiles. The script uses Remote Apple Events to tell Finder in a macOS browser VM to open the URL.

I like this adapter because no layer knows too much:

- Finicky does not know how the VM is reached.
- `SafeBrowser` does not know why this URL was selected.
- The VM does not need Finicky installed.
- macOS does not need to know that its "default browser" is a policy engine.

I can replace Apple Events with SSH without rewriting matching rules. I can replace the VM while keeping the script's one-URL contract. I can test the script directly from a terminal. I can point one rule at a different command for a temporary experiment.

I prefer composition over tight vertical integration. It lets me defer decisions, replace one layer, and hack on a new transport whenever I want. If a group of these pieces eventually stands the test of time, I can still compress them into one vertically integrated application, bundle it, and package it. I prefer that compression to be the last step, earned by experience, rather than the first architectural decision.

##### A URL is an argument, not a command

Arbitrary command support introduces a sharp edge, so the contract should stay narrow:

```text
input: exactly one absolute http or https URL
output: destination receives that URL, or the command fails visibly
```

For my adapter, I want a deliberately unforgiving contract. It should reject missing arguments, extra arguments, unsupported schemes, unreasonable size, and invalid URL syntax. I do not want it logging query strings because they may contain tokens or recipient identifiers. I want a timeout when contacting another machine and a non-zero status when delivery fails.

My hard line is that the adapter must never interpolate the URL into shell source. This applies locally and across SSH. The following shape is dangerous:

```sh
ssh browser-vm "open '$url'"
```

Quoting untrusted data across two shell parsers is easy to get wrong. A forced remote command reading an encoded value from standard input, or a small protocol that keeps data separate from command text, is safer.

The AppleScript version also needs operational hardening. Remote Apple Events credentials are sensitive. Network access to the guest should be restricted. Failure must not silently fall back to opening the URL locally, because that would erase the policy exactly when isolation is unavailable.

##### Configuration as code is the real feature

The important part of this setup is not JavaScript as a language. It is that the routing decision is text I own.

A dotfiles-native policy gives me:

- **History.** Git shows when a domain became locally trusted.
- **Review.** A broad regular expression appears as a diff.
- **Reproduction.** A new Mac receives the same handler and scripts.
- **Composition.** Rules can target browsers, profiles, applications, or commands.
- **Rollback.** One Git operation restores the prior policy.
- **Portability.** The command behind a rule can evolve independently.

It also removes mystery. When a link opens in the wrong place, I inspect one rule file and one executable. I do not search preference panes, browser extensions, and remembered prompt choices.

I do not mistake configuration as code for automatic security. Finicky executes JavaScript, so my config is executable policy. My regexes can be wrong. My script can leak URLs. My public dotfiles can expose private hostnames or machine details. What I gain is not perfection; I gain the ability to inspect, review, and correct those risks.

##### Test the whole chain

I do not consider a URL router finished after clicking one familiar link and watching the right browser open. The tests should cover each important decision:

- They should cover an exact allowlisted hostname.
- They should cover a legitimate subdomain.
- They should reject a deceptive suffix such as `trusted.example.attacker.invalid`.
- They should cover an unknown HTTPS domain.
- They should cover plain HTTP.
- They should preserve a URL with spaces or encoded characters.
- They should preserve a query string containing shell metacharacters.
- They should cover a URL shortener.
- They should reject a non-web scheme.
- They should expose an unavailable browser VM as a visible failure.

The expected destination should be explicit for each case. Testing only the JavaScript match misses command dispatch. Testing only the script misses policy errors. The useful test crosses the same boundary as a real click.

I also want failure to remain safe. If Finicky cannot load its configuration or `SafeBrowser` cannot reach the guest, it should fail visibly instead of falling back to the local browser.

##### Keep the layers boring

"URL handler to Finicky to a script" sounds more complicated than choosing Safari in System Settings. Internally, each layer is simpler than a monolithic tool that parses URLs, manages every destination, authenticates to a VM, and opens a remote browser itself.

macOS already knows how to dispatch URLs. Finicky is good at programmable routing. A script is good at adapting one selected URL to a local environment. The guest is good at containing a browser.

That division is why the setup feels hacker-friendly rather than merely clever. Every piece has a small interface. Every policy lives in a file. Every implementation can be replaced with another command.

Finicky gave me control of the decision. My command extension kept the destination open-ended. Public dotfiles made both pieces durable, while leaving me free to replace them.

### [Your default url handler is a security issue](https://www.lukasrieder.com/blog/your-default-url-handler-is-a-security-issue/)

Taking ownership of macOS URL handling lets unknown links leave the work machine before they reach a privileged browser session.

I maintained a bunch of small things in my dotfiles in February 2025. I improved language-server support, added a helper for debugging Elixir releases, restored some Go setup, and adjusted how links open on my Mac.

That last change is the one I keep wondering about. Why is everyone who takes workstation hardening seriously not also taking ownership of URL handling?

Think about how many incursions begin on a work computer. A message arrives. It contains a link. The domain looks plausible, the sender appears familiar, or the subject creates just enough urgency. One click transfers an attacker-controlled string through the operating system into a large, networked application containing authenticated sessions, saved permissions, extensions, downloads, and access to local files.

[MITRE ATT&CK calls this User Execution through a Malicious Link](https://attack.mitre.org/techniques/T1204/001/). A related path, [Drive-by Compromise](https://attack.mitre.org/techniques/T1189/), begins when a browser visits an attacker-controlled or compromised website. [CISA's phishing guidance](https://www.cisa.gov/resources-tools/resources/phishing-guidance-stopping-attack-cycle-phase-one) treats the link as a central initial-access mechanism.

For years, that path nagged me. In February, I finally dealt with it at the point before a browser opens: OS-level URL handling.

##### A click is a dispatch decision

On macOS, applications hand URLs to the operating system. Launch Services resolves the registered handler for a scheme such as `http` or `https`, then delivers the URL to that application. Normally, the handler is Safari, Firefox, Chrome, or another browser.

That default optimizes for convenience. It assumes that every link arriving from Mail, Messages, Slack, a terminal, a PDF reader, or a password manager should immediately enter the same browser environment.

My browser is not an empty document viewer. It carries work sessions. It can reach administration panels, source hosts, cloud consoles, invoicing, internal tools, and identity providers. Even with MFA, a live browser session represents authority. Sending every unknown URL directly into that context is an unnecessary default.

I consider the OS-level handoff a trust boundary. Before dispatch, the URL is data. After dispatch, a browser begins resolving, connecting, parsing, rendering, and possibly downloading. I want my own policy between those states instead of accepting one global browser choice as permanent.

One important limit: this boundary governs links handed to macOS. It does not intercept every navigation after I am already inside a browser. A malicious page opened from an allowlisted local page can still navigate elsewhere. URL brokering reduces one exposure path; it does not turn the browser into a safe viewer.

##### What can go wrong after default dispatch

A routing layer cannot know whether a page is benign. It can still reduce several kinds of risk.

###### Phishing meets an authenticated session

The most common danger is not a browser sandbox escape. It is a convincing login page, consent screen, OAuth flow, or support prompt opened in exactly the environment where I normally authenticate.

Separating unknown links from my work browser removes familiar cookies and active sessions from the first interaction. The isolation browser can still be phished, but it begins with less authority. A fake page also looks less natural when it appears in a deliberately untrusted environment.

###### Browsers process hostile input

A URL can lead to exploit code, a compromised legitimate site, malicious advertising, or a download crafted for the next stage of an attack. Modern browsers have strong sandboxes, and macOS adds quarantine, Gatekeeper, and notarization checks. None of these controls is a reason to place the browser next to every valuable host resource if I can avoid it.

Moving unknown browsing into a VM adds another containment boundary. A browser compromise must then escape both the browser and the guest before reaching the work host, unless it can abuse an integration channel.

###### Links can cross application boundaries

Web pages and messages may point to non-web schemes that launch other applications. URL-scheme handling has a long history of surprising behavior because the receiving app decides what the payload means.

A broker can reject schemes it does not expect and ensure that its fallback only forwards `http` and `https`. This does not secure every registered macOS handler; it narrows what this particular path accepts.

###### The URL itself leaks information

Tracking parameters can identify the recipient, campaign, account, or document. Some routing tools can remove known parameters or rewrite hosts before opening. That is primarily privacy hardening, but less unnecessary context reaching a remote server is also useful security hygiene.

###### Profiles collapse trust zones

Personal browsing, client systems, development tools, and privileged administration do not need the same browser profile. Routing known domains into deliberate profiles reduces accidental credential use and keeps extensions, cookies, and history from becoming one large trust zone.

##### What a URL broker can and cannot prevent

A broker can block an unexpected scheme, prompt before dispatch, rewrite a URL, route a known domain to a specific browser profile, or send everything unknown to an isolated machine. It replaces one global default with an explicit policy.

It cannot inspect a page and declare it safe. It cannot guarantee an allowlisted domain has not been compromised. It cannot stop a phishing link clicked from inside an already-open browser. It cannot compensate for an unpatched browser, weak identity controls, or a VM configured with broad host access.

I make a smaller claim. Taking ownership of URL dispatch can reduce my exposure and the possible blast radius. More importantly, it turns "open every link here" from an ambient assumption into a decision I can inspect and change.

##### Available macOS URL routers

These are the credible browser pickers and routers I found while revisiting the setup. Smaller utilities may exist outside this list.

| Tool | Approach | Useful distinction |
| --- | --- | --- |
| [Finicky](https://github.com/johnste/finicky) | JavaScript rules and rewrites | Open source and configuration-as-code; my choice |
| [Choosy](https://choosy.app/) | Prompting and configurable rules | Mature commercial "smarter default browser" |
| [Velja](https://sindresorhus.com/velja) | Native picker and routing | Polished Mac utility with tracking-link conveniences |
| [Browserosaurus](https://browserosaurus.com/) | Prompt on each link | Open-source browser chooser; repository was archived when checked |
| [OpenIn](https://loshadki.app/openin4/) | Rules across browsers, profiles, mail, and apps | Broad GUI-managed routing |
| [Bumpr](https://getbumpr.com/) | Click-time browser or mail-app choice | Simple, deliberate chooser |
| [Switchbar](https://webcatalog.io/en/switchbar/) | Browser, profile, and email-client picker | Supports macOS and Windows |
| [Yojam](https://github.com/fluffypony/yojam) | Rules for browsers, apps, and profiles | Newer open-source native option |

I also found smaller open-source implementations including [Bowzer](https://github.com/lailo/bowzer), [multi_browse](https://github.com/vismaytiwari/multi_browse), [Punt](https://github.com/mpalczew/punt), [LinkGate](https://github.com/1789984734/LinkGate), and [Usher](https://github.com/amit-t/usher). Their maturity, maintenance, and licensing vary. Browser routers are popular small projects; evaluate the code that will receive every external URL on your machine.

I could have stopped at a graphical picker if I only wanted to choose between browsers. I wanted the policy in my dotfiles, and I wanted an arbitrary command to be a valid destination. Finicky gave me the useful base without forcing the final shape of the solution.

##### My policy: known local, unknown elsewhere

I made Finicky the default `http` and `https` handler. Its rules send a small set of expected work destinations to local Safari. The default route does the opposite of a normal browser configuration: everything not explicitly matched goes to `SafeBrowser`.

A simplified version looks like this:

```js
module.exports = {
  defaultBrowser: {
    name: "SafeBrowser",
    appType: "command",
    command: "/Users/me/.bin/safebrowser",
  },
  handlers: [
    {
      match: finicky.matchHostnames([
        /.*\.?github\.com$/,
        /.*\.?example-work-service\.com$/,
      ]),
      browser: "Safari",
    },
  ],
};
```

I care more about the default than the allowlist. New and misspelled domains should not silently inherit local trust just because I have not classified them yet. I prefer them to leave the host until I deliberately add a rule.

I do not trust loose hostname matching here. A substring test for `github.com` could also match `github.com.attacker.example`. Finicky parses the URL and provides hostname matchers, but I still anchor and review my patterns. Internationalized domains, redirects, URL shorteners, and compromised subdomains are why I never want a matching hostname to become synonymous with safety.

##### Do not run the untrusted browser on the work machine

I considered a separate local browser profile, and for many workflows that may be enough. It still shares the kernel, filesystem permissions, browser installation, and many host integrations. For unknown links, I prefer paying the extra operational cost of a VM in exchange for a stronger boundary.

My fallback browser runs on another macOS machine represented as a VM endpoint. `SafeBrowser` is a short AppleScript executable:

```applescript
#!/usr/bin/osascript

on run argv
  set remoteMachine to first item of ¬
    (read (POSIX path of (path to home folder as string) & ¬
    ".config/vm1.local/eppc") using delimiter "\n")

  tell application "Finder" of machine remoteMachine ¬
    to open location (item 1 of argv)
end run
```

The script reads the Remote Apple Events address from a private file and asks Finder in the guest to open the URL. The address and credentials do not belong in dotfiles or logs.

There are other ways to deliver a URL to a browser VM:

- SSH can invoke a forced guest command that validates one URL argument and calls `open` or `xdg-open`.
- An authenticated listener can receive URLs over a host-only network.
- A one-way queue can carry URLs to a consumer in the guest.
- A hypervisor's guest-control command can open the URL directly.

If I replaced Remote Apple Events today, I would probably begin with SSH because it is portable and already understood. I would not construct `ssh guest "open $url"` and let an attacker-controlled URL become shell text. I would pass encoded data or a separate argument into a narrowly constrained command, validate the scheme and length, and avoid a general-purpose remote shell key.

I am willing to sacrifice convenience to preserve this boundary. Shared clipboard, drag-and-drop, mounted host directories, saved work credentials, broad Remote Apple Events access, and guest-control tools all create paths back toward the host. I disable what the browser does not need, keep the guest patched, and prefer snapshots or disposable instances when practical. I treat the host-to-guest opener as security-sensitive software, not harmless desktop glue.

##### The practical hardening checklist

My implementation is small, but its policy is explicit:

1. Register a broker instead of a browser as the macOS handler for `http` and `https`.
2. Keep local routing rules narrow and anchored to parsed hostnames.
3. Make the unknown case the isolated case.
4. Accept only intended schemes at the forwarding boundary.
5. Pass URLs as arguments, never interpolated shell code.
6. Keep router configuration and scripts versioned in dotfiles.
7. Protect the router config, executable, and remote-control credentials from untrusted writes.
8. Remove VM integrations that contradict the isolation goal.
9. Do not place valuable authenticated sessions in the untrusted browser.
10. Test known, unknown, malformed, shortened, and non-HTTP URLs.

There is still judgment in the allowlist. A locally opened service can link to the rest of the web. A trusted domain can be compromised. The configuration is a reduction of implicit trust, not a declaration that certain hostnames are permanently harmless.

##### Take ownership before the browser starts

The security industry spends enormous effort hardening browsers after they receive a URL. That work is essential. I had paid far less attention to the decision that sends a URL into a particular browser in the first place.

Making Finicky the OS handler gave me a policy point. Adding a script destination let that policy cross the machine boundary. Running unknown links in a VM kept routine convenience while moving hostile parsing and low-trust sessions away from my work environment.

A link is not harmless because it is small enough to fit in a message. It is an instruction to begin a network interaction in some security context. The operating system already asks which application should receive that instruction. I would rather answer with code than with one global default I stopped questioning years ago.

### [Give the child a name](https://www.lukasrieder.com/blog/give-the-child-a-name/)

How an early rename helps define the boundary of long-running software projects.

Before Plantronic became a substantial production-planning system, it needed a name that described the system rather than the organization around it.

The earliest visible milestone in 2025 is almost comically small: references in the project wiki changed from an earlier working name to **Plantronic**. No scheduler became faster. No factory data was imported. No user got a new screen. Yet the rename marked an important transition.

The name was chosen by the client's project manager, not invented by the software team as a branding exercise. It survived all the way into production and rallied dozens of early adopters behind one recognizable initiative.

A client project often begins as "the tool for this team" or "the new version of that spreadsheet." Those names are convenient while a problem is still local. They become restrictive once the software coordinates planning across teams and source systems.

A proper name creates a boundary. Plantronic was not the factory, the source system, or a single planning view. It was the application responsible for turning several kinds of operational information into an inspectable planning process.

##### A working title is already teaching the organization

Introducing software into an organization starts long before production. The name appears in meetings, invitations, tickets, demonstrations, documentation, support requests, and conversations between future users. Every repetition teaches people what the thing is and where it belongs.

That is why I hate working titles that do not make it to production. A working title does not remain an innocent placeholder once people begin using it. It builds a shared mental model. Changing it later means more than renaming modules and links: the organization must unlearn one identity while learning another. For a while, both names coexist, and every conversation must spend time establishing whether everyone is still talking about the same thing.

There are exceptions. A small discovery group may need a disposable label, and confidential work may require a code name. But once a project is introduced to the people expected to adopt, operate, or explain it, the name should be treated as durable.

Plantronic demonstrated the benefit. When someone used the name, there was never doubt about where the project was heading or who should communicate with whom about what. The name gave everyone a stable point of reference while the software itself was still taking shape. It helped materialize an idea into an organizational reality.

##### Names become architecture

A name alone cannot rescue a muddled design. It can, however, make muddle easier to see.

Once the application had its own identity, questions became sharper:

- Which concepts belong inside Plantronic?
- Which facts remain owned by external systems?
- Which changes are planning decisions rather than imported truth?
- Which reports explain Plantronic's state, and which merely repeat source data?

Those questions appeared repeatedly during implementation. Task routing, team workloads, process transitions, SAP-facing imports, public views, and reporting all needed boundaries. The project name became shorthand for one side of those boundaries.

This matters especially in consulting work. A code name borrowed from a customer or an existing system quietly encourages accidental coupling. Developers reproduce vocabulary and assumptions without deciding whether they belong in the new product. A distinct name gives the team permission to model the problem deliberately.

##### Rename while it is still cheap

Renaming later is expensive. The old term spreads into repositories, modules, database identifiers, deployment scripts, documentation, and conversation. Even when a technical rename is possible, the mental model lingers. The social migration is usually larger than the code diff.

January 2025 was early enough. The wiki change preceded most implementation milestones and happened before broad adoption hardened the earlier label. From the first adopters through production, Plantronic remained Plantronic.

Looking back after the project was completed, I like that its history begins this way. Long projects are remembered through large releases and difficult migrations, but their direction is often set by smaller acts of clarification. Giving the child a name was one of them.

### [Stop typing `git push --set-upstream`](https://www.lukasrieder.com/blog/stop-typing-git-push-set-upstream/)

One Git setting removes friction when publishing a branch for the first time.

Early in January 2025, I made a small change to my dotfiles:

```ini
[push]
    autoSetupRemote = true
```

It removed a small interruption that I had accepted for far too long.

Whenever I created a local branch and pushed it for the first time, Git used to stop and tell me that the branch had no upstream.
It also printed the command I was expected to run:

```sh
git push --set-upstream origin my-branch
```

The message was helpful once, but the repetition was not. Nearly every time, I wanted exactly what Git suggested: publish this branch to my normal remote under the same name, then track it.

Git could have gone one step further by suggesting the configuration that changes the default behavior, rather than only printing the immediate command.

With `push.autoSetupRemote` enabled, the first plain `git push` does that setup automatically.
Later pushes work as before. This is not an alias wrapping Git and it does not invent a new workflow.
It tells Git that, when no upstream exists, my usual intent is obvious enough to act on.

##### Convention over configuration, configuration over repetition

I am careful about convenience settings that hide meaningful state. Remotes matter.
Tracking relationships matter. A push can publish work. I still want to know which repository I am changing.

But typing `--set-upstream origin` was not making me reconsider any of those things.
In repositories with one conventional remote, it was ceremony after the decision had already been made.

That distinction is useful beyond Git. Automate repeated mechanics, not unresolved choices.
If I regularly worked with several equally plausible push remotes, I would leave this setting off.
Git stopping would then protect a real decision.

In my normal repositories, there is one clear remote and matching branch names are intentional. Working with multiple remotes is the exception for me.

### [How we rewrote Highsnobiety without taking it offline](https://www.lukasrieder.com/blog/how-we-rewrote-highsnobiety-without-taking-it-offline/)

Highsnobiety moved from a full WordPress site to a React frontend and an Elixir content stack over two years, with no editorial freeze and no big-bang launch. Change data capture, a PostgreSQL document model, GraphQL, and fallback-backed edge composition made the old and new systems coexist.

The best kind of production refactor is one that readers never notice.

Highsnobiety's platform rewrite took roughly two years. Throughout that period, highsnobiety.com remained a high-traffic publication operating around the clock. More than sixty editors continued writing, revising, scheduling, and publishing through the familiar WordPress administration interface. There was no content freeze while a replacement database was filled, no weekend migration window, no maintenance page, and no single launch at which the entire new stack had to be correct at once.

Behind that continuity, the site changed almost completely.

The WordPress administration interface remained the editorial source of truth. MySQL row changes were captured from the replication log and sent through Kafka. Custom Elixir consumers converted WordPress's relational rows, metadata, shortcodes, and PHP-serialized values into content components. Another Elixir service loaded those components into a document model built in PostgreSQL. A lean Elixir GraphQL API exposed that model to a server-rendered React frontend. Finally, a Cloudflare Worker allowed React to replace selected regions within live WordPress pages while retaining the old markup as a fallback.

I architected the migration and built substantial parts of the backend, API, data model, deployment path, and edge handover. It was not the work of one person: frontend, editorial-product, operations, and content-platform engineers turned the architecture into a production system and continued to refine it. The result was an engineering achievement of coordination as much as code.

##### The rewrite was constrained by the business, not the technology

"Rewrite WordPress in React" sounds like a frontend project. The real system included editorial workflows, custom fields, post types, shortcodes, media, taxonomies, embeds, scheduled publishing, caching, analytics, search, advertising, previews, and years of irregular content. WordPress was both the website renderer and the newsroom's working environment.

A big-bang plan would have treated all of that as one release:

1. Build the replacement CMS and frontend.
2. Train the editorial team.
3. Stop all writes to the old system.
4. Export and transform the database.
5. Import the transformed content into the replacement.
6. Switch the production traffic.
7. Discover the production-only assumptions under full load.

That plan optimizes the diagram rather than the business. It joins unrelated risks into one irreversible moment. A missed shortcode can corrupt an old article. A changed slug can lose search traffic. A broken preview can stop the newsroom. The new renderer and authoring interface can fail independently, yet a big-bang cutover asks both to succeed together.

We inverted the sequence. We kept the proven write path, mirrored every change into the new read path, and verified the new representations while the old site continued to serve readers. Rendering ownership moved in small pieces, and the old path could be removed only after it had become irrelevant.

This was refactoring at the scale of an entire system: preserving externally useful behavior while replacing the implementation beneath it.

##### The architecture had one source and two projections

WordPress remained the only place where editors wrote content during the migration. We did not build temporary dual-write forms or ask editors and application code to save into two schemas. Dual writes would have created a new consistency protocol at every WordPress mutation point, including plugins and scripts that we did not control.

Instead, MySQL's replication stream observed changes after WordPress made them. The local production evidence identifies **Maxwell** as the binlog reader. It emitted row-level inserts, updates, deletes, transaction IDs, timestamps, old values, and commit markers into a Kafka topic. That detail is important: the architecture used change data capture, but the historical implementation should be named accurately.

Kafka decoupled capture from interpretation. WordPress could complete its write without waiting for the new document model or React. Consumers could stop, restart, and resume from their offsets, and the lag was measurable. Raw changes and transformed content occupied separate topics, so the parsing of WordPress semantics and the loading of PostgreSQL remained independently deployable steps.

{% include diagram.html post="2026-08-22-how-we-rewrote-highsnobiety-without-taking-it-offline" diagram="live-migration" caption="Diagram: Editorial work stays on one write path. Each committed MySQL change becomes a component projection for the new read stack, while Cloudflare moves visible ownership only after the React path is ready." %}

This was continuous synchronization, not magical simultaneity. Kafka offsets and pipeline lag existed, and both were monitored. In a practical editorial sense, "perfectly in sync" meant that changes propagated continuously and deterministically without a manual export window; it did not mean that MySQL and PostgreSQL committed the same transaction at the same instant.

That distinction made the failure handling honest. Temporary lag delayed the new projection, but it did not prevent an editor from saving or take the WordPress site down.

##### The transaction boundary survived the row stream

WordPress content rarely changes in a single row. Publishing a post can update `wp_posts`, several `wp_postmeta` rows, term relationships, taxonomy counts, attachment metadata, and revision state within one MySQL transaction. Transforming each row as an isolated event could expose a half-built post or repeatedly overwrite a component with partial data.

The Elixir consumer buffered events that shared a transaction ID. A commit-marked event closed the buffer. Only then did the transformer process the statements, reduce multiple updates for the same content reference, and emit consolidated component events.

The core control flow was compact:

```elixir
def handle_event(%{"xid" => _txid, "commit" => true} = event, txbuffer) do
  handle_transaction(txbuffer ++ [event])
  {:ok, :ack, []}
end

def handle_event(%{"xid" => _txid} = event, txbuffer) do
  {:ok, txbuffer ++ [event]}
end

def handle_transaction([], events) do
  events
  |> ReduceEvents.reduce()
  |> produce()
end
```

The output records used stable references such as `wp_posts_1412981`. Kafka partitioning hashed that reference, preserving a useful order for changes to the same object. Actions were mostly `upsert` or `merge`, which allowed replays and retries without requiring every consumer to know whether the projected row already existed.

The same design instinct appears in [Connecting APIs with Flow](https://www.lukasrieder.com/posts/connecting-apis-with-flow/), where a smaller Elixir integration pipeline preserves the failed stage and its accumulated context instead of turning the whole stream into an opaque error. Here, transaction buffers and stable references extended that approach to continuous content synchronization.

The transformer was a semantic boundary. A WordPress post was not copied into PostgreSQL as a wide row with every plugin column. It became a document with a role, a stable reference, data, ordered child components, and named relationships.

For example, an article header could be an embed, an HTML body, an external video with a still image, full-screen media, a gallery, or an annotated gallery. The source values lived across `wp_posts` and many `wp_postmeta` records. The transformer assembled them into a component shape that React understood:

```json
{
  "action": "upsert",
  "role": "post-header-gallery",
  "ref": "hs_post-header_1006",
  "components": [
    {"role": "image", "ref": "wp_posts_60000"},
    {"role": "image", "ref": "wp_posts_60001"},
    {"role": "image", "ref": "wp_posts_60002"}
  ]
}
```

The parser also translated WordPress shortcodes into nested components and maintained sanity checks around word and shortcode counts. This was an essential part of the migration. The infrastructure could move every row flawlessly while silently losing the meaning of an article, so structural and content-level checks had to coexist.

##### The existing corpus entered through the same event grammar

Live capture handles the next edit, not the previous decade of posts. We needed a bootstrap process without creating a separate set of bulk-import semantics.

The management tool selected WordPress IDs or slugs, parsed post content to discover referenced attachments, and inserted bootstrap requests for the relevant `wp_posts` and `wp_postmeta` rows into Maxwell's bootstrap table. The bootstrap rows then traveled through the same transformer and loader as the live changes.

That choice reduced split-brain logic. One parser interpreted both the historical and current WordPress structures, and one loader applied both. Tests could use captured row events and expected transformed JSON. If the parser changed, selected content could be bootstrapped again rather than patched through one-off SQL.

The bootstrap and ongoing replication still required careful ordering because a backfill can race with newer edits. Stable source references, upserts, event offsets, and controlled bootstrap modes made that tractable; blind `INSERT` statements would not have done so. The operational tooling included resetting document data, requesting focused post bootstraps, and watching consumer lag. We could repair one article without rerunning the entire publication.

This is one reason the event log mattered beyond throughput: it gave the migration a repeatable unit of work.

##### PostgreSQL became a document database on purpose

The new content store used PostgreSQL 9.6 rather than a separate document product. Its core table was deliberately small:

```sql
CREATE TABLE documents (
  id serial NOT NULL PRIMARY KEY,
  role varchar(255),
  ref varchar(255),
  data jsonb
);

CREATE UNIQUE INDEX documents_ref_idx ON documents (ref);
CREATE INDEX documents_role_idx ON documents (role);
CREATE UNIQUE INDEX post_slug_idx
  ON documents ((data->>'slug'))
  WHERE role = 'post';
```

The `role` selected the component kind. The `ref` preserved the source identity and supported idempotent upserts. The `data` held role-specific fields. Ordered child IDs and named relationships represented the content tree and its cross-links.

The loader merged partial JSON updates instead of replacing an entire document:

```sql
INSERT INTO documents AS d (ref, role, data)
VALUES ($1, $2, $3)
ON CONFLICT (ref) DO UPDATE SET
  role = COALESCE(EXCLUDED.role, d.role),
  data = COALESCE(d.data, '{}'::jsonb) || COALESCE(EXCLUDED.data, d.data)
WHERE d.ref = EXCLUDED.ref
RETURNING id, ref, role;
```

The Elixir loader recursively upserted nested components, saved ordered children, and created named relationships such as `post-header`, `featured_image`, or `still`. Each transformed event could update only the fields represented by one WordPress row without erasing fields assembled from others.

PostgreSQL gave us transactions, indexes, constraints, recursive queries, and JSONB within the same system. Recursive common table expressions expanded the ordered document tree by slug while guarding against cyclic references. Partial indexes enforced post-slug uniqueness without pretending that every component had a slug.

This model deserves its own article because "PostgreSQL as a document database" can describe either a carefully considered hybrid or an unstructured dumping ground. Here, relational columns carried identity and topology, while JSONB carried the role-specific payload. The query patterns, constraints, and upsert behavior shaped the boundary.

I returned to that distinction in [I added task metadata. The next field still got a column.](https://www.lukasrieder.com/blog/i-added-task-metadata-the-next-field-still-got-a-column/). Although the later system was much smaller, the principle remained the same: JSONB could absorb externally owned variation, while explicit columns retained identity and behavior.

##### GraphQL insulated the frontend from projection internals

The new React team did not query `documents`, `document_relations`, and JSONB expressions. The Elixir content API presented a typed GraphQL schema: `Post`, `Body`, `Image`, `Gallery`, `Quote`, `PostHeaderGallery`, `ExternalVideo`, terms, authors, and component unions.

That schema was the translation layer between generic storage and the product vocabulary. The backend could retrieve a recursive tree and resolve role-specific fields while the frontend requested the exact data needed for a route or fragment. The React engineers stayed in JavaScript, Apollo, server rendering, and component design. The backend engineers stayed in Elixir, Ecto, PostgreSQL, and GraphQL execution. The contract joined the ecosystems without forcing either team into the other's implementation.

The API was intentionally lean. The repository history shows that it was reduced from a Phoenix application to Cowboy and Plug once the full framework no longer served the boundary. It loaded the GraphQL schema, parsed and validated documents, resolved them against the content store, exposed a health endpoint, and recorded errors and query metrics.

The typed schema also surfaced incomplete parts of the migration. Unknown or invalid projected components could become placeholders rather than accidentally malformed objects. The frontend could handle an explicit absence. The schema's evolution tracked which content capabilities had crossed from the WordPress representation into the new platform.

A later example, [I made translations editable without replacing Gettext](https://www.lukasrieder.com/blog/i-made-translations-editable-without-replacing-gettext/), applies the same contract-preserving approach at a smaller boundary. Callers keep one stable translation API while runtime variation remains hidden behind it, just as the React frontend depended on a content contract rather than the projection's storage details.

GraphQL was not valuable because it removed the need to model content. It was valuable because it made the model explicit, negotiated, and visible.

##### React took over the page one region at a time

By the later stages of the migration, the React frontend supported server rendering, client hydration, routing, GraphQL data, a shared component library, magazine and commerce concerns, fragment builds, and full application builds. We did not, however, wait for all of that work to be complete before replacing the first pixel.

We began with bounded regions such as the header and footer. The WordPress template emitted both the old implementation and a fragment declaration. Cloudflare fetched the server-rendered React replacement, streamed it into the page when successful, and kept the WordPress block after a timeout or error.

The detailed mechanics and full Worker source are in [how I built edge-side includes on Cloudflare Workers](https://www.lukasrieder.com/blog/how-i-built-edge-side-includes-on-cloudflare-workers/).

That edge seam created several rollout controls at once:

- WordPress feature flags enabled external fragments globally or by feature.
- Query parameters allowed deliberate checks on individual requests.
- The Worker skipped administration, uploads, includes, and other unsafe paths.
- The fragment timeout limited the damage from a failing frontend service.
- The old markup was a response-local fallback, not a remote rollback dependency.
- The React fragment server returned assets and hydration metadata for the enabled regions.
- Cache controls and tags allowed the shell and fragments to evolve independently.

After the shared chrome came larger sections and article concerns. Eventually, the React application could own an entire route rather than a region. At each stage, the old site remained both the reference implementation and the safety net.

This was the strangler pattern applied in two dimensions. The data path displaced WordPress reads by building a synchronized projection. The delivery path displaced WordPress rendering by replacing individual output regions. Neither required the immediate replacement of the editorial write path.

##### Zero downtime came from reversibility

No architecture can guarantee an incident-free migration merely by drawing redundant arrows. What made this zero-downtime rollout possible was the repeated decision to avoid irreversible steps.

If the capture process lagged, WordPress still accepted writes and served pages. If the transformation rejected an unusual shortcode, the old renderer still understood it. If the PostgreSQL loader retried after a deadlock, Kafka retained the work. If GraphQL lacked a component type, React did not need to own that page yet. If the fragment service failed, the edge wrote the WordPress fallback. If browser hydration broke, the server-rendered HTML still existed. If one region behaved badly, a feature flag removed that region rather than reverting the entire platform.

Every new stage could fail safely behind the old behavior.

Reversibility also focused the teams. The frontend team did not need to recreate a complete CMS before shipping the header. The backend team did not need to solve every historical post before the GraphQL contract could be exercised. The editorial team did not need to learn an unfinished tool. The operations team could watch lag, error rates, fragment fallbacks, and caches under real traffic while the blast radius remained bounded.

That operational responsibility also runs through [How much cloud fits into one server?](https://www.lukasrieder.com/blog/how-much-cloud-fits-into-one-server/). The deployment is smaller, but it follows the same principle: an application is not finished when its code works; its startup, migrations, monitoring, recovery, and rollout behavior are part of the design.

This is why the migration could run for two years without becoming two years of frozen product work. The old and new platforms were not competing branches waiting for a merge day. They were cooperating production systems in which the old platform owned progressively less.

##### What "no one noticed" really means

Readers did see redesigned experiences over time. Editors did receive changes where the product required them. An "invisible refactor" does not mean visual stasis. It means that the infrastructure replacement did not impose a maintenance event or an operational rupture.

The healthy signals were mundane:

- An editor saved a post in WordPress, and the content appeared through the new stack.
- Scheduled posts continued to publish.
- Existing URLs continued to resolve.
- Old and new regions shared a single page.
- A deployment changed one ownership boundary at a time.
- A rollback meant changing a flag or using a fallback, not restoring a database.
- Engineers repaired the pipeline while the publication kept moving.

The pipeline was eventually consistent, so monitoring consumer lag was a form of business monitoring. The parser's correctness mattered as much as message throughput. A cache hit ratio could not excuse stale editorial output. A successful HTTP status could still contain a semantically broken article. Each layer needed a metric tied to the reader and editor contract.

The absence of a dramatic cutover story is a marker of success. The business kept publishing street culture while the platform underneath changed its language, data model, API, rendering, and deployment shape.

##### The architecture sprint came before the implementation sprint

This result depended on deciding the migration boundaries early. WordPress would remain the write authority, and the binlog would be the integration point. Kafka would separate capture, transformation, and loading. Transformed records would use stable references and idempotent actions. PostgreSQL would store component documents, GraphQL would serve as the frontend contract, and the edge fallback would transfer rendering ownership gradually.

Those are architecture-sprint decisions. They reduce the class of future choices that engineers must make under delivery pressure. Once the boundaries held, the teams could move quickly within them.

Without that work, a rushed implementation tends to produce accidental dual writes, one-off exports, route-wide flags, coupled deployments, and a fallback that exists only in a runbook. Each shortcut appears faster for the first demonstration but becomes more expensive at the production cutover.

An architecture sprint is not months of diagrams before code. It is a short, evidence-driven effort to identify the source of truth, seams, invariants, rollback units, and observability before scaling the team. For an existing business, the key question is not "what would the ideal greenfield stack be?" It is "how can the new stack earn one responsibility without requiring the old stack to stop?"

Highsnobiety's answer worked because the migration path was part of the product architecture, not cleanup scheduled after the product work.

##### Several deeper stories remain

This overview compresses several systems that each warrant their own treatment:

- The change-data-capture pipeline deserves a closer look at transaction buffering, bootstrap, parsing, partitioning, replay, and lag. For a smaller example of an Elixir pipeline designed around explicit stages and useful failure context, read [Connecting APIs with Flow](https://www.lukasrieder.com/posts/connecting-apis-with-flow/).
- The PostgreSQL document store warrants an examination of JSONB payloads, ordered trees, named relations, indexes, and recursive CTEs. [I added task metadata. The next field still got a column.](https://www.lukasrieder.com/blog/i-added-task-metadata-the-next-field-still-got-a-column/) explores the same boundary between flexible JSONB payloads and explicit relational behavior in a later system.
- The Elixir GraphQL boundary shows how generic documents became typed content while the React team retained its ecosystem. [I made translations editable without replacing Gettext](https://www.lukasrieder.com/blog/i-made-translations-editable-without-replacing-gettext/) examines a related contract-preserving design: callers retain one stable API while an additional data source remains hidden behind it.
- The architecture-sprint story explains how to design a reversible migration before delivery pressure multiplies the wrong assumptions. [How much cloud fits into one server?](https://www.lukasrieder.com/blog/how-much-cloud-fits-into-one-server/) provides an operational counterpart in which deployment, migrations, monitoring, recovery, and rollout behavior are treated as part of the application architecture.

The edge composition already has [its own companion article, including the full Cloudflare Worker source](https://www.lukasrieder.com/blog/how-i-built-edge-side-includes-on-cloudflare-workers/), because it shows the final handover in unusually concrete form. Yet the Worker was only one detail. The main achievement was the end-to-end sequence that allowed all of those details to cooperate.

React gained its first responsibility while WordPress still owned most of the page. Editors kept working while the databases converged. Backend and frontend teams shared a contract instead of a language. We tested the new system in production before asking it to carry the whole site.

Over time, the fallback fired less often, the React regions grew, the GraphQL model covered more content, and the WordPress renderer receded. Eventually, the old path was no longer a safety net; it was obsolete.

Readers saw new designs, but they never had to wait for the platform rewrite.

### [How I built edge-side includes on Cloudflare Workers](https://www.lukasrieder.com/blog/how-i-built-edge-side-includes-on-cloudflare-workers/)

During Highsnobiety's WordPress-to-React migration, I built a streaming Cloudflare Worker that replaced selected regions of WordPress pages with server-rendered React fragments. It retained WordPress markup whenever the new path failed.

In June 2018, I placed a small parser between Highsnobiety's readers and its WordPress origin. It allowed a single page to contain working WordPress HTML alongside newer server-rendered React, replacing one region at a time without asking the browser to assemble the result.

The parser ran in a Cloudflare Worker. WordPress still produced the page, while a React service produced fragments such as the site's chrome. The Worker fetched both, replaced marked regions while the response was streaming, and sent one composed HTML document to the reader. If a fragment was slow, unavailable, or returned an error, the old WordPress markup remained in its place.

That fallback was the feature. Fast edge composition was useful; making a two-year migration safe was the reason it existed.

This article accompanies my short [Cloudflare Edge Side Includes talk](https://www.lukasrieder.com/talks/cloudflare-edge-side-includes/). The slides show the request as a sequence. Here, I will show the protocol, the complete initial implementation, the parser mechanics, the failure behavior, and the production use that justified it. The larger story is [how we rewrote Highsnobiety without taking it offline](https://www.lukasrieder.com/blog/how-we-rewrote-highsnobiety-without-taking-it-offline/).

##### These were edge-side includes, not an ESI product

"Edge Side Includes" usually refers to the ESI markup language associated with reverse proxies and content delivery networks. Cloudflare Workers did not provide an ESI engine that we enabled. We built the minimum composition protocol Highsnobiety needed on top of the Fetch and Streams APIs.

The distinction matters. We owned the syntax, discovery header, timeout policy, cache behavior, and fallback semantics. Calling the result edge-side includes describes where the composition happened and what it accomplished; it does not claim compliance with the full ESI specification.

The protocol had two channels.

First, the WordPress response advertised fragments in an HTTP header:

```http
X-Fragments: header header-7e1af762 https://frontend.example/fragments/render/header, footer footer-f810a521 https://frontend.example/fragments/render/footer
```

Each entry named a fragment, gave the occurrence a stable key, and supplied a rendering URL. The header allowed the Worker to start every fragment request as soon as the origin response headers arrived. It did not need to scan the entire body before doing useful work.

Second, WordPress marked replacement regions in its HTML:

```html

Working WordPress footer

```

The content between the markers was not a loading spinner. It was the complete old implementation. If the React fragment succeeded, the Worker replaced that block. If anything failed, the Worker wrote the block unchanged.

The first version used a simpler `` syntax. The production system later introduced paired, named markers because explicit boundaries were easier to validate and could represent multiple instances of one fragment. The full source below is the first version that accompanied the original talk. The other examples use the clearer protocol that followed.

##### One request started several pieces of work

A request took the following path:

1. The reader requested an ordinary Highsnobiety URL.
2. The Worker fetched WordPress and added an opt-in request header.
3. WordPress returned a normal HTML response with `X-Fragments` declarations.
4. The Worker started all declared fragment fetches immediately and in parallel.
5. The Worker opened a `TransformStream` and parsed complete lines from the WordPress response body.
6. Ordinary WordPress bytes flowed through as soon as they were available.
7. At a fragment marker, the Worker awaited only that fragment's already-running promise.
8. Successful fragment bytes were piped into the output. A timeout, exception, or non-success status selected the WordPress fallback bytes.
9. Parsing resumed, and a single HTML response continued to the browser.

{% include diagram.html post="2026-08-22-how-i-built-edge-side-includes-on-cloudflare-workers" diagram="edge-fragment-composition" caption="Diagram: WordPress owns the safe page shell, React owns selected fragments, and Cloudflare composes both into one stream. Failure crosses no boundary: the shell already contains its fallback." %}

This avoided the slowest possible implementation: fetching the complete WordPress response, discovering the fragments, requesting them one after another, building a new string, and only then sending the first byte. Header discovery created an early fork. Streaming preserved the time to first byte, while concurrent fragment requests hid much of their latency behind origin rendering and network transfer.

It was still an ordered composition. If the parser reached the header marker and that fragment had not resolved, the output paused there. Later WordPress bytes could not pass an unresolved earlier position without changing the document order. The parallel prefetch reduced that wait; it did not abolish causality.

##### Full source of the first Worker

Below is the complete fragment-composition Worker from the first production commit. The product-specific deployment files and sample fragment server are not part of the listing; this is the entire edge program that fetched, parsed, raced, piped, and fell back.

```javascript
/* Define regular expressions at top to have them precompiled.
 */
const htmlContentType = new RegExp('text\/html', 'i')
const fragmentStart = new RegExp(')?')
const commentEnd = new RegExp('-->')

addEventListener('fetch', event => {
  event.respondWith(main(event.request))
})

/* The main entry function
*/
async function main(request) {
  const response = await fetch(request)
  const fragments = prefetchFragments(response.headers)

  return transformResponse(response, fragments)
}

/* Build a dictionary of promises that we can evaluate later.
 * These fetch or timeout.
 *
 * The overall timeout is shared by each promise. The cumulative amount of time, that
 * all fetch-requests can spend is 10 seconds.
 *
 * Each fetch request defined in the headers gets a fair share.
 * We let the promises race and later fail gracefully when the fetch does not return in time.
 *
 * This is an important circuit-breaker mechanism, to not fail the main request.
 */
function prefetchFragments(headers) {
  const header = headers.get('X-Fragments')
  if (header === null) return {}

  const fragments = {}
  const values = header.split(',')
  const safeTimeout = 10000 / values.length

  values.forEach((entry) => {
    const [key, url] = entry.trim().split(' ')
    const request = new Request(url)
    const timeout = new Promise((resolve, reject) => {
      const wait = setTimeout(() => {
        clearTimeout(wait)
        reject()
      }, safeTimeout)
    })

    fragments[key] = Promise.race([
      fetch(request),
      timeout
    ])
  })

  return fragments
}

/*
 * Here we decide whether we are going to stream & parse the response body,
 * or just return the response as is, since the request is not eligble for fragments.
 *
 * Only Content-Type: text/html responses with one or more fragments are going to be evaluated.
 */
function transformResponse(response, fragments) {
  const contentType = response.headers.get('Content-Type')

  if (
    contentType
    && htmlContentType.test(contentType)
    && Object.keys(fragments).length > 0
  ) {
    const { readable, writable } = new TransformStream()
    transformBody(response.body, writable, fragments)
    return new Response(readable, response)
  } else {
    return response
  }
}

/*
 * This function transforms the origin response body.
 *
 * It assumes the response to be utf-8 encoded
 */
async function transformBody(body, writable, fragments) {
  const encoding = new TextDecoder('utf-8')
  const reader = body.getReader()
  const writer = writable.getWriter()

  // initialise the parser state
  let state = {writer: writer, fragments: fragments}
  let fun = parse
  let lastLine = ""

  while (true) {
    const { done, value } = await reader.read()
    if (done) break
    const buffer = encoding.decode(value, {stream: !done})
    const lines = (lastLine + buffer).split("\n")

    /* This loop is highly optimized
     * Basically it is a parse-tree keeping state between each line.
     *
     * But most important, is to not include the last line.
     * The response chunks, might be cut-off just in the middle of a line, and thus not representing
     * a full line that can be reasoned about.
     *
     * Therefore we keep the last line, and concatenate it with the next lines.
     */
    let i = 0;
    const length = lines.length - 1;
    for (; i  ON fragmentMatch with fallback
 *        > parseFragmentFallback
 *
 *       -> ON fragmentMatch without fallback
 *        > parse
 *
 * parseFragmentFallback
 *                       -> ON closing comment
 *                        > parse
 */
async function parse(state, line) {
  const fragmentMatch = line.match(fragmentStart)

  if (fragmentMatch) {
    const [match, key, fragmentEnd] = fragmentMatch
    const fragmentPromise = state.fragments[key]

    if (fragmentEnd && fragmentPromise) {
      await writeFragment(fragmentPromise, state.writer, line + "\n")
      return [parse, state]
    } else if (fragmentPromise) {
      state.fragmentPromise = state.fragments[key]
      state.fallbackBuffer = ""

      write(state.writer, line.replace(fragmentStart, ""))
      return [parseFragmentFallback, state]
    }
  }

  write(state.writer, line + "\n")
  return [parse, state]
}

/*
 * This is a sub-state, that is looking for a closing comment --> to terminate the fallback.
 * It will keep buffering the response to build the fallback buffer.
 *
 * When it finds a `-->` on a line, it will attempt to write the fragment.
 */
async function parseFragmentFallback(state, line) {
  if (commentEnd.test(line)) {
    await writeFragment(state.fragmentPromise, state.writer, state.fallbackBuffer)
    state.fragmentPromise = null
    state.fallbackBuffer = null

    write(state.writer, line.replace(commentEnd, "\n"))
    return [parse, state]
  } else {
    state.fallbackBuffer = state.fallbackBuffer + line + "\n"
    return [parseFragmentFallback, state]
  }
}

/*
 * This is called after traversing all lines.
 * If we have accumulated fallback buffer until here,
 * we might want to dump the response, because someone forgot to add an closing '-->' comment tag.
 */
async function endParse(state) {
  if (state.fallbackBuffer !== null) {
    write(state.writer, state.fallbackBuffer)
  }
}

/*
 * This function handles a fragment.
 * In order for a fragment to render, it must fetch in time and respond with a success state.
 *
 * The function will attempt to resolve the promise and pipe any successful response directly
 * to the main response. Blocking until the fragment response is consumed.
 *
 * If the fragment does not respond in time (a timeout happened), we attempt to render a fallback.
 *
 * If the fragment response is not succesful, we attempt to render a fallback.
 */
async function writeFragment(fragmentPromise, writer, fallbackResponse) {
  try {
    const fragmentResponse = await fragmentPromise

    if (fragmentResponse.ok) {
      await pipe(fragmentResponse.body.getReader(), writer)
    } else {
      write(writer, fallbackResponse)
    }

  } catch(e) {
    write(writer, fallbackResponse)
  }
}

/*
 * Helper function to pipe one stream into the other.
 */
async function pipe(reader, writer) {
  while (true) {
    const { done, value } = await reader.read()
    if (done) break
    await writer.write(value)
  }
}

/*
 * Helper function to write an utf-8 string to a stream.
 */
async function write(writer, str) {
  const bytes = new TextEncoder('utf-8').encode(str)
  await writer.write(bytes)
}
```

A small implementation did not imply trivial behavior. It coordinated two or more origins, parsed arbitrary network chunks, preserved byte order, enforced a latency budget, and retained the old output as a recovery path.

##### Network chunks are not parser tokens

The subtle part begins in `transformBody`.

A `ReadableStream` yields chunks whenever bytes happen to be available. Chunk boundaries know nothing about lines, comments, UTF-8 characters, or HTML elements. One chunk can end halfway through `<!-- frag`, and the next can begin with `ment:footer`. Treating each chunk as a parseable string would occasionally miss markers or corrupt text under real network conditions.

The Worker decoded the response incrementally, prepended the unfinished tail from the previous read, split on each newline, and retained the final partial line. Only complete lines entered the state machine. That is why the origin templates placed protocol markers on deliberate line boundaries.

The parser had two states, each represented by a function.

`parse` copied normal lines. At a one-line marker, it selected a fragment immediately. At an opening marker with fallback content, it switched to `parseFragmentFallback` and accumulated the old HTML. The fallback parser returned to `parse` at its closing marker. Returning the next function together with the new state made the transitions explicit without introducing a parser framework.

The later paired markers improved this contract. A complete opening and closing marker carried the same fragment name and ID, making malformed boundaries and repeated fragments easier to reason about. The production tests covered markers split across stream reads, complete and incomplete fragments, several fragments in one response, ordinary content before and after them, and fallback behavior.

##### The timeout was a circuit breaker, not an optimization

`prefetchFragments` stored promises rather than responses. Starting all fetches populated a lookup table immediately; parsing resolved each entry only when it reached that position.

Every promise raced its fetch against a timeout. The initial implementation divided a ten-second budget by the fragment count. Later production code used a shorter fixed timeout and logged non-success responses. Both policies enforced the same invariant: a new fragment service could not hold the established site hostage indefinitely.

This is bulkhead-like behavior at the scale of a page region. WordPress remained capable of rendering the page, while the new React service received a bounded opportunity to improve one region. A failure consumed a known amount of latency and then fell back within that region.

The fallback also handled ordinary HTTP failure. A completed `404` or `500` was not valid fragment output merely because the network request resolved. `writeFragment` required an OK response. Exceptions and timeouts followed the same old-content branch.

There was no retry within the reader's request. Retrying would have consumed more latency, complicated the budget, and risked amplifying a struggling service. The cache and the next page request provided another opportunity. The current user received working legacy HTML immediately.

##### Highsnobiety used the seam to migrate in public

The practical example was not a demonstration page assembled from a weather widget and a stock ticker. Highsnobiety was replacing a complete WordPress-rendered publication with a React frontend backed by a new Elixir content stack while the site remained busy and the editors continued publishing.

A WordPress Twig helper registered each enabled fragment in `X-Fragments`. Its template emitted the paired comments and rendered a legacy block inside `data-target="react"`. Feature flags could enable all external fragments or a single fragment type, and query parameters supported deliberate testing. Requests without Worker support simply received the WordPress output.

The React fragment server accepted routes such as `/fragments/render/footer`. It rendered the component on the server, included the Apollo cache and props needed for hydration, and exposed metadata for fragment-specific CSS, JavaScript, locale, GraphQL endpoints, session context, tracking, and error reporting. The Worker eventually fetched that metadata beside the HTML fragments and inserted assets into the document head and body.

Consider one request from beginning to end:

1. WordPress began rendering an article with its existing header, article body, and footer.
2. The header and footer templates emitted valid legacy markup between the fragment markers.
3. WordPress added two `X-Fragments` header values that pointed to React rendering routes.
4. Cloudflare received the response headers and immediately started both React requests.
5. The WordPress article bytes began flowing through the Worker toward the reader.
6. When the parser reached the header marker, the React response was ready. The Worker piped the server-rendered React header instead of the buffered WordPress header.
7. The article body did not yet have a marker, so the WordPress article continued unchanged.
8. During a simulated failure, the footer request exceeded its deadline. The Worker wrote the WordPress footer that was already present in the page.
9. The reader received one coherent document: a new React header, the old WordPress article, and the old WordPress footer.
10. On a later request, the healthy footer service replaced the footer as well. No route switch or editorial republication was required.

The same page could therefore contain migration states that would normally require separate releases or proxy routes. Ownership moved at the granularity of a component.

The header and footer were good early candidates because they appeared everywhere and had clear visual boundaries. They exercised deployment, server rendering, hydration, assets, caching, and fallback without requiring every article type to be complete. Once confidence grew, larger regions such as article rendering could cross the same seam. WordPress did not disappear in one dramatic launch. It gradually faded behind its replacements.

##### Cache composition needed explicit ownership

Edge composition only helps if the caches do not make the result incorrect.

The shell and fragments had different rates of change. A WordPress article could remain cacheable while the navigation or footer changed on another schedule. Fragment responses later carried cache-control directives and cache tags, while the metadata described the assets for the exact fragment build. The Worker code preserved the origin's response metadata while preventing browser cache behavior from bypassing the edge policy.

The keys mattered. The marker's occurrence ID joined the declaration to its position in the body. Fragment names selected the implementation and assets, while a deployment version identified the generated bundles. Mixing an old server-rendered fragment with incompatible new hydration code would turn a successful edge substitution into a client failure, so the HTML and asset metadata needed a shared release identity.

Personalization also changes the scope of a cache. Forwarding authorization during staging was necessary, but authenticated or geography-sensitive fragments cannot be shared under a cache key that ignores those dimensions. The safe default is simple: cache only output whose variation is represented in the key, and keep the fallback usable without the personalized fragment.

##### Useful beyond migrations

This pattern fits several practical cases:

- **Strangler migrations.** A team can replace a navigation bar, recommendation rail, commerce module, or article body before replacing the page's owner.
- **Independent cache lifetimes.** The edge can compose a long-lived article shell with shorter-lived navigation or availability data.
- **Polyglot delivery.** Separate stacks can render bounded HTML without moving the composition and error handling into the browser.
- **High-risk launches.** The last known server-rendered implementation can remain inline until the new service has earned trust.
- **Organizational boundaries.** Teams can own deployable page regions through an explicit protocol rather than a shared template repository.

It is a poor fit when fragments require strict atomic consistency, when many sequential boundaries create a latency waterfall, when the fallback cannot work independently, or when scripts and styles cannot be isolated. It also introduces a distributed system into page rendering. Tracing, timeout budgets, cache keys, versioning, and malformed-markup tests are not optional.

##### What I would change now

I would retain the header discovery, concurrent fetches, streaming substitution, and origin-owned fallback. Those four decisions created most of the value.

I would make the parser framing independent of line breaks. A streaming tokenizer can carry partial markers directly instead of requiring the template formatting to align with newline boundaries. I would abort timed-out fetches rather than merely reject the competing promises. I would propagate one request-scoped deadline to the origins, then allocate the remaining time instead of counting fragments. I would make output cancellation close every reader and writer. I would also specify cache variation and fragment versioning as part of the protocol rather than introducing them later through metadata.

Cloudflare's current HTML transformation APIs may remove the need for handwritten tokenization in some cases, but they do not determine the failure semantics. The difficult architectural question remains: what does the reader receive when an included service is late or wrong?

For Highsnobiety, the answer was never a blank rectangle or a failed page. It was the WordPress version that had served readers before the migration began.

That made edge-side includes more than a rendering trick. They became a reversible handover mechanism. The new React stack could take visible responsibility for one region at a time, and every request carried its own rollback. That is how a small edge detail helped a large rewrite remain invisible.

### [The tides of freelancing](https://www.lukasrieder.com/posts/the-tides-of-freelancing/)

On using freelancing to negotiate time, motivation, intense work, and necessary rest.

This week, a colleague approached me with several questions about freelancing. He asked about contracts, taxes, and the usual practical details. As we talked, he explained that he was considering freelancing as a way to gain more control over the long hours and compensation at his startup.

I gave him a long answer. This article captures its central idea.

![tides](https://raw.githubusercontent.com/Overbryd/overbryd.github.io/source/static/the-tides-of-freelancing/tides.jpg)

Freelancing can provide a stronger position from which to negotiate time and money. That position only matters when you exercise it; otherwise, you can still end up in an arrangement that satisfies neither you nor your client.

For me, this is closely connected to one of the greatest strengths I bring to a project: a **very high level of motivation**.

Time is the most precious resource in my life. If all goes well, I may have about eighty years of it. I choose to participate in society and contribute to it, so work is naturally one of the things on which I want to spend that time.

Look at a day or a week and consider how much of it goes to work. Because I charge an hourly rate, I track my productive time meticulously and record it to the minute. I can proudly point to weeks in which I logged more than fifty hours. I can be equally proud of months containing weeks with fewer than fifteen working hours.

At times, a project urgently needs my skills and I feel deeply motivated by the challenge. I will happily contribute my time, skill, and energy. At other times, I am just as content to spend most of the day with my children, read books, and do nothing conventionally described as productive.

Over the years, I have grown accustomed to this tidal way of working. The intense, fast, and long sessions are closely related to the quiet periods. For me, neither can exist sustainably without the other.

I thrive in critical situations and enjoy difficult problems. When I can commit honestly to a challenge, my mind and body support the effort. I have learned how much energy I can bring to work when I balance it with enough downtime.

Someone may see me work more than fourteen hours a day, three times in one week, and conclude that I work at an unsustainable pace. In reality, I pay close attention to how much time I can devote to work and when I need to stop.

**I need a strong position** from which to negotiate my own time. I cannot let a company or client dictate without limit how much of my life I exchange for money. That control is one of the main reasons I choose freelancing over employment.

##### Set your terms before work starts

If someone is using your willingness to work against you, decide what you are prepared to offer before they decide for you.

* How much of your time are you willing to invest in a cause other than your own?

* How much can you contribute sustainably without risking your health?

* What genuinely drives your motivation, and what do you expect from yourself at work?

Those answers should shape the terms under which you work.

### [50a875be3eac870121c72c3e0951d8c8257db1fc Initializing master branch for publication](https://www.lukasrieder.com/posts/initializing-master-branch-for-publication/)

Restarting a decade-old blog with a new static publishing pipeline.

I restarted my blog because years of project notes had nowhere to go.

I completely reset my publishing pipeline with a new static site generator, repository layout, and template.

My [overbryd.github.io](https://github.com/Overbryd/overbryd.github.io "Overbryd/overbryd.github.io") repository dates back to February 27, 2009. Several articles are now waiting in the new pipeline. Since beginning my career, I have worked professionally in three programming languages and run applications written in each of them in production.

Because I remain a hacker at heart, I will begin by examining every part of this new publishing pipeline along the way.

The restart begins at commit `50a875be3eac870121c72c3e0951d8c8257db1fc Initializing master branch for publication`. The next articles will document the pipeline as I build it.

### [Connecting APIs with Flow](https://www.lukasrieder.com/posts/connecting-apis-with-flow/)

Building a testable, failure-aware API connector with Elixir Flow and GenStage.

This article explains how to build an API connector with Elixir's `Flow`.

Many services need to consume events and act on them. A recent client needed an API connector that subscribed to AMQP events, mapped their data into the target API's structure, and posted the result.

Given the project's deadline and budget, I wanted predictable failures, useful error reports, and thorough test coverage. Elixir's `Flow` handled that combination well.

##### The flow must go on

Most examples of Elixir flows focus on ad hoc use in scripts or the console. `Flow.from_enumerable/1` connects a flow to an enumerable, while `Enum.into/2` turns it into a straightforward parallel processing pipeline.

A continuously running application can instead use `Flow.from_stage/1` and `Flow.start_link/1`. The processing pipeline looks like this:

```
defmodule Pipeline do

  def start_link() do
    pipeline = Flow.from_stage(Pipeline.Dispatcher, max_demand: 100, stages: 10)
               |> flow
    Flow.start_link(pipeline, name: __MODULE__)
  end

  def flow(input) do
    input
    |> put(:item, &Poison.decode/1)
    |> put(:transformed, &transform/1)
    |> put(:response, &post/1)
    |> Flow.each(&report/1)
  end

  def transform(%{item: item}) do
    # returns {:ok, transformed} or {:error, ...}
    TargetApi.object_from_item(item)
  end
  
  def post(%{transformed: object}) do
    # also returns {:ok, ...} or {:error, ...}
    TargetApi.post(object)
  end
  
  def report({:ok, _state}), do: Logger.info("posted successfully")
  
  def report({:error, key, error, state}) do
    Logger.error("processing errored at #{key}: #{inspect error}, #{inspect state}")
  end
  
  defp put(flow, key, fun) do
    Flow.map(flow, fn
      {:ok, state} ->
        case fun.(state) do
          {:ok, result} -> {:ok, Map.put(state, key, result)}
          error -> {:error, key, error, state}
        end
      pass -> pass
    end)
  end

end
```

This design provides two main benefits:

1. The pipeline is straightforward to test.
2. Failures retain enough context for useful error handling.

###### Testability

Passing the flow's input as an argument makes the pipeline easy to isolate in tests. There is no need to connect or start the `Pipeline.Dispatcher` GenStage. In tests, I use `Flow.from_enumerable/1` to provide input and append `Enum.into/2` to collect the pipeline's output.

My tests look like this:

```
defmodule PipelineTest do

  test "flow fails when provided invalid JSON" do
    assert {:error, :item, _error, _state} = run("not json")
  end

  def run(data) do
    [{:ok, data}]
    |> Flow.from_enumerable
    |> Pipeline.flow
    |> Enum.into([])
    |> List.first
  end
  
end
```

Because the pipeline mostly calls other modules, their respective test suites already cover most of its logic. As long as those functions honor the tagged `:ok` and `:error` return values, the pipeline can carry their results consistently into the error-handling stage.

###### Error handling

This flow is a sequence of state transformations. Each result receives its own key and can provide input for later actions. The resulting map is the protocol between functions that depend on previous output. Callers can look up successful values, while errors retain the stage at which they occurred.

Elixir offers several complementary approaches to failure. The "let it crash" philosophy relies on process isolation and supervision so that one unexpected failure does not corrupt unrelated processes. Tagged `:ok` and `:error` return values instead describe failures that callers are expected to handle directly.

Choosing between those approaches depends on the application's users. Here, the first group consists of end users, who should experience as little disruption as possible. The pipeline runs after they interact with the front-facing application and feeds a notification system. They do not see its error output, although a failure may prevent the intended follow-up action.

The second group consists of the developers who maintain that front-facing application. When processing fails and users do not receive the intended result, those developers need to identify the failed stage quickly.

I therefore chose tagged `:ok` and `:error` results for every failure I expected. Each error identifies both what went wrong and where it happened. Unexpected crashes remain under supervision, where process isolation keeps the application running and allows failed parts to restart.

##### Dispatcher

A queueing GenStage provides the pipeline's continuous input and dispatches events on demand. In this case, the AMQP consumer forwards incoming events to the dispatcher like this:

```
Dispatcher.async_push(Pipeline.Dispatcher, {:ok, data})
```

For completeness, here is the implementation of the queueing GenStage producer:

```
defmodule Dispatcher do
  use GenStage

  def start_link(name) do
    GenStage.start_link(__MODULE__, nil, name: name, id: name)
  end

  def init(_) do
    {:producer, {Queue.new, 0}, dispatcher: GenStage.DemandDispatcher}
  end

  def async_push(name, event) do
    GenStage.cast(name, {:push, event})
  end

  def queue_size(name) do
    GenStage.call(name, :queue_size)
  end

  def handle_call(:queue_size, _from, {queue, _demand} = state) do
    {:reply, Queue.size(queue), [], state}
  end

  def handle_cast({:push, event}, {queue, demand}) do
    dispatch_events(Queue.put_front(queue, event), demand, [])
  end

  def handle_demand(incoming_demand, {queue, demand}) when incoming_demand > 0 do
    dispatch_events(queue, demand + incoming_demand, [])
  end

  defp dispatch_events(queue, demand, events) do
    with true  0,
      {event, remaining_queue}  {:noreply, Enum.reverse(events), {queue, demand}}
    end
  end

end
```

### [Elli on Elixir](https://www.lukasrieder.com/posts/elli-on-elixir/)

A complete guide to running the Elli web server in an Elixir application.

This guide expands on the shorter answers I found while getting `knutin/elli` to work with Elixir. Those answers were correct, but I needed more detail before I could run Elli in a new Elixir project.

1. Implement a handler with `@behaviour :elli_handler`.

    This module is the router and controller. A minimal implementation looks like this:

        # lib/elli_handler.ex
        defmodule ElliHandler do
          @behaviour :elli_handler
          alias :elli_request, as: Request
        
          def handle(req, args) do
            handle(Request.method(req), Request.path(req), req, args)
          end
        
          def handle(:GET, _, req, args) do
            # do something with the request.
            # e.g. you can use Request.get_arg/2 to fetch a query param
            say = Request.get_arg("say", req)
            # Return a tuple with 3 elements
            # status code, list of header tuples, response body
            {200, [], "echo, #{say}"}
          end
        
          # React to events throughout the connection, request, and response cycle.
          # Elli evaluates the return value, so this callback must be implemented.
          def handle_event(event, args, config) do
            # Here would be a good point to handle logging.
            # IO.inspect([event, args, config])
            :ok
          end
        
        end

2. Create an application that starts an Elli supervisor.

        # lib/elli_supervisor.ex
        defmodule ElliSupervisor do
          use Supervisor

          def start_link(ref, options) do
            Supervisor.start_link(__MODULE__, options, name: ref)
          end

          def init(options) do
            children = [
              worker(:elli, [options], id: :elli_http_server)
            ]
            supervise(children, strategy: :one_for_one)
          end

          def shutdown(ref) do
            case Supervisor.terminate_child(ref, :elli_http_server) do
              :ok -> Supervisor.delete_child(ref, :elli_http_server)
              err -> err
            end
          end
        end

        # lib/app.ex
        defmodule App do
          use Application

          def start(_type, _args) do
            import Supervisor.Spec, warn: false
            # Start the Elli supervisor with its options.
            # The callback must implement the elli_handler behaviour.
            # The port controls where Elli listens and defaults to 8080.
            ElliSupervisor.start_link(__MODULE__, callback: ElliHandler, port: 3000)
          end

        end

        # in mix.exs
        def application do
          [
            mod: {App, []}
          ]
        end

3. Add Elli as a dependency in `mix.exs`.

    Run `mix deps.get` to install your dependencies.
    Start your server with `mix run --no-halt` or in a console using `iex -S mix`.

        # in mix.exs
        defp deps do
          [
            # elli is our web server layer
            {:elli, github: "knutin/elli"}
          ]
        end

### [The world would be a better place if...](https://www.lukasrieder.com/2009/10/05/the-world-would-be-a-better-place-if/)

A 2009 product idea connecting mite time tracking with Ballpark invoicing.

The world would be a better place if [Ballpark](http://ballparkapp.com) married [mite](http://mite.yo.lk). Mite already tracked my time; Ballpark already created invoices. I wanted them to exchange that data.

I realize that this integration may fall outside the features needed by most customers, but I hope the Ballpark team will consider it. Connecting the products could create a business opportunity for both by making them useful to more freelancers and small teams.

For more information about each app, here are the links to their tours:

The [mite tour](http://mite.yo.lk/en/tour.html) promised fast input, powerful output, while the [Ballpark tour](http://getballpark.com) described a walk-off home run moment of invoice zen.

###### Managing both without pain

Writing invoices and estimates takes considerable time. After doing it by hand for about a year, I became convinced that the process could require less effort. At the end of each month, I followed these steps:

1. Download the reports for every project of the last month from mite as a PDF.

2. Get the total and enter it into the invoice template with Pages.

3. Get the client’s complete address and enter it into the template.

4. Save the completed template as the new invoice.

5. Launch Adobe Acrobat.

6. Combine those PDFs into one.

7. Send the combined PDF per E-Mail to the client with a few nice words.

###### The integration could shorten the process

1. Create a new invoice for the client on my ballpark account.

2. Select the client’s project with all time entries from last month.

3. Add a few nice words and send the invoice to the client via ballpark.

###### The design outline

###### References for development

	The [mite API documentation](http://mite.yo.lk/en/api/) describes the service interface.
	The [mite-rb Ruby API implementation](http://github.com/yolk/mite-rb) provides a client library.

### [Stellungnahme zu der Session 'Neues Publizieren'](https://www.lukasrieder.com/2009/09/28/stellungnahme-zu-der-session-neues-publizieren/)

Eine Einordnung der Atoms-&-Bits-Diskussion über Verlage und digitales Publizieren.

Wer sich öffentlich mit einem kontroversen Thema beschäftigt, muss damit rechnen, verkürzt zitiert zu werden. Genau das ist mir nach der Session Neues Publizieren passiert. Diese Stellungnahme ergänzt den Kontext, der in der [Atoms-&-Bits-Zusammenfassung](http://www.blogpiloten.de/2009/09/29/publizierenwarten-bis-dinosaurier-aussterben/) aus meiner Sicht fehlte.

###### Worum ging es in der Session Neues Publizieren?

Durch meine Arbeit für PaperC beschäftigte mich die Zukunft des Publizierens schon länger. Ich besuchte das Atoms & Bits Camp, um Menschen kennenzulernen und neue Themen zu entdecken. Dass ich dort selbst eine Session anbot, ergab sich eher zufällig.[1](#fn1)

Ich fragte die Anwesenden, ob Interesse an einer Diskussionsrunde zum Neuen Publizieren bestand. Acht Personen nahmen schließlich teil. Nach einer kurzen Vorstellung von PaperC entwickelte sich eine offene Diskussion.

Der rote Faden hielt nicht durchgehend, was bei einem so breiten Thema kaum überrascht. Trotzdem war ich für die Beiträge der Teilnehmer und ihr anschließendes Feedback dankbar.

###### Wie ich das Thema verstand

Neues Publizieren war ein diffuser Titel für ein noch komplexeres Thema. Meine [stichwortartige Vorbereitung](https://www.lukasrieder.com/2009/09/26/atoms-bits-session-neues-publizieren/) zeigt, wie viele Fragen darin zusammenliefen.

Meine damalige Kernthese lautete: Das Internet kann dazu beitragen, bestehende Probleme der Verlagsbranche zu lösen.

###### Was an der Zusammenfassung missverständlich war[2](#fn2)

Isoliert gelesen konnten meine Aussagen so wirken, als wünschte ich mir das bedingungslose Ende aller Verlage. Das entsprach nicht meiner Position.

Ich erwartete vielmehr eine Parallelität verschiedener Modelle. Verlage würden weiterhin in vielen Bereichen bestehen, müssten sich aber verändern und die Möglichkeiten des Internets ernst nehmen. Darauf zielte das Bild von den Dinosauriern: Nicht Verlage an sich sterben aus, sondern Organisationen, die sich einer veränderten Umwelt nicht anpassen.

Daneben sah ich Raum für selbstorganisierte Autoren, die einzelne Aufgaben wie Qualitätssicherung, Layout, Druck und digitale Aufbereitung an spezialisierte Dienstleister oder Freelancer vergeben. Dieses Modell benötigt jedoch eine funktionierende und nachvollziehbare Qualitätssicherung.

Verlage verfügen gerade in der Auswahl und Qualitätssicherung über wichtige Erfahrung. Deshalb werden sie für das Publizieren relevant bleiben, auch wenn sich ihre Form und ihr Aufgabenbereich verändern.

###### Zusammenarbeit und weiterer Wandel

Ich bin den Verlagen dankbar, die mit PaperC zusammenarbeiteten. Der gemeinsame Weg war ein wichtiger Schritt zur Modernisierung des Publizierens.

Er war jedoch nicht das Ende der Entwicklung. Das Internet bot Verlagen, Autoren und neuen Dienstleistern weitere, damals noch unentdeckte Möglichkeiten. PaperC wollte an diesen Möglichkeiten aktiv forschen, arbeiten und mitentwickeln.

###### Fragen, Kritik und Diskussion

Ich freue mich weiterhin über sachliche Kritik, Ergänzungen und Diskussionen zu diesem Thema.

###### Referenzen

[1](#fnr1) Was ist eigentlich ein Bar Camp? Wie sieht eine solche Veranstaltung aus? [Wikipedia Artikel über das Bar Camp Format](http://de.wikipedia.org/wiki/BarCamp)

[2](#fnr2) Blogeintrag auf blogpiloten.de: [Publizieren: Warten, bis Dinosaurier aussterben](http://www.blogpiloten.de/2009/09/29/publizierenwarten-bis-dinosaurier-aussterben/)

### [Atoms & Bits -- Session 'Neues Publizieren'](https://www.lukasrieder.com/2009/09/26/atoms-bits-session-neues-publizieren/)

Gedanken und Diskussionspunkte zur Zukunft des digitalen Publizierens.

Dieser Beitrag diente mir zur Vorbereitung auf meine Session beim Atoms & Bits Camp 2009. Er enthält meine damaligen Thesen, Fragen aus der Diskussion und die Quellen, mit denen ich mich vorbereitet hatte.

###### Mein Hintergrund

	Ich war Gründungsmitglied und Entwickler bei der PaperC GmbH.
	PaperC bot kostenlosen Zugriff auf Fachbücher und wollte hochwertige Inhalte möglichst leicht zugänglich machen.
	Leser konnten Bücher kostenlos lesen und für eine digitale Kopie mit Funktionen zum Zitieren, Drucken und Kommentieren bezahlen.
	Vergleichbare Angebote waren unter anderem Libreka, Google Buchsuche, SpringerLink und Mendeley.

###### Probleme des konventionellen Publizierens in digitalen Medien

	Viele Verlage reagierten aus meiner Sicht defensiv auf den digitalen Wandel und überließen die Initiative dadurch anderen Akteuren.
	Wissen und Information streben nach freier Verfügbarkeit, während ihre Erstellung und Aufbereitung weiterhin finanziert werden müssen.
	Weitgehend automatisierte Produktionsprozesse schwächen einige traditionelle Begründungen für die Rolle eines Verlags.
	Publizieren ist technisch einfach geworden, aber verlässliche, referenzierte und hochwertige Informationen bleiben schwer auffindbar.
	Die Open-Access-Bewegung versucht Probleme zu lösen, die sowohl Wissenschaftler bei der Veröffentlichung als auch Fachautoren bei der Vergütung betreffen.

###### Meine damalige Vision für das digitale Publizieren

	Wissen, Erkenntnisse und Informationen sollten leicht zugänglich und qualitativ aufbereitet sein.
	Digitale Werkzeuge könnten einen wirtschaftlich orientierten Peer-Review-Prozess organisieren, Fachleute nach Themengebiet auffindbar machen und ihre direkte Bezahlung oder Erlösbeteiligung ermöglichen.
	Fundraising könnte Werke vorfinanzieren und Investoren über kleine Beteiligungen am späteren Erlös beteiligen.
	Autoren könnten die Produktion selbst steuern und spezialisierte Dienstleister für Qualitätssicherung, Layout, Druck und digitale Aufbereitung beauftragen.
	Neue Erlösmodelle könnten alle Mitwirkenden eines Werkes angemessen beteiligen.
	Sowohl renommierte als auch junge Autoren könnten häufiger selbst publizieren und neue Wege zu Anerkennung und Verbreitung finden.

###### Fragen aus der Diskussion

	Wie kann das Internet eine verlässliche Qualitätssicherung unterstützen?
	Wie könnte der Übergang zu digitalen Medien für Autoren aussehen?
	Wie könnte derselbe Übergang für Verlage aussehen?
	Welche Anreize bringen mehr Menschen dazu, neue Ideen zum Publizieren öffentlich zu diskutieren?
	Das Informationsparadoxon von Arrow bleibt dabei relevant: Der Wert einer Information lässt sich oft erst beurteilen, nachdem die Information bereits bekannt ist.

###### Weiterführendes Material

Die folgenden Quellen bildeten meine damalige Materialsammlung zum Thema Neues Publizieren:

	Zum Bibliotheksumfeld gehörten http://www.ub.uni-dortmund.de/listen/inetbib/msg40312.html, http://www.digital-scholarship.org/sepb/sepb.html und http://oebib.wordpress.com/.
	Die Verlagsperspektive behandelte der Beitrag Gefangenendilemma der Print-Verlage unter http://blog.handelsblatt.de/indiskretion/eintrag.php?id=2189.
	Als Beispiel aus dem Zeitungsmarkt diente http://nytimes.com.
	Günter Faltins Beiträge waren unter http://www.kopfschlaegtkapital.com und http://labor.entrepreneurship.de verfügbar.
	Beispiele für Gründungen und Ausgründungen waren http://paperc.de, http://libreka.de und http://epubli.de.
	Weitere Perspektiven boten http://www.blicklog.com und http://blog.paperc.de.
	Eine Erklärung des Informationsparadoxons war unter http://www.tu-chemnitz.de/wirtschaft/fsr/wiki/index.php/Informationsparadoxon_nach_Arrow verfügbar.

### [Quit your mail server admin job. Now!](https://www.lukasrieder.com/2009/09/22/quit-your-mail-server-admin-job-now/)

Why a small team should reconsider running its own mail infrastructure.

This article outlines the problems I encountered while running a mail server and describes the pragmatic Google Apps alternative I chose in 2009.

###### The mail server problem

Managing a server is not necessarily difficult when you understand the system, but it always consumes time. That cost is a reason to reconsider managing every service yourself.

I like controlling my servers and knowing that I can change them whenever necessary. That preference led me to run my own mail server. Once Postfix supported virtual users, adding an account appeared to require only one more row in a MySQL table.

###### 1. Unexpected downtime

One afternoon, part of the system stopped working. I had to abandon my other work and diagnose the problem manually. The mail server remained unavailable for five hours. Email tolerates temporary delivery failures reasonably well, but an important message could still have been delayed or lost during that window.

###### 2. Mail account management

The startup had three core team members, but I was the only person who knew how to add an account to the MySQL mail-users table. While I was away for a week, nobody else could create an account for an intern. A supposedly simple administrative task had become dependent on one person.

###### 3. Reliable mailing for your application

Most web applications need reliable communication with their users, and email remains a common channel. A mail-server outage can also break registration, password resets, and notifications.

A mail-server misconfiguration once broke user registration for about eight hours. That window was small compared with a year of operation, but exception reports indicated that it affected roughly six people. A first interaction that fails can permanently damage a user’s trust in a service.

###### What you really want

	The team needs to send and receive email.
	The service must remain reliable throughout the year because communication failures have business consequences.
	The application needs to send transactional email.
	Nontechnical colleagues should be able to manage accounts.
	Users should have webmail without requiring the team to maintain its infrastructure.
	The service should reduce both direct costs and administrative time.

###### What you could do to avoid emails completely

An application could avoid some email by using Twitter for OAuth authentication, messages, and notifications. In 2009, libraries such as the [twitter gem](http://twitter.rubyforge.org) and [Twitter4R](http://twitter4r.rubyforge.org) made that idea easy to explore. I would have welcomed fewer automated messages in my inbox.

This article still focuses on email because the application continued to depend on it.

###### What I did as a first step

As a startup, PaperC needed to control infrastructure costs. We rented servers for the core platform but outsourced services that did not differentiate the product. Our email volume was still modest, so we chose Google Apps for mail delivery. The service fit our needs at the time, with a professional mail host such as [Rackspace](http://rackspace.com) remaining an option for future growth.

###### How to integrate Google Apps into your rails app

Google Apps suited only smaller applications because each account had a daily sending limit. Before adopting it, estimate the application’s delivery volume and choose a professional provider if the limit is too restrictive. At the time of writing, Google documented these limits per account:[1](#fn1)

	Standard Edition allowed 500 external recipients per day.
	Premier and Education Edition allowed 2,000 external recipients per day.

The email addresses can be distributed among the To:, Cc:, and Bcc: fields.

1. [Sign up for Google Apps](http://apps.google.com/).

2. Set up a mail account from which the server can send messages.

3. [Follow Google’s instructions](http://www.google.com/support/a/bin/answer.py?answer=33352) to update the MX records for the domain.

4. Add an SPF record to reduce the risk of outgoing messages being classified as spam.

  The [SPF Setup Wizard](http://old.openspf.org/wizard.html) can create the record, and the [SPF Query Tool](http://www.kitterman.com/spf/validate.html) can validate it.

5. Edit config/environments/production.rb as follows.

{% highlight ruby }

ActionMailer::Base.delivery_method = :smtp

ActionMailer::Base.smtp_settings = {

  :domain => “yourdomain.com”,

  :address => “smtp.googlemail.com”,

  :tls => true,

  :authentication => :plain,

  :user_name => “your-apps-mail@yourdomain.com”,

  :password => “yourpassword”

}

{ endhighlight %}

6. Install and apply the simple-tls gem with the following commands.

{% highlight bash }

$ sudo gem install ambethia-smtp-tls -shttp://gems.github.com

{ endhighlight %}

{% highlight ruby %}

	in config/environment.rb

config.gem ‘ambethia-smtp-tls’, :lib => ‘smtp-tls’, :version => ‘1.1.2’, :source => ‘http://gems.github.com’

{% endhighlight %}

7. Deploy and relax.

  After deployment, monitor the mail volume regularly and upgrade the account or provider before reaching the documented limits.

###### References

[1](#fnr1) [Sending Limit – Google Apps Help](http://www.google.com/support/forum/p/Google+Apps/thread?tid=700114a96b58afe7&hl=en)

### [Extend your bash PS1 with a git branch without breaking your prompt](https://www.lukasrieder.com/2009/07/14/extend-your-bash-ps1/)

Adding the current Git branch to Bash PS1 without breaking prompt width calculation.

I wanted my terminal prompt to show the current branch of the Git repository in which I was working.

Many guides explain how to add a branch name, but the first one I followed left me with a broken Bash prompt. It ignored newlines and behaved strangely whenever the prompt exceeded the terminal width.

The problem was that non-printing color sequences confused Bash’s prompt-width calculation. Wrapping those sequences in \[ and \] tells Bash not to count them as visible characters.

The following commented snippet extends ~/.bash_login, or another Bash startup file that contains user-specific configuration:

{% highlight bash %}

	write a function to compute the current git branch

parse_git_branch() {

  local b=$(git symbolic-ref HEAD 2> /dev/null)

  echo -n “${b#refs/heads/}”

}

	set the PS1 variable

PS1="\w\[\e[0;33;49m\]\$(parse_git_branch)\[\e[0;0m\]$ "

	on any regular directory it will output:
	/path/to/directory$ [PROMPT]

	on a git repository it will output:
	/path/to/directory(branch_name_in_yellow)$ [PROMPT]

{% endhighlight %}

###### Understanding the PS1 components

	\w inserts the full path to the current directory.
	\W inserts the name of the current directory.
	\[ begins a non-printing-character escape sequence.
	\] ends a non-printing-character escape sequence.
	\e inserts an escape character, much like \033.
	\e[X;Y;Zm begins a Bash color sequence whose values style all subsequent characters.

  X selects the effect: 0 resets it, 1 increases brightness, 4 underlines, 5 blinks, 7 swaps the foreground and background, and 8 hides the text.

  Y selects the foreground color: 30 is black, 31 red, 32 green, 33 yellow, 34 blue, 35 magenta, 36 cyan, 37 white, and 39 the default.

  Z selects the background color by adding ten to the corresponding foreground code; for example, 40 is black.
	\e[0;0m resets the color so that subsequent characters use their previous appearance.
	\$(function_name) inserts the output of a function call.

###### The rule

Always wrap color sequences in non-printing-character escape brackets.

### [Three steps to set up vsftpd](https://www.lukasrieder.com/2009/07/07/1-2-3-easy-steps-to-setup-vsftpd/)

Configuring a read-only vsftpd service with virtual users on Debian.

This guide explains how I configured a read-only FTP service with vsftpd and virtual users. The example covers chrooting, TLS settings, transfer logs, and PAM authentication.

Update: Step 2 now highlights the configuration values that must be adapted to each environment.

###### 1. Install

Log in to the server, obtain the required administrative privileges, and install vsftpd. This example uses Debian Lenny; consult the operating system’s documentation for other installation methods. More information about the server is available at [vsftpd.beasts.org](http://vsftpd.beasts.org).

{% highlight bash }

$ aptitude install vsftpd

{ endhighlight %}

###### 2. Configure

The default /etc/vsftpd.conf contains extensive explanatory comments. I created a shorter configuration based on that file and exposed several additional options.

The example implements the following scenario:

	A standalone FTP server allows one or more users to access a single directory.
	The server denies anonymous access and disables writes for every user.
	Virtual users have separate passwords without exposing system accounts to FTP.
	Every virtual user maps to one low-privilege system account with read-only access to the exposed directory.
	An htpasswd file manages the virtual users through the modified /etc/pam.d/vsftpd configuration shown in step 3.

Some options are not required for this exact scenario. This is the general vsftpd.conf that I kept readable and adaptable.

The important values to adjust are:

	local_root identifies the directory that the service exposes.
	guest_username identifies the low-privilege account to which virtual users map; the example uses vsftp.

{% highlight bash %}

	Run the server in standalone mode

listen=YES

	Allow anonymous FTP? This will enable/disable all subsequent anonymous options

anonymous_enable=NO

	Allow anonymous users to upload files?

anon_upload_enable=NO

	Allow anonymous users to create directories?

anon_mkdir_write_enable=NO

	Allow local users to log in? (Is needed even in case of a virtual setup)

local_enable=YES

	Restrict local users to their home directories

chroot_local_user=YES

	Specify a user token

user_sub_token=$USER

	Treat all non-anonymous logins as ‘guest’ logins

guest_enable=YES

	Specify a system user to map ‘guest’ logins

guest_username=vsftp

	Enable if virtual users shall use the same privileges as locals (default: NO)

virtual_use_local_privs=NO

	Hide user/group information of files and directories

hide_ids=YES

	Specify a directory to where users get chroote’d

local_root=/var/ftp_data

	Enable SSL, this will affect all subsequent ssl options

ssl_enable=NO

	Allow SSL for anonymous users

allow_anon_ssl=NO

	Require SSL for all non-anonymous data transfers

force_local_data_ssl=YES

	Require SSL for all non-anonymous logins

force_local_logins_ssl=YES

	Enable TLS v1 protocol connections

ssl_tlsv1=YES

	Enable SSL v2 protocol connections

ssl_sslv2=YES

	Enable SSL v3 protocol connections

ssl_sslv3=YES

	Specify a RSA certificate file to use for encrypted connections

rsa_cert_file=/etc/vsftpd.d/vsftpd.pem

	Allow any form of write command?

write_enable=NO

	Enable directory messages

dirmessage_enable=NO

	Enable transfer logs, you can specify a log file and if the xferlog format shall be used

xferlog_enable=YES

#xferlog_file=/var/log/vsftpd.log

#xferlog_std_format=YES

	Allow PORT transfer connections only from the following port (default: 20)

connect_from_port_20=YES

	Chown all created files to a specific username

chown_uploads=NO

chown_username=whoever

	When to time out an idle session? (default: 600)

#idle_session_timeout=600

	When to time out a data connection? (default: 120)

#data_connection_timeout=120

	Define a totally isolated and unprivileged user

#nopriv_user=ftpsecure

	Enable asynchronous ABOR requests for backwards compability of older clients

#async_abor_enable=YES

	Enable ASCII mangling to really happen
	Not recommended

#ascii_upload_enable=YES

#ascii_download_enable=YES

	Customize your banner

#ftpd_banner=Welcome to blah FTP service.

	Enable a anonymous e-mail blacklist

#deny_email_enable=YES

#banned_email_file=/etc/vsftpd.banned_emails

	Specify a list of local users to chroot to their home directory.
	This options may become a list of users NOT to chroot if chroot_local_user is YES!

#chroot_list_enable=YES

#chroot_list_file=/etc/vsftpd.chroot_list

	Enable recursive ls, may cause excessive I/O on large sites
	Not recommended

#ls_recurse_enable=YES

	
		Debian customization
	
	Specify an empty directory to chroot vsftpd at times it doesn’t require filesystem access

secure_chroot_dir=/var/run/vsftpd

	Specify a PAM service name to use, see /etc/pam.d/ for pam services

pam_service_name=vsftpd

	Specify a RSA certificate to use for SSL encrypted connections

rsa_cert_file=/etc/ssl/certs/vsftpd.pem

{% endhighlight %}

###### 3. Manage

I manage virtual users in a file created with htpasswd. On Debian, that utility comes from the apache2-utils package and may need to be installed separately.

{% highlight bash }

$ aptitude install apache2-utils

{ endhighlight %}

With the utility installed, create the FTP user file:

{% highlight bash %}

$ mkdir /etc/vsftpd.d

$ htpasswd -c /etc/vsftpd.d/users billythekid

	note: for other users to add you have to ommit the -c flag

{% endhighlight %}

vsftpd does not know about this file automatically. Configure /etc/pam.d/vsftpd to use it:

{% highlight bash %}

	Login using a htpasswd file

auth    required pam_pwdfile.so pwdfile /etc/vsftpd.d/users

account required pam_permit.so

{% endhighlight %}

On my machine, I also had to run $ aptitude install libpam-pwdfile. Restart vsftpd to load the new configuration. If the firewall permits the required FTP ports, the newly created virtual user should then be able to log in.

{% highlight bash }

$ /etc/init.d/vsftpd restart

{ endhighlight %}

###### Credits

These resources helped me write the article:

	The [VSFTPD.CONF manual page](http://vsftpd.beasts.org/vsftpd_conf.html) documents the available options.
	[Setup Virtual Users and Directories in VSFTPD](http://howto.gumph.org/content/setup-virtual-users-and-directories-in-vsftpd/) explains the virtual-user setup.

### [Lesson learned: MySQL client character encoding matters](https://www.lukasrieder.com/2009/06/03/lesson-learned-mysql-client-character-encoding-matters/)

Recovering a large production database from a MySQL client-encoding mismatch.

While fixing a PaperC search bug, I discovered a serious configuration mistake: the production database.yml did not set encoding: utf8.

{% highlight yaml }

production:

  …

  encoding: utf8

{ endhighlight %}

Because of the production server’s MySQL defaults, every table was created with CHARSET=latin1. Rails nevertheless sent UTF-8 data, so UTF-8 bytes ended up in tables declared as Latin-1. Saving and displaying data appeared to work because the bytes remained intact, but Thinking Sphinx connected explicitly with UTF-8 expectations.

I discovered the mismatch when an important book containing umlauts failed to appear in search results. Further investigation showed that every search involving umlauts was broken. By then, the database contained about 900 MB of data.

Note to myself: Always check your database configuration for the encoding flag, both in development and production.

###### Generate the correct database configuration during deployment

I wrote a small Capistrano task to generate database.yml with an explicit encoding. The task can run after the initial setup:

{% highlight ruby }

namespace :db do

  desc “upload the result of config/database.yml.erb to \#{shared_path}/config/database.yml”

  task :configure, :roles => :db do

    db_configuration = ERB.new File.read(“#{local_root}/config/database.yml.erb”)

    set :db_host, ‘localhost’

    set :db_user, user

    set :db_password, ask("Type the mysql password on #{db_host} for #{db_user}: ")

    put db_configuration.result(binding), “#{shared_path}/config/database.yml”

  end

end

{ endhighlight %}

In config/database.yml.erb:

{% highlight yaml }

common: &common

  adapter: mysql

  username: <= db_user >

  password: <= db_password >

  host: <= db_host %>

  encoding: utf8

development:

  <<: *common

  database: <%= application %>_development

test:

  <<: *common

  database: <%= application %>_test

production:

  <<: *common

  database: <%= application %>_production

staging:

  <<: *common

  database: <%= application >_staging

{ endhighlight %}

###### How I rescued the data

After researching practical solutions and testing them against the staging database, I used this recovery procedure:

	Put the website into maintenance mode and stop all background workers.
	Dump the production database with the following command.[1](#fn1)

{% highlight bash }

$ mysqldump -u root -p —opt —default-character-set=latin1 paperc_production -r paperc_production-latin1.sql

{ endhighlight %}

	Replace every CHARSET and NAMES instruction with its UTF-8 equivalent.[2](#fn2)

{% highlight bash }

$ replace “CHARSET=latin1” “CHARSET=utf8” “SET NAMES latin1” “SET NAMES utf8” < paperc_production-latin1.sql > paperc_production-utf8.sql

{ endhighlight %}

	Restore the dump into a separate database first.

{% highlight bash }

$ mysql -u root -p paperc_staging < paperc_production-utf8.sql

{ endhighlight %}

	Connect to staging, rebuild the search index, inspect the data, and verify representative samples.
	Restore the validated dump into the production database.
	Rebuild the complete search index, leave maintenance mode, and restart the background workers.

The procedure worked.

[1](#fnr1) I used the -r option to write directly to disk because a Unix output pipe could alter the encoding.

[2](#fnr2) Search and replace in a text editor did not work well on a file of roughly 900 MB.

### [RailsWay 2009: Rails performance](https://www.lukasrieder.com/2009/05/27/railsway-berlin-rails-performance/)

Notes from RailsWay 2009 on perceived performance, HTTP caching, and Rails caching.

Perceived performance matters more to a user than an isolated application benchmark. These were my main notes from the talk:

	Use YSlow to find browser-facing performance problems.
	Send ETag and Last-Modified validators together.
	Send Expires headers where long-lived caching is safe, such as for batch-generated pages.
	Consider HTTP-cache tools such as Squid, Varnish, Akamai, and Rack::Cache.
	After exhausting HTTP caching, use Rails caching where application work remains expensive.
	Use fragment caching for expensive partials.
	Use action caching where a complete action response can be reused.
	Use model caching, available at the time through several plugins.
	Memcached can evict and rebuild entries, which makes the cache comparatively self-healing.
	Treat cache-key design as seriously as cache generation. A useful key combines a name, a unique identifier, and a version.

{% highlight ruby }

def handle_etag

  fresh_when :etag => [@user.id, @user.updated_at]

end

{ endhighlight %}

The following example uses Last-Modified explicitly:

{% highlight ruby }

def handle_last_modified

  fresh_when :last_modified => last_updated_time

end

{ endhighlight %}

### [RailsWay 2009 Berlin: Haml, Sass, and Compass](https://www.lukasrieder.com/2009/05/26/railsway-berlin-haml-and-sass/)

A concise note from RailsWay 2009 on Haml filters and embedded JavaScript.

I attended this talk to improve my use of Haml. My favorite lesson concerned filters: a JavaScript filter can keep embedded code indented and readable within a view.

{% highlight ruby }

-content_for :jquery

  :javascript

    function foo(){

      alert(‘Blah’);

    }

{ endhighlight %}

### [Meet mite.cmd, a simple command-line interface for mite](https://www.lukasrieder.com/2009/05/22/mite.cmd-a-command-line-interface-for-mite/)

Introducing mite.cmd, a terminal interface for time tracking with mite.

Time tracking has become an important part of my daily work. Starting timers, creating entries, and selecting projects is usually quick, but doing it in a browser still requires many clicks and keystrokes.

For a while, I used mite through Fluid with an iPhone user agent. The combination worked well, but it was not ideal for somebody who spends most of the day in a terminal. I still had to switch applications, click through the interface, and move a hand away from the keyboard.

DynaMite later integrated mite into the macOS preference pane and kept the current timer visible in the menu bar. The free version stopped timers after thirty minutes, however, and I could not find an uninstall button.

After another week with the Fluid application, I decided to build a client that matched my own workflow and did not stop timers arbitrarily. That decision produced mite.cmd. After one weekend and two additional nights spent working with the mite-rb gem and Bash completion, I pushed the initial commit to GitHub.

###### Use mite directly from the console

The command lets me keep TextMate and iTerm as my only open work applications. Before mite.cmd, starting work required this sequence:

	I opened a browser or the Fluid application.
	I clicked through the interface until an empty time entry was running.
	I returned to the application in which I was actually working.

My day often ended with another sequence:

	I switched back to the browser or Fluid application.
	I stopped every timer and added the missing notes.
	I opened the reporting page to review the day.

Now I can start with mite + and finish with mite + && mite today.

###### Does the terminal have to be ugly and full of gibberish?

A terminal can certainly look that way, but it does not have to. mite.cmd aligns each entry, shows only the necessary information, and uses a little color to emphasize its state. A red 0:25 indicates a stopped entry, while a yellow 1:70 indicates a running one. Each report ends with the revenue total in green.

The interface follows two principles:

	mite.cmd keeps the interaction simple.
	Color highlights the important information without overwhelming the report.

###### What about all the typing?

A select box may appear faster than entering a complete project name. Bash completion removes most of that overhead: typing mite [tab] can select a project with only a few additional keystrokes.

The real comparison is between moving a hand to the mouse and clicking through the interface, or keeping both hands on the keyboard for six to ten keystrokes. You can choose whichever interaction feels more comfortable.

	Choose mite.cmd if you enjoy terminals, administer systems, or like adapting your tools. Install it with the following command.

{% highlight bash }

  $ gem install Overbryd-mite.cmd -s http://gems.github.com

{ endhighlight %}

	You also need a mite account, which you can create at [mite.yo.lk](http://mite.yo.lk).

###### Current version and updates

At the time of writing, mite.cmd was actively maintained at version 0.1.9. Update to the most recent version with this command:

{% highlight bash }

$ gem update Overbryd-mite.cmd

{ endhighlight %}

### [How to overcome MySQL's group_concat_max_len limit in Thinking Sphinx](https://www.lukasrieder.com/2009/05/13/group_concat_max_len-limitation-and-thinking-sphinx/)

Fixing truncated Thinking Sphinx indexes caused by MySQL's default GROUP_CONCAT limit.

I recently discovered some strange search behavior in a project that used [Thinking Sphinx](http://thinking-sphinx.org). A document had many pages, each with a plain_text column that needed to be searchable through both the Page and Document models.

Here is the relevant part of the Thinking Sphinx declaration:

{% highlight ruby %}

define_index do

	…

  indexes :title, :sortable => true

  indexes :author, :sortable => true

  indexes pages.plain_text, :as => :plain_text
	…

end

{% endhighlight %}

Documents in this database generally contained between 300 and 500 pages. Thinking Sphinx used MySQL’s GROUP_CONCAT()[1](#fn1) function to join their columns, but MySQL limited the result to 1,024 bytes by default.[2](#fn2) That limit was far too small for hundreds of pages.

As a result, Thinking Sphinx indexed only the first pages of each document. Our tests had missed the problem because the test document contained only four pages. The lesson was simple: test with realistic production-scale data at least once.

###### The solution

Add the following property to the index definition for any model that joins large columns:

{% highlight ruby %}

define_index do

	…

  set_property :group_concat_max_len => 16.megabytes

end

{% endhighlight %}

After changing the property, reconfigure and rebuild the Sphinx index.

[1](#fnr1) http://dev.mysql.com/doc/refman/5.0/en/group-by-functions.html#function_group-concat

[2](#fnr2) http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html#sysvar_group_concat_max_len

### [Migrate almost any password-encryption scheme to AuthLogic](https://www.lukasrieder.com/2009/05/13/migrate-old-passwords-with-auth_logic/)

Migrating users from a legacy password scheme to AuthLogic without resetting credentials.

In the winter of 2008, I wrote the alpha version of [PaperC](http://paperc.de) with an old version of acts_as_authentic. In the spring of 2009, we discarded the untested alpha code and decided to rewrite PaperC from scratch. By then, [AuthLogic](http://github.com/binarylogic/authlogic) had replaced acts_as_authentic.

The alpha had collected more than one thousand users, and all of them needed access to the beta. We therefore had to migrate their hashed passwords without forcing account resets.

###### Configure the transition

AuthLogic was designed to replace existing authentication plugins and supports transitions from many password-encryption schemes. It can migrate some versions of acts_as_authentic automatically; this article shows how to support another salt-and-password combination.

###### A look inside

AuthLogic is modular enough to accept a custom CryptoProvider for the old encryption method. The example below uses AuthLogic 1.4.0 and depends on how that version passes passwords and salts into a provider.

AuthLogic first checks the user’s password with the current encryption method. If it does not match and a transition provider is configured, AuthLogic checks the password with the old method.

To do so it calls matches? on a CryptoProvider class with two arguments: the stored password and an array of encryption arguments (salt and password). The array looks like [raw_password, salt].

Assume that the old application stores passwords this way:

{% highlight ruby }

def encrypt_password(password)

  Digest::SHA512.hexdigest(“-#{salt}-#{password}—”)

end

{ endhighlight %}

The transition requires a small class that implements the matches? class method expected of an AuthLogic CryptoProvider:

{% highlight ruby %}

class OldApplicationSha512

	AuthLogic Credentials provides the tokens in the following order:
	[raw_password, salt]
	The old application used ‘-#{salt}-#{password}—’

  def self.matches?(crypted, *tokens)

    Digest::SHA512.hexdigest(“-#{tokens.last}-#{tokens.first}—”) == crypted

  end

end

{% endhighlight %}

Here’s how to wire it up in your user model:

{% highlight ruby }

acts_as_authentic :login_field => :email,

  :transition_from_crypto_provider => OldApplicationSha512

{ endhighlight %}

###### Complete the migration with tests

This provider completes the wiring, but production migration code should also include tests for correct passwords, incorrect passwords, representative legacy salts, and the transition to the new hash.

## Talks

### [Freelance or GmbH? A Solo Tech Founder's Story](https://www.lukasrieder.com/talks/freelance-or-gmbh/)

What fifteen years of independent software work taught me about risk, commitment, company structures, and choosing without certainty.

The event title promised a solo founder's story. The talk argues that "solo" is the least accurate part. Other people made the move to Berlin, the first company, and the later consulting work possible.

The company question depends just as much on circumstances. A GmbH can separate liability, make business runway explicit, and retain capital. A Holding can move accumulated capital away from operating risk and preserve options for future ventures. Both add fixed costs, formal duties, and limits on access to the money.

The accompanying calculator compares an Einzelunternehmen, a qualifying Freiberufler activity, a Direct GmbH, and a Holding with an operative company on the same economic basis. Its baseline does not produce a universal winner. Use the [Freelance or company? calculator](https://www.lukasrieder.com/freelance-or-company/) to find the assumption that changes your own result, then review that assumption with a German tax adviser.

The related article, [Freelance or GmbH? I built a lifetime model before answering](https://www.lukasrieder.com/blog/freelance-or-gmbh/), documents the tax model, its assumptions, and why my Holding does not lead the default scenario.

The downloadable presentation is a web-optimized export of the original Keynote deck used for the talk.

### [Privacy by Design](https://www.lukasrieder.com/talks/privacy-by-design/)

Making privacy a software-design property rather than a final compliance task.

### [PostgreSQL Friends](https://www.lukasrieder.com/talks/postgresql-friends/)

Useful PostgreSQL features hiding behind familiar application code.

### [Agentic RAG with Tilores](https://www.lukasrieder.com/talks/agentic-rag-with-tilores/)

Combining agentic retrieval with Tilores entity resolution.

### [BeyondCorp Overview](https://www.lukasrieder.com/talks/beyondcorp-overview/)

Maintaining productivity while improving security with a BeyondCorp-style model.

### [Modeling Document Databases](https://www.lukasrieder.com/talks/modeling-document-databases/)

Practical document modelling with PostgreSQL as a document store.

### [Ahoi! Complex applications on Kubernetes](https://www.lukasrieder.com/talks/ahoi-complex-applications-on-kubernetes/)

Patterns for deploying complex, stateful applications on Kubernetes.

### [Flow Based Programming in 2 minutes](https://www.lukasrieder.com/talks/flow-based-programming-in-2-minutes/)

A two-minute introduction to flow-based programming.

### [Metabase UG, November 2018](https://www.lukasrieder.com/talks/metabase-ug-november-2018/)

Opening the November 2018 Metabase community meetup.

### [Flow Based Programming in Elixir](https://www.lukasrieder.com/talks/flow-based-programming-in-elixir/)

Building explicit data-processing pipelines with Elixir and Pipette.

Pipette grew from practical work on API connectors where each transformation needed to remain small, composable, and visible. The [source code](https://github.com/suitepad-gmbh/pipette) and [API documentation](https://hexdocs.pm/pipette/Pipette.html) preserve the implementation shown in the talk.

### [Configuration in Elixir](https://www.lukasrieder.com/talks/configuration-in-elixir/)

A compact tour of safer application configuration in Elixir.

### [Cloudflare Edge Side Includes](https://www.lukasrieder.com/talks/cloudflare-edge-side-includes/)

Composing cached web responses at Cloudflare's edge.

### [Metabase: three good practices for a hosted setup](https://www.lukasrieder.com/talks/metabase-three-good-practises-for-a-hosted-setup/)

Three operational practices for a reliable hosted Metabase setup.

### [Adele: Inter-service Messaging](https://www.lukasrieder.com/talks/adele-inter-service-messaging/)

Exploring lightweight inter-service calls across a mesh of Elixir nodes.

### [Elixir Deployment](https://www.lukasrieder.com/talks/elixir-deployment/)

A whirlwind tour of options for getting an Elixir application live.

This talk compares the deployment paths available to Elixir teams and asks a practical question: which parts of release engineering should the platform own, and which parts should remain visible to the application team?

### [NIFs and C-Nodes](https://www.lukasrieder.com/talks/nifs-and-c-nodes/)

Choosing between native implemented functions and C-Nodes on the Erlang VM.

The examples led to [Nodex](https://github.com/Overbryd/nodex), an experiment in exposing Node.js modules to Elixir through a separate OS process and Erlang's distribution protocol. Keeping native work outside the BEAM changes a crash from a VM-wide failure into a node-level failure that supervisors can reason about.

### [SQL Workshop Part 1](https://www.lukasrieder.com/talks/sql-workshop-part-1/)

Foundations for getting more value from data already stored in relational databases.

### [Constraint Programming](https://www.lukasrieder.com/talks/constraint-programming/)

Solving search problems in Ruby through constraints and backtracking.

A [video recording](https://youtu.be/TxMXCkOQc7o) preserves the full session. The [example repository](https://github.com/Overbryd/constraint_programming_talk) contains the slides, backtracking exercises, logic puzzles, and Jim Weirich's `amb` implementation used during the talk.

### [Object Constancy in Visualisations](https://www.lukasrieder.com/talks/object-constancy-in-visualisations/)

How animated transitions help readers track data through a changing visualisation.

The talk's D3 demonstration and source remain available as a [Gist](https://gist.github.com/Overbryd/a93ddc3ea2ccff05f438). It connects animated transitions to a broader design goal: help readers understand change without forcing them to reconstruct the whole scene after every update.

### [MySQL Trickery](https://www.lukasrieder.com/talks/mysql-trickery/)

Tools and tactics for live MySQL database migrations.

### [Thread gem](https://www.lukasrieder.com/talks/thread-gem/)

A lightning talk about useful concurrency tools in Ruby's Thread gem.

### [App-to-User Notifications](https://www.lukasrieder.com/talks/app-to-user-notifications/)

Making game notifications less spammy while improving click-through rates.

### [PStore not dead](https://www.lukasrieder.com/talks/pstore-not-dead/)

Rediscovering PStore, a small persistence tool in Ruby's standard library.

### [Bitcoin: Why and How](https://www.lukasrieder.com/talks/bitcoin-why-and-how/)

An early technical and economic argument for Bitcoin, with a JRuby wallet example.

### [Cuba Framework](https://www.lukasrieder.com/talks/cuba-framework/)

A close look at the strengths and weaknesses of Cuba, a small Rack framework.

### [Fish: a shell for the 90s](https://www.lukasrieder.com/talks/fish-a-shell-for-the-90s/)

A lightning-talk pitch for Fish as a modern alternative to Bash.

### [Practical Functional Programming in Ruby](https://www.lukasrieder.com/talks/practical-functional-programming-in-ruby/)

Functional Ruby demonstrated through a modular system of cheat classifiers.

### [Wrapping Apache HTTPClient](https://www.lukasrieder.com/talks/wrapping-apache-httpclient/)

Designing a Ruby-friendly wrapper around a Java HTTP library for JRuby.

The wrapper was published as the [`http_client` gem](https://rubygems.org/gems/http_client), with [source code on GitHub](https://github.com/Overbryd/http_client). The core design problem was not calling Java from Ruby; it was exposing Java's behavior through an API that felt unsurprising to Ruby callers.

### [Stateful Application Server in JRuby](https://www.lukasrieder.com/talks/stateful-application-server-in-jruby/)

A conference version of the case for stateful application servers on JRuby.

This edition turns the production experience into a broader architecture argument. It explains why an in-memory object graph suited the workload, then makes the hidden costs explicit: state ownership, concurrency control, sharding, persistence, deploy handover, and recovery all move into application design.

### [Lightning in the Cloud](https://www.lukasrieder.com/talks/lightning-in-the-cloud/)

A literal lightning talk for people running applications in clouds.

### [Stateful Application Server](https://www.lukasrieder.com/talks/stateful-application-server/)

Lessons from building and operating a stateful JRuby application server.

This system kept game sessions in memory and treated the object graph as the primary application state. Static sharding assigned each session to one machine; per-session mutexes protected concurrent access; a lifecycle-aware handover moved sessions between application versions during deploys.

The approach bought high development speed and low execution time, but made ownership, persistence, JVM warm-up, garbage collection, and failure recovery application concerns. That trade was acceptable for game state where small windows of lost progress were survivable. It would be a poor fit for domains where every bit carries financial, medical, or safety consequences.