Projects · Infrastructure

A reproducible research environment

The point is reproducibility. A project should run the same way on my machine and on a server, so a result holds up when it moves from my laptop into a paper.

The problem it solves

Empirical work has a common failure mode. A result lives on one laptop, tied to whatever package versions happened to be installed when it was made. A year later the figure won't regenerate, and even the author can't reconstruct how it was produced.

The fix is to give every project its own environment. Dependencies are pinned, and a single command rebuilds every figure from the raw data. Nothing important is left sitting on the host machine, so reproducibility becomes a property of the setup rather than something I have to promise.

Architecture

The whole environment is containerized. Each service is defined in a single file and isolated from the others, and the same definition runs in both development and production. There's no “works on my machine” gap to fall through.

Containerized services

Docker Compose defines each service, the web server, language runtime, and database, as an isolated, version-pinned unit. The whole stack comes up from one file with one up.

Reproducible pipelines

Each analysis repo has a single entry point that rebuilds every output from source. No manual steps and no hidden state: raw data goes in, the figures come out.

Self-hosted

All of it runs on my own homeserver, a small cluster of machines I administer myself. TLS, database backups, and deploys are set up and under my control, with no hosting platform between me and the hardware.

Automation & CI

The repetitive work of regenerating widgets and rerunning pipelines is scripted rather than done by hand, and the analysis repos run those pipelines under continuous integration. The routine path is the one that gets checked.

Every project follows the same shape. This site's own stack is a fair example: three version-pinned services that talk only to each other over a private network, with the application built from a local image and every secret kept out of the repo.

compose.yml · this site's stack

services:
  db:                     # MariaDB, pinned
    image: mariadb:11
    env_file: .env        # secrets, never in git
    volumes: [db_data:/var/lib/mysql]
    networks: [internal]  # private to the stack

  php:                    # built from a local image
    build: { context: ., dockerfile: Dockerfile.php }
    networks: [internal]
    healthcheck:          # auto-restart if it stalls
      test: ["CMD", "php", "-r", "echo 'ok';"]

  web:                    # nginx, the only exposed service
    image: nginx:alpine
    depends_on: [php]
    networks: [internal]

volumes:  { db_data: {} } # survives a full rebuild
networks: { internal: { driver: bridge } }

What runs on it

The same environment hosts this site and the research codebases behind the papers, along with the automation that keeps them in sync, regenerating chart widgets and rerunning pipelines as the data changes. I maintain the research and the platform it runs on as one system.

Why it matters

The stack matters for what it buys the research: a result can be rebuilt on demand, and a reviewer can run the code instead of taking the figure on trust. The infrastructure is there so the economics behind it holds up.

See the research codebases for the pipelines this runs, or this site for the platform itself.