Skip to content

alimakki/tiny_ci

Repository files navigation

TinyCI

Early development. This project is a work in progress. APIs, DSL syntax, and behaviour may change between versions. Feedback and contributions are welcome.

A local CI runner for Elixir projects. Define your build pipeline as code — stages, steps, conditions, hooks — and run it from the command line. No YAML, no cloud dependency.

Quick Start

  1. Create a tiny_ci.exs file in your project root:
name :my_pipeline

on_success :notify, cmd: "echo 'Build passed on branch $TINY_CI_BRANCH'"
on_failure :alert, cmd: "curl -s -X POST $SLACK_WEBHOOK_URL -d '{\"text\":\"Build failed on $TINY_CI_BRANCH\"}'"

stage :test, mode: :parallel do
  step :unit, cmd: "mix test", timeout: 120_000
  step :lint, cmd: "mix credo"
  step :format, cmd: "mix format --check-formatted"
end

stage :deploy, mode: :serial, when: branch() == "main" do
  step :release, cmd: "mix release"
end
  1. Run it:
mix tiny_ci.run

Usage

mix tiny_ci.run [pipeline] [options]
Flag Short Description
--file PATH -f Path to a pipeline file (skips discovery)
--root DIR -r Project root for pipeline discovery
--dry-run Show what would execute without running anything
--list List all available pipelines in .tiny_ci/

The optional pipeline argument selects a named pipeline from .tiny_ci/:

mix tiny_ci.run           # discovers tiny_ci.exs or .tiny_ci/pipeline.exs
mix tiny_ci.run ci        # runs .tiny_ci/ci.exs
mix tiny_ci.run jobs/release  # runs .tiny_ci/jobs/release.exs
mix tiny_ci.run --list    # prints all available pipelines

Exit codes: 0 on success, 1 on failure — suitable for git hooks and scripts.

Pipeline Discovery

When --file is not given and no pipeline name is provided, TinyCI searches in order:

  1. tiny_ci.exs (project root)
  2. .tiny_ci/pipeline.exs

Named pipelines live in .tiny_ci/<name>.exs or nested as .tiny_ci/<dir>/<name>.exs.

DSL Reference

Pipeline files use a flat, declarative DSL. No defmodule, no use statements — just top-level directives. Files are parsed into a controlled AST rather than compiled as arbitrary Elixir modules.

name

Optional. Sets the pipeline name. Defaults to the filename stem (deploy.exs:deploy).

name :my_pipeline

Environment Variables

The env directive declares environment variables that are automatically inherited by steps. It can be used at the pipeline level (all stages and steps inherit) or inside a stage block (only steps in that stage inherit). Step-level env: values take precedence and override inherited ones.

# Pipeline-level: available to every step
env "MIX_ENV": "test"
env "DATABASE_URL": "postgres://localhost/mydb"

stage :test do
  # Stage-level: only steps in this stage inherit these
  env "NODE_ENV": "test"

  step :unit, cmd: "mix test"
  # MIX_ENV, DATABASE_URL, and NODE_ENV are all available here

  step :assets, cmd: "npm test"
  # Override a specific key for one step
  step :assets_prod, cmd: "npm run build", env: %{"NODE_ENV" => "production"}
end

Multiple variables can be declared on one line or across multiple env calls — they are merged in declaration order:

env "APP": "myapp", "REGION": "us-east-1"

--dry-run shows declared env vars at the pipeline and stage level.

Stages

By default, stages run sequentially. When any stage declares needs:, the pipeline switches to DAG execution: independent stages run in parallel while dependent stages wait for their prerequisites.

stage :name, mode: :parallel do
  # steps...
end
Option Default Description
:mode :parallel How steps within the stage execute — :parallel or :serial
:needs [] List of stage names that must complete successfully before this stage starts
:when (always run) Condition expression; stage is skipped when it evaluates to falsy
:working_dir (pipeline root) Default working directory for all steps in this stage
:matrix [] Keyword list of variable names to value lists; stage runs once per combination
:max_parallel (unlimited) Maximum number of matrix runs executing at the same time
:allow_failure false When true, a failing matrix combination does not fail the parent stage

Stage Dependencies (DAG)

The needs: option declares explicit dependencies between stages. Stages without needs: at the same level run in parallel; stages with needs: wait for all listed stages to pass.

# build and lint run in parallel (no dependencies between them)
stage :build do
  step :compile, cmd: "mix compile"
end

stage :lint do
  step :format, cmd: "mix format --check-formatted"
  step :credo,  cmd: "mix credo"
end

# test waits for build and lint to both succeed
stage :test, needs: [:build, :lint] do
  step :unit, cmd: "mix test"
end

# deploy waits for test
stage :deploy, needs: [:test], when: branch() == "main" do
  step :release, cmd: "mix release"
end

Execution topology for the above:

Level 1 (parallel): :build  :lint
Level 2:            :test          ← waits for both
Level 3:            :deploy        ← waits for test

Failure propagation: if a stage fails, all stages that needs: it (directly or transitively) are automatically skipped. Independent stages at the same level still run.

Cycle detection: circular dependencies (a needs b, b needs a) are caught at parse time with a descriptive error — the pipeline will not start.

--dry-run shows the dependency graph grouped by level, with [needs: ...] shown for each dependent stage.

Matrix Builds

The matrix: option replicates a stage across multiple variable combinations. TinyCI computes the cartesian product of all values, starts one run per combination (in parallel by default), and groups results under the parent stage name in the summary.

stage :test, matrix: [elixir: ["1.17", "1.18"], otp: ["26", "27"]] do
  step :unit, cmd: "mix test"
end

The above generates four parallel runs:

Combination Env vars injected
elixir=1.17, otp=26 ELIXIR=1.17 OTP=26
elixir=1.17, otp=27 ELIXIR=1.17 OTP=27
elixir=1.18, otp=26 ELIXIR=1.18 OTP=26
elixir=1.18, otp=27 ELIXIR=1.18 OTP=27

Each step in the stage receives its combination's values as uppercased environment variables (ELIXIR, OTP). The same values are also written into the pipeline store so module steps can read them via ctx.store.

Limiting concurrency — use max_parallel: to cap how many runs execute simultaneously:

stage :test,
  matrix: [elixir: ["1.17", "1.18"], otp: ["26", "27"]],
  max_parallel: 2 do
  step :unit, cmd: "mix test"
end

Allowing partial failure — by default any failing combination marks the entire stage as failed. Set allow_failure: true to let the pipeline continue even when some combinations fail:

stage :compatibility,
  matrix: [os: ["ubuntu", "macos", "windows"]],
  allow_failure: true do
  step :smoke, cmd: "./run_smoke_test.sh"
end

Reporter output — each combination is shown as a sub-row under the stage:

  ✓ test — passed (3.2s)
    ✓ [elixir=1.17, otp=26] (0.8s)
      ✓ unit (0.8s)
    ✓ [elixir=1.17, otp=27] (0.9s)
      ✓ unit (0.9s)
    ✓ [elixir=1.18, otp=26] (0.7s)
      ✓ unit (0.7s)
    ✓ [elixir=1.18, otp=27] (0.8s)
      ✓ unit (0.8s)

--dry-run lists all generated combinations without executing anything.

Steps

Each step is a shell command or a module callback.

# Shell command
step :test, cmd: "mix test", timeout: 60_000, env: %{"MIX_ENV" => "test"}

# Module step — module must be pre-compiled and available on the load path
step :deploy, module: MyApp.Deploy do
  set :region, "us-east-1"
  set :replicas, 3
end
Option Description
:cmd Shell command to execute
:module Module implementing execute(config, context)
:timeout Max execution time in ms; step fails if exceeded
:env Map of environment variables merged into the shell environment
:allow_failure When true, step can fail without failing the stage
:when Condition expression; step is skipped when it evaluates to falsy
:working_dir Directory to run the command in (string path)
:retry Number of times to retry on failure (e.g. retry: 3 = up to 3 retries)
:retry_delay Milliseconds to wait between retry attempts (default: no delay)

Conditions

The :when option is supported on both stages and steps. It accepts a boolean expression built from these primitives:

Expression Description
branch() Current git branch name (string)
env("VAR") Value of environment variable, or nil if unset
file_changed?("glob") true if any file matching the glob changed since last commit

Combine with standard boolean operators: and, or, not, ==, !=.

Stage-level conditions skip the entire stage when not met:

stage :deploy, when: branch() == "main" do
  step :release, cmd: "mix release"
end

stage :test, when: file_changed?("lib/**") or file_changed?("test/**") do
  step :unit, cmd: "mix test"
end

Step-level conditions skip individual steps within a running stage, leaving the rest of the stage unaffected:

stage :check do
  step :unit,     cmd: "mix test"
  step :dialyzer, cmd: "mix dialyzer", when: branch() == "main"
  step :audit,    cmd: "mix deps.audit", when: env("CI") != nil
end

A skipped step is reported with a icon in the summary and does not affect the stage outcome. --dry-run shows which steps would be skipped before any execution.

Working Directory

The working_dir: option sets the directory a shell command runs in. It can be set on a stage (applies to all steps) or on an individual step (overrides the stage value).

# Stage-level: all steps run inside frontend/
stage :frontend, working_dir: "frontend" do
  step :install, cmd: "npm install"
  step :build,   cmd: "npm run build"
  step :test,    cmd: "npm test", working_dir: "frontend/packages/core"
end

# Step-level only
stage :check do
  step :mix_test, cmd: "mix test"
  step :js_lint,  cmd: "eslint src", working_dir: "assets"
end

Relative paths are resolved from the directory containing the pipeline file. Absolute paths are used as-is. If the directory does not exist, the step fails immediately with a clear error before any command is run. --dry-run shows the resolved path for each step.

Step Retries

The retry: option retries a failed step automatically, useful for flaky network calls, intermittent package downloads, or external service timeouts.

stage :test do
  # Retry up to 3 times on failure
  step :flaky_test, cmd: "mix test --seed random", retry: 3

  # Retry with a 2-second delay between attempts
  step :fetch_deps, cmd: "mix deps.get", retry: 2, retry_delay: 2000
end
  • retry: N — retry up to N times; total max attempts = N + 1
  • retry_delay: N — wait N milliseconds between attempts (default: no delay)
  • Each attempt is logged with its number (e.g. [attempt 2/3])
  • allow_failure: true exhausts all retries before allowing the failure
  • timeout: applies per attempt, not across all attempts combined
  • --dry-run shows [retry: N] and [retry_delay: Nms] in the step plan
  • The summary reports passed on attempt N or failed after N attempts when retries were used

Hooks

Hooks run after the pipeline completes. Shell command hooks and module hooks are both supported.

# Shell command hook
on_success :notify, cmd: "say 'Build passed'"
on_failure :alert, cmd: "curl -X POST $SLACK_WEBHOOK_URL -d '{\"text\":\"Build failed\"}'"

# Module hook — module must be pre-compiled and available on the load path
on_success :slack, module: MyApp.SlackNotifier do
  set :channel, "#deploys"
end

on_failure :slack, module: MyApp.SlackNotifier do
  set :channel, "#alerts"
end

Hook failures are logged to stderr but do not change the pipeline exit code.

Module Steps and Hooks

Module steps implement execute/2; module hooks implement run/2. Both receive the config keyword list (from set/2 calls) and the pipeline context map:

defmodule MyApp.Deploy do
  def execute(config, context) do
    region = Keyword.fetch!(config, :region)
    branch = context.branch

    # deploy logic...
    :ok        # or {:error, reason}
  end
end

defmodule MyApp.SlackNotifier do
  def run(config, context) do
    emoji   = if context.pipeline_result == :on_success, do: "✅", else: "❌"
    message = "#{emoji} *#{context.branch}* — pipeline #{context.pipeline_result}"

    {_output, exit_code} =
      System.cmd("curl", [
        "-s", "-o", "/dev/null",
        "-X", "POST", config[:webhook_url],
        "-H", "Content-Type: application/json",
        "-d", ~s({"channel":"#{config[:channel]}","text":"#{message}"})
      ])

    if exit_code == 0, do: :ok, else: {:error, :curl_failed}
  end
end

Module steps return :ok or {:ok, map} to merge data into the pipeline store. Module hooks return :ok or {:error, reason}.

Note: Module steps and hooks must be pre-compiled and available on the Elixir load path before TinyCI runs. They cannot be defined inside the .exs pipeline file.

Sharing Data Between Steps

The pipeline store is a key-value map that accumulates data across steps and stages within a single pipeline run. It lets earlier steps produce values that later steps consume.

Writing to the store (module steps)

A module step writes to the store by returning {:ok, map} from execute/2:

defmodule MyApp.BuildImage do
  def execute(_config, _ctx) do
    tag = "myapp:#{System.get_env("GIT_SHA", "latest")}"
    # ... build the image ...
    {:ok, %{image_tag: tag}}   # merged into the store
  end
end

Shell steps cannot write to the store.

Reading from the store (module steps)

Module steps read prior values from ctx.store:

defmodule MyApp.PushImage do
  def execute(_config, ctx) do
    tag = ctx.store.image_tag   # written by an earlier step
    {_out, 0} = System.cmd("docker", ["push", tag])
    :ok
  end
end

Reading from the store (shell steps)

Shell steps do not receive store values automatically. Declare exactly which keys you need using store(:key) in the step's env: option:

stage :build do
  step :tag_image, module: MyApp.BuildImage    # writes image_tag to store
end

stage :deploy do
  step :push,
    cmd: "docker push $IMAGE_TAG",
    env: %{"IMAGE_TAG" => store(:image_tag)}

  step :notify,
    cmd: "echo Deployed $IMAGE_TAG to production",
    env: %{"IMAGE_TAG" => store(:image_tag)}
end

Only the keys you explicitly reference are exposed. Everything else in the store stays invisible to the shell environment — so a step that writes a computed auth token cannot accidentally leak it to unrelated steps.

Scope

The store is local to a pipeline run. It starts empty, accumulates values left to right across steps and top to bottom across stages, and is discarded when the run ends.

Stage 1 step A writes {image_tag: "myapp:abc"}
Stage 1 step B sees   store = %{image_tag: "myapp:abc"}
Stage 2 step C sees   store = %{image_tag: "myapp:abc"}   ← carries forward
Stage 2 step D writes {pushed: true}
Stage 3 step E sees   store = %{image_tag: "myapp:abc", pushed: true}

In parallel stages, all steps start with the same store snapshot; their outputs are merged after all steps finish, so two parallel steps writing the same key results in an arbitrary winner. Avoid writing the same key from parallel steps.

Hooks and the store

The same store(:key) syntax works in hook env: options:

on_success :deploy_notify,
  cmd: "echo Deployed $TAG to production",
  env: %{"TAG" => store(:image_tag)}

Hooks receive TINY_CI_RESULT, TINY_CI_BRANCH, and TINY_CI_COMMIT automatically — store values are only injected when you ask for them.

Sharing between pipelines

There is no built-in mechanism to share data between separate mix tiny_ci.run invocations. Use the filesystem or environment variables as the bridge:

# pipeline: build
stage :package do
  step :write_tag, cmd: "echo myapp:$(git rev-parse --short HEAD) > .tiny_ci_tag"
end

# pipeline: deploy  (run separately, e.g. after build)
stage :push do
  step :deploy, cmd: "docker push $(cat .tiny_ci_tag)"
end

Pipeline Context

Every pipeline run builds a context map from the git environment:

%{
  branch: "main",              # current git branch
  commit: "a1b2c3d...",       # full commit SHA
  changed_files: ["lib/..."], # files changed since last commit
  store: %{},                  # accumulated data from module steps
  timestamp: ~U[...]           # UTC timestamp
}

Module hooks also receive :pipeline_result (:on_success or :on_failure).

DSL Allowlist

Pipeline files are validated against an allowlist of permitted constructs before execution:

  • name, stage, step, on_success, on_failure, set
  • Stage options: :mode, :needs, :when, :working_dir, :matrix, :max_parallel, :allow_failure
  • Step options: :cmd, :module, :timeout, :env, :allow_failure, :when, :working_dir, :retry, :retry_delay
  • Condition expressions: branch(), env/1, file_changed?/1, ==, !=, and, or, not, if/else

Constructs outside this list (e.g. defmodule, System.cmd, File.read) are rejected at load time with a descriptive error.

Multiple Pipelines

Organize multiple pipelines in .tiny_ci/:

.tiny_ci/
  ci.exs          # mix tiny_ci.run ci
  deploy.exs      # mix tiny_ci.run deploy
  jobs/
    nightly.exs   # mix tiny_ci.run jobs/nightly
mix tiny_ci.run --list    # shows: ci, deploy, jobs/nightly
mix tiny_ci.run ci
mix tiny_ci.run jobs/nightly --dry-run

Project Structure

lib/
  mix/tasks/
    tiny_ci.run.ex        # CLI entry point (mix tiny_ci.run)
  tiny_ci/
    application.ex        # OTP application / task supervisor
    context.ex            # Git context builder
    discovery.ex          # Pipeline file discovery
    dry_run.ex            # --dry-run plan printer
    dsl/
      condition_eval.ex   # Condition expression evaluator
      interpreter.ex      # DSL file parser → PipelineSpec
      validator.ex        # AST allowlist validator
    dag.ex                # DAG level computation and cycle detection
    dsl.ex                # Macro-based DSL (internal use)
    executor.ex           # Stage/step execution engine
    hooks.ex              # Hook runner
    matrix.ex             # Matrix combination generator and helpers
    matrix_run_result.ex  # MatrixRunResult struct
    output.ex             # Command output streaming
    pipeline_spec.ex      # PipelineSpec struct
    reporter.ex           # Summary and output formatting
    tiny_ci.ex            # Step and Stage struct definitions
    step_result.ex        # StepResult struct
    stage_result.ex       # StageResult struct
test/
  mix/tasks/
    tiny_ci_run_test.exs  # Mix task integration tests
  tiny_ci/
    context_test.exs
    discovery_test.exs
    dsl/
      condition_eval_test.exs
      interpreter_test.exs
      validator_test.exs
    dsl_test.exs
    executor_test.exs
    integration_test.exs
    reporter_test.exs

Development

mix test                           # run full suite
mix format                         # format code
mix compile --warnings-as-errors   # check for warnings
mix credo                          # static analysis

Roadmap

Completed

  • Core execution — serial and parallel stage modes, fail-fast pipeline, conditional stages
  • Git context — automatic branch/commit detection passed through the pipeline
  • CLImix tiny_ci.run with discovery, --file, --root, --list, named pipelines, proper exit codes
  • Generic configset key, value for module step and hook configuration
  • Output — live streaming with per-step prefixes in parallel mode, buffered fallback in non-TTY
  • Robustness — step timeouts, --dry-run, allow_failure steps
  • Richer conditionsbranch(), env/1, file_changed?/1 with boolean combinators
  • Hookson_success / on_failure pipeline hooks (shell and module-based)
  • Step data passing — pipeline store for sharing data between module steps
  • Custom DSL — declarative pipeline format with an allowlist validator
  • Stage dependencies (DAG)needs: for fan-out/fan-in topologies with parallel independent stages, transitive skip propagation, and cycle detection at parse time
  • Matrix buildsmatrix: option for cartesian-product parallel stage runs with env var injection, max_parallel: concurrency cap, and allow_failure: for partial tolerance

Up Next

  • Secrets managementsecret "MY_KEY" reading from env or a local secrets file, with value masking in output
  • Dependency caching — skip steps when input files haven't changed, keyed by file hash
  • Artifact persistence — declare build outputs that downstream stages can consume
  • Watch modemix tiny_ci.run --watch to re-run on file changes

About

A simple CI runner with declarative pipeline DSL written in Elixir

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages