Guile Automate helps you build and run pipelines written in GNU Guile using GNU Guix and Spritely Goblins. A pipeline is written as Guile definitions, lowered to a Guix store item, and run as Goblins actors with each step in its own process. A pumphouse holds pipelines, their run history and shared resources, and serves a web interface over them.
Automate is a prototype. It runs its own pipelines, but any version may change its interfaces, the pumphouse’s databases or the form of a pipeline definition without keeping the old ones working. The Version History appendix (see Version History) says what changed in each version and what to do about it. Report bugs in the issue tracker.
This chapter explains how the pieces of Guile Automate fit together. Each section covers one concept with a short code example. For the full list of procedures, fields and options, see API.
A pipeline is set out in Guile code. The steps are the parts of the pipeline, and these reference the outputs of other steps, along with resources and parameters. The pipeline itself is a collection of steps. Lastly you turn a pipeline into something that can be run. The file below writes a greeting into a directory in one step and prints it in another.
(use-modules (guix gexp) (automate)) (define work (workspace #:name 'work)) (define who (param 'name)) (define written (step (name 'write) (inputs (list work who)) (body #~(let ((directory (attach #$work))) (call-with-output-file (string-append directory "/greeting") (lambda (port) (format port "Hello, ~a!" #$who))) directory)))) (define shown (step (name 'show) (inputs (list written work)) (body #~(let ((directory (attach #$work))) (call-with-input-file (string-append directory "/greeting") get-string-all))))) (one-shot (pipeline 'examples/hello #:steps (collect-steps shown)))
Each step’s body is a G-expression, the quoted code Guix builds with
(see G-Expressions in GNU Guix Reference Manual). It is
written at definition time and runs later, in a process of its own.
The #~ marks the body; #$ inside it splices in a value
from the definition, such as a package or one of the step’s inputs.
attach and the rest of the body vocabulary are described in the next two
sections.
The file is not run directly (see System Requirements for what the machine needs). It is built with Guix, which lowers the final expression to a program in the store:
$ guix build -f hello.scm /gnu/store/...-automate-one-shot-examples-hello
Automate’s modules have to be on Guile’s load path for this, either
through an installed package or with -L naming a checkout.
Building does not run any step: it checks the definition, writes
each step’s body into a program with its own dependencies, and
writes the structure of the pipeline beside them. The result is a
program that runs the pipeline once:
$ $(guix build -f hello.scm) --param-string=name=World
A parameter is given with --param-string=NAME=STRING, whose
value is the string after the ‘=’, or with
--param=NAME=DATUM, whose value is read as Scheme
(see Parameters).
--secrets=DIR names the directory secret files are read from
(see Resources). --repl=STEP stops that step and offers a
session on it (see Attaching to a Step). --cap=N bounds
how many step processes run at once; the default is derived from the
CPU count.
What a step prints is its log. A log is not an output: an
output is a value a step provides for other steps (see Outputs and Errors). The program copies each step’s log to its own standard
output as the step writes it, with secrets masked. Steps that run at
the same time have their lines mixed together, with no step name on
them. The program also prints each event of the run as it happens,
as a Scheme datum. --quiet leaves out the events but not the
logs. At the end it prints the run’s outcome and the status of
every step, and it exits with a non-zero status when the run errored.
Nothing is kept once the program exits, unless it is told where a
pumphouse is (see Running a Pumphouse). The logs then go to the
pumphouse rather than to the terminal, and automate run
output prints them.
The same pipeline runs from a pumphouse when the file ends in
run-daemon instead of one-shot, which Pipelines
explains. A definition file is an ordinary Guile program up to its
last expression, so steps can be generated with map, shared
helpers can live in a module of the project’s own, and a procedure
can return a whole pipeline.
The examples/ directory of the source holds more definition
files, each with the commands that build and run it beside its last
expression. hello.scm adds a secret and a limiter to the file
above, and daemon.scm serves it from a pumphouse.
linked.scm has one pipeline call another, and
channels.scm builds a package from a guix-channels
resource. slow.scm runs long enough to watch, and
pumphouse.scm runs a pumphouse for the others.
A step is declared with the step form, giving its fields by
name in any order:
(step (name 'build) (inputs (list cloned work)) (label "Build the release") (body #~(...)))
The name is a symbol, unique within the pipeline. The body is
required; every other field has a default. (inherit STEP)
copies another step’s fields before the ones given, so a variant is
declared by its differences. The field names are bound as variables
inside the form, so a definition called name, inputs
or body is shadowed there: call the variable something else.
The full list of fields is in (automate).
The inputs of a step are a list of sources: other steps, resources and parameters. A step runs once its inputs have settled, so listing a step as an input is how one step is made to run after another. Steps whose inputs allow it run at the same time, each in a process of its own.
Inside the body, an input is referred to by splicing its source with
#$, as a package is spliced. For a step, #$built is its
value output, which is what its body returned. For a
parameter, it is the value the run was given. For a resource, it is
a reference that the body attaches to get something it can
use, such as a directory or a file (see Resources).
(define work (workspace)) (define built (step (name 'build) (inputs (list work)) (body #~(attach #$work))))
A step’s other outputs are selected as Guix selects an output of a
package, with a colon: #$built:version is the version
output of built, and so is
#$(step-output built 'version). The step declares that it
wants the output with step-output in its input list:
(define reported (step (name 'report) (inputs (list (step-output built 'version))) (body #~(format #t "built version ~a~%" #$built:version))))
Listing a step plainly selects its value output. A step that
declares outputs has only the ones it declares, so a consumer of
built above lists (step-output built 'dist) rather
than built; the pipeline is refused otherwise, naming the step
and the output. A step may take several inputs from one source,
selecting different outputs. No step may declare an output called
out, since #$built:out could not be told from a plain
#$built.
(input-present? #$built) says whether the input is there,
without raising when it is not.
A splice says which input it means with the binding the file already
has in hand, so nothing has to agree on a name. This is what lets
steps written apart be put together. Two steps from two places may
each hold a workspace called work: a pipeline gives the second
of them the identifier work.2, and each body reaches its own.
A name that no other source wanted is kept as it is, so a pipeline
with no such pair is unaffected.
A step and a parameter are still named from outside the run, on the command line and in the web interface, so two steps cannot share a name and a step, a resource and a parameter cannot share one either.
The inputs a body splices are checked when the file is built. A body that splices a source its step does not declare is refused then, with the step and the name, rather than failing at run time.
A body is also compiled when the file is built, in a module of its
own called (automate step-body name). A body Guile
cannot compile is refused then, and Guile’s warnings about one, such
as a variable it cannot find, are printed as the file is built. The
backtrace on a step’s error names that module, giving the line and
column of each frame, so a failure can be placed without running the
step again.
A body runs in a process of its own, under the Guile that Guix is
built with, and it closes over nothing from the definition file. It
sees three kinds of things: values spliced in with #$ when the
file was built, its inputs, and the body vocabulary of
(automate pipeline step):
input-present?, provide, attach,
release, commit, store-item, mask,
trigger-run, list-runs, listed-run-output,
abort-run, run-id, run-url and in-vat.
run-id is the id of the run the step is part of, and
run-url the address of the run’s page on the pumphouse’s web
interface, or #f when no pumphouse with one serves the run, so
a step reporting to a forge can link back to the run.
Software is given to a body the way Guix gives it to a build: a
package spliced in with #$, usually through file-append
to name a program in it. Guix’s build utilities, which give
invoke, mkdir-p and copy-recursively, are
imported into every body, along with (ice-9 match),
(ice-9 textual-ports) and (srfi srfi-1).
(use-modules (guix gexp) (gnu packages version-control) (automate)) (define work (workspace #:name 'work)) (define repository (param 'repository)) (define cloned (step (name 'clone) (inputs (list work repository)) (body #~(let ((directory (attach #$work))) (invoke #$(file-append git-minimal "/bin/git") "clone" "--depth=1" #$repository (string-append directory "/source")) (string-append directory "/source")))))
Other modules are named in the modules field, which adds to
the default set:
(step (name 'hash) (inputs (list cloned)) (modules '((rnrs bytevectors) (ice-9 binary-ports))) (body #~(...)))
Guix’s own modules are available to every body without being listed
as software, since Guix is part of every step’s program, so a body
can talk to the Guix daemon with (guix store) once it names the
module and holds the guix-daemon resource (see Resources).
That Guix is a library, not a package collection: a body takes
packages from the guix-channels resource, which says whose
collection they are.
Helpers that several bodies share go in a module of the project’s
own, beside the definition file, and each step using it names it in
modules. Building the file copies into a step’s program every
named module that no package provides: Automate’s own, and any found
on the load path outside the store, so guix build -L . from
the project’s checkout finds both. Such a module is build-side code
in Guix’s sense: it runs in the step process, may call the body
vocabulary and Guix’s modules, and holds no package objects.
;; release/helpers.scm (define-module (release helpers) #:use-module (ice-9 textual-ports) #:export (read-first-line)) (define (read-first-line file) (call-with-input-file file (lambda (port) (let ((line (get-line port))) (if (eof-object? line) "" line)))))
(define versioned (step (name 'version) (inputs (list cloned)) (modules '((release helpers))) (body #~(read-first-line (string-append #$cloned "/VERSION")))))
Automate’s own helpers are the (automate helpers …)
modules of the API chapter: (automate helpers git) runs
git, (automate helpers secrets) reads an attached secret,
(automate helpers profiles) builds a development profile
through the Guix daemon and runs shell text inside one, and
(automate helpers notify) sums up a run’s checks and sends
messages about them (see Notifications). The programs they run
come from Guix the first time a step needs them. They are built
through the Guix daemon from the Guix of a guix-channels
resource the body attached (see Resources). A body can instead
set current-git or current-bash to a program it chose
itself, which is how a pipeline pins or replaces one. Here the
package is git-minimal, spliced in from the definition file,
and git is the helper that runs it:
(parameterize ((current-git #$(file-append git-minimal "/bin/git"))) (git (attach #$work) "fetch" "origin" #$ref))
A token for the forge stays in its secret file. A body sets
current-git-credentials to the path attach of the
secret returns, around the calls that reach the remote, and git reads
the file through a credential helper when the remote asks. The URL
is the repository’s plain one, so it can be the same parameter for a
run against the forge and a run against a clone on disk, and no token
reaches a command line, an error record, a log line or a
.git/config:
(parameterize ((current-git-credentials (attach #$forgejo-token))) (git (attach #$work) "clone" "--depth=1" #$repository "trunk"))
A body is sequential Guile on a thread of its own. It may block on
subprocesses and files for as long as it likes; the connection back to
the pipeline stays live on another thread. A step with a
timeout, in seconds, is killed when it runs out and recorded
as errored.
A step process is given an environment of Automate’s making, not the
one the pipeline process runs with. It holds LANG, the
certificate variables the machine sets, TZ and TMPDIR,
and, for a step declaring a guix-daemon resource, the
GUIX_DAEMON_SOCKET that resource names. Nothing else: a step
has no HOME and no PATH.
That is why a body names a program in full, with #$ and
file-append, rather than running it by name. Without
HOME, a step also reads none of the operator’s dotfiles, so
no ~/.gitconfig, credential helper or SSH key reaches it
because it happened to be on the machine. Where a step runs a shell
script in a profile, invoke-in-profile of
(automate helpers profiles) sources
that profile and sets PATH from it, so the script finds the
profile’s programs and nothing else.
Without HOME there is also nowhere for a program to keep a
cache, and a program that wants one usually carries on without saying
much. Guile compiles a module as it loads it and writes the result
under XDG_CACHE_HOME, or ~/.cache when that is unset; in
a step it can create neither, warns once per file, and interprets the
module instead. That is slower, and a backtrace taken from
interpreted code names no source of yours, so a step that runs Guile
code — a test suite, most of all — wants a cache directory of its own.
TMPDIR is the step’s own writable temporary directory, so this
is enough, ahead of whatever the step runs:
export XDG_CACHE_HOME="$TMPDIR/cache"
The same goes for anything else that looks for a home: point it at a
directory under TMPDIR, which lasts as long as the step and no
longer.
An operator who has to get a further variable to the steps of a run
names it with --pass-environment=NAME, which passes it on if
the pipeline process has it. Reach for it rarely: a variable passed
that way reaches every step, and it is authority that the pipeline
does not declare and a reader of it cannot see. What a step needs
from outside the run is better a secret, a parameter or a resource.
A step runs in a file system of its own. It holds the store, a temporary directory of its own at /tmp, a shell at /bin/sh, the /dev and /proc any program expects, and, of the rest of the machine, only what the step’s inputs name: its workspaces, its secrets, the Guix daemon’s socket. Everything else is absent, and the root is read-only, so a step writes only where it was given somewhere to write.
The few files the step’s environment names come with it: the zone
TZ is read from, and the certificates SSL_CERT_DIR,
SSL_CERT_FILE, GIT_SSL_CAINFO and CURL_CA_BUNDLE
point at. A step is never told where a file is and then left unable
to open it.
The paths are the machine’s, so attach answers what it always
did and a step that declares what it uses needs no change. A step
that reads a path nothing gave it fails there, with the error its
program gives for a file that is not there. That includes the other
secrets in the directory the run was given: a step holding
(secret 'token) sees that file and no other.
A step holding the guix-daemon resource is the exception worth
knowing. The socket is the whole store with write access, so what a
step can reach through it is not bounded by its file system at all.
An operator who has to give the steps of a run a directory the
pipeline does not declare names it with --expose=DIR. As with
--pass-environment, reach for it rarely: it goes to every step,
writable, and it is authority that a reader of the pipeline cannot
see. Where the directory is the pipeline’s to name rather than the
operator’s, a file-input resource says the same thing in the
definition, read-only and only to the steps that declare it.
A parameter naming a path on the machine is where this catches a
pipeline out. A repository parameter is usually a URL, which a step
with the network reaches, but a run given a local clone instead is
given a path nothing declared, and the step fails on it as it would on
any other path it was not given. --expose is the answer, since
which path it is was the operator’s to choose and not the pipeline’s.
The label is a string the web interface draws on the step’s
node in place of its name, and the group is a symbol that
gathers steps into a cluster in the drawing. Neither affects how the
step runs.
A step that declares no outputs has one, called value, provided
from what its body returns. A step that does more than one thing
declares its outputs and provides each by name:
(define built (step (name 'build) (inputs (list cloned)) (outputs '(dist version)) (body #~(let ((source #$cloned)) (invoke "make" "-C" source) (provide 'dist (string-append source "/dist")) (provide 'version "1.0")))))
provide returns once the pipeline has recorded the output, so
a step downstream can start on it while this body carries on.
Providing the same output twice is an error. When the body ends,
a declared value not yet provided takes the body’s return
value; any other declared output left unprovided errors the step,
after its work is done. So a step that declares dist and
never reaches provide for it has errored even if the body
returned normally.
An output’s value is plain data: a string, number, symbol, boolean
or list of them. A store item from the guix-daemon resource
is the one kind of reference a step may provide (see Resources).
Bulk results go into a workspace or the store, and the step provides
the path.
An input settles when the output it selects is provided, or when its step has finished without providing it. A step waits until every input has settled. It then runs if every required input was provided, and is skipped otherwise. A skipped step provides nothing, so the steps that need its outputs are skipped in turn.
#:required? #f on an input makes the step wait for it without
depending on it. In the body, input-present? says whether
the value came:
(input-present? #$built) (input-present? #$checked:status)
This is how a step runs after a fan-out whatever the outcome of each branch:
(define architectures '(x86_64 aarch64)) (define builds (map (lambda (architecture) (step (name (symbol-append 'build- architecture)) (inputs (list cloned)) (body #~(string-append #$cloned "/dist/" (symbol->string '#$architecture))))) architectures)) (define reported (step (name 'report) (inputs (map (lambda (build) (step-output build 'value #:required? #f)) builds)) (body #~(begin #$@(map (lambda (architecture build) #~(format #t "~a: ~a~%" '#$architecture (if (input-present? #$build) #$build "not built"))) architectures builds)))))
The list of builds is known when the file is built, so the body is
made there: #$@ splices in one expression per build, each
splicing its own step. Every input the body refers to is then checked
when the file is built.
When a body raises, the step is errored and its implicit
error output is provided with a record of the error: its kind,
its message, its irritants printed as strings, and the printed
representation of the original. Outputs provided before the raise
stand. Any other step may take the error output as an input,
and runs only when the error happens:
(define published (step (name 'publish) (inputs (list (step-output built 'dist))) (body #~(upload #$built:dist)))) (define rolled-back (step (name 'rollback) (inputs (list (step-output published 'error))) (modules '((automate pipeline errors))) (body #~(begin (format #t "publish failed: ~a~%" (error-record-message #$published:error)) (undo-upload)))))
The record is a list, and (automate pipeline errors) has the
accessors for it. A step that wants to run whether or not another
failed takes both value and error with
#:required? #f, and asks input-present? which came.
A step is pending until its inputs settle, running
while its process lives, and then one of succeeded,
errored, skipped or abandoned, the last meaning
its process died before it finished. These are the words
automate run show and the run’s page use.
A run is errored when a step was abandoned, or when a step
errored without (error-permitted? #t), or when the run was
aborted. It is abandoned when the daemon serving it died or was
restarted before it finished (see Running a Pumphouse); its
running steps are then abandoned and its pending ones skipped.
Otherwise it succeeded.
Skipped steps never fail a run. This is deliberate, and it has a
consequence to watch for: a pipeline whose final step is skipped
because an earlier one errored is only failed by that earlier error.
If the earlier step is marked error-permitted?, the run
succeeds with its last step never having run. So error-permitted?
is a claim about the step that raises: its error is expected, and
something downstream handles it. Put it on the step whose failure is
routine, not on the handler. A handler taking the error
output keeps the default, so that a failure in the handling itself
fails the run.
A step’s timeout counts as an error of the step, not as abandonment.
automate run abort RUN-ID ends a run early: its pending steps
are skipped and the run is errored.
A resource is something a step needs that the pipeline provides: a
directory to work in, a secret, a file to read, the network, a slot,
the Guix daemon, the Guix its packages come from, a decision from
outside the run. Resources are declared once,
with a name, and appear in the input lists of the steps that use
them. A body gets at one with attach:
(define work (workspace #:name 'work)) (define token (secret 'forge-token)) (define fetched (step (name 'fetch) (inputs (list work token)) (body #~(let ((directory (attach #$work)) (token-file (attach #$token))) (fetch-into directory (call-with-input-file token-file get-string-all))))))
attach returns whatever identifies the resource locally: a
path for a workspace, a secret, a file input or the Guix daemon’s socket,
and nothing of use for a limiter or a gate, where attaching is the
point.
Attaching twice from the same step returns the same thing. Everything
a step attached is released when its process exits; release
lets it go earlier, which matters for limiters. A failure to attach,
such as a secret whose file is missing, raises in the body and errors
the step.
The one binding is what identifies a resource. Two steps sharing the
binding work share the directory. Two separate
(workspace) forms are two resources: a pipeline holding both
gives the second the identifier workspace.2 and keeps them
apart, and a body reaches the one it means by splicing it
(see Steps and Inputs). The identifier is what the run shows and
what a workspace’s directory is named after.
A resource’s identifier starts from its name, which is its type unless
#:name gives another, and is numbered where a step, a parameter
or another resource of the pipeline has that name already. So the
name is a label, for the run’s pages and events. It is more only
where something outside the definition picks the resource by it: a
secret, found by name among the files the run is given, a gate, which
automate gate approve names, and a pumphouse limiter, which
the pipelines sharing it agree on by name.
Those take the name as an argument of their own.
Most resources are materialised once per run and gone at its end. Their constructors are in (automate).
(workspace) is a directory, made when first attached and
shared by every step holding it. It is the usual place for a checkout
and everything built from it. Steps holding a workspace see the same
directory, so one step’s files are the next step’s inputs by path.
A workspace is scratch, and belongs to the run that made it. Every
run makes a directory of its own to hold its workspaces, under
TMPDIR or under /var/tmp where the environment names
none, and removes it when the run ends, whatever the run’s outcome.
Nothing a run leaves in a workspace outlives it: what has to stay on
the machine is a file output, below, and what has to come from the
machine is a file input. Point TMPDIR at a file
system with room for what the steps build; a /tmp that is a
tmpfs is memory.
#:seen-at says where a step sees it, rather than leaving it
where the run put it:
(define work (workspace #:seen-at "/src"))
attach then answers /src, in every step holding the
workspace, and where the directory is on the machine is the run’s
business rather than the pipeline’s. Put it on the resource and not
on a step, so that every step agrees: a path one step provides as an
output is a path the next one opens. A pipeline naming one place for
two resources a step holds is refused, since the step could only see
one of them. #:seen-at has no effect on a run whose steps are
not confined, which has no file system of its own to put it in.
(secret NAME) is a read-only file, found under the name in the
directory the program was given with --secrets=DIR. Attaching
returns its path; the step reads it. The file has to exist when the
step attaches it, and the step errors otherwise. Where the secret
material comes from is outside Automate: an operator puts the files
in place. A secret is never a parameter, since parameters are
recorded and shown.
#:seen-at says where a step sees it, as it does for a
workspace:
(define token (secret 'forge-token #:seen-at "/etc/forge-token"))
A step is given the secrets it declares whether it attaches them or
not, so the run masks a declared secret’s contents from the moment the
step starts. Attaching it masks them too, for a secret whose file
appears later. Wherever the value would appear in what the run shows,
the marker
*** stands in its place: in the step’s log as it is written,
in every event, in every output value a step provides, and in every
error record, including the irritants of an error a body raised. The
whole value is masked, whitespace trimmed, and so is each of its lines
when it has several, as a private key does. The masking is exact: a
value printed in another encoding, or in pieces, is not recognised.
And a value that was printed at all has already left the step
process, so the masking is a net under the rule that a body does not
print a secret, not a reason to relax it.
A value under eight bytes is not masked, since replacing it would
corrupt ordinary output, and the run records a mask-refused
event naming the secret, as it does for a value over four kilobytes
without lines short enough, or past the run’s limit of 128 values.
A body that derives a further secret, such as a short-lived token it
exchanged the secret for, masks it with mask before printing or
providing it:
(define exchanged (step (name 'exchange) (inputs (list token)) (modules '((automate helpers secrets))) (body #~(let ((session (exchange-token (read-secret (attach #$token))))) (mask session) session))))
mask raises, with the reason, for a value the run cannot
take, so a body finds out at once rather than leaking quietly.
(file-input #:source PATH) is a file or directory on the
machine the run is on, which the steps holding it read and cannot
write. Attaching returns its path, and the step opens it as it would
any other:
(define sources (file-input #:name 'sources #:source "/srv/sources")) (define built (step (name 'build) (inputs (list sources work)) (body #~(build-from (attach #$sources) (attach #$work)))))
The source has to be there when a step attaches it, and the step errors otherwise: the resource makes nothing, it only lets a step in. A step that holds the file input sees it and no more of the machine than it saw before, and writing there is refused by the kernel rather than by convention, so a pipeline can hand a step a checkout, a cache or a directory of fixtures without also handing it the means to change them.
#:seen-at says where a step sees it, as it does for a
workspace or a secret:
(define sources (file-input #:source "/srv/sources" #:seen-at "/src"))
Reach for it rarely. A pipeline naming a place on the machine runs
only on a machine that has it, which is what a workspace, a step’s
output or a store item avoid; and where the place is the operator’s to
choose rather than the pipeline’s, --expose is still the
answer. What a file input is for is the case where the place is the
pipeline’s to name, the same on every machine it runs on, and the
steps have no business writing to it.
Neither the read-only view nor #:seen-at has any effect on a
run whose steps are not confined: such a step has no file system of
its own, and reaches the source whether it declares it or not.
(file-output #:destination PATH) is a place on the machine
that the run replaces, and the one thing a step writes whose result
outlives the run. The step never holds the destination. Attaching
answers a path in a staging directory of the run’s own, the body makes
the content there, and commit publishes it:
(define site (file-output #:name 'site #:destination "/srv/www/site")) (define published (step (name 'publish) (inputs (list site checked)) (body #~(let ((out (attach #$site))) (build-the-site-into out) (commit #$site)))))
The publishing is the pipeline process’s, not the step’s. A confined
step never has the destination’s directory in its file system at all,
so it can write what it stages and nothing else; the authority to
replace the place on the machine stays with the process the operator
started. That is the difference from --expose, which hands a
step the directory itself and says nothing a reader of the pipeline
can see.
Commit publishes in one step that a reader of the destination cannot see the middle of. The destination is a symbolic link that Automate keeps: each commit puts a generation beside it and renames a new link over it, so a reader following the destination gets the generation a run published or the one before it and never a half-written tree. Renaming the content itself onto the destination would not do, since that is refused for a directory that is not empty, and replacing a built directory is what this is for. A destination that is already a directory is in the way: an operator moves it aside once, and the run makes the link.
commit raises where nothing was staged, where the file output
was committed already—it is committed once—and where the
destination cannot be published to. The raise happens in the step, so
a publish that fails errors that step, skips what depends on it and
lands in that step’s log with a backtrace, rather than surfacing
somewhere the run has no name for. Whether the run publishes only
after everything else passed is the step graph’s to say, by what the
publishing step depends on, as it is for a step that pushes to a
forge.
#:commit-by says what commit publishes:
renameThe default. The generation is the staged content, moved. It costs nothing, and no path the step can still name reaches what was published—but a descriptor the body left open on a staged file still writes to the published one.
copyThe generation is a copy, so the body shares nothing with what was published and a descriptor it kept reaches only the staged tree, which the next commit reaps. The copy is copy-on-write where the file system does that and a plain copy where it does not.
Reach for copy where the guarantee matters more than the work:
a body that hands a file to something else before committing, or one
whose behaviour after the commit you would rather not have to reason
about.
#:seen-at says where a step sees the content, as it does for a
workspace or a secret. It names the content rather than the directory
holding it, so it has to have a directory above it, and that directory
is what the step is given to write in: a step holding two file outputs
under one directory is refused, as one told to see two resources at
one place is.
(define site (file-output #:destination "/srv/www/site" #:seen-at "/out/site"))
Everything the resource makes lives in one hidden directory beside the destination, which is where it has to be, since neither a rename nor a copy-on-write copy crosses file systems. Each commit reaps what is there but the generation it published, the one it replaced and its own staging directory, so a destination holds two generations and not a history. Two runs publishing to one destination at once is what a limiter is for, as two runs pushing one branch are.
As with a file input, a pipeline naming a place on the machine runs only on a machine that has it. Reach for a file output where a run has to leave something on the machine it ran on and a store item or a step’s output will not do.
(host-network) gives a step the host’s network. A step
that declares it reaches out as the pipeline process could; a step
that does not runs in a network namespace of its own, a private view
of the system’s network holding nothing but a loopback device. Such a
step can still talk to itself over 127.0.0.1, so a test that
starts a server and connects to it works, and it cannot open a
connection to anything else. Attaching returns nothing of use: the
declaration is what does the work, and a step need not attach the
resource to use the network it names.
So the steps that clone, fetch, publish or call an API declare it, and
the steps that build and test do not. A step that builds through the
guix-daemon resource needs no network of its own, because the
Guix daemon downloads on its behalf and runs outside the step.
(define work (workspace #:name 'work)) (define net (host-network)) (define fetched (step (name 'fetch) (inputs (list work net)) (modules '((automate helpers git))) (body #~(git (attach #$work) "fetch" "origin"))))
A step that reaches the network without declaring it fails where it connects, with whatever error its program gives for an unreachable address. The certificates needed to verify a TLS connection are among the variables every step is given, and the files they name are in the step’s file system, so a step that has the network can use it. Where a kernel gives an unprivileged process no namespaces at all, no step can be confined: the run fails, and --unconfined on a one-shot program or a daemon spawns every step the way they were spawned before Automate confined them.
(limiter #:max N #:scope SCOPE) is a set of N
slots. attach returns when a slot is free and holds it until
release or the end of the step. With #:max 1, the
default, it serialises the steps that hold it.
The scope says how far the limit reaches. process, the
default, is one pipeline process: a daemon serving the pipeline, or a
one-shot program. pumphouse is every pipeline attached to the
pumphouse that declares a limiter of the name #:name gives it,
which a pumphouse limiter must have, so it serialises across pipelines
and across machines. A pumphouse limiter needs the
pumphouse to be reachable: a step blocked on one errors after thirty
seconds without it.
A step blocked on a limiter has already started, and its process sits idle. Attach the limiter as late as the body allows and release it as soon as the limited work is done. Late means after the work that has nothing to do with what the limiter covers, not after reading the state the limited work is built on: a body that reads outside the slot and writes inside it has excluded nothing, since what it read may have changed by the time it writes:
(define publish-slot (limiter #:name 'publishing #:max 1 #:scope 'pumphouse)) (define published (step (name 'publish) (inputs (list (step-output built 'dist) publish-slot)) (body #~(let ((dist #$built:dist)) (prepare dist) (attach #$publish-slot) (upload dist) (release #$publish-slot) (tidy dist)))))
A limiter gives a step exclusion, which is not the same as order.
The steps holding it run one at a time, in the order they asked,
and that is the order their runs happened to reach the step rather
than the order the runs were started in: two runs of one pipeline
publishing two commits can publish them oldest last. Nor can a
limiter be made to order them, since a run cannot know whether a
newer run exists or will get as far as publishing. What a step can
do is hold the slot across both the reading and the writing and
stand aside when it finds itself overtaken, which leaves the final
state the newest run’s, whatever order they arrived in.
.automate/pipelines.scm in the source does this: its
publishing step names the branch the site is built from and pushes
only while that branch is still at the commit the run built, which
git-remote-tip in
(automate helpers git) answers.
Two steps that attach two limiters in opposite orders can deadlock each other. Nothing detects this; a step timeout is the only way out.
A step’s program carries a Guix, so that a body can compute
derivations and talk to the Guix daemon. That Guix is a library.
Its package collection is the one Guix’s own package pinned when the
pipeline was lowered. Nobody chose that collection, and it lags
behind guix pull, so it is never where a step’s packages
come from. Packages come from a guix-channels resource:
(define channels (guix-channels #:name 'channels))
By default that is the Guix that lowers the definition file, at the
channels and commits its guix pull profile was made from,
which is what guix describe prints on the machine running
guix build. Lowering copies nothing and builds nothing
for it: the definition refers to the Guix already there, and the
channels are recorded in the structure, so a run shows which
collection its steps built with. A pipeline that pins its own
collection passes #:channels, a list of channel records of
(guix channels), and gets a Guix built from them, which needs
the channels’ checkouts when the file is lowered.
Attaching returns the directory of that Guix. Its bin/guix
is the command, and the helpers in (automate helpers …)
take packages from it once current-guix of (automate
helpers guix) names it, which a body does at its start:
(define fetched (step (name 'fetch) (inputs (list work channels repository)) (modules '((automate helpers git))) (body #~(let ((source (attach #$work))) (current-guix (attach #$channels)) (git source "clone" "--depth=1" #$repository ".") (git-output source "rev-parse" "HEAD")))))
Without that, a helper that needs a package raises no-guix
rather than take one from the library: the git the helpers run, and
anything a body asks build-package for, come from the channels
or not at all. The Guix is opened as an inferior, a guix
repl of it that the step asks for packages, the first time a helper
needs it.
Taking a package from the channels means building it, so such a step
needs a guix-daemon input as well: the daemon’s socket is in a
step’s file system only where the step declares it. That is a great
deal of authority for a step whose need is one program. A body that
wants a program it can name takes it from the Guix that lowers the
definition instead, and holds neither resource:
(current-git #$(file-append git-minimal "/bin/git"))
The program is then the lowering Guix’s rather than the channels’, and it is in the definition’s closure, so the step builds nothing and talks to no Guix daemon. The channels are still what a step wants where the collection has to be the one the pipeline pinned, as the step building the development environment does.
A step that only runs a script in a profile needs no Guix at all when
the profile carries a bash: invoke-in-profile uses the
profile’s own. The profile is the whole of that script’s
PATH, since a step has none of the machine’s, so what the
profile lacks the script cannot run. A profile made from the
development inputs of a package on gnu-build-system carries the
standard tools; one on guile-build-system carries tar, gzip,
bzip2 and xz and no others, so a manifest for one adds coreutils,
grep, sed and whatever else configure and make
reach for.
(guix-daemon) is the Guix daemon’s socket, and with it the whole
store: a step holding it can build and add to the store, and read
back anything there. That is a lot of authority, comparable to
unconfined execution, so hold it in the steps that build and not in
the rest. Attaching returns the socket path, taken from
GUIX_DAEMON_SOCKET or Guix’s default, which
open-connection of (guix store) accepts.
What a step builds is only a path until something roots it, and the
Guix daemon’s garbage collector may take it between one step and the next.
(store-item DAEMON PATH) asks the resource to keep the item:
the resource makes a garbage collector root for it, and answers a
store item, an object the step provides as an output. Call it while
the connection that built the path is still open, so that nothing
can collect the item in between. A later step takes the item as an
input and attaches it, which returns the path. The root
lasts until every step that could use the item has finished.
The example below builds on the cloned step, the work
workspace and the channels resource defined earlier in this
chapter.
(define daemon (guix-daemon #:name 'daemon)) (define environment (step (name 'environment) (inputs (list cloned work daemon channels)) (modules '((automate helpers profiles))) (body #~(build-development-profile #$daemon #$channels (string-append (attach #$work) "/source/guix.scm"))))) (define built (step (name 'build) (inputs (list environment work)) (modules '((automate helpers profiles))) (body #~(let ((source (string-append (attach #$work) "/source"))) (invoke-in-profile (attach #$environment) source "make")))))
This is what guix shell -D -f guix.scm would assemble, done
inside a step: the Guix of the channels evaluates guix.scm, so
the packages it names are the ones the developers get, the profile is
built through the Guix daemon, kept as a store item, and attached by the
steps that build in it. A run whose steps declare no
guix-daemon has nothing to root with, and a store path passed
around in such a run is a string like any other.
A store item is recorded, and shown on the run’s page, as the list
(store-item PATH). Passed to another run as a parameter,
through trigger-run, that list is adopted by the receiving
run’s own guix-daemon resource: the path is rooted again and
the step gets an item to attach. A run without such a resource gets
the list as it is.
(gate NAME #:needed N) is a decision made from outside the run.
Attaching it waits until N distinct parties have approved it
with automate gate approve RUN-ID NAME, and then returns. If
someone rejects it instead, with automate gate reject,
attach raises and the step errors unless the body handles the
exception. automate gate list RUN-ID lists a run’s open
gates, each with how many parties it needs and who has approved so
far.
(define approval (gate 'approve-release #:needed 2)) (define released (step (name 'release) (inputs (list (step-output built 'dist) approval)) (timeout 86400) (body #~(begin (attach #$approval) (publish #$built:dist)))))
A step waiting on a gate holds its process open for as long as the decision takes. Give such a step a timeout, which bounds the wait. Gates need a pumphouse: the approval travels from the command line through the pumphouse to the run. Attaching the same gate from several steps shares one decision.
(pipeline-trigger #:pipeline PIPELINE) is a live reference
to another pipeline attached to the same pumphouse. A step attaches
it and calls trigger-run to start a run of that pipeline,
getting the new run’s id back. The child run records this run as
its cause, which its page shows.
(define deploy (pipeline-trigger #:name 'deploy #:pipeline 'site/deploy)) (define handed-over (step (name 'hand-over) (inputs (list (step-output built 'version) deploy)) (body #~(let ((version #$built:version)) (trigger-run (attach #$deploy) #:parameters `((version . ,version)))))))
A pipeline cannot trigger itself, directly or through a chain of triggers, and a chain has a maximum depth.
(param NAME) declares something a run is given when it is
started: a branch, a tag, a repository URL. A step lists the
parameter among its inputs and reads its value by splicing it into
the body with #$, as it does any other input.
A parameter’s value may be any plain data: a string, a number, a
symbol, a list. A run started by a webhook gets what the delivery
carried (see Forge Webhooks). A run started by trigger-run
gets what the triggering step passed.
A run has to be given every parameter that any step of the pipeline
takes as an input. This holds even for a parameter that only a step
the run will skip takes. A run lacking one is refused before any
step starts, with a missing-parameter error naming it. A
parameter has no default value, so a pipeline that can run without
a value takes one that says so, such as an empty string, and its
steps check for it.
On the command line, --param=NAME=DATUM reads the value as one
Scheme datum, and --param-string=NAME=STRING takes the text as
it is. Most values a pipeline is given are strings, such as a
repository URL or a ref, and those need --param-string. Read
as Scheme, ‘refs/heads/main’ is a symbol and ‘1.10’ is the
number 1.1, so a body expecting a string would raise, or would get the
wrong value.
$ automate run start project/trunk \
--param-string=ref=refs/heads/main --param=jobs=4 \
--param='targets=("x86_64-linux" "aarch64-linux")'
A value is compared by how it is written, so the string
"12" and the number 12 are two different values. This
matters when runs are listed by their parameters (see Run History): a run started with --param-string=number=12 is not
one of the runs a Forgejo webhook started for pull request 12, whose
number is the number 12.
Parameters cannot change the structure of a pipeline. The set of steps and the edges between them are fixed when the file is built; what a parameter can do is change what a step does with them.
Parameters are recorded on the run and shown on its page, so they carry no secrets. A token goes in a secret file (see Resources) and a parameter says which repository to use it for.
A step can list the runs of a pipeline that the pumphouse has recorded. This is how a run compares itself with earlier runs of the same thing. For example, a step can announce that the tests of a pull request have started failing or are fixed, and stay quiet when nothing changed.
The runs come from a run-history resource. A step attaches it
and passes it to list-runs:
(define history (run-history)) (define number (param 'number)) (define checked (step (name 'check) (outputs '(status)) (body #~(begin (run-tests) (provide 'status 'passed))))) (define verdict (step (name 'verdict) (inputs (list (step-output checked 'status #:required? #f))) (body #~(if (input-present? #$checked:status) 'passed 'failed)))) (define previous (step (name 'previous) (inputs (list verdict history number)) (body #~(match (list-runs (attach #$history) #:matching '(number) #:before (run-id) #:provided '(verdict) #:outputs '(verdict) #:limit 1) ((run) (listed-run-output run 'verdict)) (() #f)))))
previous answers the verdict of the newest earlier run of the
same pull request, or #f for the first one. A run is listed
only when it matches every keyword given. The common ones are these:
#:matching names parameters of this run, and keeps the runs
given the same values. #:parameters gives the values itself,
as an alist, which is how a pull request’s run finds the runs of the
branch it targets.
#:before and #:after take a run id and keep the runs
requested before or after it. (run-id) is this run’s.
#:provided keeps the runs in which each of the listed step
outputs was provided. A step’s name stands for its value
output, and a (STEP OUTPUT) list for another.
#:outcome keeps the finished runs that ended in one of the
listed outcomes, such as '(succeeded).
#:finished keeps the runs that have finished when it is
#t, and those still waiting or going when it is #f.
#:outputs says which recorded output values to return with each
run, and listed-run-output reads one of them.
#:order, #:direction and #:limit say which runs
come first and how many are returned. The default is newest request
first, with no limit.
Each run is an alist of its id, pipeline,
parameters, outcome, the times it was requested,
started and finished, and its outputs.
(run-history) lists the runs of the pipeline that declares
it. #:pipeline names another pipeline:
(define trunk-history (run-history #:pipeline 'project/trunk))
A run may list the runs of its own pipeline and of the pipelines its
run-history resources name, and no others. The pumphouse reads
the list from the definition, as it does for pipeline triggers.
Runs overlap. The newest earlier run may still be going, and its
steps may not have got as far as the one you want to compare with.
Use #:provided or #:outcome to keep only the runs that
have an answer. Without either, #:limit 1 can return a run
that has none.
A run can finish after a run that was requested later. This happens
when a pull request gains a commit while the previous commit is still
being checked. Before announcing a change, the older run can ask for
runs #:after (run-id) that already provided a verdict. If
there is one, the newer run has made the comparison and the older run
stays quiet.
Parameters match by how their values are written, so the string
"12" does not match the number 12 (see Parameters).
A run that lacks a parameter the query names is never listed.
#:matching raises when this run lacks the parameter.
Only an output whose value is plain data is recorded.
#:outputs returns nothing for an output whose value is a store
item. Put the value to compare in a small output of its own, as
verdict is above.
The runs are the pumphouse’s records. In a one-shot run without a
pumphouse, the step can still attach the resource, but
list-runs raises.
A step can abort other runs. The usual reason is that they are out of date. When a pull request gains a commit, the run checking the previous commit is wasted work, and its result will be replaced anyway. The newer run can abort it.
A step aborts a run through a run-control resource. It finds
the runs to abort through a run-history resource
(see Run History), then passes each id to abort-run:
(define history (run-history)) (define control (run-control)) (define number (param 'number)) (define supersede (step (name 'supersede) (inputs (list history control number)) (body #~(let ((control (attach #$control))) (for-each (lambda (run) (abort-run control (assq-ref run 'id) #:reason "superseded")) (list-runs (attach #$history) #:matching '(number) #:before (run-id) #:finished #f))))))
supersede aborts every unfinished run of the same pull request
that was requested before this one. #:before (run-id) matters.
Without it, two runs requested close together could each abort the
other.
Aborting a run kills its running steps and skips its pending ones.
The run finishes as errored. Its record shows the reason, and
which pipeline and run asked for the abort.
abort-run answers aborting, or finished when the
run had already finished. A run can finish between the step listing
it and the step aborting it, so a finished run is not an error.
(run-control) aborts runs of the pipeline that declares
it. #:pipeline names another pipeline:
(define deploy-control (run-control #:pipeline 'site/deploy))
A run may abort the runs of its own pipeline and of the pipelines its
run-control resources name, and no others. The pumphouse reads
the list from the definition, as it does for pipeline triggers.
abort-run raises a not-permitted error for a run of any
other pipeline.
A run may not abort itself, and abort-run raises a
not-permitted error if it tries. Aborting kills every running
step, including the step that asked. To end its own run, a step
fails.
The steps of an aborted run are killed with SIGKILL. They get
no chance to clean up, so a step that was part way through publishing
something leaves it part way. If a pipeline publishes, abort its runs
only from a step that runs before any publishing could start in them,
or keep publishing in a pipeline of its own that nothing aborts.
Aborting a run does not abort the runs it started. A run started by a step that calls a pipeline carries on, because another run may be waiting on it too.
Aborting goes through the pumphouse. In a one-shot run without a
pumphouse, the step can still attach the resource, but
abort-run raises.
A pipeline that runs on every push or pull request should say something when its result changes: the tests started failing, or they pass again. Saying something on every run is noise that people learn to ignore. (automate helpers notify) builds this from three parts, each a step or a call in one:
checks-verdict. Its value is recorded, so later runs can read
it.
verdict-change, which reads the earlier runs through a
run-history resource (see Run History). The answer is a
transition: first, broken, fixed,
still-passing, still-failing, changed-failure or
superseded.
send-email, post-json for a chat service’s web hook, or
a comment on the pull request.
(define history (run-history)) (define net (host-network)) (define token (secret 'forge-token)) (define repository (param 'repository)) (define number (param 'number)) (define built (step (name 'build) (body #~(begin (run-build) 'built)))) (define checked (step (name 'check) (inputs (list built)) (body #~(begin (run-tests) 'ok)))) (define verdict (step (name 'verdict) (inputs (list (step-output built 'value #:required? #f) (step-output checked 'value #:required? #f))) (modules '((automate helpers notify))) (body #~(checks-verdict `((build . ,(input-present? #$built)) (check . ,(input-present? #$checked))))))) (define notified (step (name 'notify) (inputs (list verdict history net token repository number)) (error-permitted? #t) (modules '((automate helpers notify) (automate helpers secrets) (automate forge forgejo))) (body #~(let ((transition (verdict-change (attach #$history) #$verdict #:matching '(repository number))) (token (read-secret (attach #$token)))) (unless (eq? transition 'superseded) (forgejo-sticky-comment #$repository token #$number (format #f "Checks ~a~a" (verdict-state #$verdict) (if (run-url) (string-append ": " (run-url)) "")) #:marker "checks")) (when (memq transition '(broken fixed changed-failure)) (forgejo-comment #$repository token #$number (format #f "The checks are ~a." transition))) transition))))
The sticky comment is one comment on the pull request, edited by each run, so the pull request always says where it stands. Editing a comment notifies nobody. The second comment is new, so the forge tells everyone watching, and it is made only when the result changed.
A pipeline for a branch compares the runs of that branch, with
#:matching '(repository ref), and sends mail or posts to chat
instead of commenting. send-email runs msmtp from the
Guix a guix-channels resource attaches, so the step sets
current-guix first (see Resources). msmtp’s settings,
including the server’s password, go in a secret file, which msmtp
refuses unless only its owner can read it.
(define channels (guix-channels)) (define mail-settings (secret 'mail-settings)) (define ref (param 'ref)) (define mailed (step (name 'mail) (inputs (list verdict history net channels mail-settings repository ref)) (error-permitted? #t) (modules '((automate helpers notify))) (body #~(let ((transition (verdict-change (attach #$history) #$verdict #:matching '(repository ref)))) (current-guix (attach #$channels)) (when (memq transition '(broken fixed changed-failure)) (send-email #:to "team@example.org" #:subject (format #f "trunk is ~a" transition) #:body (format #f "~a~%" (run-url)) #:configuration (attach #$mail-settings))) transition))))
The verdict step and the notify step must run when the checks fail. Give them the checks as inputs that are not required (see Outputs and Errors); a required input that is never provided skips the step, and nobody hears about the failure.
Mark the notify step error-permitted?. A mail server that is
down should not turn a passing run into a failed one.
A run that is aborted, or whose pipeline process dies, never reaches its notify step and records no verdict. The next run compares itself with the last run that did, so a change is reported late, not lost.
superseded means a run requested later has already recorded its
verdict, which happens when a pull request gains a commit while the
previous commit is still being checked. The later run does the
comparing, so a run that is superseded tells nobody, and does not
update the sticky comment with an older result.
The notify step needs the network, so it lists a host-network
resource.
(pipeline NAME #:steps STEPS) gathers steps into a pipeline
named by a symbol. A name with slashes, such as
site/deploy, is grouped by its prefix on the front page of
the web interface. collect-steps walks the inputs of the
steps it is given and returns every step reachable from them, so a
pipeline is usually declared from its final steps:
(pipeline 'site/deploy #:steps (collect-steps rolled-back notified))
The pipeline is checked as it is made: no two steps with one name, no cycle, every input selecting an output its source has, every step input among the steps given. A problem is reported with the step and the name concerned.
A step’s outputs belong to the pipeline’s inside. Which step
provided a value, and what it called it, is a detail the author
changes freely. #:outputs names what the pipeline as a whole
produced, so that whoever reads a finished run need not know the
steps at all:
(pipeline 'site/deploy #:steps (collect-steps notified) #:outputs `((url . ,(step-output deployed 'url)) (report . ,report)))
An output selects a step’s output the way an input does, so a step
given plainly stands for its value output. A one-shot
program prints these when the run ends, and a pumphouse records them
against the run.
Name the few values someone outside the run needs and no more. The
names are what the pipeline is read by, so keep them still while the
steps beneath them move: a step renamed, split in two or replaced
leaves the outputs as they were. error is not a name an
output may take, since it is what a step’s own failure is called
(see Outputs and Errors).
An output whose step never provided it is absent, not false. A step
that was skipped provides nothing, so a pipeline whose deployment
only some runs reach answers without its url on the runs that
stopped short. Read the result with that in mind: a missing name
means the run did not get that far, which is not the same as a step
that ran and produced #f.
A step usually runs a body. It can instead run another pipeline, and take what that pipeline answers with as its own outputs:
(define package (param 'package)) (define plan (step (name 'plan) (inputs (list package)) (body #~(provide 'value `((package . ,#$package)))))) (define built (step (name 'built) (inputs (list plan)) (outputs '(derivation)) (calls (call-pipeline 'guix/build-package #:parameters plan))))
Such a step is no process. The pipeline process starts the run, waits
for it, and ends the step when it ends: succeeded where that run
succeeded, and errored otherwise, with an error output naming the run
and what became of it. A step calling a pipeline does not count against
--cap, which bounds the step processes on this machine and not
what other machines are doing.
The step provides each output it declares, taken from the output of that
name the called pipeline answers with, as its #:outputs names
them. It provides
run besides, the id of the run it started, and provides it as
soon as the run starts rather than when it ends. A step waiting only on
run therefore runs while the called run is still going, which is
how a build is reported as it begins.
#:parameters says what the run is given. A call computes
nothing: it takes either an alist fixed when the pipeline is built, or a
step whose value is such an alist, worked out by an ordinary body. Put
the computing in a body upstream, as plan does above, and name
that step. It has to be among the calling step’s inputs, so that the
call waits for it.
The pipeline called is named, not built in. What runs is whatever is attached under that name when the call is made, which is what lets the two pipelines be built, reviewed and deployed apart, and lets the called one run on another machine. The cost is that the calling pipeline’s definition no longer covers everything that ran: the run page links the two runs, and that link is what says which definition answered.
Calling needs a pumphouse. A pipeline’s name means nothing without the registry that holds it, so a one-shot program run with no pumphouse errors such a step rather than guessing.
A pipeline may start the pipelines it declares and no others. The
pumphouse works that out from the structure a pipeline process
registers: every step that calls a pipeline, and every
pipeline-trigger resource, names a target, and those are what a
run of it may start. There is nothing to configure, and a pipeline
cannot be given a reach its own definition does not show. A run asking
for anything else is refused.
Nothing else can check a call. The called pipeline is named rather than built in, so it is not in the caller’s closure and the build cannot see it. The pumphouse holds both structures, so the pipeline’s page says what is unsettled: a called pipeline it does not know, and an output a step takes that the called pipeline does not answer with. It says so and refuses nothing. A called pipeline that has not registered yet is the ordinary state of affairs while a system is being brought up, and the caller is right either way once it has.
A pipeline’s steps are fixed when it is built. There is no making a
step per package that changed, per pull request open, or per test to
bisect, because none of those is known then. A run can make a run
apiece instead: #:each takes a step whose value is a list of
parameter alists and calls the pipeline once for each of them.
(define packages (param 'packages)) (define plans (step (name 'plans) (inputs (list packages)) (body #~(provide 'value (map (lambda (package) `((package . ,package))) (string-split #$packages #\,)))))) (define built (step (name 'built) (inputs (list plans)) (calls (call-pipeline 'guix/build-package #:each plans #:width 8))))
Such a step declares no outputs. It provides runs, the ids of
the runs it started in the order of the list, and results, one
entry per element in that same order:
((run . "a17c93ff21") (outcome . succeeded) (outputs . ((derivation . "/gnu/store/..."))))
#:width bounds how many of those runs are going at once, the next
starting as one ends. Without it they all start together, which is
fine for a handful and unkind to a pumphouse for five hundred.
The step errors where its runs did not all succeed. Its results
still say what each run did, so the ones that worked are not lost with
the one that did not: a step downstream can read them and report, and
error-permitted? on the calling step is what decides whether the
run it is part of fails. An empty list starts no run at all, and the
step succeeds with empty runs and results, which is worth
knowing when the list comes from a search that found nothing.
The file’s last expression says how the pipeline runs.
(one-shot PIPELINE) makes a program that runs it once, with
its parameters on the command line, as Getting Started shows.
(run-daemon (list PIPELINE …)) makes a program that
attaches the pipelines to a pumphouse and serves runs of them until
stopped:
(run-daemon (list website pull-request))
$ $(guix build -f pipelines.scm) --pumphouse=/var/lib/automate/address \
--secrets=/etc/automate/secrets
The daemon takes --pumphouse=ADDRESS, or reads
AUTOMATE_PUMPHOUSE, and the same --secrets,
--unconfined, --pass-environment
and --expose as a one-shot program. --output=DIR is
where each step’s log is written before it is uploaded to the
pumphouse; a temporary directory by default. --cap=N bounds
the number of step processes running at once, derived from the CPU
count by default, which is an operator’s safety valve rather than a
limiter. Runs are started at the pumphouse: by the command line, a
timer, another pipeline or a webhook.
The steps are the same programs either way. Building the file with
guix build gives a store path, and that path identifies the
definition: the structure, every step’s program and everything they
depend on. A pumphouse numbers the definitions it sees of each
pipeline, and a daemon started from a rebuilt file registers a new
one, which applies to runs started after it. Runs under way keep
the definition they started with. Since Automate’s own modules are
copied into each program, upgrading Automate means rebuilding the
pipelines.
A pipeline is itself something Guix can lower: #$PIPELINE in
a G-expression is the definition’s directory, holding the structure
and every step’s program. A program of the author’s own can be
built around a pipeline that way, as one-shot and
run-daemon are.
A pipeline’s code is not read from the repository the pipeline builds. The file is built into a definition beforehand, and a daemon registers what was built. A run is then given the repository, the reference and the commit as parameters, and its steps fetch that commit themselves (see Forge Webhooks). The file may live anywhere: in the repository it builds, in another repository, or beside the pumphouse’s configuration. Automate keeps its own under .automate/, beside the code they check, but nothing reads them from there. Three things follow.
A commit cannot change the steps that run on it. The structure and each step’s inputs are fixed in the definition the run started with. So are the resources each step holds, and the secrets it can attach. A pull request that edits the pipeline file edits a file in the checkout, and a step that reads that file reads it as text, like any other file there.
What a commit does change is what those steps do. A step that runs
make check over the checkout runs the code in the checkout.
So the definition bounds what a run can reach, and the commit decides
what happens within those bounds. A pipeline checking pull requests
holds the author’s secrets and runs a contributor’s code, so divide
the steps along that line. Automate’s own pipelines declare the
forge token and the network in the steps that fetch, publish and
report, and the steps that build and test the checkout hold neither.
Changing a pipeline takes a rebuild. A commit to the pipeline file changes nothing by itself. Build the file again and start the daemon from what was built. The new definition applies to runs started after it, and runs under way keep the one they started with.
A pipeline that takes its repository as a parameter runs against any clone of it. That is how one is tried before a forge is told about it, as Real Pipelines shows.
#:web-ui-options is an alist the web interface reads and
nothing else interprets. It is part of the definition. The one key
so far is front-page: with latest-run, the front page
draws the pipeline with the latest run’s statuses on it, which suits
a pipeline whose state is its latest run, such as one that publishes
a site. By default the drawing is plain and the recent runs are
listed beneath it, which suits a pipeline whose runs are each about
something of their own, such as pull requests.
(pipeline 'site/publish #:steps (collect-steps published) #:web-ui-options '((front-page . latest-run)))
Every step records where in the source its (step …) form
is, and so does the pipeline, and the web interface shows the file and
line beside each step of a run. #:source makes those into
links. It takes a procedure answering the URL where a piece of the
pipeline’s code is published, and (automate) has the
constructors that make one for a forge:
(pipeline 'site/publish #:steps (collect-steps published) #:source (forgejo-source "https://forge.example/owner/site"))
The links are worked out when the pipeline is lowered, not when a page is drawn, so nothing at the pumphouse has to be told where the code lives, and a run from months ago still links to the code that ran. Lowering asks git about each file: a form is linked at the commit the file was at, or on the branch when the file had changes that were not in any commit, since the commit does not hold the code that ran.
forgejo-source serves Forgejo and Gitea. For any other forge,
template-source fills a URL with the position:
(template-source "https://git.example/site.git/tree/{file}?id={commit}#n{line}")
Both link only forms in the pipeline’s own repository, so a step taken
from a library elsewhere is shown without a link rather than linked to
a file that is not there. A procedure of your own may do as it likes
with the two locations it is given, the form’s and the pipeline’s; it
answers a URL string, or #f for no link.
A pipeline without #:source still shows where each step is
defined, without linking. There is one cost to weigh: a location
carries the commit, so any commit touching the file the forms are in
changes the definition’s store path, and the pumphouse numbers a new
definition where it would once have recognised the old one.
A pumphouse holds pipelines, their definitions and every run, keeps
the resources shared between pipelines, and serves the web interface.
It runs no step itself. It is started by a script that ends by
calling run-pumphouse from
(automate pumphouse process):
(use-modules (automate pumphouse process)) (run-pumphouse #:state-directory "/var/lib/automate" #:http (http-configuration (port 8080)))
$ guile pumphouse.scm pumphouse address: /var/lib/automate/address
The state directory holds the databases, one for the pumphouse and
one per pipeline, and the address file that everything else
starts from: the daemons that attach pipelines, and the
automate command. Both take --pumphouse=FILE or
read AUTOMATE_PUMPHOUSE. A one-shot program given the same
records its definition and run there too, with one-shot as
the run’s cause, without disturbing a daemon serving the same
pipeline. It uploads its steps’ logs as a daemon would, waiting up
to thirty seconds after the run for the upload to finish, and warns
when something was not sent.
Everything about a pipeline’s runs is in that pipeline’s database, under pipelines/ in the state directory. So one pipeline’s history can be copied, dropped or trimmed on its own, with the pumphouse stopped, by dealing with the one file. The pumphouse’s own database holds what crosses pipelines: which pipeline each run id belongs to, the links between runs, and the registrations.
Configuration is code. The script can compute what it passes, read
a file, or check the environment, before it calls
run-pumphouse, which runs on the calling thread until the
process is told to stop.
#:http takes an http-configuration. Its port is
where the web interface, step log uploads and webhooks are served;
zero means any free port, and the port bound is written to
http-port in the state directory. Without #:http there
is no web interface and step logs are not kept.
The server listens on every address of both families by default.
address binds it to one IPv4 or IPv6 address instead, and
public-url names the server as daemons on other machines reach
it. Both are for a pumphouse behind a proxy that terminates TLS: the
server is bound to loopback and named by the proxy’s URL. The server
speaks plain HTTP itself, and closes a connection that has been idle
between requests for a minute.
The server can face the internet. An upload of a step’s log is refused before its body is read when it has no content length, when the body is over 256 kibibytes, or when the token it bears is not the one the run was given. So whoever can reach the port cannot make the pumphouse hold bytes. What the web interface shows to everyone is described in The Web Interface.
metrics? adds /metrics to the server, where Prometheus,
a time series database that collects metrics over HTTP, reads them.
(http-configuration (port 8080) (metrics? #t))
The pumphouse counts runs as it records them, so reading the metrics queries nothing. Runs requested, started and finished are counted by pipeline and by outcome, with how long each run waited before starting and how long it then took, and how many are requested and not yet finished. Each pipeline also carries when its last run of each outcome finished, which state its daemon’s attachment is in, and when that daemon was last heard from. The server’s own requests are counted by route, method and status code, and the process reports its memory, processor time and garbage collection.
The series worth an alert first is
automate_run_last_finish_timestamp_seconds. A pipeline that
has quietly stopped running shows nothing on the pages, since there is
no failed run to see, and an alert on how long ago that pipeline last
succeeded catches it, whether the cause is a paused timer, a daemon
that never came back or a webhook that stopped arriving.
The metrics are served to whoever reaches the port, as the pages are. They name pipelines, outcomes and timings, which the pages show already, so a pumphouse whose pages are public can serve them on the same port; one behind a proxy keeps both behind it.
#:tcp takes a (HOST . PORT) pair, and the pumphouse then
also listens for daemons and the command line over TCP with TLS, the
transport layer security protocol, under a key and certificate it
keeps in the state directory. The address file gains
remote-registrar and remote-admin, sturdyrefs that name
the pumphouse by its certificate. A daemon on another machine is
given one such sturdyref as its --pumphouse, and it dials out;
nothing listens on the daemon’s host. A sturdyref carries the port,
so a pumphouse reached from other machines is given a fixed port
rather than zero, or every sturdyref handed out stops working at
its next restart.
Whoever holds a sturdyref holds what it names. The registrar and
admin sturdyrefs are for the operator. For a daemon that should
attach some pipelines and no others, automate registration
grant PATTERN... mints a registration, a sturdyref that accepts
those pipelines alone. Each pattern is a pipeline name, accepted
exactly, or a prefix ending in a slash, such as automate/,
accepting every pipeline under it; a slash alone is refused, since
what covers every pipeline is the registrar. The patterns are the
registration’s identity: asking for the same ones in any order
answers the same sturdyref, automate registration revoke
PATTERN... with the same ones withdraws it, and automate
registration list lists them.
Registrations survive a restart of the pumphouse. A daemon that
serves pipelines outside its registration has those refused, which it
prints once per connection, and serves the rest; a daemon whose every
pipeline is refused tries again later.
A timer starts a run of one pipeline at a fixed interval. Timers are
part of the script, as a list of timer-configuration records
under #:timers, so they are there again after every restart
and are read alongside the rest of the configuration:
(run-pumphouse #:state-directory "/var/lib/automate" #:timers (list (timer-configuration (pipeline 'website) (every 3600)) (timer-configuration (name 'website-full) (pipeline 'website) (every 86400) (parameters '((full . #t))))))
every is in seconds, and the first run starts that long after
the pumphouse does. A tick while no daemon is attached under the
pipeline’s name starts nothing, so a timer can be configured before
the pipeline it names has ever registered. A timer is known by its
name, which defaults to the pipeline’s, so two timers of one
pipeline need names of their own. Each run a timer starts records
the timer as its cause.
automate timer list lists the timers with their state.
automate timer pause NAME stops one starting runs and
automate timer resume NAME lets it again; the pause is held in
the pumphouse’s memory, so a restart resumes every timer.
Every run starts from a run request carrying an id. A request repeated with the same id, from anything that starts runs, answers the run already started, or already finished, rather than starting another. This is what makes a webhook delivered twice one run (see Forge Webhooks).
A run records what caused it. A timer’s runs have the cause
timer, a forge’s webhook, a one-shot program’s
one-shot, and a run started by trigger-run names the
pipeline and run that started it. A run started with
automate run start has no cause. automate run show
prints the cause, and the pumphouse keeps the links from one run to
another in both directions.
A daemon sends a heartbeat every ten seconds, or as
--heartbeat says. One unanswered for three intervals means
the daemon has lost the pumphouse. On the pumphouse’s side a
pipeline is attached when its daemon registers, live
once a heartbeat has arrived, stale after #:stale-after
seconds without one, thirty by default, lost after
#:lost-after seconds without one, ten minutes by default, and
detached once the daemon has gone.
A daemon that has lost the pumphouse keeps running its runs and
spools what it has to report, up to ten thousand messages. When the
spool fills, every run the daemon has under way is aborted with the
reason outbox full. When the pumphouse is back, the daemon
attaches again and replays the spool. Dialling it is given limits of
its own, since a link that dies in the middle of an attempt may never
answer and never fail either: a pumphouse silent for fifteen seconds
about a pipeline being registered ends the attempt, as does an attempt
that has neither attached nor failed within thirty seconds, and another
is made two seconds later. An address file rewritten under an attempt
sends the daemon to the new pumphouse at once. Pumphouse limiters,
gates and triggers need the pumphouse, so a step that needs one of
those errors while it is away.
The pumphouse keeps the same distance from a daemon. Asking a pipeline
process for a run, aborting one and unlocking or rejecting a gate each
fail after fifteen seconds of silence rather than waiting on a process
that may never answer, so automate run start comes back, a
webhook delivery is answered and a timer’s tick is skipped. A run asked for
that way is recorded before the process is asked, so one the process
starts late is still the same run.
A run does not survive its daemon. When the daemon’s process dies,
or is restarted with a new definition, the runs it had under way are
not resumed. They are closed the next time a daemon registers the
pipeline. A daemon registers with the runs it is still reporting on,
and the pumphouse finishes every other unfinished run of the pipeline
as abandoned. The run’s running steps are abandoned, its
pending steps skipped, and its last event is lost, which
automate run show prints.
A daemon that is simply gone is caught by its silence. Once its
pipeline is lost, the pumphouse abandons every unfinished run
of it the same way. So the records show a dead daemon’s runs under
way for #:lost-after seconds at most. A one-shot program’s
runs are never closed either way, since no daemon reports on them. Restart a daemon when automate run list
shows nothing under way, or accept that those runs are abandoned.
A pipeline that is no longer wanted, renamed or set up by mistake,
stays on the pumphouse’s pages until automate pipeline forget
NAME removes it with every run of it. The pumphouse refuses while a
daemon is attached under the name, so stop the daemon, or take the
pipeline out of its definitions, first.
automate talks to a pumphouse. A command is a noun and a
verb: the noun names what the command acts on, and automate
NOUN prints the commands of that noun. Every command takes
--pumphouse=ADDRESS, an address file or an admin sturdyref,
defaulting to AUTOMATE_PUMPHOUSE; --help on any command
prints its options.
automate pipeline listThe pipelines the pumphouse knows: each one’s attachment, its current
definition number, how many runs it has and the ids of the runs still
under way. It names no run that has finished, so its size does not
grow with the history; automate run list NAME is where the rest
are.
automate pipeline forget NAMERemove a pipeline and every run of it from the pumphouse, with their output and their links to other runs. Refused while a daemon is attached under the name.
automate definition list NAMEThe definitions the pumphouse has seen of a pipeline, numbered in the order they were seen, each with its store path.
automate run start NAME [--param=NAME=DATUM]… [--param-string=NAME=STRING]… [--wait]Start a run and print its id. --param reads a parameter’s
value as Scheme and --param-string takes it as a string
(see Parameters). With --wait, follow it to its
outcome, print the status of each step, and exit non-zero when the
run errored.
automate run list NAME [--limit=N] [--param=NAME=DATUM]… [--param-string=NAME=STRING]…The newest runs of a pipeline, summarised: id, definition, parameters,
outcome and timestamps. --limit says how many, twenty by
default. --param and --param-string keep the runs
given those values, read as they are for run start, so
--param=number=12 lists the runs a Forgejo webhook started for
pull request 12.
automate run show RUN-IDA run’s record: its parameters, cause, outcome and the status of each step.
automate run output RUN-ID STEP [--from=N] [--follow]A step’s log, as the pumphouse holds it, from byte N on.
With --follow the command goes on printing what arrives until
the run is over; it asks the pumphouse again every second, since the
admin object has no way to be told.
automate run abort RUN-ID [--reason=TEXT]End a run: its pending steps are skipped and the run is errored.
automate gate list RUN-IDThe gates a run has open, each with its state, the number of parties it
needs and who has approved so far. A gate whose pipeline process does
not answer is listed with an unreachable state rather than
failing the whole command.
automate gate approve RUN-ID GATE [--as=WHO]automate gate reject RUN-ID GATE [--as=WHO] [--reason=TEXT]Advance or refuse a gate. --as names the party, defaulting
to the user’s name, and a gate needing several parties counts
distinct names.
automate repl request NAME STEP [--at=start|end] [--as=WHO]Ask for a step of a pipeline a daemon serves to be stopped, and drive the session it offers (see Attaching to a Step). It names a pipeline rather than a run because the request is armed in advance, and the next run to reach that step takes it.
automate timer listautomate timer pause NAMEautomate timer resume NAMEThe pumphouse’s timers, and stopping and restarting one (see Running a Pumphouse).
automate registration grant PATTERN…automate registration listautomate registration revoke PATTERN…Registrations for daemons on other machines (see Running a Pumphouse).
The answers are printed as Scheme data, so they can be read back by
a script. automate run output, which prints a step’s log as
it was written, is the one exception.
automate --version says which Automate this is and where its
manual is.
An error is one line on the error port, and the command exits 64 when
the command line itself was not usable and 1 otherwise. Setting
AUTOMATE_BACKTRACE in the environment leaves the exception to
Guile instead, which prints the backtrace a report of a fault in
Automate wants. An error raised inside the pumphouse reaches the
command line without its message: Goblins deliberately carries nothing
of an exception across a connection, so the command can only say that
there was one.
A step can be stopped and held while someone looks at it. Nothing enters the step: the run is asked in advance to stop a step by name, and the step, which already holds a link back to the pipeline process, dials out when it reaches the point named and waits there until it is let go. So there is no moment to catch and nothing to keep alive afterwards. A step nobody asked about runs exactly as it always does.
A one-shot program takes --repl=STEP. The program is the
pipeline process itself, so it stops a step of the run it starts:
$ $(guix build -f hello.scm) --repl=write --param-string=name=World waiting to attach to write
For a pipeline a daemon serves, automate repl request asks
the pumphouse. The pipeline process dialled out to the pumphouse and
cannot be reached from outside, so the pumphouse relays the request to
it and relays the session back:
$ automate repl request website build --as=ada waiting to attach to build of website
Either command blocks. The request is armed first, and the next run to reach that step takes it; a step already running cannot be told to stop. The request is used once, and a second request for a step already asked about is refused. When the step runs to its end without stopping, or the run ends without the step starting, the command says so and exits.
--as=WHO says who is attaching, as it does for a gate
(see Resources), and defaults to the user’s name.
--at=start, the default, stops the step before its body runs.
--at=end stops it once the body has finished, however it
finished. A one-shot program takes --at beside --repl
in the same way.
--at=end is where a step that failed is looked at. The
session says how the body ended, and hands back the error record with
its backtrace. The workspace, the secrets and everything else the
step holds are as the body left them, since the step has not
finished. Its outputs are settled by then, so a session reads what
the step provided and cannot rewrite it.
When the step stops, the command prints where it stopped, the module the body runs in, the inputs the step was given, and the path of a Unix socket:
attached to write at start module: (automate step-body write) input: work value input: name value nrepl: /run/user/1000/automate-repl-I5tQTF/nrepl.sock connect a client to that socket; write is let go when it disconnects
That socket speaks nREPL, the protocol
Arei and other editors’
Scheme clients connect over. What is evaluated there is evaluated in
the body’s module, with everything the body had: (input 'name)
answers what the step was given, and attach reaches the
resources it holds. Beyond the operations every nREPL client uses, a
session answers automate/inputs and automate/outputs,
saying what the step was given and what it owes. Disconnecting lets
the step go, and the run carries on.
The socket is on the machine the command was typed on, and it is the only socket in any of this. Nothing listens inside the step: what the client sends travels to the step as data, over the link the step already holds.
A session holds everything the step holds, secrets included, so
masking (see Resources) does not survive one. Asking for a step
is an authority over the pipeline, and a pumphouse grants it to
whoever it grants automate run abort and automate gate
approve to. Every operation that runs in a session is recorded as
an event of the run, along with who was attached, so what was done in
one is in the run’s record and not only in the memory of whoever was
there.
A stopped step keeps its place under the process cap and whatever its resources gave it, and other runs wait behind it: the step has not finished with what it holds. Its timeout does not count the time it spends stopped. A run’s page marks a stopped step and says who has it. The web interface stays read-only, and there is no session in a browser.
--repl stops a step of the run the program itself is running,
so a one-shot program given a pumphouse address refuses it.
A pumphouse with an HTTP server serves a read-only web interface at
its root. The front page lists the pipelines, grouped by the part
of the name before its last slash: libs/publish-library is
shown as publish-library under the heading libs/, and
names without a slash come first, under no heading. Each pipeline
is drawn there as its steps alone, left to right, with its
attachment state and active runs. Beneath the drawing are its five
most recent runs, newest first, or, when its web-ui-options
say so, the latest run alone with its statuses on the drawing
(see Pipelines). A run is told by its id, its state and when it
came to be so, with the parameters it was given beneath, a name and a
value to a line.
A pipeline’s page, at /pipelines/NAME, draws the steps of
its current definition and lists its runs. Each parameter value on
the pages links to the runs of the pipeline given that value, which
is how to see every run of one pull request. A value longer than
512 characters, such as a webhook’s payload, is folded away
beneath its length instead, and not linked. The link’s query holds
param=NAME=DATUM, the value written as Scheme, and may repeat
to give several; a string is written with its quotes, so
param=ref="refs/heads/main". The resources and
parameters the steps draw on are in the definition’s structure, which
the pages leave out of the drawing. A step that calls another
pipeline is drawn as one: its node carries a second frame inside the
first and says which pipeline it calls, beneath the step’s own name. The called pipeline is never drawn inside the
caller’s diagram, its structure not being known there and differing
from one run to the next. A pipeline’s page says which pipelines it
calls and which call it.
A run’s page, at /runs/RUN-ID, draws the run over its
definition, each step marked by status, and lists its events. Where
any of its steps calls a pipeline, the steps gain a column naming the
runs each started, linked. It
says what the run answered with, where its pipeline declares outputs
(see Pipelines). It also says where the run came from and what
came of it. The run that caused this one is named beside its
pipeline and linked. The runs this one caused are listed under
theirs, grouped by pipeline and counted, since one run may cause
many. A run started by a webhook delivery, a timer or a one-shot
program was caused by no run, and its cause is shown as it was
recorded instead. A run that caused none says nothing at all, which
is most of them. A step’s page, at
/runs/RUN-ID/steps/STEP, shows its log.
Pages update while a run is under way.
?orientation=lr on a pipeline or run page draws the diagram
left to right rather than top to bottom. A status is never shown by
colour alone, for readers who cannot tell red from green. Each has
a line style, solid for what went to plan, dashed for what did not
or has not yet, and dotted for groups, resources and parameters, a
mark on the node’s corner, and its word, in a title on the node for
assistive technology and beside the mark in the tables. A running
step’s line marches, unless the reader’s browser asks for reduced
motion. The pages follow the system’s choice of a light or dark
scheme unless the reader picks one with the button in the header,
which is remembered in the browser’s local storage.
Nothing on the interface starts, aborts, approves or rejects anything. It is meant to face the public, and everything recorded is visible to whoever can reach the port: parameters, output values, error messages, who approved a gate, and everything a step printed. What the pipeline records is the boundary, so a step that prints a secret has published it.
The pages update from server-sent events, a one-way stream of
messages from a server that a browser reads with
EventSource. Anything else that speaks HTTP can read the
same streams:
/events/runs/RUN-IDThe run’s record as a state event, then each event of the run
as it happens, its id being the event’s position within the run,
then finished, after which the stream ends.
/events/output/RUN-ID/STEPThe step’s log held so far, then each chunk as it arrives, the id being
the number of bytes held after it. ?from=N starts at a byte
offset.
/events/pumphouseAn activity event for each run requested, started or
finished, for each event of a run under way, and for each pipeline
whose attachment changes state. An attachment is published when its
state changes and not on every heartbeat.
Payloads are JSON. A client that reconnects sends the last id it
saw as the Last-Event-ID header, and the stream resumes from
there, so nothing is missed across a dropped connection.
A forge starts runs by sending webhook deliveries to the pumphouse.
The HTTP server accepts Forgejo’s and Gitea’s at /webhook when
the script gives it a webhook-configuration: the file holding
the secret the forge signs deliveries with, rules saying which
pipeline each kind of delivery starts, and optionally the path the
forge posts to in place of /webhook. The rules are made by
push-rule and pull-request-rule of
(automate forge forgejo).
(use-modules (automate pumphouse process) (automate forge forgejo)) (run-pumphouse #:state-directory "/var/lib/automate" #:http (http-configuration (port 8080) (webhooks (webhook-configuration (secret-file "/etc/automate/webhook-secret") (rules (list (push-rule (repository "cbaines/automate") (branch "trunk") (pipeline 'automate/trunk)) (pull-request-rule (repository "cbaines/automate") (pipeline 'automate/pull-request))))))))
On the forge, the webhook’s target is the server’s URL with
/webhook appended, its content type is JSON, and its secret is
what the file holds. A delivery whose signature does not match is
refused.
A push rule starts its pipeline for pushes to the named branch, or
to every branch when it names none. The run’s parameters
are repository, the clone URL, ref, the full reference
pushed, commit, each a string, and payload, the
delivery the forge sent, as a string of JSON. A pull request rule
starts its pipeline when a pull request is opened, reopened or gains
new commits, with the same four parameters plus number, the
pull request’s number as a number. Its ref is
refs/pull/N/head, which the forge serves for fetching. A
pipeline started this way declares some or all of those parameters
and no others: a run lacking a parameter the pipeline declares is
refused.
The payload holds everything else the forge says about the push or
the pull request, such as who opened a pull request, its address on
the forge and the branch it would merge into, so a pipeline decides
for itself what to read. A step declaring it decodes it with
guile-json and reads it with payload-ref:
(let ((payload (json-string->scm #$payload))) (values (payload-ref payload "pull_request" "user" "login") (payload-ref payload "pull_request" "html_url") (payload-ref payload "pull_request" "base" "ref")))
It is the payload of the delivery that started the run. A later delivery for the same commit, after a label is added or the title is edited say, maps to that run and leaves it as it was.
The payload is a parameter like any other, so whoever can read the web interface can read it, folded away beneath its length on a run’s page. For a private repository, that is its commit messages, the names of the files a push changed, and a pull request’s title and description. Most of its text was written by whoever pushed or opened the pull request, so a step treats it as data: a title passed to a shell as part of a command line runs as a command.
Every delivery from a forge arrives at the same path, whichever
repository it is from, so the repository of a rule is what
keeps the pipelines of several repositories apart. It names the
repository as owner/name, and the rule then applies only to
deliveries from that repository. A rule without one applies to
every repository the forge delivers for. Take a pumphouse serving
two repositories whose push rules both name trunk and no
repository. A push to either starts both pipelines, each with the
other’s clone URL as its repository parameter.
A pull request rule without criteria runs the code of anyone who can open a pull request. That code runs on the pumphouse’s machines, with the resources the pipeline gives its steps, such as the network or a token for the forge. On a forge open to the public, a rule should run only the pull requests someone trusted has looked at.
A rule’s criteria is a procedure deciding this. It is called
with the delivery’s payload, the JSON the forge sent, and the rule
starts its pipeline only when it returns true. The usual choice is
to trust people who can push to the repository, and to have one of
them add a label to any other pull request after reading it:
(pull-request-rule (repository "cbaines/automate") (criteria (lambda (payload) (or (not (from-fork? payload)) (label-present? payload "run-pipeline")))) (pipeline 'automate/pull-request))
from-fork? is true for a pull request from a repository other
than the one it is made to. A pull request that is not from a fork
comes from a branch of the repository itself, which only people who
can push to it can create. On Forgejo only people with write access
can add a label. The API chapter lists the other
helpers, author-among?, targets-branch? and
draft?, and payload-ref, which reads any other field of
the payload.
A rule with criteria is considered again when the labels, title or description of a pull request change, as well as when it is opened, reopened or gains commits. Adding the label therefore starts the pipeline for the commit the pull request is at. The forge only sends label changes to a webhook subscribed to them. In the webhook’s settings, choose custom events and tick the pull request label event along with the pull request events.
A pull request stays trusted while it carries the label. Commits pushed to it after that run as they arrive, and nobody has read them. Remove the label when a pull request needs reading again, and add it back once it has been read. A run for a commit that already ran is not started again, so adding the label back only starts runs for new commits.
Criteria are called in the pumphouse’s HTTP server while the forge waits for an answer. They must return quickly and must not make network requests. When criteria raise an exception, their rule starts nothing and the other rules start their runs as usual. The answer to the delivery names that rule’s pipeline with the exception’s message, which the forge shows in its delivery log.
A push rule takes criteria too, called with the push’s
payload. It can skip commits whose message asks for it, as the
example under push-rule in the API chapter does. It can also
ignore the deletion of a branch, which the forge delivers as a push
whose after commit is all zeros.
Each forge signs with a secret of its own, so a pumphouse taking
deliveries from two gives webhooks a list of configurations,
one per forge. A delivery is accepted when its signature matches
one of the secrets, and starts runs by the rules of that
configuration only. This is what keeps the forges apart: a rule’s
repository names an owner and a repository, which two
forges may both have, but a forge cannot start a pipeline configured
under another forge’s secret.
(webhooks (list (webhook-configuration (secret-file "/etc/automate/codeberg-secret") (path "/hooks/codeberg") (rules (list (push-rule (repository "cbaines/automate") (branch "trunk") (pipeline 'automate/trunk))))) (webhook-configuration (secret-file "/etc/automate/work-forge-secret") (path "/hooks/work") (rules (list (pull-request-rule (repository "team/service") (pipeline 'service/check)))))))
The path field is optional and defaults to /webhook.
Two configurations may share a path, since the secrets tell them
apart; a path per forge makes the request log say which forge a
delivery came from, which matters when one is refused for a bad
signature and the answer says nothing else.
The run’s id is derived from the pipeline, the reference and the commit, so a delivery the forge sends twice maps to the run already started rather than starting another. The server answers a delivery with the runs it started, which the forge shows in its delivery log.
(automate forge forgejo) has what a
step needs to talk to the forge. forgejo-report-status posts
a commit status, which the forge shows on the pull request. A body
names the module in its modules field:
(define reported (step (name 'report) (inputs (list fetched net token repository (step-output checked 'value #:required? #f))) (modules '((automate forge forgejo))) (body #~(forgejo-report-status #$repository (call-with-input-file (attach #$token) get-string-all) #$fetched (if (input-present? #$checked) "success" "failure") #:context "automate/pull-request"))))
Other forges get a module of the same shape.
Automate’s own pipelines are under .automate/ in its
repository, where a forge would keep its workflow files. They are
the reference for a pipeline that fetches, builds in a development
environment assembled through the Guix daemon from the channels
resource, and talks to a forge. automate/trunk and
automate/pull-request, in pipelines.scm, are one set of
steps under two names, one started by a push to trunk and one by a
pull request, so that each reports its commit status under its own
context.
automate/pull-requestFetches the pull request’s reference and builds the environment of
manifest.scm, which adds to the package’s development inputs
the Emacs that make format runs. The formatting is checked
first, because make format rewrites the checkout that every
later step shares. That step prints what it changed, restores the
tree, and then fails if anything had changed. The build takes the
check’s outcome as an input that is not required, so it waits for the
check to settle but runs on whatever it found: a badly formatted
commit is still built and tested. The test suite and this manual are
then built from the same tree, neither waiting for the other. A
change that breaks either one fails the commit status, which the last
step reports with a link to the run. The test log is provided as an
output before the suite runs, so it can be read while the step is
still going.
automate/trunkThe same steps for a push to trunk, and a publishing step that
follows the manual build alone. It pushes doc/index.html and
its logo to the pages branch when they changed, under a
pumphouse limiter and only while trunk is still at the commit
the run built, so that a run overtaken by a newer one leaves the site
to it. The manual documents the commit
as it stands, so the push waits for neither the test suite nor the
formatting check.
Both are written on the (automate helpers …) modules,
for running git, building a profile through the Guix daemon and running a
script inside it, with the packages of a guix-channels
resource, so a pipeline of the same shape for another project starts
from them.
The pipelines take the repository as a parameter, so they run
against a bare clone on disk, which is how they are tried before they
are pointed at the forge. The steps see none of the machine’s files
that they do not declare, so the daemon is given the clone with
--expose (see Steps and Inputs). Keep the clone out of
/tmp, since each step has a /tmp of its own. The
secrets directory still needs a forgejo-token file, although
nothing reads it for a clone on disk, so any text will do:
$ git clone --bare https://forge.example.org/owner/project ~/project.git
$ mkdir ~/secrets && echo not-a-real-token > ~/secrets/forgejo-token
$ $(guix build -L . -f .automate/pipelines.scm) --pumphouse=ADDRESS \
--secrets=$HOME/secrets --expose=$HOME/project.git &
$ automate run start automate/trunk \
--param-string=repository=$HOME/project.git \
--param-string=ref=refs/heads/trunk --wait
The reporting step posts a commit status only for a repository whose
URL starts with ‘https://’, so a run against the clone reports
nothing. automate/trunk still pushes the manual, to the
pages branch of the clone.
The forge token is the secret file forgejo-token. Git reads
it through a credential helper, and the reporting step sends it with
the commit status. On another machine, the daemon is attached with
the sturdyref that automate registration grant automate/
mints, which covers both pipelines and nothing else.
Automate runs on Linux. Only building a definition file needs Guix as a whole; running the result needs less, as below. A pumphouse runs no step, so its machine needs only Automate itself.
A definition file is built with guix build, so the machine
building it needs Guix and a Guix daemon. The machine running the
result needs the store items the build made, and nothing else from
Guix. That is the same machine unless the items are copied to
another one.
A Guix daemon is needed where the program runs only when a step uses
one. That is a step holding the guix-daemon resource, or a
step that builds packages from a guix-channels resource,
which it does through the Guix daemon.
A pipeline with a guix-channels resource takes its packages
from the Guix that builds the definition file, unless the resource
is given channels of its own (see Resources). That Guix has to
be one guix pull made, since the resource records the
channels it was pulled from. A Guix run from a checkout or a
guix shell has no such record, and building the file fails
with no-current-guix.
A step holding the guix-daemon resource connects to the Guix
daemon as the pipeline process’s user. That user needs access to
the daemon’s socket, which is GUIX_DAEMON_SOCKET or Guix’s
default socket.
Each step runs in Linux namespaces of its own, which give it a
private file system, process table and network (see Steps and Inputs). The pipeline process runs as an ordinary user, so the
kernel has to let an unprivileged process create a user namespace.
Most kernels do. Some distributions turn this off or restrict it:
Debian’s older kernels with the kernel.unprivileged_userns_clone
setting, Ubuntu from 23.10 with
kernel.apparmor_restrict_unprivileged_userns, and any system
with user.max_user_namespaces set to zero.
Where the kernel refuses, each step dies as it starts, before its
body runs, with the error no-confinement in its log. The
step is abandoned and the run errors. --unconfined, given to a
one-shot program or a daemon, then runs every step as an ordinary
process. Such a step sees the whole machine the pipeline process
can see, with its file system, network and every secret file in the
secrets directory. Its declared resources are only a description,
not a limit.
A run keeps its workspaces in a directory under TMPDIR, or
under /var/tmp where that is unset, and removes it when the
run ends. Point TMPDIR at a file system with room for what the
steps build. Where /tmp is a tmpfs, it is held in memory.
What steps build through the Guix daemon goes into the store
instead.
The following is the list of modules provided by this library.
Declare a pipeline named name whose full set of steps is
steps: (pipeline name #:steps steps #:outputs
outputs #:web-ui-options options #:source source).
The structure is validated, and the pipeline’s steps are held in
dependency order.
outputs says what the pipeline answers with when something outside
it runs the pipeline: an alist from a name of the pipeline’s own to the
step output it stands for, selected as a step’s input is, so that a bare
step means its value output. The names are the pipeline’s
interface, and a step can be renamed without disturbing them.
error is not among the names an output may take. Default
(), so the pipeline answers with nothing.
(pipeline 'guix/build-package #:steps (list plan build report) #:outputs `((derivation . ,(step-output build 'drv)) (log . ,report)))
options is an alist the web interface reads; the meaning of each
key is the interface’s. source says where the pipeline’s code is
published: a procedure called, when the pipeline is lowered, with the
source location of each step and of the pipeline itself, and with the
pipeline’s own location, answering the URL of that code or #f for
no link. forgejo-source and template-source make such
procedures. Default #f, so the web interface shows where each
step is defined without linking there.
Return the branch LOCATION’s repository had checked out, or #f
when its file is in no repository or the repository has a detached head.
Return true when LOCATION’s file differed from the commit when the pipeline was lowered: changed in the working tree, staged, or never committed at all. The commit then does not hold the code that runs, so a link should name the branch instead.
Return the column LOCATION’s form is at, counted from zero, or #f
when the form carried none.
Return the commit LOCATION’s repository was at when the pipeline was
lowered, or #f when its file is in no repository or the
repository has no commit yet. The file itself may differ from that
commit; see source-location-changed?.
Return the working directory of LOCATION’s repository, or #f when
its file is in none. Two locations with the same directory are in the
same repository.
Return the file LOCATION’s form is in, relative to its repository’s working directory when it is in one, and the path it was loaded as when it is not.
Return the line LOCATION’s form is on, counted from one, or #f
when the form carried none.
Return the URL of the origin remote of LOCATION’s repository, or
#f when there is none. A source procedure serving several
repositories can tell them apart by this.
Return true if OBJECT is a source location, as a pipeline’s source procedure is called with.
Declare a step: (step (name n) (inputs l) (body
b) field...). The fields are:
nameA symbol, unique among the steps and parameters of a pipeline, which no resource of it may take either.
inputsA list of steps, resources, parameters and step-output
selections. A step or parameter given plainly stands for its
value output; a resource stands for itself. Default ().
bodyA G-expression evaluated in the step’s own process once every input is
satisfied. It refers to an input by splicing it: #$build for its
value output, and #$build:dist, as Guix writes an output
of a package, for its dist output, the same as
#$(step-output build 'dist). The body checks a non-required
input with input-present?, attaches a resource with
attach, and provides outputs with provide. Its value is
the step’s value output unless the step declares outputs.
callsA call, as call-pipeline makes, saying that the step runs another
pipeline instead of a body of its own. Such a step is no process: the
pipeline process starts the run, waits for it, and takes the outputs the
called pipeline answers with. A step has a body or a call, never both
and never neither, which is checked when the pipeline holding it is
made.
outputsThe list of output names the body provides with provide, instead
of value. A step that calls a pipeline takes each of them from
the output of that name the called pipeline answers with. error
is implicit in every step and cannot be declared, as is run in a
step that calls a pipeline. out cannot be declared either, since
#$step:out in a body would read as a plain splice of the
step.
error-permitted?#t when an error in this step is an outcome the pipeline handles,
through the error output, rather than a failure of the run.
Default #f.
timeoutSeconds after which the step’s process is killed and the step errors, or
#f for no limit.
labelA string shown in place of the name in the web interface.
groupA symbol naming the group the step is drawn in.
modulesModule names the body needs beyond the default set of (guix build
utils), (ice-9 match), (ice-9 textual-ports) and
(srfi srfi-1).
A step may take its fields from another with (inherit
step), overriding those it gives. The step records where in the
source its form is, and the web interface shows that, linked when the
pipeline says where its source is published (see Pipelines).
Declare that a step runs the pipeline named TARGET rather than a body of
its own, and return the call to put in the step’s calls field.
parameters says what the one run of TARGET is given. It is either
the parameters themselves, an alist fixed when the pipeline is built, or
a step whose value is such an alist, worked out while the run is going.
The step then provides the outputs it declares, each taken from the
output of that name TARGET answers with, and run besides, which
is the id of the run it started.
(step (name 'build) (inputs (list plan)) (outputs '(derivation)) (calls (call-pipeline 'guix/build-package #:parameters plan)))
each runs TARGET once for each element of a step’s value, which is
a list of such alists, instead. How many elements there are is settled
while the run is going, which is what this is for: a pipeline’s steps
are fixed when it is built, so a step cannot be made per package that
changed, but a run can. Such a step declares no outputs. It provides
runs, the ids of the runs it started in the order of the list,
and results, one entry per element in that same order:
((run . "a17c93ff21") (outcome . succeeded) (outputs . ((derivation . "/gnu/store/..."))))
width bounds how many of those runs are going at once, the next starting as one ends; without it they all start together. It belongs only to a call made for each of a list.
A step whose runs did not all succeed errors, its results still
carrying what each run did. error-permitted? on the step is then
what decides whether the run it is part of fails (see Outputs and Errors).
A step given to parameters or each must be among the calling step’s inputs, so that the call waits for it. Calling needs a pumphouse: the name of a pipeline means nothing without the registry that holds it.
Return every step reachable from ROOTS through their inputs, each once, dependencies before dependents. Traversal stops at resources and parameters.
Declare the file or directory at SOURCE, which the steps holding it read
and cannot write; NAME labels it in what a run shows. SOURCE is an
absolute path on the machine the run is on, and has to be there when a
step attaches it: the resource makes nothing, it only lets a step in.
With SEEN-AT, an absolute path, that is where a step sees it and what
attach answers; without, a step sees it where it is on the
machine. SEEN-AT has no effect on a run whose steps are not confined,
which has no file system of its own to put it in; neither does the
reading being read-only, a step that is not confined reaching the source
whether it declares it or not.
Prefer an input a step computes, a workspace or a store item wherever
one will do: a pipeline naming a place on the machine runs only on a
machine that has it. Where the place is the operator’s to choose rather
than the pipeline’s, --expose is still the answer.
(define sources (file-input #:source "/srv/sources" #:seen-at "/src"))
Declare the place at DESTINATION on the machine that this run replaces;
NAME labels it in what a run shows. DESTINATION is an absolute path,
and its directory has to be there when a step attaches the resource; the
step is given a staging directory of the run’s own instead, and
attach answers the path in it where the body makes the content.
commit of (automate pipeline step), called from the body,
publishes what was staged and points DESTINATION at it, with one rename
of a symbolic link, so that a reader of DESTINATION sees the content a
run published or the one before it and never a half-written tree. A
step that is confined never holds DESTINATION’s directory at all: the
putting in place is the pipeline process’s, which is the authority
--expose hands over and this does not.
COMMIT-BY says what commit publishes. With rename, the default,
it is the staged content itself, which is free; a step can no longer
name it afterwards, but a descriptor the body kept open still writes to
the published files. With copy, it is a copy, so the body shares
nothing with what was published; the copy is copy-on-write where the
file system does that and a plain copy where it does not.
With SEEN-AT, an absolute path, that is where a step sees the content
and what attach answers; without, a step sees it where it is
staged on the machine. SEEN-AT has no effect on a run whose steps are
not confined, which has no file system of its own to put it in; neither
has the destination being out of reach, such a step reaching it whether
it declares this resource or not.
A pipeline naming a place on the machine runs only on a machine that has it, so reach for this where a run has to leave something on the machine it ran on and a store item or a step’s output will not do.
(define site (file-output #:destination "/srv/www/site" #:seen-at "/out/site"))
Return a source procedure for a pipeline whose repository is at URL, its
page on a Forgejo, or Gitea, forge such as
https://codeberg.org/owner/name. A form in the pipeline’s own
repository is linked to its line, at the commit the file was at, or on
the branch when the file had changes not in any commit. A form in
another repository, or outside any, is not linked.
Declare a gate named NAME: attaching it waits for NEEDED distinct parties to unlock it from outside the run, and raises if it is rejected.
Declare the Guix that steps take packages from: the one built from
CHANNELS, a list of channel records of (guix channels), or, when
CHANNELS is not given, the Guix that lowers the definition file, at the
channels and commits its guix pull made it from. NAME labels
it in what a run shows. Attaching returns the directory of that Guix,
whose bin/guix is its command; the helpers take packages from it
once current-guix of (automate helpers guix) names it. The
structure records the channels, so a run shows which collection its
steps built with.
(define channels (guix-channels))
Declare the guix daemon’s socket, and with it the whole store. NAME labels it in what a run shows.
Declare the host’s network, and give it to the steps that list it. NAME
labels it in what a run shows. A step that does not list it runs in a
network namespace of its own, holding a loopback device and nothing
else: it can still talk to itself over 127.0.0.1, and it can open
a connection to nothing else. Attaching returns nothing of use, since
the declaration is what gives the step the network.
(define net (host-network))
Declare a slot that at most MAX steps hold at once. SCOPE is process or pumphouse. A pumphouse limiter bounds across every pipeline declaring one of its NAME, which it must be given; a process limiter’s NAME only labels it in what a run shows.
(define publish-slot (limiter #:name 'publishing #:max 1 #:scope 'pumphouse))
Return the source location of a form at LINE, counted from one, and COLUMN, counted from zero, of FILE. When the file is in a repository, DIRECTORY is its working directory, FILE is relative to it, COMMIT and BRANCH are those the repository was at, REMOTE is the URL of its origin and CHANGED? is whether the file differed from the commit; otherwise FILE is as loaded and the rest are #f. Lowering makes these, and a test of a source procedure can.
Return a program that runs PIPELINE once. Parameters, the secrets directory and the workspace root are given on its command line.
Declare a parameter named NAME, a symbol: a value a run is given when it
is started, such as a branch or a repository URL. A step lists the
parameter among its inputs and reads its value by splicing it,
#$repository for the one below. A run has to be given every
parameter a step of its pipeline takes, or it is refused; a parameter
has no default value.
(define repository (param 'repository))
Declare a live reference to the trigger of the pipeline named PIPELINE. NAME labels it in what a run shows.
Declare a resource of TYPE named NAME, materialised once per SCOPE. ARGS is an alist of build-time arguments interpreted by the resource type.
Declare the power to abort the runs of the pipeline named PIPELINE, or
of the pipeline declaring it when PIPELINE is not given. NAME labels it
in what a run shows. A step attaches it and passes what attach
returns to abort-run of (automate pipeline step). Aborting
goes through the pumphouse, so it needs a run under a pumphouse, and a
pipeline aborts the runs of no other pipeline than those its run-control
resources name. Finding the runs to abort is a run-history resource’s
job.
(define control (run-control)) (define deploy-control (run-control #:pipeline 'site/deploy))
Return a program that attaches PIPELINES to a pumphouse and serves runs until stopped. The pumphouse address file is given on its command line.
Declare the runs of the pipeline named PIPELINE, or of the pipeline
declaring it when PIPELINE is not given, as a step can list them. NAME
labels it in what a run shows. A step attaches it and passes what
attach returns to list-runs of (automate pipeline
step). The runs are the pumphouse’s records, so listing them needs a
run under a pumphouse, and a pipeline lists the runs of no other
pipeline than those its run-history resources name.
(define history (run-history)) (define trunk-history (run-history #:pipeline 'project/trunk))
Return true if LOCATION and OTHER are in the same repository. A location outside any repository is in the same one as nothing, and neither is #f, a form whose location was not recorded.
Declare a read-only file holding the secret named NAME, once per run.
With SEEN-AT, an absolute path, that is where a step sees it and what
attach answers; without, a step sees it where it is in the
directory the run was given. SEEN-AT has no effect on a run whose steps
are not confined.
(define token (secret 'forge-token #:seen-at "/etc/forge-token"))
Select OUTPUT of SOURCE as an input. A required input whose output is never provided skips the consuming step; a non-required one is merely waited for.
Return a source procedure filling TEMPLATE, a URL with placeholders, for
a form in the pipeline’s own repository: {file},
{line} and {column} are its position,
{commit} the commit the file was at and {branch} the
branch. A form whose template needs a value it has not got, such as the
commit of a file with changes not in any commit, is not linked, nor is a
form in another repository or outside any.
(template-source "https://git.example.org/site.git/tree/{file}?id={commit}#n{line}")
Declare a read-write directory, shared by every step holding it and
materialised once per run. NAME labels it in what a run shows. With
SEEN-AT, an absolute path, that is where a step sees it and what
attach answers; without, a step sees it where it is on the
machine. SEEN-AT has no effect on a run whose steps are not confined,
which has no file system of its own to put it in.
A workspace is scratch. The run makes it in a directory of its own,
under TMPDIR or under /var/tmp, and removes that directory
when the run ends, whatever the outcome; what has to stay on the machine
is a file-output.
(define work (workspace #:seen-at "/src"))
Return a rule starting a pipeline on pull requests:
(pull-request-rule (repository owner/name) (pipeline
pipeline) (criteria procedure)). Only pipeline, a
symbol, is required. Without repository the rule applies to pull
requests of every repository the forge delivers for.
Without criteria the rule starts its pipeline when a pull request
is opened or reopened or gains new commits. With it, the rule also
considers a pull request whose labels, title or description change,
calls procedure with the delivery’s payload, and starts its
pipeline only when that returns true. label-present?,
from-fork?, author-among?, targets-branch? and
draft? answer the common questions of a payload, and
payload-ref reads the rest. This runs the pull requests of
people who can push to the repository, and those of anyone else once
someone has added the label run-pipeline:
(pull-request-rule (repository "cbaines/automate") (criteria (lambda (payload) (or (not (from-fork? payload)) (label-present? payload "run-pipeline")))) (pipeline 'automate/pull-request))
The procedure runs in the pumphouse’s HTTP server while the forge waits for an answer, so it must return quickly and must not make network requests.
Return the procedure deciding whether RULE starts its pipeline for a payload, or #f when it starts it for every pull request it applies to.
Return the name of the pipeline RULE starts.
Return the repository RULE applies to as owner/name, or #f when it applies to every repository.
Return true if OBJECT is a pull request rule.
Return a rule starting a pipeline on pushes: (push-rule
(repository owner/name) (branch name) (pipeline
pipeline) (criteria procedure)). Only pipeline, a
symbol, is required. Without repository the rule applies to
pushes to every repository the forge delivers for, and without
branch, or with the branch "*", to every branch.
criteria, when given, is called with the push’s payload, as
payload-ref reads it, and the rule starts its pipeline only when
it returns true. This skips commits whose message asks for it:
(push-rule (repository "cbaines/automate") (branch "trunk") (criteria (lambda (payload) (not (string-contains (or (payload-ref payload "head_commit" "message") "") "[skip ci]")))) (pipeline 'automate/trunk))
Return the branch RULE applies to, "*" for every branch.
Return the procedure deciding whether RULE starts its pipeline for a payload, or #f when it starts it for every push it applies to.
Return the name of the pipeline RULE starts.
Return the repository RULE applies to as owner/name, or #f when it applies to every repository.
Return true if OBJECT is a push rule.
Return true if the pull request of PAYLOAD was opened by one of LOGINS, a list of user names on the forge.
Return true if the pull request of PAYLOAD is a draft: the forge says
so, or its title starts with WIP: or [WIP], Forgejo’s
default prefixes for a work in progress.
Comment TEXT, Markdown, on pull request or issue NUMBER of the repository at URL, as the user TOKEN belongs to, and return the new comment’s id. Everyone watching the pull request is told, as for any comment.
Return the (PIPELINE . REQUEST) pairs to start for a delivery of EVENT, the event header, with PAYLOAD, the parsed JSON body, under RULES, a list of push and pull request rules. A rule with a repository only applies to deliveries from that repository, so one webhook listener serves the pipelines of several. An exception raised by a rule’s criteria is not caught.
Post a commit status for SHA in the repository at URL: STATE is
"success", "failure", "error" or "pending", shown under CONTEXT with
DESCRIPTION, and linking to TARGET-URL when given, which is what
run-url of the body vocabulary answers. Return the response
code; raise when the forge refuses.
Return the host, owner and name of the repository at URL, an https clone URL, as three values, or three times #f for anything else.
Return the run id a delivery for COMMIT at REF of PIPELINE maps to. The same delivery repeated maps to the same run.
Return true if SIGNATURE, the hex HMAC-SHA256 a forge sends with a delivery, matches BODY, a bytevector, under SECRET.
Show TEXT, Markdown, in one comment on pull request or issue NUMBER of the repository at URL, editing the comment an earlier call with the same MARKER made, or making it when there is none, and return its id. A run reporting on a pull request calls this every time, so the pull request carries one comment saying where it stands rather than a comment a run. The comment carries MARKER in an HTML comment, which the forge does not show, and only a comment of the user TOKEN belongs to is edited.
Editing a comment tells nobody, where making one tells everyone
watching, so a run that should be heard, having broken or fixed
something, calls forgejo-comment as well.
(forgejo-sticky-comment repository token number (format #f "Checks ~a: [run](~a)" state (run-url)) #:marker "checks")
Return true if the pull request of PAYLOAD comes from a repository other than the one it is made to, as when its author cannot push to the repository. A pull request whose source repository the forge does not name, such as one from a fork since deleted, counts as from a fork.
Return true if the pull request of PAYLOAD carries the label NAME. Only people with write access to the repository can label a pull request on Forgejo, so a label is how one of them says a pull request may run.
Return the value under KEYS, strings, in PAYLOAD, a delivery’s body
parsed by guile-json, or #f when there is none. JSON objects are alists
with string keys and arrays are vectors, so (payload-ref payload
"pull_request" "head" "sha") is the commit a pull request is at. A
criteria procedure uses this for what no helper here reads.
Return true if the pull request of PAYLOAD would merge into BRANCH, a
branch name such as "trunk".
Return RULES, raising an automate error unless they are a list of rules,
as push-rule and pull-request-rule of (automate
forge forgejo) make them.
Return a handler for the Knots web server that accepts forge deliveries from FORGES, a list of (SECRET . RULES) pairs, one per forge sending deliveries. A delivery is accepted when its signature matches one of the secrets, mapped to runs by the rules paired with that secret, and TRIGGER is called with a pipeline name and a run request for each run to start, expecting the run id. The handler answers with the runs started, and names each rule whose criteria raised with the exception’s message, which the forge shows in its delivery log. It is called with a request and a port for its body.
Default value:
#f
Default value:
#f
Default value:
#f
Run git in DIRECTORY with ARGUMENTS, as git -C DIRECTORY
ARGUMENTS..., raising unless it exits with zero. Its output goes where
the step’s does, and it takes its password from
current-git-credentials when that is set.
Put a checkout of BRANCH of the repository at URL under DIRECTORY and
return its path. When the remote has the branch this is git clone
--depth=1 of it; when it has not, it is an empty repository on that
branch with origin at URL, so that a body pushing to a branch
works whether or not the repository has one yet. The checkout is named
after the branch, or after #:into when that says otherwise.
(let ((pages (git-clone-or-init work url "pages"))) (install-file (string-append source "/doc/index.html") pages) (git pages "add" "."))
Run git in DIRECTORY with ARGUMENTS and return the first line it prints,
without its line ending, or "" when it prints nothing. Its exit
status is not checked, so (git-output directory "status"
"--porcelain") answers "" for a clean tree.
Whether the repository at URL has BRANCH, asked from DIRECTORY with
git ls-remote, so a body can clone it when it is there and start
it when it is not. Only a remote that answers without the branch gives
#f; an unreachable remote or a refused token raises as git
does, so a missing branch is never mistaken for one.
git-clone-or-init is this question and both of its answers
together.
(when (git-remote-branch? work url "pages") (git work "clone" "--depth=1" "--branch=pages" url "pages"))
The commit REF is at in the repository at URL, asked from DIRECTORY with
git ls-remote, or #f when the remote has no such ref. REF
is a whole ref name, such as "refs/heads/trunk". An unreachable
remote or a refused token raises as git does, so a ref that could
not be read is never mistaken for one that is gone.
A step publishing something built from a branch asks for the branch again, inside whatever serialises the publishing, and stands aside when it has moved: of several runs publishing at once, the one whose commit is still the branch’s is the one whose work remains.
(attach #$publishing) (if (equal? (git-remote-tip work url "refs/heads/trunk") (git-output source "rev-parse" "HEAD")) (publish-the-site) 'superseded)
Return the path of OUTPUT of the package called NAME, a string such as
"git-minimal", in the Guix current-guix names, building it
through the daemon the first time this process asks for it. The package
is the one that Guix defines, at the channels the guix-channels
resource captured, rather than one from the Guix in the step’s program,
which is a library for talking to the daemon whose collection nobody
chose. The daemon is the one GUIX_DAEMON_SOCKET names, or Guix’s
default.
(current-guix (attach #$channels)) (string-append (build-package "coreutils") "/bin/ls")
Return an inferior of the Guix current-guix names: a
guix repl of it, started the first time this process asks and
kept for the rest of the step. Raises no-guix when
current-guix is not set.
Default value:
#f
Default value:
#f
Return the verdict of CHECKS, an alist from the name of each check to whether it passed: plain data to provide as a step’s output and compare with an earlier run’s. The verdict has passed when every check did, and failed otherwise, naming the checks that did not, in order.
(checks-verdict `((build . ,(input-present? #$built)) (tests . ,(input-present? #$checked)))) ⇒ ((state . failed) (failed tests))
Post PAYLOAD, an alist or vector as guile-json writes it, as JSON to
URL, with HEADERS added to the request, an alist from header name to
value. Return the response code; raise post-refused when it is
not a success. The step needs the host’s network, a host-network
input. This is the shape of the web hooks chat services take:
(post-json (read-secret (attach #$chat-hook)) `((text . ,(format #f "~a is ~a: ~a" ref transition (run-url)))))
Mail BODY under SUBJECT to TO, an address or a list of them, from FROM
when given, with HEADERS added, an alist from header name to value. The
message goes to the program current-sendmail names, which reads
the recipients from it. CONFIGURATION is a file of msmtp’s settings,
such as what attach of a secret input returns, passed as
--file: the server to send through, its account and its
password. msmtp refuses a file others can read, so the secret is kept
with mode 600. Raises mail-failed when the program fails. The
step needs the host’s network, a host-network input.
(send-email #:to "team@example.org" #:subject "trunk is broken" #:body (format #f "See ~a~%" (run-url)) #:configuration (attach #$mail-settings))
Return the transition, as verdict-transition names it, from the
verdict of the newest earlier run of the same thing to CURRENT, this
run’s, and that earlier run as a second value, or #f for none.
HISTORY is an attached run-history resource. The runs compared are
those given the same values of the parameters MATCHING names, and the
values PARAMETERS gives, whose OUTPUT of STEP was provided; STEP is the
step providing the verdict.
Runs overlap, so a run requested later than this one may already have
provided its verdict. Then this one answers superseded and no
run, and says nothing: the later run compared itself with this one, or
with a run before it, and a transition is told once, by the newer run,
rather than twice or out of order. The question is asked at one moment
and not held, which is enough for that: whichever of two runs asks
second sees the other.
(let ((transition (verdict-change (attach #$history) #$verdict #:matching '(repository number)))) (when (memq transition '(broken fixed changed-failure)) (announce transition)))
Return the names of the checks VERDICT says failed, empty for a verdict that is a symbol alone.
Return the state of VERDICT, passed or failed. A verdict
is what checks-verdict returns, or one of those two symbols
alone.
Return what changed from PREVIOUS, the verdict of an earlier run or
#f when there is none, to CURRENT, this run’s:
firstthere was no earlier verdict;
brokenit passed and now fails;
fixedit failed and now passes;
still-passingit passed and still does;
still-failingit failed and the same checks still fail;
changed-failureit failed and still does, but other checks fail now.
Default value:
#f
Default value:
#f
Build the profile FILE describes through DAEMON, the step’s
guix-daemon input, with the packages of GUIX, the step’s
guix-channels input, and return it as a store item the resource
keeps while the run holds it: the development inputs of the package FILE
evaluates to, what guix shell -D -f FILE would assemble, or
the manifest it evaluates to, what guix shell -m FILE would.
FILE is evaluated by that Guix, so the packages it names are the ones
its channels define, as they would be for a developer running
guix shell from it. A later step takes the item as an input
and attaches it to get the profile’s path. Inputs may collide, as when
a library is propagated both as Guix builds it and as rebuilt with
another Guile, and that is allowed as guix shell allows it.
It prints the file it evaluates, the derivation it computed and the profile it built, so the step’s output says which profile the steps after it run in. When the build fails, that output names the derivation to build again while looking into it.
Run SCRIPT, shell text, with bash in DIRECTORY with the environment of
PROFILE loaded, stopping at the first failing command and raising unless
the script exits with zero. The bash is the profile’s own when it has
one, so a step whose profile carries a shell needs no
guix-channels input; otherwise it is current-bash, or
bash-minimal from the Guix current-guix names. The
profile’s own etc/profile is sourced, which is why the shell does
not run with -u.
Return true if the spliced input was provided: (input-present?
#$built) for the value output of built, or
(input-present? #$built:dist) for its dist output. An
input declared with #:required? #f may settle without being
provided, and this is how a body finds out before using it. The splice
is asked after rather than fetched, so this does not raise when the
input is absent.
(if (input-present? #$checked:status) "success" "failure")
Abort run ID, one of the pipeline CONTROL, an attached run-control
resource, names, for REASON: its running steps are killed, its pending
ones skipped, and it finishes as errored. Return aborting, or
finished when the run has already finished, so that a run racing
to finish is no error. Raises when ID is this run, which may not abort
itself, or a run of another pipeline than CONTROL’s.
(for-each (lambda (run) (abort-run (attach #$control) (assq-ref run 'id) #:reason "superseded")) (list-runs (attach #$history) #:matching '(number) #:before (run-id) #:finished #f))
Attach RESOURCE, a resource or store item received as an input, to this
step and return what identifies it locally: the path of a workspace, a
secret or a file input, the staging path of a file output, the socket of
a guix-daemon, the directory of a guix-channels Guix, the
path of a store item. Attaching a limiter waits for a slot, and a gate
for its decision. Attaching twice returns the same thing, and what a
step attached is released when its process exits.
Publish what this step staged in RESOURCE, a file output received as an input, and return the place on the machine it was published to. The pipeline process does the publishing, so a step that is confined never holds that place itself; the body makes the content at the path attach answered, and commit puts it there in one step a reader cannot see the middle of. Raises where nothing was staged, where the file output was committed already, or where the place cannot be published to, so a body that has to know commits before it finishes.
Run MAKE-VOW, a thunk, inside the step’s vat, the Goblins event loop holding the step’s connection to the pipeline process, and wait for the promise it returns. Only a body that uses Goblins directly needs it.
Return OUTPUT of input SOURCE, the name of a step, resource or parameter
this step declares as an input. For a step, that is the value it
provided, its value output unless OUTPUT names another; for a
parameter, the value the run was given; for a resource, a reference to
pass to attach. Raises if the output was not provided, which
input-present? asks first.
(input 'repository) (input 'build 'version)
Return the runs HISTORY, an attached run-history resource, holds that
match the keywords given, newest first unless ORDER and DIRECTION say
otherwise. Each run is an alist of its id, pipeline,
definition, parameters, outcome, when it was
requested, started and finished, and the
outputs OUTPUTS asks for; listed-run-output reads one of
those.
PARAMETERS keeps the runs given each of its (NAME . VALUE) pairs, a
value matching only a value written the same way: the string "12"
is not the number 12. MATCHING names parameters of this run, and
keeps the runs given the same value of each. BEFORE and AFTER, each a
run id such as (run-id) answers, keep the runs requested before
or after that run. PROVIDED keeps the runs where each of its step
outputs was provided. OUTCOME, a list such as '(succeeded
errored), keeps the finished runs that ended so. FINISHED, when
#t, keeps the runs that have finished, and when #f those
that have not; by default, any, it keeps both. ORDER is
requested, started or finished, and a run that has
not yet started or finished is left out when ordering by that time.
DIRECTION is ascending or descending; LIMIT keeps that
many runs at most.
A step output is a step’s name for its value output, or a
(STEP OUTPUT) list for another. A run’s outputs are
(STEP OUTPUT VALUE) lists. Only an output whose value is plain
data is recorded, so OUTPUTS answers nothing for one whose value is a
store item.
(list-runs (attach #$history) #:matching '(repository number) #:before (run-id) #:provided '(verdict) #:outputs '(verdict) #:limit 1)
Return the value OUTPUT of STEP recorded in RUN, one of the runs
list-runs answers, or #f when the run recorded none or the
query did not ask for it.
Keep VALUE, a string, out of what the run shows from now on: the marker replaces it in this step’s log, in every event, output value and error record of the run. Every attached secret is masked this way without being asked; mask is for a value the body derived, such as a token it exchanged the secret for, and is called before the value is printed or provided. A value the run cannot take raises with the reason: too-short, under eight bytes, too-long, over four kilobytes, or too-many, past the run’s limit. Masking is a net, not a control: a value printed before it was masked, or printed transformed, goes through, and a value that was printed at all has left the step.
Provide OUTPUT of the running step as VALUE, waiting for the pipeline process to record it, so a step taking the output can start while this body carries on. OUTPUT is one the step declares, and VALUE is plain data or a store item. Raises if the step does not declare OUTPUT, or has provided it already.
Release RESOURCE, which this step attached, before the step ends, so that a step waiting for the limiter slot it holds can take it.
Return the id of the run this step is part of, as the pumphouse and
automate run show name it, or #f for a run without one.
Run BODY, a thunk, as this process’s step. The program lowering makes of a step calls this with its body, and a body never does. ARGUMENTS holds the socket path and the sturdyref the pipeline process passed. The process connects back to the pipeline process, receives its inputs, and runs BODY on the main thread while a vat on another thread keeps the connection live. It exits only once the pipeline process has recorded the outcome.
Return the address of this run’s page on the pumphouse’s web interface,
or #f when no pumphouse with one serves the run: what a commit
status links back to.
(forgejo-report-status repository token commit "success" #:target-url (run-url))
Ask DAEMON, a guix-daemon resource received as an input, to keep the store item at PATH for the steps after this one, and return the item, an object to provide as an output. The daemon roots it before answering, so call this while the connection that built it is still open.
Start a run of the pipeline TRIGGER, an attached pipeline-trigger
resource, points at, with PARAMETERS, an alist of names and values.
Return the new run’s id. The new run records this one as its cause.
Return the admin sturdyref string of ADDRESS.
Return the registrar sturdyref string of ADDRESS.
Return the admin sturdyref string of ADDRESS over TCP, or #f.
Return the registrar sturdyref string of ADDRESS over TCP, or #f.
Return the introduction server socket path of ADDRESS, or #f.
Connect to the pumphouse at ADDRESS and call PROCEDURE with the vat and the admin object, halting the vat afterwards.
Return a fresh vat and its mycapn, reaching the pumphouse at ADDRESS.
Return the pumphouse address stored in FILE.
Inside a vat, spawn a netlayer that reaches the pumphouse at ADDRESS, over its introduction server when the address names one and over TCP otherwise. Goblins 0.18 stops accepting incoming connections on a netlayer once any of its introduction servers goes away, so the link to a pumphouse, which may die, gets a netlayer of its own.
Return four values: the netlayer, a mycapn for it, a promise fulfilled once the netlayer is installed in that mycapn and ready to be dialled through, and a promise that settles when the introduction server behind it goes away, or #f where there is none to lose. The netlayer is installed rather than handed to spawn-mycapn so that there is a readiness promise at all; whoever dials waits on it, since a mycapn installing a netlayer does not yet know it.
Return the address STRING names: the contents of an address file, or, for a sturdyref, an address whose registrar and admin are both that object, whatever it is.
Write the address of a pumphouse to FILE: FIELDS is an alist of socket, the introduction server’s path, registrar and admin, sturdyref strings, and remote-registrar and remote-admin, their sturdyref strings over TCP, when there are any.
Configure the HTTP server of a pumphouse, as run-pumphouse takes
under #:http: (http-configuration (port port)
field...). The fields are:
portThe port to listen on; zero means any free one, and the port bound is
written to the state directory’s http-port file.
addressAn IPv4 or IPv6 address to bind to, or #f, the default, for every
address of both families.
public-urlThe URL this server is reached at when it is not the address bound:
daemons on other machines post step output under it, and a step’s
run-url, which a commit status links to, is built from it.
Default #f.
webhooksA webhook-configuration, a list of them when deliveries come from
several forges each signing with a secret of its own, or #f, the
default, for no webhook listener. A delivery is accepted when its
signature matches the secret of one of them, and starts runs by that
one’s rules only.
metrics?Whether to serve this pumphouse’s metrics at /metrics, for
Prometheus, a time series database that reads them over HTTP, to
collect. Default #f. They are served to whoever asks, as the
pages are: they name pipelines, outcomes and timings, which the pages
show already.
Return the address CONFIGURATION binds to, or #f for every
address.
Return true if CONFIGURATION serves metrics at /metrics.
Return the port CONFIGURATION listens on, zero meaning any free one.
Return the URL daemons elsewhere reach CONFIGURATION’s server at, or
#f.
Return the webhook configuration of CONFIGURATION, a list of them, or
#f.
Return true if OBJECT is an HTTP configuration.
Return the fields of PUMPHOUSE’s address, as write-address of
(automate pumphouse client) takes them.
Return the file publish-pumphouse! writes PUMPHOUSE’s address to,
in its state directory.
Return true if OBJECT is a running pumphouse, as start-pumphouse
returns.
Configure one timer of a pumphouse, in the list run-pumphouse
takes under #:timers: (timer-configuration (pipeline
name) (every seconds) field...). The timer starts a
run of the pipeline every seconds seconds for as long as the
pumphouse runs, skipping a tick while no daemon is attached under the
name or while the timer is paused. The fields are:
pipelineThe name of the pipeline to start, a symbol.
everyThe seconds between one run and the next, a positive number. The first run starts that long after the pumphouse does.
parametersAn alist of parameter name to value that every run is given. Default empty.
nameThe symbol automate timer list, automate timer pause and
automate timer resume know the timer by. Defaults to the
pipeline name, so two timers of one pipeline need names of their own.
(run-pumphouse #:state-directory "/var/lib/automate" #:timers (list (timer-configuration (pipeline 'website) (every 3600)) (timer-configuration (name 'website-full) (pipeline 'website) (every 86400) (parameters '((full . #t))))))
Return the seconds between the runs timer CONFIGURATION starts.
Return the name of timer CONFIGURATION, a symbol.
Return the parameters of the runs timer CONFIGURATION starts, an alist.
Return the name of the pipeline timer CONFIGURATION starts.
Return true if OBJECT is a timer configuration.
Configure the forge webhook listener of a pumphouse’s HTTP server:
(webhook-configuration (secret-file file) (rules
rules) field...). file holds the secret the forge
signs its deliveries with. rules is a list of the rules
push-rule and pull-request-rule of (automate forge
forgejo) make, saying which pipelines a delivery starts. The optional
path is where the forge posts deliveries, an absolute path such
as "/hooks/codeberg"; the default is "/webhook".
Configurations sharing a path are told apart by their secrets, so giving
each forge a path of its own is a matter of telling from the request log
which forge sent a delivery.
Return the path the forge posts deliveries to CONFIGURATION at,
"/webhook" by default.
Return the rules of CONFIGURATION, a list of push and pull request rules.
Return the file holding the secret the forge signs deliveries to CONFIGURATION with.
Return true if OBJECT is a webhook configuration.
Halt PUMPHOUSE’s vats, closing its databases first.
Write PUMPHOUSE’s address file, after which pipeline processes and the command line can reach it.
Return the file in PUMPHOUSE’s state directory naming the port its HTTP server is bound to, written when the server starts.
Run a pumphouse keeping its records under STATE-DIRECTORY until told to stop, on the calling thread. A pipeline process unheard of for STALE-AFTER seconds is marked stale, and one unheard of for LOST-AFTER seconds, ten minutes by default, is marked lost and every unfinished run of its pipeline abandoned, since nothing else will finish them. With TCP, a (HOST . PORT) pair, it listens for CapTP over TCP+TLS on that port as well, zero meaning any free one, at every address of both families, naming HOST in its sturdyrefs. With HTTP, an http-configuration, it serves output uploads, the web interface, any webhooks and, where the configuration asks for them, the metrics at /metrics, on that port, at the address it gives or every one, and writes the port bound to DIR/http-port. TIMERS is a list of timer-configurations, each starting a run of its pipeline at its interval while the pumphouse runs. Fibers runs on the main thread alone, so requests are handled one at a time.
Start a pumphouse keeping its records under STATE-DIRECTORY, creating it and its parents as needed, and return it. A pipeline process unheard of for STALE-AFTER seconds is marked stale, and for LOST-AFTER seconds lost, its unfinished runs abandoned. With TCP, a (HOST . PORT) pair, it listens on that port as well, zero meaning any free one, naming HOST in the sturdyrefs it hands out. TIMERS is a list of timer-configurations to run. Nothing can find it until publish-pumphouse! writes its address.
Automate depends on the libraries below. The package propagates all of them, so there is nothing further to install.
Lowers a definition file to store items and builds what the steps build. Step bodies are G-expressions. See GNU Guix Reference Manual.
The actors every process is made of, and the CapTP protocol that connects them. A pumphouse and the daemons connecting to it over TCP must run the same version, since the wire format changes between releases. See The Spritely Goblins Manual.
The pumphouse’s HTTP server and web interface. Fibers is at https://doc.guix.gnu.org/fibers/latest/en/fibers.html, Knots at https://docs.cbaines.net/guile-knots/ and Safsaf at https://docs.cbaines.net/safsaf/.
The pumphouse’s records, with step logs stored compressed.
Webhook deliveries, commit statuses and the web interface’s event streams. https://github.com/aconchillo/guile-json.
Webhook signatures, run ids, and TLS on the pumphouse’s TCP listener. https://codeberg.org/guile-gcrypt/guile-gcrypt.
The pumphouse’s metrics at /metrics.
https://forge.cbaines.net/cbaines/guile-prometheus.
The nREPL server a step offers on automate repl request.
https://git.sr.ht/~abcdw/guile-ares-rs.
Copyright © 2026 Christopher Baines <mail@cbaines.net>
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
| Jump to: | A B C D F G H I L M O P R S T V W |
|---|
| Jump to: | A B C D F G H I L M O P R S T V W |
|---|