Beyond started with the Guix workflow language
Recap
In the previous section we defined a simple process. Now it's time to look more closely at workflows.
Defining workflows
A workflow describes how processes
relate to each other. So before we can write the workflow, we must
define some processes. In this example we will create a file with a
process named create-file
, and we will compress that file
using a process named compress-file
.
process: create-file
outputs "/tmp/file.txt"
run-time
complexity
space 20 MiB
time 10 seconds
# { echo hello > {{outputs}} }
process: compress-file
packages "gzip"
inputs "/tmp/file.txt"
outputs "/tmp/file.txt.gz"
run-time
complexity
space 20 mebibytes
time 2 minutes
# { gzip {{inputs}} -c > {{outputs}} }
With these definitions in place, we can run both in one go by defining a workflow.
workflow: file-workflow
processes
auto-connect create-file compress-file
The workflow specifies all processes that should run.
The auto-connect
procedure links up all inputs and outputs of
all specified processes and ensures that the processes are run in the
correct order. Later we will see other ways to specify process
dependencies.
Process templates
We can parameterize the inputs and outputs for a process, so
that the same process template can serve for different inputs and
outputs. Here is a process template that is parameterized
on input
:
process: (compress-file input)
name
string-append "compress-file-"
basename input
packages "gzip"
inputs input
outputs
string-append input ".gz"
run-time
complexity
space 20 mebibytes
time 10 seconds
# {
gzip {{input}} -c > {{outputs}}
}
Dynamic workflows
We can now dynamically create compression processes by
instantiating the compress-file
template with specific
input file names. We use Scheme's let
,
and map
to simplify the work for us:
process: (create-file filename)
name
string-append "create-file-"
basename filename
outputs filename
run-time
complexity
space 20 mebibytes
time 10 seconds
# { echo "Hello, world!" > {{filename}} }
process: (compress-file input)
name
string-append "compress-file-"
basename input
packages "gzip"
inputs input
outputs
string-append input ".gz"
run-time
complexity
space 20 mebibytes
time 10 seconds
# { gzip {{inputs}} -c > {{outputs}} }
;; All inputs files. The leading dot continues the previous line.
define files
list "/tmp/one.txt"
. "/tmp/two.txt"
. "/tmp/three.txt"
;; Map process templates to files to generate a list of processes.
define create-file-processes
map create-file files
define compress-file-processes
map compress-file files
workflow: dynamic-workflow
processes
auto-connect compress-file-processes create-file-processes
In the GWL, we can define process dependencies explicitly.
This is useful when processes don't have explicit outputs
or inputs
. Processes can do something other than
producing output files, such as inserting data in a database, so
process dependencies can be specified manually.
Restrictions can be specified as an association list mapping
processes to their dependencies, or via the
convenient graph
syntax.
workflow: graph-example
processes
graph
A -> B C
B -> D
C -> B
Reusing workflows in new workflows
On the next page, we
will reuse parts of dynamic-workflow
above in a new
workflow.