Orderly Beginnings

Install orderly

From the mrc-ide r-universe (recommended)

install.packages(
  "orderly",
  repos = c("https://mrc-ide.r-universe.dev", "https://cloud.r-project.org"))

From GitHub using remotes:

remotes::install_github("mrc-ide/orderly")

From PyPi (for Python)

pip install pyorderly

Check your version

packageVersion("orderly")
## [1] '2.0.3'

My first orderly report / task

There is a discussion to have here about naming. We might have this in a break…

The setup

First, load the package and create a new empty orderly root.

library(orderly)
orderly_init("workdir/part1")
## ✔ Created orderly root at '/home/runner/work/orderly-tutorial/orderly-tutorial/workdir/part1'
## Warning: Can't check if files are correctly gitignored
## ℹ Your outpack repo is in a subdirectory ''workdir/part1'' of your git repo
## ℹ To disable this check, set the option 'orderly.git_error_ignore' to TRUE by
##   running `options(orderly.git_error_ignore = TRUE)`

(for the rest of this section, we have setwd() into this directory; you should create an RStudio “Project” here.)

What’s in the box?

fs::dir_tree("workdir/part1")
## workdir/part1
## └── orderly_config.json

Really:

fs::dir_tree("workdir/part1", all = TRUE)
## workdir/part1
## ├── .outpack
## │   ├── config.json
## │   ├── location
## │   └── metadata
## └── orderly_config.json

But leave everything in .outpack/ alone, just like .git/

Create an empty report

orderly_new("example")
## Warning: Can't check if files are correctly gitignored
## ℹ Your outpack repo is in a subdirectory ''workdir/part1'' of your git repo
## ℹ To disable this check, set the option 'orderly.git_error_ignore' to TRUE by
##   running `options(orderly.git_error_ignore = TRUE)`
## ✔ Created 'src/example/example.R'

Our contents now:

fs::dir_tree("workdir/part1")
## workdir/part1
## ├── orderly_config.json
## └── src
##     └── example
##         └── example.R

The name example.R is important; this always has the form src/<name>/name.R

Hello orderly world

We have edited src/example/example.R to contain:

d <- data.frame(greeting = "hello", to = "world")
write.csv(d, "hello.csv", row.names = FALSE)

Now we run

id <- orderly_run("example")
## Warning: Can't check if files are correctly gitignored
## ℹ Your outpack repo is in a subdirectory ''workdir/part1'' of your git repo
## ℹ To disable this check, set the option 'orderly.git_error_ignore' to TRUE by
##   running `options(orderly.git_error_ignore = TRUE)`
## ℹ Starting packet 'example' `20260602-103223-3a8e33b1` at 2026-06-02 10:32:23.236281
## > d <- data.frame(greeting = "hello", to = "world")
## > write.csv(d, "hello.csv", row.names = FALSE)
## ✔ Finished running 'example.R'
## ℹ Finished 20260602-103223-3a8e33b1 at 2026-06-02 10:32:23.288577 (0.05229616 secs)

Files created

fs::dir_tree("workdir/part1")
## workdir/part1
## ├── archive
## │   └── example
## │       └── 20260602-103223-3a8e33b1
## │           ├── example.R
## │           └── hello.csv
## ├── draft
## │   └── example
## ├── orderly_config.json
## └── src
##     └── example
##         └── example.R
  • Directory named after the id (in archive/example)
  • We have copied example.R into the directory
  • Output sits next to inputs
  • Metadata is stored in a hidden location

Contents of hello.csv:

read.csv(file.path("workdir/part1/archive/example", id, "hello.csv"))
##   greeting    to
## 1    hello world

Every packet has a unique id

id
## [1] "20260602-103223-3a8e33b1"

and a bunch of metadata:

orderly_metadata(id)
## Warning: Can't check if files are correctly gitignored
## ℹ Your outpack repo is in a subdirectory ''workdir/part1'' of your git repo
## ℹ To disable this check, set the option 'orderly.git_error_ignore' to TRUE by
##   running `options(orderly.git_error_ignore = TRUE)`
## $schema_version
## [1] "0.1.1"
## 
## $name
## [1] "example"
## 
## $id
## [1] "20260602-103223-3a8e33b1"
## 
## $time
## $time$start
## [1] "2026-06-02 10:32:23 UTC"
## 
## $time$end
## [1] "2026-06-02 10:32:23 UTC"
## 
## 
## $parameters
## NULL
## 
## $files
##        path size
## 1 example.R   95
## 2 hello.csv   32
##                                                                      hash
## 1 sha256:541682d8b8dba9b2ddb4ac5809c03e6bedd58b52ab3e64a662f3f48e66a9639f
## 2 sha256:b9f0704f459f7ad9785ddee01a281d81f95a461dbb682436a263e0b7252e92b7
## 
## $depends
## [1] packet query  files 
## <0 rows> (or 0-length row.names)
## 
## $git
## $git$sha
## [1] "2a19c597853024e2e6f9f8f82f0d7894dbbbb07c"
## 
## $git$branch
## [1] "main"
## 
## $git$url
## [1] "https://github.com/mrc-ide/orderly-tutorial"
## 
## 
## $custom
## $custom$orderly
## $custom$orderly$artefacts
## [1] description paths      
## <0 rows> (or 0-length row.names)
## 
## $custom$orderly$role
##        path    role
## 1 example.R orderly
## 
## $custom$orderly$description
## $custom$orderly$description$display
## NULL
## 
## $custom$orderly$description$long
## NULL
## 
## $custom$orderly$description$custom
## NULL
## 
## 
## $custom$orderly$shared
## [1] here  there
## <0 rows> (or 0-length row.names)
## 
## $custom$orderly$session
## $custom$orderly$session$platform
## $custom$orderly$session$platform$version
## [1] "R version 4.6.0 (2026-04-24)"
## 
## $custom$orderly$session$platform$os
## [1] "Ubuntu 24.04.4 LTS"
## 
## $custom$orderly$session$platform$system
## [1] "x86_64, linux-gnu"
## 
## 
## $custom$orderly$session$packages
##        package version attached
## 1      orderly   2.0.3     TRUE
## 2       crayon   1.5.3    FALSE
## 3        vctrs   0.7.3    FALSE
## 4          cli   3.6.6    FALSE
## 5        knitr    1.51    FALSE
## 6        rlang   1.2.0    FALSE
## 7         xfun    0.58    FALSE
## 8     jsonlite   2.0.0    FALSE
## 9         glue   1.8.1    FALSE
## 10     openssl   2.4.1    FALSE
## 11     askpass   1.2.1    FALSE
## 12   htmltools   0.5.9    FALSE
## 13         sys   3.4.3    FALSE
## 14   rmarkdown    2.31    FALSE
## 15    evaluate   1.0.5    FALSE
## 16      tibble   3.3.1    FALSE
## 17     fastmap   1.2.0    FALSE
## 18        yaml  2.3.12    FALSE
## 19   lifecycle   1.0.5    FALSE
## 20    compiler   4.6.0    FALSE
## 21          fs   2.1.0    FALSE
## 22   pkgconfig   2.0.3    FALSE
## 23      digest  0.6.39    FALSE
## 24        gert   2.3.1    FALSE
## 25          R6   2.6.1    FALSE
## 26      pillar  1.11.1    FALSE
## 27 credentials   2.0.3    FALSE
## 28    magrittr   2.0.5    FALSE
## 29       withr   3.0.2    FALSE
## 30       tools   4.6.0    FALSE

What is a hash?

A one-way transformation from data to a fairly short string

orderly_hash_data("hello", "sha256")
## [1] "sha256:2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824"

Very small changes to the string give large changes to the hash

orderly_hash_data("hel1o", "sha256")
## [1] "sha256:28ad6a376c5c22d1a3ad7f115c0cd8f4b8a9c94f55325c17f9302b6a4e41b29c"

This means we can compare hashes and be confident we are looking at the same file (git does a lot of this)

Run it again, Sam

orderly_run("example")
## Warning: Can't check if files are correctly gitignored
## ℹ Your outpack repo is in a subdirectory ''workdir/part1'' of your git repo
## ℹ To disable this check, set the option 'orderly.git_error_ignore' to TRUE by
##   running `options(orderly.git_error_ignore = TRUE)`
## ℹ Starting packet 'example' `20260602-103223-6498921b` at 2026-06-02 10:32:23.397277
## > d <- data.frame(greeting = "hello", to = "world")
## > write.csv(d, "hello.csv", row.names = FALSE)
## ✔ Finished running 'example.R'
## ℹ Finished 20260602-103223-6498921b at 2026-06-02 10:32:23.422707 (0.02543044 secs)
## [1] "20260602-103223-6498921b"
  • We have a new id with the new copy

A copy saved every time we run

Stop naming files data_final-rgf (2).csv, please

fs::dir_tree("workdir/part1")
## workdir/part1
## ├── archive
## │   └── example
## │       ├── 20260602-103223-3a8e33b1
## │       │   ├── example.R
## │       │   └── hello.csv
## │       └── 20260602-103223-6498921b
## │           ├── example.R
## │           └── hello.csv
## ├── draft
## │   └── example
## ├── orderly_config.json
## └── src
##     └── example
##         └── example.R

A high-level overview of of packets:

orderly_metadata_extract()
## Warning: Can't check if files are correctly gitignored
## ℹ Your outpack repo is in a subdirectory ''workdir/part1'' of your git repo
## ℹ To disable this check, set the option 'orderly.git_error_ignore' to TRUE by
##   running `options(orderly.git_error_ignore = TRUE)`
##                         id    name parameters
## 1 20260602-103223-3a8e33b1 example           
## 2 20260602-103223-6498921b example

(more on this later).

Next steps