library(odin2)
7 Packaging odin models
So far, we have compiled odin code as we have needed it, using the odin()
function. This works well for many uses, but has its limitations:
- It requires that anyone who uses your code has a working C++ compiler; they will need to compile your model and load it into R to run things
- Compilation of the model is quite slow; large models can take more than 10s to compile. We generally will only compile a system once per R session, but this is still quite a large fixed cost to pay
- If you need to work with HPC systems, then your tasks all need to compile your code. This slows down tasks, but also means you need to take care that everyone writes to different places
- We try to paper over this, but the models that we compile on-the-fly may not always play nicely with parallel frameworks (e.g.,
future
) - As your model grows, you will accumulate a number of coupled support functions that go along with it, and a package is the easiest mechanism for supporting this
- Packages are much easier for other people to use than standalone R scripts
- There is nice tooling available to packages for configuring dependencies, tests and other automatic checks (see below).
7.1 A basic package skeleton
Many tools exist to create packages; here we will use usethis
, but there’s nothing magic here - you could create these files by hand if you prefer (see Writing R Extensions for the official guide on R packages if this is the approach you prefer).
<- tempfile()
path ::create_package(
usethis
path,fields = list(Package = "pkg",
Title = "My Odin Model",
Description = "An example odin model"))
#> ✔ Creating '/tmp/RtmpzGtnYb/file1c652ed74a82/'.
#> ✔ Setting active project to "/tmp/RtmpzGtnYb/file1c652ed74a82".
#> ✔ Creating 'R/'.
#> ✔ Writing 'DESCRIPTION'.
#> Package: pkg
#> Title: My Odin Model
#> Version: 0.1.0
#> Authors@R (parsed):
#> * An Author <a.author@example.com> [aut, cre]
#> Description: An example odin model
#> License: CC0
#> Encoding: UTF-8
#> Language: en-GB
#> Roxygen: list(markdown = TRUE)
#> RoxygenNote: 7.3.2
#> ✔ Writing 'NAMESPACE'.
#> ✔ Setting active project to "<no active project>".
::proj_set(path)
usethis#> ✔ Setting active project to "/tmp/RtmpzGtnYb/file1c652ed74a82".
Our package now contains:
::dir_tree(path)
fs#> /tmp/RtmpzGtnYb/file1c652ed74a82
#> ├── DESCRIPTION
#> ├── NAMESPACE
#> └── R
usethis
has created some skeleton files for us. The DESCRIPTION
contains:
Package: pkg
Title: My Odin Model
Version: 0.1.0
Authors@R:
person("An", "Author", , "a.author@example.com", role = c("aut", "cre"))
Description: An example odin model
License: CC0
Encoding: UTF-8
Language: en-GB
Roxygen: list(markdown = TRUE) RoxygenNote: 7.3.2
and the NAMESPACE
file contains
# Generated by roxygen2: do not edit by hand
7.2 Adding odin code
The odin code needs to go within this package in the inst/odin
directory. Code saved as inst/odin/myname.R
will create a generator called myname
. Here, we create a file inst/odin/sir.R
containing the SIR model from Chapter 1:
deriv(S) <- -beta * S * I / N
deriv(I) <- beta * S * I / N - gamma * I
deriv(R) <- gamma * I
initial(S) <- N - I0
initial(I) <- I0
initial(R) <- 0
<- parameter(1000)
N <- parameter(10)
I0 <- parameter(0.2)
beta <- parameter(0.1) gamma
We also need to make some changes to our package:
- We need to include
dust2
as anImports
dependency - We need to include
dust2
,monty
andcpp11
asLinkingTo
dependencies - We need to arrange our package so it loads the shared library that we build
If you run odin_package()
before setting this up, it will error and indicate where the problem lies:
odin_package(path)
#> ℹ Found 1 odin code file in 'inst/odin'
#> ✔ Wrote 'inst/dust/sir.cpp'
#> Error:
#> ! Expected package 'dust2' as 'Imports' in DESCRIPTION
::use_package("dust2", "Imports")
usethis#> ✔ Adding dust2 to 'Imports' field in DESCRIPTION.
#> ☐ Refer to functions with `dust2::fun()`.
::use_package("dust2", "LinkingTo")
usethis#> ✔ Adding dust2 to 'LinkingTo' field in DESCRIPTION.
::use_package("monty", "LinkingTo")
usethis#> ✔ Adding monty to 'LinkingTo' field in DESCRIPTION.
Setting up to use cpp11
is a bit more involved because of the changes that we need to make to ensure that everything links together correctly; for details see the packaging section of the cpp11
“Getting started” vignette
::use_package_doc()
usethis#> ✔ Writing 'R/pkg-package.R'.
#> ☐ Run `devtools::document()` to update package-level documentation.
::use_cpp11()
usethis#> ✔ Creating 'src/'.
#> ✔ Adding "*.o", "*.so", and "*.dll" to 'src/.gitignore'.
#> ✔ Adding "@useDynLib pkg, .registration = TRUE" to 'R/pkg-package.R'.
#> ☐ Run `devtools::document()` to update 'NAMESPACE'.
#> ✔ Adding cpp11 to 'LinkingTo' field in DESCRIPTION.
#> ✔ Writing 'src/code.cpp'.
::document(path)
devtools#> ℹ Updating pkg documentation
#> Writing 'NAMESPACE'
#> ℹ Loading pkg
#> ℹ 1 functions decorated with [[cpp11::register]]
#>
#> ✔ generated file 'cpp11.R'
#>
#> ✔ generated file 'cpp11.cpp'
#>
#> ℹ Re-compiling pkg (debug build)
#> ── R CMD INSTALL ───────────────────────────────────────────────────────────────
#> * installing *source* package ‘pkg’ ...
#> ** using staged installation
#> ** libs
#> using C++ compiler: ‘g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0’
#> g++ -std=gnu++17 -I"/opt/R/4.4.2/lib/R/include" -DNDEBUG -I'/home/runner/work/_temp/Library/cpp11/include' -I'/home/runner/work/_temp/Library/dust2/include' -I'/home/runner/work/_temp/Library/monty/include' -I/usr/local/include -fpic -g -O2 -UNDEBUG -Wall -pedantic -g -O0 -c code.cpp -o code.o
#> g++ -std=gnu++17 -I"/opt/R/4.4.2/lib/R/include" -DNDEBUG -I'/home/runner/work/_temp/Library/cpp11/include' -I'/home/runner/work/_temp/Library/dust2/include' -I'/home/runner/work/_temp/Library/monty/include' -I/usr/local/include -fpic -g -O2 -UNDEBUG -Wall -pedantic -g -O0 -c cpp11.cpp -o cpp11.o
#> g++ -std=gnu++17 -shared -L/opt/R/4.4.2/lib/R/lib -L/usr/local/lib -o pkg.so code.o cpp11.o -L/opt/R/4.4.2/lib/R/lib -lR
#> installing to /tmp/RtmpzGtnYb/devtools_install_1c655f161ec1/00LOCK-file1c652ed74a82/00new/pkg/libs
#> ** checking absolute paths in shared objects and dynamic libraries
#> * DONE (pkg)
#> Writing 'pkg-package.Rd'
::file_delete(file.path(path, "src/code.cpp")) fs
We can now generate our odin code:
odin_package(path)
#> ℹ Found 1 odin code file in 'inst/odin'
#> ℹ 'inst/dust/sir.cpp' is up to date
#> ℹ Working in package 'pkg' at '/tmp/RtmpzGtnYb/file1c652ed74a82'
#> ℹ Found 1 system
#> ✔ Wrote 'src/sir.cpp'
#> ✔ Wrote 'R/dust.R'
#> ✔ Wrote 'src/Makevars'
#> ℹ 13 functions decorated with [[cpp11::register]]
#> ✔ generated file 'cpp11.R'
#> ✔ generated file 'cpp11.cpp'
The package now contains more files:
::dir_tree(path)
fs#> /tmp/RtmpzGtnYb/file1c652ed74a82
#> ├── DESCRIPTION
#> ├── NAMESPACE
#> ├── R
#> │ ├── cpp11.R
#> │ ├── dust.R
#> │ └── pkg-package.R
#> ├── inst
#> │ ├── dust
#> │ │ └── sir.cpp
#> │ └── odin
#> │ └── sir.R
#> ├── man
#> │ └── pkg-package.Rd
#> └── src
#> ├── Makevars
#> ├── code.o
#> ├── cpp11.cpp
#> ├── cpp11.o
#> ├── pkg.so
#> └── sir.cpp
Almost every new file here should not be edited directly (and all contain a line at the start to that effect).
Our DESCRIPTION
now contains
Package: pkg
Title: My Odin Model
Version: 0.1.0
Authors@R:
person("An", "Author", , "a.author@example.com", role = c("aut", "cre"))
Description: An example odin model
License: CC0
Encoding: UTF-8
Language: en-GB
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.3.2
Imports:
dust2
LinkingTo:
cpp11,
dust2, monty
and NAMESPACE
contains
# Generated by roxygen2: do not edit by hand
useDynLib(pkg, .registration = TRUE)
In R/
:
cpp11.R
is the glue code generated bycpp11
dust.R
is glue code generated bydust2
pkg-package.R
was generated byusethis::use_package_doc()
and holds the specialroxygen2
comments that causeddevtools::document()
to write ourNAMESPACE
file (you can edit this file!)
#' @keywords internal
"_PACKAGE"
## usethis namespace: start
#' @useDynLib pkg, .registration = TRUE
## usethis namespace: end
NULL
In inst/dust
, sir.cpp
contains the dust interface for our model (see the “Writing dust2 systems” vignette if you are curious)
In man/
, pkg-package.Rd
contains help files generated by roxygen2
In src/
Makevars
contains code to allow OpenMP to work to parallelise the systemcpp11.cpp
is glue code generated bycpp11
sir.cpp
is the full system code generated bydust2
code.cpp
was added bycpp11
and can be removed- The
.o
and.so
(or.dll
on Windows) files are generated by the compiler
7.3 Development of the package
Once your odin code is in a package, you will want to iterate over it. Previously you might have had scripts that you used source()
on to load into the session. Now your workflow looks like:
- Edit the odin code in
inst/odin
- Run
odin_package()
- Load the package with
pkgload::load_all()
If you edit code in R/
you don’t need to run step 2, and running step 3 is enough.
::load_all(path)
pkgload#> ℹ Loading pkg
#> ℹ 13 functions decorated with [[cpp11::register]]
#>
#> ✔ generated file 'cpp11.R'
#>
#> ✔ generated file 'cpp11.cpp'
#>
#> ℹ Re-compiling pkg (debug build)
#> ── R CMD INSTALL ───────────────────────────────────────────────────────────────
#> * installing *source* package ‘pkg’ ...
#> ** using staged installation
#> ** libs
#> using C++ compiler: ‘g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0’
#> g++ -std=gnu++17 -I"/opt/R/4.4.2/lib/R/include" -DNDEBUG -I'/home/runner/work/_temp/Library/cpp11/include' -I'/home/runner/work/_temp/Library/dust2/include' -I'/home/runner/work/_temp/Library/monty/include' -I/usr/local/include -fopenmp -fpic -g -O2 -UNDEBUG -Wall -pedantic -g -O0 -c cpp11.cpp -o cpp11.o
#> g++ -std=gnu++17 -I"/opt/R/4.4.2/lib/R/include" -DNDEBUG -I'/home/runner/work/_temp/Library/cpp11/include' -I'/home/runner/work/_temp/Library/dust2/include' -I'/home/runner/work/_temp/Library/monty/include' -I/usr/local/include -fopenmp -fpic -g -O2 -UNDEBUG -Wall -pedantic -g -O0 -c sir.cpp -o sir.o
#> g++ -std=gnu++17 -shared -L/opt/R/4.4.2/lib/R/lib -L/usr/local/lib -o pkg.so cpp11.o sir.o -fopenmp -L/opt/R/4.4.2/lib/R/lib -lR
#> installing to /tmp/RtmpzGtnYb/devtools_install_1c6514a93627/00LOCK-file1c652ed74a82/00new/pkg/libs
#> ** checking absolute paths in shared objects and dynamic libraries
#> * DONE (pkg)
If you are using RStudio then Ctrl-Shift-l
will load the package for you.
7.4 Next steps
Once you have a model in a package, then you are within the realms of normal R package development, and nothing here is specific to odin. However, if you are not familiar with package development we hope these tips will be useful.
A good place to look for information on package development is Hadley Wickham and Jenny Bryan’s “R packages” book, and to avoid repeating their material we’ll just link to it:
- Create an RStudio project for your new package
- Write some tests for your model and its support code. We’ll document some basic ideas for this later.
- Create a git repository for your package and put it on GitHub. For more discussion see this chapter in “R packages”, and this guide on configuring R, RStudio and git to work well together.
- Set up continuous integration so that your tests are run automatically when you make changes to your package
- Put your package into an R universe to make it easy for others to install. Your organisation may already have one (for DIDE users, please see this repo), or you can start your own
This looks like a lot of work, but most of the setup here can be configured in a few lines. Writing tests is the only part that requires more than configuration, and that is something that you will tend to get better at over time.