Build a parameter packer

Build a parameter packer, which can be used in models to translate between an unstructured vector of numbers (the vector being updated by an MCMC for example) to a structured list of named values, which is easier to program against. We refer to the process of taking a named list of scalars, vectors and arrays and converting into a single vector "packing" and the inverse "unpacking".

Usage

monty_packer(scalar = NULL, array = NULL, fixed = NULL, process = NULL)

Arguments

scalar: Names of scalar parameters. This is similar for listing elements in array with values of 1, though elements in scalar will be placed ahead of those listed in array within the final parameter vector, and elements in array will have generated names that include square brackets.
array: A list, where names correspond to the names of array parameters and values correspond to the lengths of parameters. Multiple dimensions are allowed (so if you provide an element with two entries these represent dimensions of a matrix). Zero-length integer vectors or NULL values are counted as scalars, which allows you to put scalars at positions other than the front of the packing vector. In future, you may be able to use strings as values for the lengths, in which case these will be looked for within fixed.
fixed: A named list of fixed parameters; these will be added into the final list directly. These typically represent additional pieces of data that your model needs to run, but which you are not performing inference on.
process: An arbitrary R function that will be passed the final assembled parameter list; it may create any additional entries, which will be concatenated onto the original list. If you use this you should take care not to return any values with the same names as entries listed in scalar, array or fixed, as this is an error (this is so that pack() is not broken). We will likely play around with this process in future in order to get automatic differentiation to work.

Value

An object of class monty_packer, which has three elements:

parameters: a character vector of computed parameter names; these are the names that your statistical model will use.
unpack: a function that can unpack an unstructured vector (say, from your statistical model parameters) into a structured list (say, for your generative model)
pack: a function that can pack your structured list of parameters back into a numeric vector suitable for the statistical model. This ignores values created by a preprocess function.
index: a function which produces a named list where each element has the name of a value in parameters and each value has the indices within an unstructured vector where these values can be found.

Details

There are several places where it is most convenient to work in an unstructured vector:

An MCMC is typically discussed as a the updating of some vector x to another x'
An optimisation algorithm will try and find a set of values for a vector x that minimises (or maximises) some function f(x)
An ode solver works with a vector x(t) (x at time t) and considers x(t + h) by computing the vector of derivatives dx(t)/dt

In all these cases, the algorithm that needs the vector of numbers knows nothing about what they represent. Commonly, these will be a packed vector of parameters. So our vector x might actually represent the parameters a, b and c in a vector as [a, b, c] - this is a very common pattern, and you have probably implemented this yourself.

In more complex settings, we might want our vector x to collect more structured quantities. Suppose that you are fitting a model with an age-structured or sex-structured parameter. Rather than having a series of scalars packed into your vector x you might have a series of values destined to be treated as a vector:

| 1  2  3  4  5  6  7  |
| a  b  c  d1 d2 d3 d4 |

So here we might have a vector of length 7, where the first three elements will represent be the scalar values a, b and c but the next four will be a vector d.

Unpacked, this might be written as:

list(a = 1, b = 2, c = 3, d = 4:7)

The machinery here is designed to make these transformations simple and standardised within monty, and should be flexible enough for many situations. We will also use these from within dust2 and odin2 for transformations in and out of vectors of ODE state.

When to use `process`

The process function is a get-out-of-jail function designed to let you do arbitrary transformations when unpacking a vector. In general, this should not be the first choice to use because it is less easy to reason about by other tooling (for example, as we develop automatic differentiation support for use with the HMC algorithm, a process function will be problematic because we will need to make sure we can differentiate this process). However, there are cases where it will be only way to achieve some results.

Imagine that you are packing a 2x2 covariance matrix into your vector in order to use within an MCMC or optimisation algorithm. Ultimately, our unpacked vector will need to hold four elements (b11, b12, b21, b22), but there are only three distinct values as the two off-diagonal elements will be the same (i.e., b12 == b21``). So we might write this passing in b_raw = 3toarray, so that our unpacked list holds b_raw = c(b11, b12, b22). We would then write process` as something like:

process <- function(x) {
  list(b = matrix(x$b_raw[c(1, 2, 2, 3)], 2, 2))
}

which creates the symmetric 2x2 matrix b from b_raw.

Unpacking matrices

If you do not use fixed or process when defining your packer, then you can use $unpack() with a matrix or higher-dimensional output. There are two ways that you might like to unpack this sort of output. Assume you have a matrix m with 3 rows and 2 columns; this means that we have two sets of parameters or state (one per column) and 3 states within each; this is the format that MCMC parameters will be in for example.

The first would to be return a list where the ith element is the result of unpacking the ith parameter/state vector. You can do this by running

apply(m, 2, p$unpack)

The second would be to return a named list with three elements where the ith element is the unpacked version of the ith state. In this case you can pass the matrix directly in to the unpacker:

p$unpack(m)

When you do this, the elements of m will acquire an additional dimension; scalars become vectors (one per set), vectors become matrices (one column per set) and so on.

This approach generalises to higher dimensional input, though we suspect you'll spend a bit of time head-scratching if you use it.