Retry a task

Retry one or more tasks. This creates a new task that copies the work of the old one. Most of the time this is transparent. We'll document this in the "advanced" vignette once it's written.

Usage

task_retry(id, driver = NULL, resources = NULL, root = NULL)

Arguments

id

The identifier or identifiers of tasks to retry.

driver

Name of the driver to use to submit the task. The default (NULL) depends on your configured drivers; if you have no drivers configured no submission happens (or indeed is possible). If you have exactly one driver configured we'll submit your task with it. If you have more than one driver configured, then we will error, though in future versions we may fall back on a default driver if you have one configured. If you pass FALSE here, submission is prevented even if you have no driver configured.

resources

A list generated by hipercow_resources giving the cluster resource requirements to run your task.

root

A hipercow root, or path to it. If NULL we search up your directory tree.

Value

New identifiers for the retried tasks

Details

This ends up being a little more complicated than ideal in order to keep things relatively fast, while keeping our usual guarantees about race conditions etc. Basically; retrying is the only way a task can move out of a terminal state but it still does not modify the existing task. Instead, we keep a separate register of whether a task has been retried or not. Each time we retry we write into this register. When you query about the status etc of a task you can then add a follow argument to control whether or not we check the register. We assume that you never call this in parallel; if you do then retries may be lost. You can run task_retry(NULL) to refresh the cached copy of the retry map if you need to.

Examples

cleanup <- hipercow_example_helper()
#> ℹ This example uses a special helper

# For demonstration, we just generate random numbers as then it's
# more obvious that things have been rerun:
id1 <- task_create_expr(runif(1))
#> ✔ Submitted task 'e266f83c7c59e0034b36d45ef4e1dda9' using 'example'
task_wait(id1)
#> [1] TRUE
task_result(id1)
#> [1] 0.460328

# Now retry the task and get the retried result:
id2 <- task_retry(id1)
#> ✔ Submitted task '116387fcb2f6a64fca90aff275a800c8' using 'example'
task_wait(id2)
#> [1] TRUE
task_result(id2)
#> [1] 0.4039083

# After a retry, both the original and derived tasks know about
# each other:
task_info(id1)
#> 
#> ── task e266f83c7c59e0034b36d45ef4e1dda9 (success) ─────────────────────────────
#> ℹ Submitted with 'example'
#> ℹ Task type: expression
#>   • Expression: runif(1)
#>   • Locals: (none)
#>   • Environment: default
#>     R_GC_MEM_GROW: 3
#> ℹ Created at 2024-08-16 20:23:06.714589 (moments ago)
#> ℹ Started at 2024-08-16 20:23:06.893481 (moments ago; waited 179ms)
#> ℹ Finished at 2024-08-16 20:23:06.981577 (moments ago; ran for 89ms)
#> ℹ 1st of a chain of a task retried 1 time, most recently '116387fcb2f6a64fca90aff275a800c8'
task_info(id2)
#> 
#> ── task 116387fcb2f6a64fca90aff275a800c8 (success) ─────────────────────────────
#> ℹ Submitted with 'example'
#> ℹ Task type: expression
#>   • Expression: runif(1)
#>   • Locals: (none)
#>   • Environment: default
#>     R_GC_MEM_GROW: 3
#> ℹ Created at 2024-08-16 20:23:06.714589 (moments ago)
#> ℹ Started at 2024-08-16 20:23:07.786233 (moments ago; waited 1.1s)
#> ℹ Finished at 2024-08-16 20:23:07.787101 (moments ago; ran for 1ms)
#> ℹ Last of a chain of a task retried 1 time

# By default every task will "follow" and access the most recent
# task in the chain:
task_result(id1) == task_result(id2)
#> [1] TRUE

# You can prevent this by passing follow = FALSE to get the value
# of this particular attempt:
task_result(id1, follow = FALSE)
#> [1] 0.460328

# Tasks can be retried as many times as needed, creating a
# chain. It does not matter which task you retry as we always
# follow all the way to the end of the chain before retrying:
id3 <- task_retry(id1)
#> ✔ Submitted task '85b9fc71ee9510a3ac73c63c6d4cdd0c' using 'example'
task_info(id1, follow = FALSE)
#> 
#> ── task e266f83c7c59e0034b36d45ef4e1dda9 (submitted) ───────────────────────────
#> ℹ Submitted with 'example'
#> ℹ Task type: expression
#>   • Expression: runif(1)
#>   • Locals: (none)
#>   • Environment: default
#>     R_GC_MEM_GROW: 3
#> ℹ Created at 2024-08-16 20:23:06.714589 (moments ago)
#> ℹ Started at 2024-08-16 20:23:06.976878 (moments ago; waited 263ms)
#> ! Not finished yet (waiting to start)
#> ℹ 1st of a chain of a task retried 2 times, most recently '85b9fc71ee9510a3ac73c63c6d4cdd0c'
task_info(id3)
#> 
#> ── task 85b9fc71ee9510a3ac73c63c6d4cdd0c (success) ─────────────────────────────
#> ℹ Submitted with 'example'
#> ℹ Task type: expression
#>   • Expression: runif(1)
#>   • Locals: (none)
#>   • Environment: default
#>     R_GC_MEM_GROW: 3
#> ℹ Created at 2024-08-16 20:23:06.714589 (moments ago)
#> ℹ Started at 2024-08-16 20:23:08.892922 (moments ago; waited 2.2s)
#> ℹ Finished at 2024-08-16 20:23:08.893866 (moments ago; ran for 1ms)
#> ℹ Last of a chain of a task retried 2 times

cleanup()
#> ℹ Cleaning up example