Troubleshooting • orderly2

Outpack files accidentally committed to git

As discussed in the orderly introduction, you do not want to commit any files from .outpack/, drafts/ or archive/ (if used) to git as this will create all sorts of problems down the line.

If you were directed here, it is probably because you have ended up with these files in git and want to undo this situation. The least painful way depends on your situation.

We have now put in guard rails to try and prevent this happening, but it could still happen to you if you modify the .gitignore file or force-add files for example.

Once you are in this situation, orderly2 will shout at you:

orderly2::orderly_run("data")

## Error in `orderly2::orderly_run()`:
## ! Detected 6 outpack files committed to git
## ✖ Detected files were found in '.outpack/' and 'archive/'
## ℹ For tips on resolving this, please see
##   <https://mrc-ide.github.io/orderly2/articles/troubleshooting.html>
## ℹ To turn this into a warning and continue anyway set the option
##   'orderly_git_error_is_warning' to TRUE by running
##   options(orderly_git_error_is_warning = TRUE)

which may have directed you to this very page. If you just want to continue working anyway, then run the suggested command:

options(orderly_git_error_is_warning = TRUE)

after which things will work with a warning the first time that session:

orderly2::orderly_run("data")

## Warning in orderly2::orderly_run("data"): Detected 6 outpack files committed to git
## ✖ Detected files were found in '.outpack/' and 'archive/'
## ℹ For tips on resolving this, please see
##   <https://mrc-ide.github.io/orderly2/articles/troubleshooting.html>
## This warning is displayed once per session.

## ✔ Wrote '.gitignore'

## ℹ Starting packet 'data' `20240913-130535-5805bfe6` at 2024-09-13 13:05:35.346253

## > orderly2::orderly_artefact("data.rds", description = "Final data")
## > saveRDS(mtcars, "data.rds")

## ✔ Finished running data.R

## ℹ Finished 20240913-130535-5805bfe6 at 2024-09-13 13:05:35.382333 (0.03608012 secs)

## [1] "20240913-130535-5805bfe6"

subsequent calls will not display the warning:

orderly2::orderly_run("data")

## ℹ Starting packet 'data' `20240913-130535-777b64a2` at 2024-09-13 13:05:35.469147

## > orderly2::orderly_artefact("data.rds", description = "Final data")
## > saveRDS(mtcars, "data.rds")

## ✔ Finished running data.R

## ℹ Finished 20240913-130535-777b64a2 at 2024-09-13 13:05:35.490855 (0.02170777 secs)

## [1] "20240913-130535-777b64a2"

The rest of this section discusses how you might permanently fix the issue.

I don’t care about my history at all

This is the case if you have just started a project, and are not yet collaborating on it with anyone else (or if that person is willing to re-clone their sources). The simplest thing to do is:

Delete the .git directory entirely
Run orderly2::orderly_gitignore_update("(root)") to set up a reasonable .gitignore that will prevent this situation happening again
Run git add . to add everything back in (review this with git status to make sure you’re happy)
Run git commit -m "Initial commit" to create a single commit that contains all the files in currently in your repo with no history, and also with no .outpack files

If you have previously pushed this repo to GitHub or similar then you will need to set that up again

git remote add origin https://github.com/user/repo (replacing user/repo with your path, or using git@github.com:user/repo if you use ssh to talk with GitHub)
git branch -M main assuming you are using main for your default branch, which is now most common
git push --force -u origin main

Note that this is destructive and will require coordination with any collaborators as you have changed history.

I just want this to go away and nothing I have committed is very large

If you do care about your history, but you also have only committed a few files (e.g., you have committed files from .outpack/ which are small but not a large 100MB file in archive/ that is preventing you pushing to GitHub) then you could just delete the offending files from git without updating the history, or affecting your local copies.

Run git rm --cached .outpack (repeating with draft and archive as needed)
Run orderly2::orderly_gitignore_update("(root)") to set up a reasonable .gitignore that will prevent this situation happening again
Run git add .gitignore to also stage this
Run git commit -m "Delete unwanted outpack files"

You can then push this without any issues.

I care about my history but want this stuff gone

If you are working on a branch, and the unwanted files were committed on that branch, the simplest thing to do is to copy the changes you have made somewhere safe, create a new branch against the current main and copy those changes over there. You could do this somewhat automatically by generating and applying a patch:

git diff -- src > changes.patch
git checkout main
git checkout -b changes-attempt2
git apply changes.patch
git push -u origin changes-attempt2

If the unwanted files have been committed onto your default branch, then you will have to do some potentially gory history rewriting. See this StackOverflow question, the git docs and the currently recommended tool for doing this. Good luck!