Support exit codes

[nilsvu]

Proposed changes

Define exit code 2 as "program incomplete, continue from checkpoint". This can be used to resubmit the job (working on that).

Upgrade instructions

Code review checklist

• The code is documented and the documentation renders correctly. Run
  make doc to generate the documentation locally into BUILD_DIR/docs/html.
  Then open index.html.
• The code follows the stylistic and code quality guidelines listed in the
  code review guide.
• The PR lists upgrade instructions and is labeled bugfix or
  new feature if appropriate.

Further comments

[nilsdeppe]

Why not have the values be hard-coded in an enum and then convert them to an int where we need them to be an int? I worry that the current approach requires us to match something in this documentation block. E.g.

enum class ExitCode : int {
  Complete = 0,
  Abort = 1,
  ContinueFromCheckpoint = 2
};

and then

static_cast<int>(ExitCode::ContinueFromCheckpoint)

If you choose to make the enum not an enum class you can just do an implicit cast.

[kidder]

I would also prefer using enums in this manner...

[nilsvu]

ok done!

[nilsdeppe]

Nice :D