Linting crashes when file deleted but not staged

[reganhayward]

Linting issue with 'push_dockerhub.yml'

When linting, I get the following error:

[✗] 1 Test Failed                                                                                                  │
│ https://nf-co.re/errors#1: File must be removed: .github/workflows/push_dockerhub.yml

So when I remove the file, I get this:

FileNotFoundError: [Errno 2] No such file or directory: './.github/workflows/push_dockerhub.yml'

Tried running version 1.11, 1.12 and 1.13.dev0 of nf-core tools, but they all give the same errors

Mentioned this to Phil Ewels on Slack and he suggested raising a bug report.

• This test goes through all pipeline files looking for comments. It gets that list of files by doing git ls-files - this is so that it ignores all the stuff that’s in the pipeline directory that should be ignored (temporary files, cache files, other stuff mentioned in .gitignore)
• But, if you did rm instead of git rm, then git will still think that the file is there, so then it tries to open the file and fails when it doesn’t exist
• If you do git add . then you’ll stage the deletion and I think it should work again
• Possibly you’ll need to commit the change
• Would be easy to add an extra check / handle the error if the file doesn’t exist

Full output

nf-core lint .

                                          ,--./,-.
          ___     __   __   __   ___     /,-._.--~\
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'

    nf-core/tools version 1.13.dev0



INFO      Testing pipeline: .                                                                              lint.py:201
╭──────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────╮
│ /home/r/miniconda3/bin/nf-core:11 in <module>                                                                      │
│                                                                                                                    │
│ /home/r/miniconda3/lib/python3.7/site-packages/nf_core-1.13.dev0-py3.7.egg/nf_core/__main__.py:56 in run_nf_core   │
│                                                                                                                    │
│    53 │   stderr.print("\n\n")                                                                                     │
│    54 │                                                                                                            │
│    55 │   # Lanch the click cli                                                                                    │
│ ❱  56 │   nf_core_cli()                                                                                            │
│    57                                                                                                              │
│    58                                                                                                              │
│    59 # Customise the order of subcommands for --help                                                              │
│                                                                                                                    │
│ /home/r/miniconda3/lib/python3.7/site-packages/click/core.py:829 in __call__                                       │
│                                                                                                                    │
│    826 │                                                                                                           │
│    827 │   def __call__(self, *args, **kwargs):                                                                    │
│    828 │   │   """Alias for :meth:`main`."""                                                                       │
│ ❱  829 │   │   return self.main(*args, **kwargs)                                                                   │
│    830                                                                                                             │
│    831                                                                                                             │
│    832 class Command(BaseCommand):                                                                                 │
│                                                                                                                    │
│ /home/r/miniconda3/lib/python3.7/site-packages/click/core.py:782 in main                                           │
│                                                                                                                    │
│    779 │   │   try:                                                                                                │
│    780 │   │   │   try:                                                                                            │
│    781 │   │   │   │   with self.make_context(prog_name, args, **extra) as ctx:                                    │
│ ❱  782 │   │   │   │   │   rv = self.invoke(ctx)                                                                   │
│    783 │   │   │   │   │   if not standalone_mode:                                                                 │
│    784 │   │   │   │   │   │   return rv                                                                           │
│    785 │   │   │   │   │   # it's not safe to `ctx.exit(rv)` here!                                                 │
│                                                                                                                    │
│ /home/r/miniconda3/lib/python3.7/site-packages/click/core.py:1259 in invoke                                        │
│                                                                                                                    │
│   1256 │   │   │   │   Command.invoke(self, ctx)                                                                   │
│   1257 │   │   │   │   sub_ctx = cmd.make_context(cmd_name, args, parent=ctx)                                      │
│   1258 │   │   │   │   with sub_ctx:                                                                               │
│ ❱ 1259 │   │   │   │   │   return _process_result(sub_ctx.command.invoke(sub_ctx))                                 │
│   1260 │   │                                                                                                       │
│   1261 │   │   # In chain mode we create the contexts step by step, but after the                                  │
│   1262 │   │   # base command has been invoked.  Because at that point we do not                                   │
│                                                                                                                    │
│ /home/r/miniconda3/lib/python3.7/site-packages/click/core.py:1066 in invoke                                        │
│                                                                                                                    │
│   1063 │   │   """                                                                                                 │
│   1064 │   │   _maybe_show_deprecated_notice(self)                                                                 │
│   1065 │   │   if self.callback is not None:                                                                       │
│ ❱ 1066 │   │   │   return ctx.invoke(self.callback, **ctx.params)                                                  │
│   1067                                                                                                             │
│   1068                                                                                                             │
│   1069 class MultiCommand(Command):                                                                                │
│                                                                                                                    │
│ /home/r/miniconda3/lib/python3.7/site-packages/click/core.py:610 in invoke                                         │
│                                                                                                                    │
│    607 │   │   args = args[2:]                                                                                     │
│    608 │   │   with augment_usage_errors(self):                                                                    │
│    609 │   │   │   with ctx:                                                                                       │
│ ❱  610 │   │   │   │   return callback(*args, **kwargs)                                                            │
│    611 │                                                                                                           │
│    612 │   def forward(*args, **kwargs):  # noqa: B902                                                             │
│    613 │   │   """Similar to :meth:`invoke` but fills in default keyword                                           │
│                                                                                                                    │
│ /home/r/miniconda3/lib/python3.7/site-packages/nf_core-1.13.dev0-py3.7.egg/nf_core/__main__.py:306 in lint         │
│                                                                                                                    │
│   303 │   """                                                                                                      │
│   304 │                                                                                                            │
│   305 │   # Run the lint tests!                                                                                    │
│ ❱ 306 │   lint_obj = nf_core.lint.run_linting(pipeline_dir, release, show_passed, markdown,                        │
│       json)                                                                                                        │
│   307 │   if len(lint_obj.failed) > 0:                                                                             │
│   308 │   │   sys.exit(1)                                                                                          │
│   309                                                                                                              │
│                                                                                                                    │
│ /home/r/miniconda3/lib/python3.7/site-packages/nf_core-1.13.dev0-py3.7.egg/nf_core/lint.py:61 in run_linting       │
│                                                                                                                    │
│     58 │                                                                                                           │
│     59 │   # Run the linting tests                                                                                 │
│     60 │   try:                                                                                                    │
│ ❱   61 │   │   lint_obj.lint_pipeline(release_mode)                                                                │
│     62 │   except AssertionError as e:                                                                             │
│     63 │   │   log.critical("Critical error: {}".format(e))                                                        │
│     64 │   │   log.info("Stopping tests...")                                                                       │
│                                                                                                                    │
│ /home/r/miniconda3/lib/python3.7/site-packages/nf_core-1.13.dev0-py3.7.egg/nf_core/lint.py:240 in lint_pipeline    │
│                                                                                                                    │
│    237 │   │   │   for fun_name in check_functions:                                                                │
│    238 │   │   │   │   progress.update(lint_progress, advance=1, func_name=fun_name)                               │
│    239 │   │   │   │   log.debug("Running lint test: {}".format(fun_name))                                         │
│ ❱  240 │   │   │   │   getattr(self, fun_name)()                                                                   │
│    241 │   │   │   │   if len(self.failed) > 0:                                                                    │
│    242 │   │   │   │   │   log.error("Found test failures in `{}`, halting lint                                    │
│        run.".format(fun_name))                                                                                     │
│    243 │   │   │   │   │   break                                                                                   │
│                                                                                                                    │
│ /home/r/miniconda3/lib/python3.7/site-packages/nf_core-1.13.dev0-py3.7.egg/nf_core/lint.py:1244 in                 │
│ check_cookiecutter_strings                                                                                         │
│                                                                                                                    │
│   1241 │   │   num_files = 0                                                                                       │
│   1242 │   │   for fn in list_of_files:                                                                            │
│   1243 │   │   │   num_files += 1                                                                                  │
│ ❱ 1244 │   │   │   with io.open(fn, "r", encoding="latin1") as fh:                                                 │
│   1245 │   │   │   │   lnum = 0                                                                                    │
│   1246 │   │   │   │   for l in fh:                                                                                │
│   1247 │   │   │   │   │   lnum += 1                                                                               │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
FileNotFoundError: [Errno 2] No such file or directory: './.github/workflows/push_dockerhub.yml'

[ewels]

Fixed in #804