Runtime error for certain combinations of notebook names

[ivanmkc]

I have two notebooks:

• [UJ.1].ipynb
• [UJ.1 NEW].ipynb

For some reason, if you run them in the same nbqa command:

nbqa black \[UJ.1\].ipynb \[UJ.1\ NEW\].ipynb

You get the following error:

Traceback (most recent call last):
  File "/env2/lib/python3.8/site-packages/nbqa/__main__.py", line 618, in _run_on_one_root_dir
    REPLACE_FUNCTION[configs.nbqa_diff](
  File "/env2/lib/python3.8/site-packages/nbqa/replace_source.py", line 180, in mutate
    cell["source"] = _get_new_source(code_cell_number, notebook_info, next(pycells))
StopIteration

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/env2/bin/nbqa", line 8, in <module>
    sys.exit(main())
  File "/env2/lib/python3.8/site-packages/nbqa/__main__.py", line 691, in main
    output_code = _run_on_one_root_dir(cli_args, configs, project_root)
  File "/env2/lib/python3.8/site-packages/nbqa/__main__.py", line 624, in _run_on_one_root_dir
    raise RuntimeError(
RuntimeError: Error reconstructing [UJ.1 NEW].ipynb

From debugging, if you change the name of [UJ.1].ipynb to [UJ.1.ipynb, it works fine.

Also, if you run them one at a time, it works fine.

The contents of the notebook matters. I tried with trivial content and it passed. I think there is some issue with the combination of the filenames and the replacement function.

Archive.zip

[MarcoGorelli]

Hi @ivanmkc

Thanks for your report!

I couldn't reproduce this though, I ran

nbqa black \[UJ.1\].ipynb \[UJ.1\ NEW\].ipynb --nbqa-diff

and it ran fine.

Could you provide some more diagnostics please, and/or try to come up with a smaller reproducible example?

[MarcoGorelli]

Hi @ivanmkc - could you provide some more info please? I can't reproduce

[ivanmkc]

Sorry, busy with some deadlines. Let me retry again.

[MarcoGorelli]

no hurry 😄

[ivanmkc]

Try these two notebooks.

I ran: nbqa black \[UJ.1\ NEW\]\ AutoML\ Vision\ Image\ Classification\ \(1\).ipynb \[UJ.1\ OLD\]\ AutoML\ Vision\ Image\ Classification\ \(1\).ipynb

Archive.zip

nbqa version is nbqa==0.6.0

Error was:

Traceback (most recent call last):
  File "/Users/ivanmkc/Documents/code/ai-platform-samples/env2/lib/python3.8/site-packages/nbqa/__main__.py", line 618, in _run_on_one_root_dir
    REPLACE_FUNCTION[configs.nbqa_diff](
  File "/Users/ivanmkc/Documents/code/ai-platform-samples/env2/lib/python3.8/site-packages/nbqa/replace_source.py", line 180, in mutate
    cell["source"] = _get_new_source(code_cell_number, notebook_info, next(pycells))
StopIteration

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "env2/bin/nbqa", line 8, in <module>
    sys.exit(main())
  File "env2/lib/python3.8/site-packages/nbqa/__main__.py", line 691, in main
    output_code = _run_on_one_root_dir(cli_args, configs, project_root)
  File "env2/lib/python3.8/site-packages/nbqa/__main__.py", line 624, in _run_on_one_root_dir
    raise RuntimeError(
RuntimeError: Error reconstructing [UJ.1 NEW] AutoML Vision Image Classification (1).ipynb
Please report a bug at https://github.com/nbQA-dev/nbQA/issues 

If you run them separately, it strangely works fine. I wonder if it's confusing the two files when it attempts reconstructing them.

Appreciate the help.

[MarcoGorelli]

Thanks @ivanmkc - I presume you also have some configuration in pyproject.toml - could you show that please?

I'm not seeing any error here either. This works fine for me:

$ nbqa black \[UJ.1\ NEW\]\ AutoML\ Vision\ Image\ Classification\ \(1\).ipynb  --nbqa-diff

[MarcoGorelli]

Sorry, I hand't used your full command - now I can reproduce the bug!

Traceback (most recent call last):
  File "/home/marco/nbQA-dev/nbqa/__main__.py", line 618, in _run_on_one_root_dir
    REPLACE_FUNCTION[configs.nbqa_diff](
  File "/home/marco/nbQA-dev/nbqa/replace_source.py", line 239, in diff
    new_source = _get_new_source(code_cell_number, notebook_info, next(pycells))
StopIteration

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/marco/nbQA-dev/venv/bin/nbqa", line 33, in <module>
    sys.exit(load_entry_point('nbqa', 'console_scripts', 'nbqa')())
  File "/home/marco/nbQA-dev/nbqa/__main__.py", line 691, in main
    output_code = _run_on_one_root_dir(cli_args, configs, project_root)
  File "/home/marco/nbQA-dev/nbqa/__main__.py", line 624, in _run_on_one_root_dir
    raise RuntimeError(
RuntimeError: Error reconstructing [UJ.1 NEW] AutoML Vision Image Classification (1).ipynb
Please report a bug at https://github.com/nbQA-dev/nbQA/issues 

(venv) marco@marco-Predator-PH315-52:~/nbQA-dev$ 

Thank you so much, I'll look into it!

[MarcoGorelli]

@all-contributors please add @ivanmkc for bugs

[allcontributors]

@MarcoGorelli

I've put up a pull request to add @ivanmkc! 🎉

[ivanmkc]

Wow that was quick! Will take a look.

[ivanmkc]

Oh I see, added as a contributor. Thanks!

[ivanmkc]

I just integrated this into our automated testing at https://github.com/GoogleCloudPlatform/ai-platform-samples btw. Works great!

I removed the pre-commit integration as this bug was blocking but may consider adding it again.

[MarcoGorelli]

Ooh, I see the issue now!

(Pdb) new_parent / new_stem
PosixPath('[UJ.1 NEW] AutoML Vision Image Classification (1)_93044693')
(Pdb) (new_parent / new_stem).with_suffix('.py')
PosixPath('[UJ.py')

That's it, I'm ditching pathlib forever now, gonna stick to os.path from now on 😆

[MarcoGorelli]

@ivanmkc thanks for your excellent report - have released 0.6.1, would appreciate it if you could confirm if it's fixed for you!

[ivanmkc]

Just tested it and it works fine. Thanks for looking into it so quickly.