QUESTION: how do you measure the complexity of a block of code or blocks of code in the entire notebook?

[neomatrix369]

I'm raising this just to discuss if there is a way to measure complexity of Python (or other language supported by a notebook)?

There's https://en.wikipedia.org/wiki/Cyclomatic_complexity that many languages have libraries for, I'm thinking there is one for Python as well.

In case your other analysis already covers this then please close this. Otherwise, it would be a helpful feature to have.

Maybe related to #259

[MarcoGorelli]

Hi @neomatrix369 - thanks for opening this issue!

nbQA simply applies any standard Python code quality tool to a Jupyter Notebook - if you have a Python tool which calculates complexity, then using nbQA it should be possible to apply it to a notebook.

Do you know of any Python tool for calculating complexity?

[neomatrix369]

Hi @neomatrix369 - thanks for opening this issue!

nbQA simply applies any standard Python code quality tool to a Jupyter Notebook - if you have a Python tool which calculates complexity, then using nbQA it should be possible to apply it to a notebook.

Do you know of any Python tool for calculating complexity?

Cool so it's configurable! Nice!

I came across Radon but there might be others. As long as we can get some score it helps.

[girip11]

You might find this useful Radon usage with Jupyter Notebooks

[neomatrix369]

You might find this useful Radon usage with Jupyter Notebooks

Thanks for the response but I raised this question, I was inquiring to see if this would/could be a part of the QA checks already performed by nbQA! I can run these separately but an option to do it via the tool itself can be helpful.

Any thoughts on this?

[MarcoGorelli]

Thanks for the suggestion @neomatrix369 !

If radon already supports Jupyter Notebooks out of the box, then IMO we don't need to support it here.

nbQA is meant as an adapter which allows Python-only tools to be run on Jupyter Notebooks - ideally, all the tools I love using (e.g. pylint, flake8, pyupgrade, etc.) would work out-of-the-box on Jupyter Notebooks, but until they do, I'm stuck running them via nbQA :)

Anyway, I'm grateful for the suggestion, but as radon is a rare example of a tool which works automatically on Jupyter Notebooks, I think we should consider it out-of-scope

[s-weigand]

Maybe extending the aim of the project from:

nbQA is meant as an adapter which allows Python-only tools to be run on Jupyter Notebooks

To:

A Tool wrapper to allow using your favorite Python QA tools with jupyter notebooks

Might be worth a thought.

I.e. for tools that already support jupyter notebooks like radon, nbQA just skip the parsing+change file steps and simply call the tool with the appropriate flag.

The big con with an approach like that, is that nbQA will get issues, caused by another tool, causing issue redirect work for the devs.

[MarcoGorelli]

Thanks @s-weigand for the suggestion! Your idea makes sense, but I don't have much time to work on this project and would prefer to not expand the scope too much - if a tool already supports Jupyter Notebooks out-of-the-box, IMO there's no need to duplicate work.

Closing for now as out-of-scope then, but I'd like to re-iterate that I really appreciate the suggestions made here

[MarcoGorelli]

@all-contributors please add @neomatrix369 for ideas

[allcontributors]

@MarcoGorelli

I've put up a pull request to add @neomatrix369! 🎉

[neomatrix369]

Here are some formatting related resources I found, which may help with our discussions (the ones I raised previously):

• Python Zero to Hero - Ep.12 - Python linting and auto-formating: https://dev.to/cheukting_ho/python-zero-to-hero-ep-12-python-linting-and-auto-formating-p55
• Joe Gordon - Syntax Trees and Python - Automated Code Transformations - PyCon 2019: https://www.youtube.com/watch?v=viNzD1zD-Fg
• Microsoft Research • Gather is a notebook cleaning tool which use a dependency graph to analyzes and determines the necessary code within a notebook and performs code cleanup, automating this difficult, annoying, and time-consuming task.
  https://www.linkedin.com/posts/philipvollet_python-python3-analyst-activity-6693389309015412736-yyJq
  VSCode Extension https://lnkd.in/gQVZQuN
  Blogpost https://lnkd.in/gxJiPni
• The Uncompromising Python Code Formatter also for your Jupyter Notebook!: https://www.linkedin.com/posts/philipvollet_python-jupyternotebook-programming-activity-6722049722435158017-3XuZ

[MarcoGorelli]

Thanks @neomatrix369 !

flake8, autoflake, black and isort.

We already have support for flake8, black, and isort, but we should add autoflake as it can be pretty useful

[neomatrix369]

@MarcoGorelli I updated the above with more links

[neomatrix369]

Suggestion

It's me again, this time I found something that will help our code base itself, I have installed @sourcery-ai as a github action, see the report it generates neomatrix369/nlp_profiler#27 (comment), pretty cool hey!

Maybe some or all of the features of @sourcery-ai could also be integrated into nbQA itself (just another thought), and let that tool do the heavy-lifting.

[MarcoGorelli]

Wow, nice, thanks for bringing this to our attention! Do you want to add it as a github action here?

BTW pylint issue is resolved if you want to update your caching PR

[neomatrix369]

Wow, nice, thanks for bringing this to our attention! Do you want to add it as a github action here?

BTW pylint issue is resolved if you want to update your caching PR

Yes it's worth adding it as a github action, let me do it. I know the two founders from my past endeavours - I'm testing their offerings and also reporting with feedback.

Also look at this one https://github.com/sourcery-ai/python-best-practices-cookiecutter - lots of things to look at and use to verify our own flow

[neomatrix369]

I just updated my branch, lets see what happens!

[neomatrix369]

@MarcoGorelli just requested Sourcery-ai to be installed on nbQA-dev, one of the owners/admin users will need to approve it and then we have it integrated as part of the workflow just like it has done for my repo.

[neomatrix369]

Adding one more to the list https://www.codeac.io/ - its things Sourcery-ai does not do I have already added to my repo (I know the devs to this one as well)

[MarcoGorelli]

Cool, thanks for all these resources you've been linking, they'll be really useful!

[neomatrix369]

Adding one more to the list https://www.codeac.io/ - its things Sourcery-ai does not do I have already added to my repo (I know the devs to this one as well)

Their analysis is very comprehensive, they are basing it on their own tools and also external ones I think. Have a place to see what you think about it.

[neomatrix369]

Did I share this ever?
https://github.com/analysis-tools-dev/static-analysis/blob/master/README.md#python

It's highly likely you have covered a bunch of them and others might be worth looking into as well when time permits.

[MarcoGorelli]

that's a lot of tools, thanks! I'd like to see how many we can support