Add functionality for ignoring notebooks

[natanlao]

Closes #1. Allows ignoring notebooks with a .treonignore file. It mimics how .gitignore files work, except a little simpler.

[amit1rrr]

Thanks for your contribution @natanlao

We can actually simplify this quite a bit. From requirements perspective, we don't need to support multiple .treonignore files, just one file at the base of test PATH specified to treon should suffice. What we actually need though is to be able to specify individual file paths to ignore e.g. run all tests except ignore this scratch notebook.

With these requirements in mind we can simply implement the following,

1. read all entries in a single .treonignore file (if present at base of test PATH)
2. From our list of notebooks to test, remove the notebooks whose path overlaps with any entry in our ignore list

Edit: Updated my own suggestions.

[natanlao]

From requirements perspective, we don't need to support multiple .treonignore files, just one file at the base of test PATH specified to treon should suffice.

I'm not sure that only looking for a single .treonignore file would make this code materially simpler. I think that both approaches are similar in terms of both complexity and LOC. That said, I'm happy to change it if you still think it's a good idea.

What we actually need though is to be able to specify individual file paths to ignore e.g. run all tests except ignore this scratch notebook.

That works!

treon/tests/test_ignore.py

Line 30 in 833ccde

assert TEST_NOTEBOOK_PATH.joinpath('basic.ipynb').samefile(ignored[0])

[amit1rrr]

@natanlao I'm sorry I haven't been able to find free time to review this properly. I want to clone your changes and try it out locally for a meaningful review. I will get to it early next week. Sorry again for the delay.

[natanlao]

@amit1rrr All good! Thanks for your time.

[amit1rrr]

Ignore paths are not working recursively.

E.g. Even if I have following rule in .treonignore
site/en/r2/*

all these following files are not-ignored

/workspace/treon/tmp/docs/site/en/r2/tutorials/eager/custom_layers.ipynb
/workspace/treon/tmp/docs/site/en/r2/tutorials/load_data/images.ipynb
...

[natanlao]

Ah, good catch. It looks like pathlib isn't working predictably...

[amit1rrr]

This is still not working for me.

I have following rules in .treonignore -

site/en/r2/*
site/en/r2/tutorials/load_data/tf_records.ipynb

and the following files are still not ignored

'/workspace/treon/tmp/docs/site/en/r2/tutorials/eager/custom_layers.ipynb', '/workspace/treon/tmp/docs/site/en/r2/tutorials/load_data/images.ipynb', '/workspace/treon/tmp/docs/site/en/r2/tutorials/load_data/tf_records.ipynb'

[natanlao]

Thanks for the review @amit1rrr! I've addressed your comments and added tests cases where appropriate.

[amit1rrr]

I tried it again. Somehow the ignore list is still not working for me. Posted example in a separate comment.

I honestly think that this approach is getting too complicated in our quest to be identical as .gitignore (which we don't need to be). How about we simply read a single .treonignore file at base of the path. And exclude that rule list from notebooks_to_test by simply comparing whether the notebook path starts with any rule (if yes, ignore).

Some benefits over current approach,

• Our ignore file can accept absolute paths (currently absolute paths are not supported I believe)
• Single ignore file is easy to keep track of for a user (no unintended ignoring of notebooks)

[natanlao]

Okay! Sounds good to me.

[amit1rrr]

Discarding this PR now that we have --exclude command line option.

[natanlao]

Sorry, this got stuck on my backlog! Thanks for following up.

[amit1rrr]

No worries @natanlao