Parso Cache Garbage Collection

[davidhalter]

Currently people using parso might be using a lot of disk space after a while. ~/.cache/parso might get quite big. (or whatever it is on other OS's, see parso/cache.py.

IMO we should probably clean up in parso.cache.save_module. That function should just check "once a day" if files are older than a month. In that case they should be deleted. It should be fine to delete stuff that we might parse again, since the cache is not extremely important and just there to speed up some things.

As a small additional thing, we should probably use scandir, see also https://github.com/davidhalter/jedi/blob/7fd5c8af8ff04711599fea10ca8babe51b280464/jedi/_compatibility.py#L29-L47 for Python 2.7/3.5. However we could also just remove Python 2.7/3.5 support for this feature. I don't plan on maintaining it for a long time.

[isidentical]

If we are going to drop 2.7/3.5 support, what would be your opinion about switching to pathlib (completely)? And maybe start by dropping the 2.7/3.5 completely and then resolve this issue afterwards

[isidentical]

Also if the target will become 3.6+, we might use some kind of formatters to keep the source style consistent (black + isort and since we do have annotations already + mypy with pre-commit enforcement)?

[davidhalter]

If we are going to drop 2.7/3.5 support, what would be your opinion about switching to pathlib (completely)? And maybe start by dropping the 2.7/3.5 completely and then resolve this issue afterwards

I'm fine with that, but I think we should at least wait until Python 3.5 is EOL. (Also Jedi has to drop Python 3.5 first, because it's by far the biggest parso user). So i guess this we could drop it this fall.

Also if the target will become 3.6+, we might use some kind of formatters to keep the source style consistent (black + isort

I'm probably fine with flak8 being enforced, but I don't particularly like black. Same kind of goes for isort. I don't think formatting is that big of an issue, especially for the limited amount of contributors parso has. In general I think it's important to not obsess with formatting. BTW: I liked Raymond Hettingers talk about discovering the actual issues: https://www.youtube.com/watch?v=wf-BqAjZb8M

But as I said, if you want to add a flake8 checker in the CI, I'm happy with that, it can also be pre-commit or just CI.

since we do have annotations already + mypy with pre-commit enforcement)?

I'm fine with having mypy enabled (pre-commit is fine, except if it's not installed). I just don't want parso to have annotations everywhere in the Python code. I consider that pretty annoying to read Python code, but I'm fine with stubs and annotations here and there to make the intention of the programmer clearer.

[isidentical]

I see. Your points makes sense. In the first round, I'll go for removing python2 support and migrating to pathlib. Afterwards we can resolve this issue, and when the 3.5 goes out of EOL, we can simply drop it and use some great features (like f-strings). If you are OK with this migration plan?

[davidhalter]

Well, please also wait with dropping Python 2, until Jedi drops it (hopefully this summer).

If you want, you can of course create a branch and we'll work there, I leave that up to you.

[davidhalter]

The branch could also just drop 3.5. I just want to avoid you being disappointed, because that branch will probably not be merged to master until summer/fall.

[isidentical]

If you want, you can of course create a branch and we'll work there, I leave that up to you.

I see, my primary objective was using pathlib but yeah, of course we can wait for summer. It would be awesome if you can ping me when Jedi drops their support for python2, and I can start the work.

I'll directly try to solve this issue, and we can discuss the migration plan later.

[davidhalter]

Do you intend to drop parsing for Python 2/3.5 as well or are you just interested in making it impossible to run it on Python 2?

[isidentical]

Well, I'd go for keeping the 2.7 grammar and errors, I guess it wont hurt much

…

On Sun, May 24, 2020, 12:11 AM Dave Halter ***@***.***> wrote: Do you intend to drop parsing for Python 2/3.5 as well or are you just interested in making it impossible to run it on Python 2? — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#120 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALJKHQL6ROW2VIEWO2O3H53RTA3XZANCNFSM4NIQ5XWQ> .