Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python[3]-simplejson: doesn't remove tests folder #10136

Closed
BKPepe opened this issue Oct 2, 2019 · 8 comments
Closed

python[3]-simplejson: doesn't remove tests folder #10136

BKPepe opened this issue Oct 2, 2019 · 8 comments

Comments

@BKPepe
Copy link
Member

@BKPepe BKPepe commented Oct 2, 2019

Maintainer: @commodo (and @jefferyto might be interesting in it as well)
Environment: Turris Omnia, OpenWrt 19.07, mvebu

Description:

When you do opkg install python3-simplejson, there will appear two folders on your router with files.

root@turris:~# tree /usr/lib/python3.7/site-packages/simplejson*
/usr/lib/python3.7/site-packages/simplejson
├── __init__.pyc
├── _speedups.cpython-37.so
├── compat.pyc
├── decoder.pyc
├── encoder.pyc
├── errors.pyc
├── ordered_dict.pyc
├── raw_json.pyc
├── scanner.pyc
├── tests
│   ├── __init__.pyc
│   ├── test_bigint_as_string.pyc
│   ├── test_bitsize_int_as_string.pyc
│   ├── test_check_circular.pyc
│   ├── test_decimal.pyc
│   ├── test_decode.pyc
│   ├── test_default.pyc
│   ├── test_dump.pyc
│   ├── test_encode_basestring_ascii.pyc
│   ├── test_encode_for_html.pyc
│   ├── test_errors.pyc
│   ├── test_fail.pyc
│   ├── test_float.pyc
│   ├── test_for_json.pyc
│   ├── test_indent.pyc
│   ├── test_item_sort_key.pyc
│   ├── test_iterable.pyc
│   ├── test_namedtuple.pyc
│   ├── test_pass1.pyc
│   ├── test_pass2.pyc
│   ├── test_pass3.pyc
│   ├── test_raw_json.pyc
│   ├── test_recursion.pyc
│   ├── test_scanstring.pyc
│   ├── test_separators.pyc
│   ├── test_speedups.pyc
│   ├── test_str_subclass.pyc
│   ├── test_subclass.pyc
│   ├── test_tool.pyc
│   ├── test_tuple.pyc
│   └── test_unicode.pyc
└── tool.pyc
/usr/lib/python3.7/site-packages/simplejson-3.16.0-py3.7.egg-info
├── PKG-INFO
├── SOURCES.txt
├── dependency_links.txt
└── top_level.txt

1 directory, 45 files

Which is alright, but the issue appears when you do opkg remove python3-simplejson, it will not remove folder with egg-info neither tests folder, but files inside it yes.

root@turris:~# tree /usr/lib/python3.7/site-packages/simplejson*
/usr/lib/python3.7/site-packages/simplejson
└── tests
/usr/lib/python3.7/site-packages/simplejson-3.16.0-py3.7.egg-info

1 directory, 0 files

It seems that this is seems to be cause of the issue. I try to import simplejson and it works.

root@turris:~# python3
Python 3.7.4 (default, Oct 01 2019, 23:50:45) 
[GCC 7.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import simplejson
>>> simplejson.loads('{}')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'simplejson' has no attribute 'loads'
>>> 

Because when I remove those two files manually rm -rf /usr/lib/python3.7/site-packages/simplejson*, and try to import simplejson again, it can not find it, which is correct.

root@turris:~# python3
Python 3.7.4 (default, Oct 01 2019, 23:50:45) 
[GCC 7.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import simplejson
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'simplejson'
@commodo

This comment has been minimized.

Copy link
Contributor

@commodo commodo commented Oct 3, 2019

Will take a look.
Thanks :)

@commodo

This comment has been minimized.

Copy link
Contributor

@commodo commodo commented Oct 3, 2019

Just as a note.
My main thought it to try to remove the tests. They aren't typically used. It's what's being done for Python's sub-packages

@jefferyto

This comment has been minimized.

Copy link
Contributor

@jefferyto jefferyto commented Oct 4, 2019

I've tested this a little, and even with the tests directory removed, the same "error" occurs.

If there is a simplejson folder in site-packages, then import simplejson will work (in Python 3) because simplejson (or any directory that does not have a __init__.py file) is considered a namespace package. This appears to be standard/expected behaviour, so I don't think we should try to change it.

As for why opkg remove doesn't remove these empty directories, I haven't been able to find any documentation/bug reports, but IIRC this behaviour has always existed (for years at least). It would be nice to fix this.

@commodo

This comment has been minimized.

Copy link
Contributor

@commodo commodo commented Oct 8, 2019

I've spinned up PR #10181 for dropping the tests
That at least makes the packages lighter a bit.

As for why opkg remove doesn't remove these empty directories, I haven't been able to find any documentation/bug reports, but IIRC this behaviour has always existed (for years at least). It would be nice to fix this.

I was not aware of this.
Thanks for the input.

[Curios] Does it make sense to keep this issue open in that case?

@jefferyto

This comment has been minimized.

Copy link
Contributor

@jefferyto jefferyto commented Oct 8, 2019

I don't have a strong opinion on whether to keep this issue open or not, but ultimately it's an issue with opkg (I think...).

@BKPepe

This comment has been minimized.

Copy link
Member Author

@BKPepe BKPepe commented Oct 21, 2019

I think, we should close the issue here and report it to OpenWrt developers at https://bugs.openwrt.org

@BKPepe

This comment has been minimized.

Copy link
Member Author

@BKPepe BKPepe commented Jan 11, 2020

Created bug in OpenWrt's Flyspray: https://bugs.openwrt.org/index.php?do=details&task_id=2725

@neheb

This comment has been minimized.

Copy link
Contributor

@neheb neheb commented Jan 19, 2020

Closing since this seems to be an opkg bug.

@neheb neheb closed this Jan 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.