Reorganize astropy.utils.misc a bit #2857

embray · 2014-08-15T17:40:48Z

Following on from #2849, it became apparent to me that astropy.utils.misc was getting a bit cluttered, so this introduces a few new astropy.utils modules so that astropy.utils.misc could be broken up a bit.

Specifically it moves:

make_func_with_sig into the codegen submodule
deprecated, deprecated_attribute, lazyproperty, and wraps into the decorators submodule
find_current_module and find_mod_objs into the introspection submodule

Any other moves seem worthwhile? Note again, this has #2849 as a prerequisite.

<astropy#2835 (comment)> this adds the make_func_with_sig function to astropy.utils.misc. Taking advantage of this, a new astropy.utils.compat.misc.wraps function is provided as a replacement for functools.wraps, which provides the same functionality but while also preserving the original wrapped function's signature (so that it displays properly in help())

… thought. And most of them are in utilties/backwards compatibility features that aren't terribly important. But I found this one case where the new wraps implementation is useful. And there could be others in the future.

…_sig, rather than only allowing the name of the wrapped function to be used

…eplacement, such that the new function gets the same *name* as the function being wrapped as well

… number into the module make_func_with_sig was called from, and a test to prove that it works.

…mplementation available on Python 3.4 as well. As such I moved it out of astropy.utils.compat since it's really providing a new function rather than some backward/forward compatibility.

….utils.misc was getting a bit cluttered, so this introduces a few new astropy.utils modules so that astropy.utils.misc could be broken up a bit.

embray · 2014-08-15T19:35:24Z

Any opinions on this? Part of what motivated this, by the way, was a branch in which I've introduced yet another new decorator called sharedmethod.

pllim · 2014-08-15T19:47:10Z

👍 but I do have minor concern on how much this would impact affiliated packages.

embray · 2014-08-15T19:51:53Z

Me too--really they should just use from astropy.utils import ... to get to most of these. Worse comes to worse I can provide backwards compat.

embray · 2014-08-18T22:10:53Z

Went ahead and added a changelog entry clarifying which imports moved. I don't think this will have much impact but it can't hurt to have the change clearly documented.

embray · 2014-08-18T22:12:29Z

#2849 has been merged, so I'll go ahead and merge this too. If it ends up affecting any affiliated packages we'll find out quickly and I can take steps to make their lives easier.

Reorganize astropy.utils.misc a bit

cdeil · 2014-08-19T10:55:28Z

In Gammapy we used from astropy.utils.misc import lazyproperty so this broke our test.
It's OK, I'll just add a try ... except to import it from the new or old location.

But could someone please wipe readthedocs so that this shows up at the right location (and hopefully with a docstring) in the development version of the docs?
http://astropy.readthedocs.org/en/latest/search.html?q=lazyproperty
http://astropy.readthedocs.org/en/latest/api/astropy.utils.misc.lazyproperty.html?highlight=lazyproperty

cdeil · 2014-08-19T10:56:19Z

CHANGES.rst

+  - Some members of ``astropy.utils.misc`` were moved into new submodules.
+    Specifically:
+
+    - ``deprecated`, ``deprecated_attribute``, and ``lazyproperty`` ->


Missing double-backtick

``deprecated`

should be

``deprecated``

astrofrog · 2014-08-19T11:00:21Z

@cdeil - I've wiped latest and started a new build.

cdeil · 2014-08-19T11:22:48Z

@astrofrog Build log is here:
https://readthedocs.org/builds/astropy/1613002/

I"ve seen this error before, but never figured out the reason ... any idea?

Traceback (most recent call last):
  File "setup.py", line 112, in <module>
    **package_info
  File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/lib/python2.7/dist-packages/setuptools/command/install.py", line 73, in run
    self.do_egg_install()
  File "/usr/lib/python2.7/dist-packages/setuptools/command/install.py", line 88, in do_egg_install
    self.run_command('bdist_egg')
  File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/lib/python2.7/dist-packages/setuptools/command/bdist_egg.py", line 177, in run
    self.run_command("egg_info")
  File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 163, in run
    self.find_sources()
  File "/usr/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 186, in find_sources
    mm.run()
  File "/usr/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 246, in run
    self.add_defaults()
  File "/usr/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 285, in add_defaults
    rcfiles = list(walk_revctrl())
  File "/usr/lib/python2.7/dist-packages/setuptools/command/sdist.py", line 18, in walk_revctrl
    for item in ep.load()(dirname):
  File "/usr/lib/python2.7/dist-packages/setuptools/command/sdist.py", line 65, in _default_revctrl
    for item in finder(dirname):
  File "/usr/lib/python2.7/dist-packages/setuptools/svn_utils.py", line 468, in svn_finder
    info = SvnInfo.load(dirname)
  File "/usr/lib/python2.7/dist-packages/setuptools/svn_utils.py", line 264, in load
    with TemporaryDirectory() as tempdir:
  File "/usr/lib/python2.7/dist-packages/setuptools/py31compat.py", line 27, in __init__
    self.name = tempfile.mkdtemp()
AttributeError: 'NoneType' object has no attribute 'mkdtemp'

cdeil · 2014-08-19T11:24:16Z

Fixed in Gammapy at https://github.com/gammapy/gammapy/pull/168/files without try .. except by simply importing from the top-level astropy.utils after I saw @embray's comment in the change log ... thanks!

embray · 2014-08-19T15:29:13Z

@cdeil Thanks, sorry for the trouble. And yes, import from astropy.utils is the way to go for most of those.

kbarbary · 2014-09-24T14:56:04Z

I just ran into this change, with an sncosmo user working on astropy master. Glad to see the flatter astropy.utils namespace.

However, the docs look like they don't reflect this still. That is, http://astropy.readthedocs.org/en/latest/api/astropy.utils.decorators.lazyproperty.html

has the path as astropy.utils.decorators.lazyproperty rather than astropy.utils.lazyproperty.

embray · 2014-09-24T16:36:03Z

@kbarbary I agree that's a problem. The docs aren't wrong because that is the correct fully-qualified name for that function. This was discussed a bit in #2873

kbarbary · 2014-09-24T17:50:29Z

Thanks, I hadn't seen #2873. Looks like the docs issue will be resolved there.

Reorganize astropy.utils.misc a bit

embray added 8 commits August 13, 2014 12:30

Allow specifying the name of the function generated by make_func_with…

f5e57f6

…_sig, rather than only allowing the name of the wrapped function to be used

Take advantage of the new name-specification feature in the wraps() r…

744b62c

…eplacement, such that the new function gets the same *name* as the function being wrapped as well

Ensure that the filename for the code object is not a .pyc file

85074c0

Trick the compiled code object into also pointing to the correct line…

f845283

… number into the module make_func_with_sig was called from, and a test to prove that it works.

Following discussion in astropy#2849, decided to make the new wraps i…

c121150

…mplementation available on Python 3.4 as well. As such I moved it out of astropy.utils.compat since it's really providing a new function rather than some backward/forward compatibility.

Following on from astropy#2849, it became apparent to me that astropy…

9831ad6

….utils.misc was getting a bit cluttered, so this introduces a few new astropy.utils modules so that astropy.utils.misc could be broken up a bit.

embray added the utils label Aug 15, 2014

Added a changelog entry

652b0b1

embray added this to the v1.0.0 milestone Aug 18, 2014

embray added a commit that referenced this pull request Aug 18, 2014

Merge pull request #2857 from embray/utils/reorg

2bf73db

Reorganize astropy.utils.misc a bit

embray merged commit 2bf73db into astropy:master Aug 18, 2014

embray deleted the utils/reorg branch August 18, 2014 22:12

cdeil reviewed Aug 19, 2014
View reviewed changes

This was referenced Aug 20, 2014

Build stuck in building state for a day readthedocs/readthedocs.org#895

Closed

Generate API docs for some missing sub-modules in astropy.utils #2873

Merged

embray mentioned this pull request Sep 24, 2014

ImportError: cannot import name lazyproperty #2973

Closed

embray added the Affects-release label Feb 18, 2015

astrofrog pushed a commit to astropy/sphinx-automodapi that referenced this pull request Jan 27, 2018

Merge pull request astropy/astropy#2857 from embray/utils/reorg

573b133

Reorganize astropy.utils.misc a bit

ntessore pushed a commit to glass-dev/cosmology that referenced this pull request Dec 7, 2020

Merge pull request astropy/astropy#2857 from embray/utils/reorg

9a82e4c

Reorganize astropy.utils.misc a bit

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reorganize astropy.utils.misc a bit #2857

Reorganize astropy.utils.misc a bit #2857

embray commented Aug 15, 2014

embray commented Aug 15, 2014

pllim commented Aug 15, 2014

embray commented Aug 15, 2014

embray commented Aug 18, 2014

embray commented Aug 18, 2014

cdeil commented Aug 19, 2014

cdeil Aug 19, 2014

astrofrog commented Aug 19, 2014

cdeil commented Aug 19, 2014

cdeil commented Aug 19, 2014

embray commented Aug 19, 2014

kbarbary commented Sep 24, 2014

embray commented Sep 24, 2014

kbarbary commented Sep 24, 2014

Reorganize astropy.utils.misc a bit #2857

Reorganize astropy.utils.misc a bit #2857

Conversation

embray commented Aug 15, 2014

embray commented Aug 15, 2014

pllim commented Aug 15, 2014

embray commented Aug 15, 2014

embray commented Aug 18, 2014

embray commented Aug 18, 2014

cdeil commented Aug 19, 2014

cdeil Aug 19, 2014

Choose a reason for hiding this comment

astrofrog commented Aug 19, 2014

cdeil commented Aug 19, 2014

cdeil commented Aug 19, 2014

embray commented Aug 19, 2014

kbarbary commented Sep 24, 2014

embray commented Sep 24, 2014

kbarbary commented Sep 24, 2014