table: change 'units' to 'unit'

[adrn]

I suspect this will make some people angry, but we need to decide on plural or singular. My vote goes for singular, and that's what we decided on for Quantity and in coordinates. I've changed all the units instances to unit (where appropriate).

[mdboom]

No anger from me. 😉 Haven't really read this line-by-line, but I agree in principle. I know that saying "the units of a value" is common English usage, but in code, I generally don't like to see plural unless something is a collection of something.

[astrofrog]

If we do this we should probably deprecate this properly so that it doesn't break in 0.3. Is this the only place the plural appears?

[taldcroft]

I'm OK with this, but I do think there should be a deprecation release. Maybe we can make a Column.units property that issues a warning and then returns self.unit? There are some function argument name changes in table (e.g. pformat) that should probably be deprecated, I guess by means of a show_units kwarg that isn't documented and but which issues a warning and sets show_unit accordingly.

There are a number of units in io.ascii that need fixing. Note that the units kwarg in the LaTex class should not be changed since that refers to a dict of unit values. Just quickly scanning the test failures, the first one I saw was in io.fits.

I'm not sure what you mean about a bunch of tests also failing on master. I didn't do a comprehensive scan, but the first five failures in the Travis output were all about a Column object not having a units attribute.

[hamogu]

Once we are at it, I also suggest changing dtypes to dtype in tab = Table(dtypes = [np.float, 'S4']). After all, in numpy I do array.dtype not array.dtypes even if array is a record array with multiple types.

I think it would be good to do all changes Plural -> Singular in one PR, that's why I bring it up here, even if it does not strictly fit under the title of this issue.

[eteq]

I'm fine with the change to unit as well, but I agree a deprecation release is a important.

@hamogu - I think a monolathic change of doing all plural -> singular throughout astropy is a bit tough because it would touch a lot of different files and lead to many other PRs needing to be rebased (or do you mean just in table? In that case maybe it's fine, I'm not sure exactly how inter-mixed that is). I do agree we should make this all consistent, though (coordinates in particular needs to be made more consistent with other places)

[hamogu]

Yes, I agree, maybe not everywhere, but everywhere in table - let's
aim for one PR per module to make things singular.

[adrn]

Great, ok, so my action item is to add units back in the relevant places but issue a deprecation warning?

[embray]

All you have to do is make units a property that returns self.unit and put a @deprecated decorator on top of it.

[adrn]

Hm, not getting a DeprecationWarning when I call .units -- any idea why?

[adrn]

Ah, right, so Python 2.7 hides DeprecationWarnings by default... ok

[astrofrog]

@adrn - yeah, I think that's a little bit silly given that most users will therefore get no warning and things will suddenly break.

[adrn]

Made the change from dtypes -> dtype too.

[embray]

@astrofrog I agree. I've been thinking for a while of ensuring that Deprecation Warnings are displayed by default, even on Python 2.7. This can just go somewhere in astropy.__init__. I need to do this on PyFITS too because a lot of people haven't been aware of the deprecation warnings since they're using 2.7 (I do say in the documentation to check for deprecation warnings but who reads the docs?)

[taldcroft]

Made the change from dtypes -> dtype too.

Strictly speaking the dtypes arg is a list of dtype specifiers, and that was the reason it was put in as plural. You cannot provide a dtype or dtype-compatible object:

>>> dat = Table([[1], [2]], dtypes=[('a', int), ('b', int)])
ValueError: mismatch in size of old and new data-descriptor
>>> dat = Table([[1], [2]], dtypes=np.dtype([('a', int), ('b', int)]))
ValueError: dtypes must be a list or None
>>> dat = Table([[1], [2]], dtypes=[int, int])
# OK
>>> dat = Table([[1], [2]], dtypes=[np.dtype(int), np.dtype(int)])
# OK

Now maybe the above errors should be fixed, but as-designed there was a specific reason for dtypes instead of dtype as the arg name.

[taldcroft]

Interestingly pandas takes a dtype argument, but it doesn't behave like it should, and the corresponding DataFrame attribute is dtypes:

In [49]: x = pd.DataFrame([[1, 2]], dtype=np.dtype([('a', int), ('b', int)]))

In [50]: x
Out[50]: 
        0       1
0  (1, 1)  (2, 2)

In [51]: x.dtypes
Out[51]: 
0    object
1    object
dtype: object

Basically pandas is doing something weird relative to what I'd expect from numpy array.

[hamogu]

My point here was the comparison with numpy.rec.arrays, where it is also
called dtype (singular), although a more complex objects (a list)
needs to be passed in.

[taldcroft]

In the end I agree we should probably change dtypes to dtype since it probably an unnecessary source of confusion. But at the same time I think that the new dtype arg should accept anything that works to initialize np.dtype.

[taldcroft]

@adrn - every function or class initializer that previously accepted units or dtypes needs to still work with the plural forms in 0.3, but issue a DeprecationWarning. Basically the rule is that no existing code should break because of this change.

The basic plan is that you need to add back the old (plural) keyword arg at the end of the argument list. If that is specified by name in the call then set unit = units (e.g.) and issue a deprecation warning.

Maybe we could make some sort of keyword argument change decorator that handles the deprecation warning and the argument swap. I guess a decorator could be a good thing also because (I think) we could really hide the fact in the call signature and docs that the plural form is still an acceptable keyword arg. So you would do

@deprecate_keyword_arg('units', 'unit')
def func(arg1, arg2, unit=None, arg3=None):
    pass

@iguananaut - do you know of any available decorators that would fit this bill? It actually seems like a generic thing that has probably already been invented.

[adrn]

Ah, right -- deprecated the attributes but forgot about the kwargs.

[embray]

I don't have anything like that on hand for arguments. When I've needed to do that (rarely) I've just hand-coded it the function.

[adrn]

Ok, I think I added the plural kwargs back where necessary, and I added some tests to make sure they raise DeprecatingWarnings.

[adrn]

Rebased on master.

[adrn]

@iguananaut any idea why the build is failing? can't make any sense of that...

[eteq]

@adrn - I think this is unrelated to your changes and is some Travis quirk with the parallel testing. I just restarted the last set of tests and hopefully that'll do it.

Quick note: there should probably be an entry in the changelog about this. Maybe also add this to the section about API breaking, indicating that the API isn't going to break in this version, but will in the next?

[adrn]

Cool, done!

[adrn]

Argh, docs bug...how is such a simple PR such a hassle :)

[astrofrog]

@adrn - the issue is in CHANGES.rst:

../CHANGES.rst:112: ERROR: Unexpected indentation.

There were also a few other failed builds, but I don't think that's related to this pull request.

[taldcroft]

@adrn - just one final check that you grepped the entire doc tree for units and there were no remaining instances that need to be changed. Otherwise looks good to me, thanks for opening this can of worms and getting it closed again!

[adrn]

@astrofrog thanks for the direct pointer!

@taldcroft ok, double checked and found a few that I forgot to change! fixed.

[adrn]

As noted before, build failures are unrelated.

[adrn]

Someone merge this before I have to rebase again :)

[astrofrog]

@taldcroft - good to merge?

[taldcroft]

I think this somehow got changed from table_types (the fixture that does Masked and Unmasked tables) to the plain table (package).

[taldcroft]

Other than the change above with table_types this looks good.

[adrn]

Oh yea, that's weird... fixed!

[adrn]

Build failure due to network failure I think?

[taldcroft]

Well this is probably as close as this will get to success... there is a travis build failure unrelated to testing. Everything else passed, so I'm merging.

[taldcroft]

Thanks @adrn, hopefully you've learned your lesson to never become owner of an issue like this again. 😸

[mdboom]

Sorry to just be getting to this now -- I could have saved everyone a lot of grief: There is a helper function in astropy.tests.helper called catch_warnings that resets all of the warning state first so that the order of tests is not important as to how the warnings come out. This was critical to get things to work when I did all the work to support parallel testing.

See #1265.

[taldcroft]

@mdboom - can you please not go on vacation in the future. :-)