Exclude columns by index in io.ascii.read()

[Gabriel-p]

Currently the read() module allows to exclude columns from the output table via the exclude_names parameter.

I'd like to be able to exclude columns by index number also. Can this be done at all? If not, could it be added?

[pllim]

Just to understand the use case, why is excluding by name not useful for you?

[Gabriel-p]

I have a code that processes data files in batch mode. Many of these files have either incomplete headers, or no header at all. But most (or all) of them share the column positioning.

I understand the importance of a proper header, and I plan to enforce this in future versions of my code. But in the meantime, accessing columns by index is almost the only way.

[pllim]

I thought Table assigns default column names like col_0, col_1, etc? Maybe you can exclude based on those for now until someone has time to implement this?

[Gabriel-p]

It assigns names as col1, col2, etc.. (yes, it starts from 1 not 0 as you'd expect)

I already do something like that by creating the column names manually:

col_names = ['col' + str(_ + 1) for _ in col_ids]

[pllim]

Glad to know you have a workaround. You are always welcome to contribute if you want to see this feature implemented sooner than later. 😄

[Gabriel-p]

I'd love to but I think my Python skills are nowhere near the level of the guys developing this (great) package.

[hamogu]

@Gabriel-p Thanks for the suggestion, however, I don't see much benefit given that it only saves you one line of code (the col_names=... that you showed in the comment).

[taldcroft]

I think this could be done in a relatively clean way (from the API perspective) by allowing for exclude_names (and include_names and fill_include_names and fill_exclude_names) to accept integer values along with string values. The integer values would then be an index into the table column names list.

From a quick glance at the pure-Python and fast-C reader code this would not be too messy, but that is just a quick glance. To do this in a consistent way will require making the interface uniform for all those exclude/include options for read and write. So with all the implementation, testing, and docs this would be a moderate effort patch. I would propose to leave this open if someone wants to tackle it.

[taldcroft]

BTW @Gabriel-p, there is no better way to improve your Python skills than trying to make a patch to astropy. You will learn a lot and get plenty of free feedback on writing better code!

[taldcroft]

And you just start by grepping for exclude_names in every file in io.ascii. That will show you where to think about applying a patch.