Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add Inventory.sort() #2087

Open
wants to merge 14 commits into
base: master
from

Conversation

Projects
None yet
3 participants
@megies
Copy link
Member

commented Mar 29, 2019

It might be useful to have a sorting method on Inventory, something like..

inv = read_inventory()
inv.networks = sorted(inv.networks, key=lambda x: x.code)
for net in inv:
    net.stations = sorted(net.stations, key=lambda x: x.code)
    for sta in net:
        sta.channels = sorted(sta.channels, key=lambda x: x.code)

Looking at the contents of inv before sorting:

for net in inv:
    for sta in net:
        for cha in sta:
            print net.code, sta.code, cha.code
GR FUR HHZ
GR FUR HHN
GR FUR HHE
GR FUR BHZ
GR FUR BHN
GR FUR BHE
GR FUR LHZ
GR FUR LHN
GR FUR LHE
GR FUR VHZ
GR FUR VHN
GR FUR VHE
GR WET HHZ
GR WET HHN
GR WET HHE
GR WET BHZ
GR WET BHN
GR WET BHE
GR WET LHZ
GR WET LHN
GR WET LHE
BW RJOB EHZ
BW RJOB EHN
BW RJOB EHE
BW RJOB EHZ
BW RJOB EHN
BW RJOB EHE
BW RJOB EHZ
BW RJOB EHN
BW RJOB EHE

..and after:

BW RJOB EHE
BW RJOB EHN
BW RJOB EHZ
BW RJOB EHE
BW RJOB EHN
BW RJOB EHZ
BW RJOB EHE
BW RJOB EHN
BW RJOB EHZ
GR FUR BHE
GR FUR BHN
GR FUR BHZ
GR FUR HHE
GR FUR HHN
GR FUR HHZ
GR FUR LHE
GR FUR LHN
GR FUR LHZ
GR FUR VHE
GR FUR VHN
GR FUR VHZ
GR WET BHE
GR WET BHN
GR WET BHZ
GR WET HHE
GR WET HHN
GR WET HHZ
GR WET LHE
GR WET LHN
GR WET LHZ

We could offer to sort by epoch start or end time or also sampling rate etc..

@megies megies added this to the 1.2.0 milestone Mar 9, 2018

@megies megies modified the milestones: 1.2.0, 1.3.0 Apr 19, 2018

@megies megies modified the milestones: 1.3.0, 2.0.0 Feb 15, 2019

@megies

This comment has been minimized.

Copy link
Member Author

commented Mar 6, 2019

@DominikStr this could be something you can work on.

@DominikStr

This comment has been minimized.

Copy link
Contributor

commented Mar 12, 2019

I have a few questions regarding this one.

What exactly is the aim of the function:
Is it to return a sorted (by whatever criteria) list of all the channels(or stations) in the inventory?
Or is the aim to return a sorted inventory object, where the networks, stations and channels are sorted separately?

I'm not completely sure if it's possible to return an inventory object, which is sorted by sample_rate for example. The only way I could think of would be multiple network objects with the same name. These network objects contain all the stations/channels with the same value of the search criteria and are ordered accordingly.

The example looks like the aim is to return a list of channels but I'm not completely sure.

@megies

This comment has been minimized.

Copy link
Member Author

commented Mar 15, 2019

I guess we could maybe start by having an Inventory.sort() without any parameters that sorts the full hierarchy (so the Networks / Stations / Channels) first by code and then by starttime.

DominikStr added some commits Mar 20, 2019

First try of a sorting function.
This commit contains the first try for a sorting function. Until now its only a simple sort first by code and then by start_date.
First try on a sorting function (Issue #2087)
Basic sorting function sorting first by code and then by starting date.
@DominikStr

This comment has been minimized.

Copy link
Contributor

commented Mar 20, 2019

I committed the first try on a basic sorting function.

It would be pretty easy to make the sorting parameters adjustable, so I could add that if needed.

If this is the direction you want I will write a test case, description and make sure it's stable.

@megies

This comment has been minimized.

Copy link
Member Author

commented Mar 28, 2019

@DominikStr please push your commit to a differently named branch. Only alphanumeric characters, dashes, underscores please. Brackets in the branch name are real ugly and a mess to work with on command line.. I'll make this issue a PR then for better review options

I can't even open you branch in my browser because it hickups with the round brackets..

@megies

This comment has been minimized.

Copy link
Collaborator

commented on obspy/core/inventory/inventory.py in c9b84e4 Mar 28, 2019

I think it should work in place. It should be mentioned in the docstring then as well.

@megies

This comment has been minimized.

Copy link
Collaborator

commented on c9b84e4 Mar 28, 2019

Looks good to me in principle! Now go for a test case.

@DominikStr

This comment has been minimized.

Copy link
Contributor

commented Mar 28, 2019

I renamed the branch, hopefully it is working.

Sorting only inplace.
Small change to only sort in place plus docstring extension.
@megies

This comment has been minimized.

Copy link
Member Author

commented Mar 29, 2019

@DominikStr looks like github changed the issue->PR conversion API. I can't convert this issue into a PR because I don't have write access on your fork. So I guess, you can just open a separate PR (or add me as collaborator on your fork).

@DominikStr

This comment has been minimized.

Copy link
Contributor

commented Mar 29, 2019

I added you as a collaborator and I will open the pull request as soon as I finish the test case.

@megies

This comment has been minimized.

Copy link
Member Author

commented Mar 29, 2019

I added you as a collaborator and I will open the pull request as soon as I finish the test case.

No need, I made this issue into a PR now. It will dynamically update whenever you push to your branch again. :-)

Test case for Inventory sort method #2087
Contains a test case for the sorting method and to new inventory files. One to sort and one to compare.
@krischer

This comment has been minimized.

Copy link
Member

commented Apr 11, 2019

Instead of basing the test case on two StationXML files it would probably be better to just create some dummy Inventory objects and perform the test on these. Avoids having to commit test files but also is IMHO easier to reason about, especially for different cases like reverse sorting.

@DominikStr

This comment has been minimized.

Copy link
Contributor

commented Apr 27, 2019

The new test is a bit long but I tried to keep it as short as possible. I couldn't think of a way to make it more compact.

DominikStr added a commit to DominikStr/obspy that referenced this pull request Apr 27, 2019

First try on a sorting function (Issue obspy#2087)
Basic sorting function sorting first by code and then by starting date.

DominikStr added a commit to DominikStr/obspy that referenced this pull request Apr 27, 2019

Test case for Inventory sort method obspy#2087
Contains a test case for the sorting method and to new inventory files. One to sort and one to compare.

DominikStr added a commit to DominikStr/obspy that referenced this pull request Apr 27, 2019

First try on a sorting function (Issue obspy#2087)
Basic sorting function sorting first by code and then by starting date.

DominikStr added a commit to DominikStr/obspy that referenced this pull request Apr 27, 2019

Test case for Inventory sort method obspy#2087
Contains a test case for the sorting method and to new inventory files. One to sort and one to compare.

DominikStr added some commits Apr 27, 2019

Merge remote-tracking branch 'origin/add_2087_Inventory_sort' into ad…
…d_2087_Inventory_sort

# Conflicts:
#	obspy/core/tests/test_inventory.py
@krischer
Copy link
Member

left a comment

One long in-line comment with a few suggestions.

Also I think it would be good to replace the test files with some generated test objects (e.g. manually create some inventory/network/... objects and run the tests on them).

@@ -1117,6 +1117,27 @@ def plot_response(self, min_freq, output="VEL", network="*", station="*",

return fig

def sort(self):

This comment has been minimized.

Copy link
@krischer

krischer Apr 29, 2019

Member

How about something like this:

Suggested change
def sort(self):
def sort(self, keys=("code", "start_date"), max_level="channel"):
sort_fct = lambda x: [getattr(x, i) for i in keys]
self.networks = sorted(self.network, key=sort_fct)
...

Then it works for all keys and any number of keys (at least those that are present on all object types which could also checked before any in-place sorting takes place). I also added a max_level argument so one could for example only sort the networks and not the stations/channels/...

I also think the .sort() method should be part of every network/station/... object. If you implement it at the BaseNode (https://github.com/obspy/obspy/blob/master/obspy/core/inventory/util.py#L26) this would just work I think.

Changed the inventory sort to support keys and max_level. Implemented…
… a sorting function for networks and stations at the base node.
@DominikStr

This comment has been minimized.

Copy link
Contributor

commented May 1, 2019

I implemented the sort at the base node and changed the inventory sort according to your suggested change. If this is what you have in mind I will add a few lines to deal with wrong input and test cases for the base node implementation.

I'm still not entirely sure what you mean by manually creating the objects. Something like commit b3d7b13 ?

DominikStr added some commits May 1, 2019

Changed the inventory sort to support keys and max_level. Implemented…
… a sorting function for networks and stations at the base node.
Merge remote-tracking branch 'origin/add_2087_Inventory_sort' into ad…
…d_2087_Inventory_sort

# Conflicts:
#	obspy/core/inventory/inventory.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.