Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merging variables with overlapping not but not conflicting values #835

Closed
shoyer opened this issue Apr 20, 2016 · 3 comments
Closed

Merging variables with overlapping not but not conflicting values #835

shoyer opened this issue Apr 20, 2016 · 3 comments

Comments

@shoyer
Copy link
Member

shoyer commented Apr 20, 2016

It should be possible to safely merge together variables with values [NaN, 1, 2] and [0, 1, NaN] by using methods such as combine_first (which should also be OK with conflicting values, like the pandas method) and merge (which should raise if values conflict).

See this stackoverflow post for a merge example: http://stackoverflow.com/questions/36731870/how-to-merge-xarray-datasets-with-conflicting-coordinates

@shoyer shoyer changed the title combine_first and merging variables with overlapping not but conflicting variables Merging variables with overlapping not but not conflicting values Apr 20, 2016
@shoyer
Copy link
Member Author

shoyer commented Apr 20, 2016

I'm currently in the process of refactoring the merge code so this should be pretty easy to add in.

A few snippets that should come in handy:

def combine_first(arrays):
    notnulls = [ops.notnull(array) for array in arrays]
    first_notnull = ops.argmax(ops.stack(notnulls), axis=0)
    return ops.choose(first_notnull, arrays)

def combine_if_equal(arrays):
    combined = combine_first(arrays)
    combined_null = ops.isnull(combined)
    if not all(((combined == array) | combined_null).all() for array in arrays):
        raise ValueError
    return combined

@shoyer
Copy link
Member Author

shoyer commented Nov 6, 2016

Actually, we can just write:

def combine_first(left, right):
    return ops.where(ops.notnull(left), left, right)

@shoyer
Copy link
Member Author

shoyer commented Jan 23, 2017

Fixed by #1204

@shoyer shoyer closed this as completed Jan 23, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant