New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Enable unary math operations for pandas, sqlite #1071

Closed
wants to merge 3 commits into
base: master
from

Conversation

Projects
None yet
3 participants
@cpcloud
Member

cpcloud commented Jul 19, 2017

Implement decimal for pandas
Add SQLite unary ops
Fix operations in postgres that require numeric

@cpcloud cpcloud force-pushed the cpcloud:unary-ops branch 3 times, most recently from 0dc84c3 to 13a0422 Jul 20, 2017

@cpcloud cpcloud self-assigned this Jul 20, 2017

@cpcloud cpcloud added the enhancement label Jul 20, 2017

@cpcloud cpcloud added this to the 0.11.3 milestone Jul 20, 2017

@cpcloud cpcloud force-pushed the cpcloud:unary-ops branch 5 times, most recently from d6c9085 to 3322844 Jul 20, 2017

@cpcloud cpcloud requested a review from wesm Jul 21, 2017

@cpcloud

This comment has been minimized.

Member

cpcloud commented Jul 21, 2017

@wesm can you take a look here? bit of a rabbit hole with decimals, otherwise just adding unary ops to series.

@cpcloud cpcloud force-pushed the cpcloud:unary-ops branch from 3322844 to e059f7f Jul 22, 2017

@wesm

Minor comments, but otherwise LGTM

if column_name in schema:
ibis_type = dt.validate_type(schema[column_name])
elif dtype == np.object_:
inferred_dtype = infer_dtype(df[column_name].dropna())

This comment has been minimized.

@wesm

wesm Jul 24, 2017

Member

Yikes. I guess we should make a NaN-friendly type inference function someplace (seems like an oversight in infer_dtype originally)

This comment has been minimized.

@jreback

jreback Jul 24, 2017

Contributor

can u post an issue in pandas tracker about this

This comment has been minimized.

@cpcloud

This comment has been minimized.

@cpcloud
def execute_series_unary_op(op, data, scope=None):
function = getattr(np, type(op).__name__.lower())
if data.dtype == np.dtype(np.object_):
return data.apply(functools.partial(execute_node, op, scope=scope))

This comment has been minimized.

@wesm

wesm Jul 24, 2017

Member

Is Series.apply different from Series.map (in behavior or performance)?

This comment has been minimized.

@cpcloud

cpcloud Jul 25, 2017

Member

Don't think so, @jreback any idea here?

This comment has been minimized.

@cpcloud

cpcloud Jul 27, 2017

Member

So, it looks like Series.map accepts a dict, to support a simple CASE-statement-like operation, as well as callables, whereas apply only deals with callables. For callables, both methods use the same underlying function lib.map_infer to call the passed in callable in a Cython loop. Here we could use either since we're only dealing with callables.

def execute_series_log_with_base(op, data, base, scope=None):
if data.dtype == np.dtype(np.object_):
func = np.vectorize(functools.partial(execute_node, op, scope=scope))
return pd.Series(func(data, base), index=data.index, name=data.name)

This comment has been minimized.

@wesm

wesm Jul 24, 2017

Member

Perhaps this bit could be factored out into a helper function since it's repeated a couple times (in case it's useful in future execution rules)

This comment has been minimized.

@cpcloud

cpcloud Jul 25, 2017

Member

Yeah, good idea!

def _floor_divide(t, expr):
left, right = map(t.translate, expr.op().args)
return sa.func.floor(left / right)

This comment has been minimized.

@wesm

wesm Jul 24, 2017

Member

Does integer division in postgres yield doubles?

This comment has been minimized.

@cpcloud

cpcloud Jul 24, 2017

Member

No, it yields integers. This is implemented so that it works regardless of the type of left and right.

This comment has been minimized.

@wesm

wesm Jul 24, 2017

Member

Got it, I was just curious =)

cpcloud added some commits Jul 6, 2017

ENH: Enable unary math operations for pandas, sqlite
Implement decimal for pandas
Add SQLite unary ops
Fix operations in postgres that require numeric

@cpcloud cpcloud force-pushed the cpcloud:unary-ops branch from e46867f to d94b0c6 Jul 27, 2017

@cpcloud

This comment has been minimized.

Member

cpcloud commented Jul 27, 2017

Merging on green.

@cpcloud cpcloud closed this in 9882b5a Jul 27, 2017

@cpcloud cpcloud deleted the cpcloud:unary-ops branch Jul 27, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment