Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

issue with select_all usage in regression methods #4941

Closed
tpoterba opened this issue Dec 11, 2018 · 3 comments

Comments

Projects
None yet
1 participant
@tpoterba
Copy link
Collaborator

commented Dec 11, 2018

mt = hl.utils.range_matrix_table(1, 1).annotate_cols(y=0, cov1=1).annotate_entries(x=1)

# x_expr uses a col field
x_expr = hl.case().when(mt.cov1 == 1, 2).default(mt.x)

hl.logistic_regression_rows(y=mt.y, x=x_expr, test='wald', covariates=[1])
Java stack trace:
java.lang.AssertionError: assertion failed: cov1 not in struct{__y: float64, __cov0: float64}
	at scala.Predef$.assert(Predef.scala:170)
	at is.hail.expr.ir.Infer$.apply(Infer.scala:74)
	at is.hail.expr.ir.InferIR$class.typ(IR.scala:62)
	at is.hail.expr.ir.GetField.typ(IR.scala:215)
	at is.hail.expr.ir.IRParser$.ir_value_expr_1(Parser.scala:561)
	at is.hail.expr.ir.IRParser$.ir_value_expr(Parser.scala:511)
	at is.hail.expr.ir.IRParser$.ir_value_expr_1(Parser.scala:536)
	at is.hail.expr.ir.IRParser$.ir_value_expr(Parser.scala:511)
	at is.hail.expr.ir.IRParser$$anonfun$ir_value_children$1.apply(Parser.scala:507)
	at is.hail.expr.ir.IRParser$$anonfun$ir_value_children$1.apply(Parser.scala:507)
	at is.hail.expr.ir.IRParser$.repUntil(Parser.scala:281)
	at is.hail.expr.ir.IRParser$.ir_value_children(Parser.scala:507)
	at is.hail.expr.ir.IRParser$.ir_value_expr_1(Parser.scala:714)
	at is.hail.expr.ir.IRParser$.ir_value_expr(Parser.scala:511)
	at is.hail.expr.ir.IRParser$.named_value_ir(Parser.scala:494)
	at is.hail.expr.ir.IRParser$$anonfun$named_value_irs$1.apply(Parser.scala:489)
	at is.hail.expr.ir.IRParser$$anonfun$named_value_irs$1.apply(Parser.scala:489)
	at is.hail.expr.ir.IRParser$.repUntil(Parser.scala:281)
	at is.hail.expr.ir.IRParser$.named_value_irs(Parser.scala:489)
	at is.hail.expr.ir.IRParser$.ir_value_expr_1(Parser.scala:670)
	at is.hail.expr.ir.IRParser$.ir_value_expr(Parser.scala:511)
	at is.hail.expr.ir.IRParser$.matrix_ir_1(Parser.scala:920)
	at is.hail.expr.ir.IRParser$.matrix_ir(Parser.scala:885)
	at is.hail.expr.ir.IRParser$.matrix_ir_1(Parser.scala:923)
	at is.hail.expr.ir.IRParser$.matrix_ir(Parser.scala:885)
	at is.hail.expr.ir.IRParser$$anonfun$parse_matrix_ir$2.apply(Parser.scala:1023)
	at is.hail.expr.ir.IRParser$$anonfun$parse_matrix_ir$2.apply(Parser.scala:1023)
	at is.hail.expr.ir.IRParser$.parse(Parser.scala:1007)
	at is.hail.expr.ir.IRParser$.parse_matrix_ir(Parser.scala:1023)
	at is.hail.expr.ir.IRParser$.parse_matrix_ir(Parser.scala:1022)
	at is.hail.expr.ir.IRParser.parse_matrix_ir(Parser.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
	at py4j.Gateway.invoke(Gateway.java:280)
	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
	at py4j.commands.CallCommand.execute(CallCommand.java:79)
	at py4j.GatewayConnection.run(GatewayConnection.java:214)
	at java.lang.Thread.run(Thread.java:748)



Hail version: 0.2.5-e017bef1c26c
Error summary: AssertionError: assertion failed: cov1 not in struct{__y: float64, __cov0: float64}
@tpoterba

This comment has been minimized.

Copy link
Collaborator Author

commented Dec 11, 2018

This is an issue with MT._select_all. The operation needs to modify all schemas atomically, which is not an easy fix.

@tpoterba

This comment has been minimized.

Copy link
Collaborator Author

commented Dec 11, 2018

I can see a fix using _annotate_all and then a select directly before entering the logreg, though.

@tpoterba

This comment has been minimized.

Copy link
Collaborator Author

commented Dec 11, 2018

I'll claim this

@tpoterba tpoterba self-assigned this Dec 11, 2018

tpoterba added a commit to tpoterba/hail that referenced this issue Dec 11, 2018

tpoterba added a commit to tpoterba/hail that referenced this issue Dec 11, 2018

danking added a commit that referenced this issue Dec 16, 2018

Fix MatrixTable._select_all to address atomicity constraint (#4942)
* Fix MatrixTable._select_all to address atomicity constraint

fixes #4941

* use uids in select all

* fix keys

* Fix gen test

* fix

* fix

bcajes added a commit to bcajes/hail that referenced this issue Jan 4, 2019

Fix MatrixTable._select_all to address atomicity constraint (hail-is#…
…4942)

* Fix MatrixTable._select_all to address atomicity constraint

fixes hail-is#4941

* use uids in select all

* fix keys

* Fix gen test

* fix

* fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.