Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

maximal_independent_set MatchError #4857

Closed
mike-w-wilson opened this issue Nov 29, 2018 · 7 comments

Comments

Projects
None yet
4 participants
@mike-w-wilson
Copy link

commented Nov 29, 2018

Hail version:

0.2-29fbaeaf265e

What you did:

Ran hail's maximal independent set method with the following code:

related_samples_to_drop_ranked = hl.maximal_independent_set(related_pairs.id1_rank, related_pairs.id2_rank,keep=False, tie_breaker=tie_breaker)

where related pairs is structured as:

----------------------------------------
Global fields:
    None
----------------------------------------
Row fields:
    'i': struct {
        data_type: str, 
        s: str
    } 
    'j': struct {
        data_type: str, 
        s: str
    } 
    'id1_rank': struct {
        id: struct {
            data_type: str, 
            s: str
        }, 
        rank: int32
    } 
    'id2_rank': struct {
        id: struct {
            data_type: str, 
            s: str
        }, 
        rank: int32
    } 
----------------------------------------
Key: ['i', 'j']
----------------------------------------

and tie_breaker is :

def tie_breaker(l, r):
    return hl.or_else(l.rank, max_rank + 1) - hl.or_else(r.rank, max_rank + 1)

What went wrong (all error messages here, including the full java stack trace):

FatalError                                Traceback (most recent call last)
<ipython-input-220-88e7ce1066ed> in <module>
     11     return hl.or_else(l.rank, max_rank + 1) - hl.or_else(r.rank, max_rank + 1)
     12 
---> 13 related_samples_to_drop_ranked = hl.maximal_independent_set(related_pairs.id1_rank, related_pairs.id2_rank,keep=False, tie_breaker=tie_breaker)
     14 #return related_samples_to_drop_ranked.select(**related_samples_to_drop_ranked.node.id).key_by('data_type', 's')

<decorator-gen-1024> in maximal_independent_set(i, j, keep, tie_breaker)

/home/hail/hail.zip/hail/typecheck/check.py in wrapper(__original_func, *args, **kwargs)
    558     def wrapper(__original_func, *args, **kwargs):
    559         args_, kwargs_ = check_all(__original_func, args, kwargs, checkers, is_method=is_method)
--> 560         return __original_func(*args_, **kwargs_)
    561 
    562     return wrapper

/home/hail/hail.zip/hail/methods/misc.py in maximal_independent_set(i, j, keep, tie_breaker)
    142 
    143     edges = t.key_by().select('i', 'j')
--> 144     nodes_in_set = Env.hail().utils.Graph.maximalIndependentSet(edges._jt.collect(), node_t._jtype, joption(tie_breaker_str))
    145 
    146     nt = Table._from_java(nodes._jt.annotateGlobal(nodes_in_set, hl.tset(node_t)._jtype, 'nodes_in_set'))

/usr/lib/spark/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py in __call__(self, *args)
   1131         answer = self.gateway_client.send_command(command)
   1132         return_value = get_return_value(
-> 1133             answer, self.gateway_client, self.target_id, self.name)
   1134 
   1135         for temp_arg in temp_args:

/home/hail/hail.zip/hail/utils/java.py in deco(*args, **kwargs)
    208             raise FatalError('%s\n\nJava stack trace:\n%s\n'
    209                              'Hail version: %s\n'
--> 210                              'Error summary: %s' % (deepest, full, hail.__version__, deepest)) from None
    211         except pyspark.sql.utils.CapturedException as e:
    212             raise FatalError('%s\n\nJava stack trace:\n%s\n'

FatalError: MatchError: cmg_exomes (of class java.lang.String)

Java stack trace:
scala.MatchError: cmg_exomes (of class java.lang.String)
	at is.hail.annotations.RegionValueBuilder.addAnnotation(RegionValueBuilder.scala:537)
	at is.hail.annotations.RegionValueBuilder.addRow(RegionValueBuilder.scala:298)
	at is.hail.annotations.RegionValueBuilder.addAnnotation(RegionValueBuilder.scala:541)
	at is.hail.utils.Graph$$anonfun$2$$anonfun$apply$2$$anonfun$apply$3.apply(Graph.scala:60)
	at is.hail.utils.Graph$$anonfun$2$$anonfun$apply$2$$anonfun$apply$3.apply(Graph.scala:54)
	at is.hail.utils.package$.using(package.scala:587)
	at is.hail.annotations.Region$.scoped(Region.scala:20)
	at is.hail.utils.Graph$$anonfun$2$$anonfun$apply$2.apply(Graph.scala:54)
	at is.hail.utils.Graph$$anonfun$2$$anonfun$apply$2.apply(Graph.scala:53)
	at is.hail.utils.BinaryHeap.isLeftFavoredTie(BinaryHeap.scala:16)
	at is.hail.utils.BinaryHeap.is$hail$utils$BinaryHeap$$bubbleUp(BinaryHeap.scala:161)
	at is.hail.utils.BinaryHeap.insert(BinaryHeap.scala:40)
	at is.hail.utils.Graph$$anonfun$maximalIndependentSet$1.apply(Graph.scala:91)
	at is.hail.utils.Graph$$anonfun$maximalIndependentSet$1.apply(Graph.scala:90)
	at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
	at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
	at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
	at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
	at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
	at is.hail.utils.Graph$.maximalIndependentSet(Graph.scala:90)
	at is.hail.utils.Graph$.maximalIndependentSet(Graph.scala:76)
	at is.hail.utils.Graph.maximalIndependentSet(Graph.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
	at py4j.Gateway.invoke(Gateway.java:280)
	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
	at py4j.commands.CallCommand.execute(CallCommand.java:79)
	at py4j.GatewayConnection.run(GatewayConnection.java:214)
	at java.lang.Thread.run(Thread.java:748)



Hail version: 0.2-29fbaeaf265e
Error summary: MatchError: cmg_exomes (of class java.lang.String)
@tpoterba

This comment has been minimized.

Copy link
Collaborator

commented Nov 29, 2018

randomly assigned @jbloom22

@tpoterba

This comment has been minimized.

Copy link
Collaborator

commented Dec 3, 2018

bump

@jbloom22

This comment has been minimized.

Copy link
Collaborator

commented Dec 3, 2018

Looking now

@jbloom22

This comment has been minimized.

Copy link
Collaborator

commented Dec 5, 2018

This is top of my list for tomorrow and I'll ask for help if needed

@danking

This comment has been minimized.

Copy link
Collaborator

commented Dec 7, 2018

bump

@danking

This comment has been minimized.

Copy link
Collaborator

commented Dec 7, 2018

I think we need to annotate two fields called i and j onto the table. This code assumes the i and j python arguments are top-level field references named i and j.

@danking

This comment has been minimized.

Copy link
Collaborator

commented Dec 12, 2018

bump

@tpoterba tpoterba assigned tpoterba and unassigned jbloom22 Dec 13, 2018

tpoterba added a commit to tpoterba/hail that referenced this issue Dec 13, 2018

danking added a commit that referenced this issue Dec 13, 2018

tpoterba added a commit to tpoterba/hail that referenced this issue Dec 13, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.