Fixed the endpoint comparison bug

[MarkDana]

Updated files:

• causallearn/graph/Endpoint.py: reload __eq__.
• causallearn/graph/Edge.py;Edges.py and causallearn/utils/GraphUtils.py: replace all the "is Endpoint" with "== Endpoint", since identity comparison is unsafe.

Reproduce the issue (as of 446b9a2):

>>> from causallearn.graph import Endpoint
>>> arr1 = Endpoint.Endpoint.ARROW
>>>
>>> # module reload sometimes happens unknowingly; e.g., during discovery algorithms in causallearn
>>> import importlib; importlib.reload(Endpoint)
>>> arr2 = Endpoint.Endpoint.ARROW
>>>
>>> print(arr1, arr2, type(arr1), type(arr2), id(arr1), id(arr2))
ARROW ARROW <enum 'Endpoint'> <enum 'Endpoint'> 4302557296 4302743968
>>> arr1 == arr2 # oh no..
False

According to Python doc, equality comparisons for Enum are defined by identity comparison (i.e., is). Here we need to explicitly reimplement __eq__ for attributed value comparison. See stackoverflow 1 2.

Incorrect SHD caused:

I experienced an incorrect SHD result due to this bug. The truth contains X1 --> X2 and the estimation contains X2 --> X1 (given by GES), but the returned SHD is 0, because this line is evaluated as False (i.e., ARROW != ARROW).

Potentially there might be more bugs related:

Might be a fatal bug:

Endpoint comparisons are in the basic building blocks, while they're not used uniformly:

• Sometimes identity comparison is used (bad!), e.g.,

  causal-learn/causallearn/graph/Edges.py

  Line 37 in 446b9a2

  if edge.get_endpoint1() is Endpoint.TAIL:

• Sometimes equality comparison (bad! but will be corrected by this pr), e.g.,

  causal-learn/causallearn/graph/SHD.py

  Line 44 in 446b9a2

  truth.get_node(nodes_name[j])) == Endpoint.ARROW and est.get_endpoint(

• Sometimes value string comparison (good!), e.g.,

  causal-learn/causallearn/graph/Edges.py

  Line 59 in 446b9a2

  if str(edge.get_endpoint1()) == "TAIL" and str(edge.get_endpoint2()) == "ARROW":

• Sometimes value integer comparison (well, unclear but correct), e.g.,

  causal-learn/causallearn/graph/Edge.py

  Line 84 in 446b9a2

  if self.numerical_endpoint_1 == 1 and self.numerical_endpoint_2 == 1:

Let's take care of this if there will be a graph class refactoring.

Merge plan

I'm not sure whether this pr should be merged right away. On the one hand, we need to ensure that the users receive accurate results (especially SHDs for evaluation in papers). On the other hand, we need to regenerate all the test benchmarks (which takes time) so that all the tests can pass. What do you think? @kunwuz @tofuwen

[tofuwen]

Thanks for identifying this bug! This is a really fatal bug! This is a great great finding!

Now I am really worried about the correctness of some of our algorithms, and very surprised why we didn't identify this bug before. Let's have a joint meeting to discuss next steps?