neomodel closes sessions only after object is deleted

[liranbg]

Using neomodel 3.2.5

I figured when using transaction with db.begin() and db.commit() it does not close the session immediately but when the object itself is being destroyed.

I used the following test to determine when the session is being close by using logging's output to check when the object is being deleted (before the tearDown() or after)
in test_session_closing_neomodel -> it closes session only after deleted
in test_session_closing_neodriver -> it closes session when transaction is ended

import unittest
import logging
from neomodel import db
from neo4j.v1 import GraphDatabase
logging.basicConfig(level=logging.DEBUG)


class NeoTest(unittest.TestCase):
    def setUp(self):
        pass

    def tearDown(self):
        pass

    def test_session_closing_neomodel(self):
        db.set_connection("bolt://neo4j:neo4j@localhost:7687")
        db.begin()
        db.cypher_query("MATCH (n) RETURN n")
        db.commit()

    def test_session_closing_neodriver(self):
        driver = GraphDatabase.driver("bolt://localhost:7687", auth=("neo4j", "neo4j"))
        with driver.session() as session:
            with session.begin_transaction() as tx:
                tx.run("MATCH (n) RETURN n")
                tx.commit()

[robinedwards]

Thanks for raising this issue. Coupled to it is also the issue with the connection clean up causing some strange exceptions in neo4j_drivers destructor. Spent a while trying to figure it out to no avail but will let you know as soon as I find a solution

[liranbg]

@robinedwards Thanks a lot. let me know if there's any help needed

[aanastasiou]

@liranbg Would this still be of interest? Looks like adding an explicit close here (?)

[liranbg]

@aanastasiou - That indeed would be more declarative rather than let python's gc do it

[aanastasiou]

@liranbg Thanks for such a quick response. Just to understand the severity of the problem, what is the worst that can happen from leaving this up to the system? Besides the exceptions on exit (which I have experienced myself as well) do you think this could be causing any transaction related problems too? (for example).

[liranbg]

sorry for this late response.
I believe, that for a huge transaction loop, the behavior expectation would be freeing the allocated fd once an iteration is done. and since it is being dangled, we might hit the maximum fd limit

since I did not test it, I am not certain about it.

[aanastasiou]

@liranbg Right, sounds good, I have marked it as an enhancement and hope to see it taken care of relatively soon.

All the best

[aanastasiou]

I have noticed that after repeatedly running pytest through a project that uses neomodel, I am consistently getting the following driver level error that I think might be relevant to this issue.

Exception ignored in: <bound method Driver.__del__ of <neo4j.v1.direct.DirectDriver object at 0x7f42f6e5cd68>>
Traceback (most recent call last):
  File "lib/python3.5/site-packages/neo4j/v1/api.py", line 152, in __del__
  File "lib/python3.5/site-packages/neo4j/v1/api.py", line 191, in close
  File "lib/python3.5/site-packages/neo4j/bolt/connection.py", line 536, in close
  File "lib/python3.5/site-packages/neo4j/bolt/connection.py", line 521, in remove
  File "lib/python3.5/site-packages/neo4j/bolt/connection.py", line 386, in close
  File "/usr/lib/python3.5/socket.py", line 407, in close
  File "/usr/lib/python3.5/ssl.py", line 979, in _real_close
AttributeError: 'NoneType' object has no attribute '_real_close'
Exception ignored in: <bound method Connection.__del__ of <neo4j.bolt.connection.Connection object at 0x7f42f67d9cf8>>
Traceback (most recent call last):
  File "lib/python3.5/site-packages/neo4j/bolt/connection.py", line 212, in __del__
  File "lib/python3.5/site-packages/neo4j/bolt/connection.py", line 386, in close
  File "/usr/lib/python3.5/socket.py", line 407, in close
  File "/usr/lib/python3.5/ssl.py", line 979, in _real_close
AttributeError: 'NoneType' object has no attribute '_real_close'
sys:1: ResourceWarning: unclosed <ssl.SSLSocket fd=8, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 35322), raddr=('127.0.0.1', 7687)>