DBCreator.py clear_and_generate command fails

[2xyo]

clear then generate commands work as expected but clear_and_generate command always fails.

Tested with :

• Python 2.7.13 with neo4j-driver-1.6.2 neotime-1.0.0
• neo4j from https://hub.docker.com/r/specterops/bloodhound-neo4j/
• DBCreator.py from https://github.com/BloodHoundAD/BloodHound-Tools.git at commit 095fb82181a28b51fb1cfeda9f852524061a0649

Test 1

$ python DBCreator.py 
================================================================
BloodHound Sample Database Creator
================================================================

Documented commands (type help <topic>):
========================================
clear_and_generate  cleardb  connect  dbconfig  exit  generate  help  setnodes

(Cmd) dbconfig
Current Settings:
DB Url: bolt://localhost:7687
DB Username: neo4j
DB Password: neo4jj

Enter DB URL [bolt://localhost:7687] 
Enter DB Username [neo4j] 
Enter DB Password [neo4jj] BloodHound

New Settings:
DB Url: bolt://localhost:7687
DB Username: neo4j
DB Password: BloodHound

Testing DB Connection
Database Connection Successful!
(Cmd) connect
Database Connection Successful!
(Cmd) cleardb
Clearing Database
Resetting Schema
DB Cleared and Schema Set
(Cmd) generate
Starting data generation with nodes=500
Populating Standard Nodes
Adding Standard Edges
Generating Computer Nodes
Creating Domain Controllers
Generating User Nodes
Generating Group Nodes
Adding Domain Admins to Local Admins of Computers
Creating 25 Domain Admins (5% of users capped at 30)
Applying random group nesting
Adding users to groups
Calculated 7 groups per user with a variance of - 6
Adding local admin rights
Adding RDP/ExecuteDCOM/AllowedToDelegateTo
Adding sessions
Adding Domain Admin ACEs
Creating OUs
Creating GPOs
Adding outbound ACLs to 3 objects
Marking some users as Kerberoastable
Adding unconstrained delegation to a few computers
Database Generation Finished!
(Cmd) clear_and_generate
Database Connection Successful!
Clearing Database
Resetting Schema
DB Cleared and Schema Set
Starting data generation with nodes=500
Populating Standard Nodes
Adding Standard Edges
Traceback (most recent call last):
  File "DBCreator.py", line 692, in <module>
    MainMenu().cmdloop()
  File "DBCreator.py", line 56, in cmdloop
    cmd.Cmd.cmdloop(self)
  File "/usr/lib/python2.7/cmd.py", line 142, in cmdloop
    stop = self.onecmd(line)
  File "/usr/lib/python2.7/cmd.py", line 221, in onecmd
    return func(arg)
  File "DBCreator.py", line 166, in do_clear_and_generate
    self.generate_data()
  File "DBCreator.py", line 231, in generate_data
    'MERGE (n:Domain {name:"TESTLAB.LOCAL"}) MERGE (m:Group {name:"ENTERPRISE ADMINS@TESTLAB.LOCAL"}) MERGE (m)-[:GenericAll {isacl:true}]->(n)')
  File "/usr/local/lib/python2.7/dist-packages/neo4j/v1/api.py", line 331, in run
    self._connection.fetch()
  File "/usr/local/lib/python2.7/dist-packages/neo4j/bolt/connection.py", line 287, in fetch
    return self._fetch()
  File "/usr/local/lib/python2.7/dist-packages/neo4j/bolt/connection.py", line 327, in _fetch
    response.on_failure(summary_metadata or {})
  File "/usr/local/lib/python2.7/dist-packages/neo4j/v1/result.py", line 70, in on_failure
    raise CypherError.hydrate(**metadata)
neo4j.exceptions.TransientError: Database constraints have changed (txId=849) after this transaction (txId=848) started, which is not yet supported. Please retry your transaction to ensure all constraints are executed.

Test 2

$ python DBCreator.py 
================================================================
BloodHound Sample Database Creator
================================================================

Documented commands (type help <topic>):
========================================
clear_and_generate  cleardb  connect  dbconfig  exit  generate  help  setnodes

(Cmd) connect
Database Connection Failed. Check your settings.
(Cmd) dbconfig
Current Settings:
DB Url: bolt://localhost:7687
DB Username: neo4j
DB Password: neo4jj

Enter DB URL [bolt://localhost:7687] 
Enter DB Username [neo4j] 
Enter DB Password [neo4jj] BloodHound

New Settings:
DB Url: bolt://localhost:7687
DB Username: neo4j
DB Password: BloodHound

Testing DB Connection
Database Connection Successful!
(Cmd) connect
Database Connection Successful!
(Cmd) cleardb
Clearing Database
Resetting Schema
DB Cleared and Schema Set
(Cmd) generate
Starting data generation with nodes=500
Populating Standard Nodes
Adding Standard Edges
Generating Computer Nodes
Creating Domain Controllers
Generating User Nodes
Generating Group Nodes
Adding Domain Admins to Local Admins of Computers
Creating 20 Domain Admins (4% of users capped at 30)
Applying random group nesting
Adding users to groups
Calculated 7 groups per user with a variance of - 6
Adding local admin rights
Adding RDP/ExecuteDCOM/AllowedToDelegateTo
Adding sessions
Adding Domain Admin ACEs
Creating OUs
Creating GPOs
Adding outbound ACLs to 2 objects
Marking some users as Kerberoastable
Adding unconstrained delegation to a few computers
Database Generation Finished!
(Cmd) cleardb
Clearing Database
Resetting Schema
DB Cleared and Schema Set
(Cmd) clear_and_generate
Database Connection Successful!
Clearing Database
Resetting Schema
DB Cleared and Schema Set
Starting data generation with nodes=500
Populating Standard Nodes
Traceback (most recent call last):
  File "DBCreator.py", line 692, in <module>
    MainMenu().cmdloop()
  File "DBCreator.py", line 56, in cmdloop
    cmd.Cmd.cmdloop(self)
  File "/usr/lib/python2.7/cmd.py", line 142, in cmdloop
    stop = self.onecmd(line)
  File "/usr/lib/python2.7/cmd.py", line 221, in onecmd
    return func(arg)
  File "DBCreator.py", line 166, in do_clear_and_generate
    self.generate_data()
  File "DBCreator.py", line 210, in generate_data
    session.run("MERGE (n:Group {name:'ENTERPRISE READ-ONLY DOMAIN CONTROLLERS@TESTLAB.LOCAL'})")
  File "/usr/local/lib/python2.7/dist-packages/neo4j/v1/api.py", line 331, in run
    self._connection.fetch()
  File "/usr/local/lib/python2.7/dist-packages/neo4j/bolt/connection.py", line 287, in fetch
    return self._fetch()
  File "/usr/local/lib/python2.7/dist-packages/neo4j/bolt/connection.py", line 327, in _fetch
    response.on_failure(summary_metadata or {})
  File "/usr/local/lib/python2.7/dist-packages/neo4j/v1/result.py", line 70, in on_failure
    raise CypherError.hydrate(**metadata)
neo4j.exceptions.TransientError: Database constraints have changed (txId=1100) after this transaction (txId=1099) started, which is not yet supported. Please retry your transaction to ensure all constraints are executed.

[thomai]

It looks like this is caused by a timing issue. A sleep of three seconds before data generation solved it for me: https://github.com/BloodHoundAD/BloodHound-Tools/blob/master/DBCreator/DBCreator.py#L204

It seems like the database is busy with deleting the old nodes. This is what the docs state about transient errors:

Transaction functions are the recommended form for containing transactional units of work. This form requires minimal boilerplate code and allows for a clear separation of database queries and application logic.

and

Transaction functions are also able to handle connection problems and transient errors using an automatic retry mechanism. This retry capability can be configured on Driver construction.
https://neo4j.com/docs/developer-manual/current/drivers/sessions-transactions/#driver-transactions-transaction-functions

I tried to find a clean solution by setting the retry time but it didn't work:

self.driver = GraphDatabase.driver(self.url, auth=(self.username,self.password), max_retry_time=15)

[rvazarkar]

Can you go ahead and make a pull request with your change?