Issue with _p_mtime when mocking time with python-libfaketime

[azmeuk]

python-libfaketime is a great python wrapper for libfaketime. It catches system-calls related to date and time, and allows you to mock them as you want:

>>> # after eval $(python-libfaketime)
>>> import libfaketime, datetime
>>> with libfaketime.fake_time("2015-01-01"):
...     datetime.datetime.now()
datetime.datetime(2015, 1, 1, 0, 0)

The code of python-libfaketime is outrageously short and simple (< 300 LOC), and for what I have played with, it works great in many contexts, including C extensions. However, I encountered an unexpected behavior with ZODB and the persistent _p_mtime attribute.

If a ZODB database object is created in a libfaketime context, after editing some persistent objects, _p_mtime value is the mocked time, as expected:

>>> import datetime, libfaketime, ZODB
>>> with libfaketime.fake_time("2015-01-01"):
...     db = ZODB.DB(None)
...     connection = db.open()
...     connection.root()["a"] = 1
...     connection.transaction_manager.commit()
...     timestamp = connection.root()._p_mtime
...     db.close()
...     datetime.datetime.fromtimestamp(timestamp)
datetime.datetime(2015, 1, 1, 0, 0)

But if the ZODB database object is created outside the libfaketime context, _p_mtime value is not mocked:

>>> import datetime, libfaketime, ZODB
>>> db = ZODB.DB(None)
>>> with libfaketime.fake_time("2015-01-01"):
...     connection = db.open()
...     connection.root()["a"] = 1
...     connection.transaction_manager.commit()
...     timestamp = connection.root()._p_mtime
...     db.close()
...     datetime.datetime.fromtimestamp(timestamp)
datetime.datetime(2019, 8, 25, 10, 25, 15, 652691)

I would have expected the latter example to print a mocked time since the modification and the commit are done in the libfaketime context.

So I know that bug report related to different libraries that were not designed to work together are sometimes tricky, but I though someone here might have a clue.

• What happens during the ZODB.DB creation that would need to happen in a libfaketime context?
• Is it something we can easily fix in ZODB?

Thank you for your help

[jamadden]

_p_mtime is just a wrapper (which loses precision, by the way) around _p_serial. In ZODB, _p_serial is the transaction ID, and transaction IDs always increase; they can never go backwards. When you create the DB, ZODB creates the first transaction to initially create the root object, and thus establishes the baseline for all transaction IDs going forward.

>>> db = ZODB.DB(None)
>>> db._storage.lastTransaction()
b'\x03\xd1\xffFF\xd1+\x00'
>>> db.open().root()._p_serial
b'\x03\xd1\xffFF\xd1+\x00'

When it's time to commit, ZODB essentially says to the TimeStamp class, "give me a TimeStamp based on the current time that's later than the last one I committed." If the clock has gone backwards, it'll be ignored.

[icemac]

@jamadden Thank you for your excellent explanation.
@azmeuk Does this explanation answer your question so the issue can be closed?

[d-maurer]

All "_p_*" attributes are special. When I remember right, then `_p_mtime` is just a different represantation of `_p_serial`. The latter contains the transaction id, the transaction id which caused the current object state. Transaction ids are mostly timestamps. However, the ZODB enforces that they strictly increase. This means you cannot fake arbitrary times in the past.

[azmeuk]

@jamadden Thank you. Your answer helped me a lot understand what is going on.
@icemac Yep!