Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Changed DB root-object-initialization to use an open connection and more
This fixes #84. Also: - Added missing transaction ``begin`` call in the DB ``transaction()`` context manager. - Added the ability to add a transaction note whan calling ``transaction()``. This is useful (as would be the ability to pass in other transaction meta data, but this was needed by (and this tested) to make an existing test pass.
- Loading branch information
Jim Fulton
committed
Jul 23, 2016
1 parent
dedd5a2
commit 03b8183
Showing
8 changed files
with
63 additions
and
44 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
A change in the way databases were initialized affected tests | ||
============================================================= | ||
|
||
Originally, databases added root objects by interacting directly with | ||
storages, rather than using connections. As storages transaction | ||
interaction became more complex, interacting directly with storages | ||
let to duplicated code (and buggy) code. | ||
|
||
See: https://github.com/zopefoundation/ZODB/issues/84 | ||
|
||
Fixing this had some impacts that affected tests: | ||
|
||
- New databases now have a connection with a single object in it's cache. | ||
This is a very slightly good thing, but it broke some tests expectations. | ||
|
||
- Tests that manipulated time, had their clocks off because of new time calls. | ||
|
||
This led to some test fixes, in many cases adding a mysterious | ||
``cacheMinimize()`` call. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
03b8183
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jimfulton ouch! It breaks our build because we don't have thread transactions, we are getting the transaction from the frame looking for a Request/View object.
03b8183
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bloodbare I don't understand what you're trying to say. Can you unpack that please?
03b8183
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jimfulton on the experimental plone.server (https://github.com/plone/plone.server.git) we have a RequestAwareDBConnection, so objects get registered on a data manager on the request object, not on a local thread var in order to allow mutiple transactions/thread. On the CMS REST API is supposed that all operations will be tied to a request object, minus the creation of the root object that is created on startup of the application. We can wrap the creation of the DB with a Request object so it works or add the specific use case on _register if its the root object, but we didn't need it before because the root was not transaction managed :)
03b8183
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bloodbare I don't see how this change affects what you describe.
transaction
method to open a connection with a dedicated transaction manager. The transaction manager isn't tied to a thread in any way. If this is breaking your code, it's almost certainly exposing a latent bug in your code. (This did expose a bug in ZEO, which was recently fixed, where invalidations weren't sent to clients for transactions, like this one, that only added objects.)If you want to discuss, I'm often in #zodb on on freenode.
03b8183
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From irc:
bloodbare, a) why would database initialization using a standard tm break this and b) why don't you just hang a standard TM on a request.
In fact, there are existing tools that do this for you.
For example, zc.zodbwsgi attaches a ZODB connection and has an option for that connection to use a transaction manager that's not connected to a thread.
In fact: https://github.com/zc/twotieredkanban/blob/master/buildout.cfg illustrates using this option for a ZODB web app that uses gevent.
03b8183
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe the missing point is that we store the transaction and the datamanager on the request., and that we are reusing the connection for all the requests. This last feature it will be configurable (to open connection/request or reuse), so there is less RAM on using caches. Reusing the connection makes us to lose MVCC and transaction on the connection but we can have multiple transactions on the same thread.
the main file for this subject is :
https://github.com/plone/plone.server/blob/master/src/plone.server/plone/server/transactions.py
Sorry maybe we are missing something we don't know!
03b8183
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I get that you store the transaction and data manager on the request. As I pointed out, that's easily done without any custom software.
"Reusing the connection makes us to lose MVCC" WTF? (Maybe I'll pretend you didn't say that. :) )
In any case, the change to ZODB shouldn't really affect your code other than when you ask for a connection the first time, you may get a connection that was used to add the root object. If this somehow reveals breakage, the breakage was already there waiting to be revealed.