Added update_metadata() to write_ adapters #716
Conversation
|
|
||
| class DocumentRevision(BaseDocument): | ||
| revision: int | ||
|
|
There was a problem hiding this comment.
It may be useful to add a classmethod constructor here to do what you were trying to do in __init__.
@classmethod
def from_document(cls, document)
return cls(key=document.key, ...)| @@ -62,8 +63,9 @@ def inner(self, *args, **kwargs): | |||
| class WritingArrayAdapter: | |||
| structure_family = "array" | |||
|
|
|||
| def __init__(self, collection, directory, doc): | |||
| def __init__(self, collection, revisions, directory, doc): | |||
There was a problem hiding this comment.
I think it would make sense to pass in the database here rather than separately passing in each of its collections.
There was a problem hiding this comment.
Fixed in the latest commit
| updated_at = datetime.now(tz=timezone.utc) | ||
| self.doc.updated_at = updated_at | ||
|
|
||
| if len(metadata) > 0: |
There was a problem hiding this comment.
If I want to update metadata to be empty {} or specs to be empty [], shouldn't that update be processed?
There was a problem hiding this comment.
Fixed in the latest commit
| ) | ||
|
|
||
| if result.matched_count != result.modified_count: | ||
| raise ValueError("Error while writing to database") |
There was a problem hiding this comment.
I suggest classifying this as a RuntimeError.
danielballan
force-pushed
the
Add-timestamps-to-experimental-document
branch
from
5583ff5
to
0518b1d
Compare
jmaruland
changed the title
|
Now that we are adding indexes, I think we should also add an index to the |
| def __len__(self): | ||
| return self._collection.count_documents( | ||
| {"key": self._key} | ||
| ) # maybe wrong MongoDB usage here... |
There was a problem hiding this comment.
Delete comment (assuming this usage is now correct).
| {"key": self._key} | ||
| ) # maybe wrong MongoDB usage here... | ||
|
|
||
| def __getitem__(self, item_): |
There was a problem hiding this comment.
This looks mixed up and likely needs testing.
The usage r[i:j] should lead to skip(offset).limit(j - i). The usage r[i:] or r[i:None] or r[:-1] (all equivalent) should lead to skip(offset) with no limit. Pymongo also accept skip(offset).limit(0) where 0 means "no limit", which is an option if you find it leads to cleaner code.
| if now > self.deadline: | ||
| self._doc = Document( | ||
| **self.collection.find_one({"key": self.key}) | ||
| ) # run query |
There was a problem hiding this comment.
This comment seems superfluous. :-)
| def create_indexes(self): | ||
| self.revision_coll.create_index( | ||
| [("key", pymongo.ASCENDING), ("revision", pymongo.DESCENDING)], unique=True | ||
| ) |
There was a problem hiding this comment.
While we're creating indexes, we should also create an index on the nodes collection to ensure that key is unique.
This PR started as an approach to add more information to the document schema that will help to keep track of all the updates made to the metadata and specs of the samples.
In addition, we implemented a revisions system with the mongo database that will keep track of old versions of documents. Every time that
update_metadata()is run, the active document that is saved in collections is copied to revisions where every entry is protected by the samekeyid plus arevisionnumber. There two parameters is used as a unique identifier for every entry.