[obsolete] pyDKB storages #244
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sometimes we have to interact with the same storage in different scripts. When it happens we got to implement same functionality multiple times: read configuration, check that Python module is available, create client, ... This submodule is a place to implement this once and then reuse whwnever it is needed.
It would be wierd to manage module global variable from class; it would also be wierd to get errors like ``StorageAlreadyExists`` if one just uses the Storage functionality and doesn't want any extra logic around it.
…`client()`). * `log()` -- quite useful method for all the derived classes; * `client()` -- given that one day we may want to make multiple simultaneous requests to the storage, it doesn'y look good to use instance variable `c` directly; method that returns ready-to-use client seems to be a better way.
There may be more than one valuable parameter for record detection (e.g. record ID and index in ES).
Added concept of "raw" queries: it is useful when query must be stored, yet can not be converted to the format acceptable by storage client. Here it is JSON with %(parameter)s values: it may fail to be parsed as proper JSON if there is something like `"taskname": %(name)s`.
mgolosova
force-pushed
the
pyDKB-storages
branch
from
729f40e
to
14cf1ca
Compare
|
Force-push was just to fix codestyle in the original commits, instead of a separate 'Codestyle fix' commit. |
This was referenced Apr 28, 2019
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rewritten with #277
This PR introduces to
pyDKBlibrary new submodulestorages.The module is intended to accumulate all the functionality related to the interaction with the storages we use, including both external and internal ones. It basically provides unified way to interact with any of them: object of
Storageclass, that understands basic set of methods, like:get()request (get item by ID);It is needed as we have at least 4 pieces of (Python) code that requires interaction with ES@DKB or Chicago ES:
Also, almost same way as in the last case, we need to update Stage 95 (AMI-related stage) (update: see #246).
To avoid copy-and-paste code cloning, this common module is developed.
ToDo / to think about:
Maybe it would be better to place it inside the(discussed here);pyDKB.commonmodule.If there should be something DKB-specific (or, better to say, DKB@ATLAS-specific) -- it should be in a separate module, which is aware of the data scheme, etc.(discussed here).For now I am going to leave this PR as is in favor of safe update issue (for details please refer to Trello).