-
Notifications
You must be signed in to change notification settings - Fork 0
AdvancedConnectorTutorial
The YamlEngine defined in the Basic Connector
Tutorial can retrieve complete items from a YAML file,
assuming the structure of objects in the file matches exactly the structure of
the expected DatabaseItem. But what if we want to load a custom DatabaseItem
from an already existing YAML file that does not match the expected Item
specification, e.g. some config files used by other application?
As a first simple example, we will define a YamlAttribute that allows us to
fetch a value from a field at a different path from the current one in the
DatabaseItem model. For example, we want the login field of the DatabaseItem to
be retrieved from the name attribute of the object in the YAML file. To do so,
we define a custom YamlItem as follows
database_item = YamlItem(
model={
"login": YamlAttribute(["name"])
}
)A custom DatabaseItem attribute must support the same request operations as
the ItemMapper. Indeed, requests built by attribute will be executed as any
other request by the database connection. Requests performed by the ItemMapper
are called "base" request. Each attribute of the model can either modify the
base request, or return new requests that will be handled independently by the
database connection. Both approaches are valid, and depends on the requests
structure, performances, and the logic required by each attribute.
Here is the detail of each DatabaseAttribute operation:
-
search_request(base_request, _id): build a new request or update thebase_requestto retrieve the value of the attribute. -
load(base_response, attribute_response): returns the value of the attribute.attribute_responseis the response to the request built bysearch_request, if any. -
create_request(base_request, value): build a new request or update thebase_requestto create the attribute with the specifiedvalue. -
delete_request(base_request, _id): build a new request or update thebase_requestto delete the attribute in the item associated to_id -
update_request(base_request, _id, value): build a new request or update the base request to update the attribute tovaluein the item associated to_id.
The following examples will demonstrate how to implement the YamlAttribute
using two different methods: modifying the base request or creating new
requests.
This is the recommended method for best performance, when requested attributes
can easily be added to the base request (for example in the SELECT field of an
SQL request or in the list of attributes of an LDAP SEARCH).
# backo/yaml/item.py
from ..database.item import ItemMapper, DatabaseItem, DatabaseAttribute
...
class YamlAttribute(DatabaseAttribute):
def __init__(self, path):
"""
:param attribute: Name of the field to load from the Yaml file
"""
self.path = path
def search_request(self, base_request, _id):
# Nothing to do, the requested field will already be included in the
# response of the base_request, that include the complete object
# associated to _id
pass
def load(self, base_response, _attribute_response):
# _attribute_response is None, because no request was returned by
# search_request. But the field can be retrieved from the response of
# the base request
value = find(base_response.value, self.path)
# The value must be deleted from the response so this field is not
# included in the generated JSON-like dict
delete(base_response.value, self.path)
return value
def create_request(self, base_request: YamlCreateRequest, value):
# Adds the value of the attribute to the base create request
update(base_request.value, self.path, value)
# Removes the replaced path from the original user item
delete(base_request.value, self.attribute_path)
def update_request(self, base_request, _id, value):
# Adds the value of the attribute to the base create request
update(base_request.value, self.path, value)
# Removes the replaced path from the original user item
delete(base_request.value, self.attribute_path)
def delete_request(self, base_request, _id):
# Nothing to do, the complete item will be deleted by the base request
passThe custom YamlAttribute can now be used as follows:
# examples/yaml/attributes.py
from backo.yaml.engine import YamlEngine
from backo.yaml.item import YamlItem, YamlAttribute
if __name__ == "__main__":
yaml_engine = YamlEngine(
"test.yaml", database_item=YamlItem(model={"login": YamlAttribute(["name"])})
)
item_id = yaml_engine.create(
{"login": "pipo", "gid": 12, "description": "Example user"}
)
print(f"Item {item_id} created.")
print(yaml_engine.search(item_id))
print()
yaml_engine.save(
item_id, {"login": "molo", "gid": 13, "description": "Updated user"}
)
print(f"Item {item_id} updated.")
print(yaml_engine.search(item_id))
print()
yaml_engine.delete(item_id)
print(f"Item {item_id} deleted.")$ uv run examples/yaml/attributes.py
Item c9b4149e-d1ba-466b-b574-474fa4aeaec2 created.
{'description': 'Example user', 'gid': 12, 'login': 'pipo', '_id': 'c9b4149e-d1ba-466b-b574-474fa4aeaec2'}
Item c9b4149e-d1ba-466b-b574-474fa4aeaec2 updated.
{'description': 'Updated user', 'gid': 13, 'login': 'molo', '_id': 'c9b4149e-d1ba-466b-b574-474fa4aeaec2'}
Item c9b4149e-d1ba-466b-b574-474fa4aeaec2 deleted.
Notice how the login field is included in engine.search responses, but not
name. More however, if you create some items with the engine, the database
file looks as follows:
e8642beb-12f8-477d-b93d-bfa0eee1b964:
description: Example user
gid: 12
name: pipo
ef7f4920-78cc-4a07-97e8-ea64e4eeb3fa:
description: Updated user
gid: 13
name: moloIn the database, name is included but not login.
This is the recommended method for complex attributes for which the base request cannot be enough. For example, look for an attribute in an other YAML file, in an other SQL table, or in an other LDAP entry.
class YamlAttribute(DatabaseAttribute):
def __init__(self, path):
"""
:param attribute: Name of the field to load from the Yaml file
"""
self.path = path
def search_request(self, base_request, _id):
# Request the attribute at path, relative the base search
return YamlSearchRequest(base_request.path + self.path)
def load(self, base_response, attribute_response):
# The value must be deleted from the response so this field is not
# included in the generated JSON-like dict
delete(base_response.value, self.path)
# Returns the value of the attribute obtained from the attribute search request
return attribute_response.value
def create_request(self, base_request: YamlCreateRequest, value):
# Removes the replaced path from the original user item
delete(base_request.value, self.attribute_path)
# Creates the attribute at path, relative the base search
return YamlCreateRequest(base_request.path + self.path, value)
def update_request(self, base_request, _id, value):
# Removes the replaced path from the original user item
delete(base_request.value, self.attribute_path)
# Updates the attribute at path, relative the base search
return YamlUpdateRequest(base_request.path + self.path, value)
def delete_request(self, base_request, _id):
# Deletes the attribute at path, relative the base search
# This is actually not necessary as the complete object is deleted by
# the base request
return YamlDeleteRequest(base_request.path + self.path)The same example as before can be used without any modification, and the result is the same:
$ uv run examples/yaml/attributes.py
Item 49025c47-f509-4e16-82d3-5d40b76b3aca created.
{'description': 'Example user', 'gid': 12, 'login': 'pipo', '_id': '49025c47-f509-4e16-82d3-5d40b76b3aca'}
Item 49025c47-f509-4e16-82d3-5d40b76b3aca updated.
{'description': 'Updated user', 'gid': 13, 'login': 'molo', '_id': '49025c47-f509-4e16-82d3-5d40b76b3aca'}
Item 49025c47-f509-4e16-82d3-5d40b76b3aca deleted.