Partial map updates #121
Closed
Partial map updates #121
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In the current implementation, every time an element of either a map or set column has to be modified, the entire content of the column is sent to the server to be updated. This is not a major problem if the information contained in the column for the corresponding row is small, but there are cases where these columns can have a significant amount of elements per row, or these values are updated frequently, therefore the cost of the modifications becomes high in terms of time and bandwidth. In this solution, the ovsdb-idl code is modified to use the RFC 7047 'mutate' operation, to allow sending partial modifications on map columns to the server. The functionality is exposed to clients in the vswitch idl. This was implemented through map operations. A map operation is defined as an insertion, update or deletion of a key-value pair inside a map. The idea is to minimize the amount of map operations that are send to the OVSDB server when a transaction is committed. In order to keep track of the requested map operations, structs map_op and map_op_list were defined with accompanying functions to manipulate them. These functions make sure that only one operation is send to the server for each key-value that wants to be modified, so multiple operation on a key value are collapsed into a single operation. As an example, if a client using the IDL updates several times the value for the same key, the functions will ensure that only the last value is send to the server, instead of multiple updates. Or, if the client inserts a key-value, and later on deletes the key before committing the transaction, then both actions cancel out and no map operation is send for that key. To keep track of the desired map operations on each transaction, a list of map operations (struct map_op_list) is created for every column on the row on which a map operation is performed. When a new map operation is requested on the same column, the corresponding map_op_list is checked to verify if a previous operations was performed on the same key, on the same transaction. If there is no previous operation, then the new operation is just added into the list. But if there was a previous operation on the same key, then the previous operation is collapsed with the new operation into a single operation that preserves the final result if both operations were to be performed sequentially. This design keep a small memory footprint during transactions. When a transaction is committed, the map operations lists are checked and all map operations that belong to the same map are grouped together into a single JSON RPC "mutate" operation, in which each map_op is transformed into the necessary "insert" or "delete" mutators. Then the "mutate" operation is added to the operations that will be send to the server. Once the transaction is finished, all map operation lists are cleared and deleted, so the next transaction starts with a clean board for map operations. Using different structures and logic to handle map operations, instead of trying to force the current structures (like 'old' and 'new' datums in the row) to handle then, ensures that map operations won't mess up with the current logic to generate JSON messages for other operations, avoids duplicating the whole map for just a few changes, and is faster for insert and delete operations, because there is no need to maintain the invariants in the 'new' datum. Signed-off-by: Edward Aymerich <edward.aymerich@hpe.com> Signed-off-by: Arnoldo Lutz <arnoldo.lutz.guevara@hpe.com> Co-authored-by: Arnoldo Lutz <arnoldo.lutz.guevara@hpe.com>
Code inserted that autogenerates corresponding map functions to set and delete elements in map columns. Inserts description to the functions that are autogenerated. Signed-off-by: Edward Aymerich <edward.aymerich@hpe.com> Signed-off-by: Arnoldo Lutz <arnoldo.lutz.guevara@hpe.com> Co-authored-by: Arnoldo Lutz <arnoldo.lutz.guevara@hpe.com>
Insert basic functionality for testing partial update column and add a new test table named "simple2". Signed-off-by: Edward Aymerich <edward.aymerich@hpe.com> Signed-off-by: Arnoldo Lutz <arnoldo.lutz.guevara@hpe.com> Co-authored-by: Arnoldo Lutz <arnoldo.lutz.guevara@hpe.com>
Fix non-ANSI declaration of a function inside 'ovsdb-map-op.c'. Signed-off-by: Edward Aymerich <edward.aymerich@hpe.com> Signed-off-by: Arnoldo Lutz <arnoldo.lutz.guevara@hpe.com> Co-authored-by: Arnoldo Lutz <arnoldo.lutz.guevara@hpe.com>
Signed-off-by: Edward Aymerich <edward.aymerich@hpe.com> Signed-off-by: Arnoldo Lutz <arnoldo.lutz.guevara@hpe.com> Co-authored-by: Arnoldo Lutz <arnoldo.lutz.guevara@hpe.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In the current implementation, every time an element of either a map or set column has to be modified, the entire content of the map/set is sent to the server to be updated. This improvement allows to partially update a map, sending just the modifications (inserts, deletes or updates) to the OVSDB server.
This submission includes a fix to a bug found in a previous submission, in which the IDL crashed when a new row was created and used as reference inside a map in the same transaction.