fix(xmlupload): fix performance slowdown during stash applying (DEV-2717) #533
Conversation
DEV-2717 xml_upload_stash: improve "_replace_internal_ids_with_iris" function
The function _replace_internal_ids_with_iris function in the file xml_upload_stash.py is slowing down the entire xmlupload. The function iterates over an entire dictionary to find the id to replace. The bigger the dictionary, the longer the function takes. in a file with 10 links this replace function takes 415ns. In a file with 1000 links this function takes 278ms 420ns. This is longer than by a factor of nearly 670'000. This slow dictionary function will be replaced. |
BaseError is built differently from KeyError. Therefore how the message has to be retrieved is different too.
Nora-Olivia-Ammann
changed the title
New methods to extract and replace string in KnoraStandoffXml
Rewrite _replace_internal_ids_with_iris method
Fix docstring remove separate method to replace (testing should be with the class)
The IRI prefix is only if it is an internal id
…ce_internal_ids_with_iris-function
mypy complained that the unittest (next commit) were not typed
make function private
Add unittest for new replace function
Nora-Olivia-Ammann
changed the title
|
@Nora-Olivia-Ammann I just realized that there was already a Linear ticket for this: DEV-2667. Once your PR is merged, could you close that ticket with a comment that it was resolved by this PR? And could you also close the ticket DEV-1981 with a comment that it was probably resolved by this PR? (But only after this PR is merged) |
…ce_internal_ids_with_iris-function
Nora-Olivia-Ammann
deleted the
wip/dev-2717-xml_upload_stash-improve-_replace_internal_ids_with_iris-function
branch
No description provided.