[FIX] Consistent import/export on db ids, no external ids #34
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…g the capabilities of CSV, it needs to be discussed whether it is worth it to add it
… and discussion about delimiters and required features
…import and export
…t export would stay stuck in pending with no additional info
When importing a huge file that take a long time it's quiet borring to need to reimport all valid line Here we add the possibility to commit every valid line, then you can do a smaller file with only failling line Add also the possibility to configure the flush step. Indeed when doing an import of a huge file if one line fail then odoo will import the line one by one and it will so fucking long doing smaller flush reduce this issue
kevinkhao
changed the title
kevinkhao
force-pushed
the
12.0-extid-temp-fix
branch
2 times, most recently
from
ed4c069
to
ccca603
Compare
kevinkhao
commented
Oct 16, 2020
| @@ -284,7 +296,7 @@ def _generate_import_with_pattern_job(self, patterned_import): | |||
| datas = self._read_import_data(attachment_data) | |||
| except Exception as e: | |||
| patterned_import.status = "fail" | |||
| patterned_import.info = _("Failed (check details)") | |||
| patterned_import.info = "Failed (check details)" | |||
| patterned_import.info_detail = e | |||
| res = ( | |||
There was a problem hiding this comment.
@sebastienbeau as we discussed before: the issue is that we presume that load([]) will always fail as we want it to fail, which is not the case. For example:
- We try to load example.users.fail.csv : our first column is id. Instead of putting an integer, we put a string. This will cause an error with psycopg2 (bad request in pg). This is not caught and will raise outside of the patterned.import.export.
- We try to load example.users.ok.csv with bad Pattern config: we change separators to something like %, then the entire first line will be considered one value (because we don't find any % in the line and then we have \n). So our first value is: "id,name,company_ids". This will raise too.
There was a problem hiding this comment.
I just push a fix and also created a PR for Odoo V14 : odoo/odoo#60260
Codecov Report
@@ Coverage Diff @@
## 12.0 #34 +/- ##
==========================================
+ Coverage 87.65% 88.31% +0.65%
==========================================
Files 10 12 +2
Lines 729 847 +118
==========================================
+ Hits 639 748 +109
- Misses 90 99 +9
Continue to review full report at Codecov.
|
sebastienbeau
force-pushed
the
12.0-extid-temp-fix
branch
from
8e15e07
to
66292dc
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Because of issues, in the current module functionality, we must only consider in-DB ids, not external IDs. Here are the issues with extids:
This is a temporary measure and should be fixed later on.
Contains #32