TP-841 - Order and partition transactions into partitions: SEED_FILE, REVISION, UPDATE.

[stuaxo]

This PR introduces partitioning to Transactions.

Global ordering is achieved by ordering on partition, order.

Partitions are:

SEED_FILE: Transactions imported from the initial seed file.
REVISION: Transactions imported or updated later.
DRAFT: Transactions in a workbasket.

During seed file update transactions may transitions are in the SEED_FILE partition.
During ordinary operation transactions are created as DRAFT, then when the WorkBasket is approved the transactions are changed to UPDATE and the order is rewritten to start after the highest REVISION transaction.

[codecov-commenter]

Codecov Report

Merging #334 (c0b6a9c) into master (ef6dc60) will increase coverage by 0.02%.
The diff coverage is 92.37%.

@@            Coverage Diff             @@
##           master     #334      +/-   ##
==========================================
+ Coverage   91.26%   91.29%   +0.02%     
==========================================
  Files         276      276              
  Lines       16322    16572     +250     
  Branches     1326     1348      +22     
==========================================
+ Hits        14897    15129     +232     
- Misses       1193     1206      +13     
- Partials      232      237       +5     

Impacted Files	Coverage Δ
additional_codes/models.py	95.23% <ø> (ø)
common/business_rules.py	97.69% <ø> (ø)
common/tests/test_serializers.py	100.00% <ø> (ø)
exporter/tests/test_exporter_tasks.py	100.00% <ø> (ø)
importer/forms.py	78.12% <50.00%> (+0.70%)	⬆️
importer/management/commands/run_import_batch.py	63.15% <66.66%> (+3.15%)	⬆️
exporter/sqlite/tasks.py	92.30% <75.00%> (-7.70%)	⬇️
workbaskets/models.py	88.60% <81.25%> (-5.40%)	⬇️
importer/tasks.py	66.66% <85.71%> (+0.51%)	⬆️
common/models/transactions.py	86.73% <88.46%> (+2.11%)	⬆️
... and 16 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ef6dc60...c0b6a9c. Read the comment docs.

[simonwo]

I like where this is going – will need some tests ofc.

To what extent should we be using the DRAFT/FINALISED flag elsewhere? Would it be more efficient to use it in TrackedModelQuerySet.approved_up_to_transaction(…) for example, instead of referring to the workbasket status?

[simonwo]

I'm worried that UPDATE already has another use in our system as one of UPDATE, CREATE and DELETE update types. Is there another word?

Maybe REVISION?

[stuaxo]

@simonwo one (non essential) thing I wanted to get working on this, was using RowNumber in the update(...) statement that corrects the order this would mean we could write transactions without any gaps, in the end I couldn't get this working, probably/hopefully missing something obvious there.

I guess gaps shouldn't be a problem - (though it opens a question about deletions: can transactions in a draft workbasket be deleted (probably) - and in that case should we consider removing them before making putting them in the REVISION partition ?

[stuaxo]

I like where this is going – will need some tests ofc.

To what extent should we be using the DRAFT/FINALISED flag elsewhere? Would it be more efficient to use it in TrackedModelQuerySet.approved_up_to_transaction(…) for example, instead of referring to the workbasket status?

Yes, probably everywhere - I think this should try and land and we can move everything else over that hasn't been as future PRs.