DHIS2-8065:Require a specific authority to attribute data values

[jason-p-pickering]

As detailed in DHIS2-8065, the current behavior of the system allows for storedBy, created and lastUpdated fields to be overridden by values contained in the data value set payload. This creates significant issues for accountability, auditing and debugging in systems who receive data via the dataValueSets endpoint.

This change in the API behavior will respect the current behavior for superusers who need to import data on behalf of other users or for other purposes. However, for non-superusers, the values specified for these fields in the payload will be ignored. The storedBy will be set to the current user name of the user importing the data. The created value will not be able to be be overridden. If the value is an existing value, the created date will be maintained, otherwise it will default to the current date.
For lastUpdated this will always default to the current time regardless of what is specified in the payload for non-superusers.

[codecov]

Codecov Report

Attention: 321 lines in your changes are missing coverage. Please review.

Comparison is base (886690f) 58.58% compared to head (3e97b7b) 65.13%.
Report is 185 commits behind head on 2.39.

Additional details and impacted files

@@             Coverage Diff              @@
##               2.39   #16494      +/-   ##
============================================
+ Coverage     58.58%   65.13%   +6.54%     
- Complexity    25817    28916    +3099     
============================================
  Files          3207     3218      +11     
  Lines        121397   122261     +864     
  Branches      14158    14257      +99     
============================================
+ Hits          71118    79630    +8512     
+ Misses        44340    36126    -8214     
- Partials       5939     6505     +566     

Flag	Coverage Δ
integration	49.28% <49.91%> (+0.68%)	⬆️
integration-h2	34.47% <33.11%> (?)
unit	29.77% <40.37%> (+0.39%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
...main/java/org/hisp/dhis/appmanager/AppManager.java	100.00% <ø> (ø)
...src/main/java/org/hisp/dhis/category/Category.java	65.21% <100.00%> (ø)
.../src/main/java/org/hisp/dhis/common/AuditType.java	100.00% <100.00%> (+10.00%)	⬆️
...ava/org/hisp/dhis/common/BaseAnalyticalObject.java	70.95% <100.00%> (+5.81%)	⬆️
...a/org/hisp/dhis/common/BaseIdentifiableObject.java	80.20% <100.00%> (+1.37%)	⬆️
...s-api/src/main/java/org/hisp/dhis/common/Grid.java	0.00% <ø> (ø)
.../src/main/java/org/hisp/dhis/common/QueryItem.java	68.66% <100.00%> (ø)
...rg/hisp/dhis/dataanalysis/DataAnalysisService.java	0.00% <ø> (ø)
.../org/hisp/dhis/dataanalysis/DataAnalysisStore.java	0.00% <ø> (ø)
...isp/dhis/dataapproval/DataApprovalPermissions.java	91.66% <ø> (+22.43%)	⬆️
... and 125 more

... and 610 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b6906ee...3e97b7b. Read the comment docs.

[sonarcloud]

Quality Gate passed

Issues
0 New issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[jason-p-pickering]

Thinking more about this, it would seem we should introduce a separate authority "Allow data value attribution". This would allow users which have this authority to attribute either via the dataValue or dataValueSets end points the storedBy, created and lastUpdated fields of data values. Considering that this behavior has been in the code base for a very long time, its not really known how people are using it. However, by granting this authority (without granting super) this would allow what is essentially the current behavior.

It is worth pointing out however that overriding the data value attribution is not possible via the dataValue endpoint, so the question might be whether we maintain this behavior or also allow overriding of these attributes via that point as well via the proposed authority.

[sonarcloud]

Quality Gate failed

Failed conditions
2 New issues
2 New Code Smells (required ≤ 0)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint