Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not accept invalid units #982 #986

Merged
merged 2 commits into from
Apr 29, 2021
Merged

Do not accept invalid units #982 #986

merged 2 commits into from
Apr 29, 2021

Conversation

abdelr
Copy link
Member

@abdelr abdelr commented Apr 29, 2021

Do not accept invalid units #982

@abdelr abdelr added the Importer-Redesign Redesign of observed data import label Apr 29, 2021
@abdelr abdelr added this to the Version 10 milestone Apr 29, 2021
@abdelr abdelr self-assigned this Apr 29, 2021
@abdelr abdelr linked an issue Apr 29, 2021 that may be closed by this pull request
@@ -39,6 +44,7 @@ protected override UnitDescription ExtractUnits(string description, IUnformatted
.Substring(1, units.Length - 2) //remove the brackets
.Trim() //remove whitespace
.Split(',') //split comma separated list
.Where(unitName => _dimensionFactory.Dimensions.SelectMany(dimension => dimension.Units).Any(unitFromDimension => unitFromDimension.Name == unitName)) //only accepts valid units
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think this is correct here. We have a lot of methods checking whether a unit is valid or not (taking care of case, synonyms etc...)
This needs to change

Copy link
Member

@msevestre msevestre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checking of units is not what I expected. Look at existing method

For instance
IDimension DimensionForUnit(string unitName)

if the retruend value is null., it is not a valid dimension.

@abdelr abdelr requested a review from msevestre April 29, 2021 13:18
@@ -39,6 +44,7 @@ protected override UnitDescription ExtractUnits(string description, IUnformatted
.Substring(1, units.Length - 2) //remove the brackets
.Trim() //remove whitespace
.Split(',') //split comma separated list
.Where(unitName => _dimensionFactory.DimensionForUnit(unitName) != null) //only accepts valid units
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same logic should be used for other unit recognition by the way (in column headeR)
Is this the case @abdelr and @georgeDaskalakis

Copy link
Member

@msevestre msevestre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merging. Please review my comment

@msevestre msevestre merged commit 062aa03 into develop Apr 29, 2021
@msevestre msevestre deleted the 982_invalid_unit branch April 29, 2021 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Importer-Redesign Redesign of observed data import
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Data import: Invalid units in column headers are not properly handled
3 participants