country.name can not deal with special characters

[kwa20]

Bug Description

when defining a country.name in the properties, only specific formats can be read. This is a problem when countries are missing in the standard import (cf. #4782 (comment)) or were generally added to the list. These countries, to count as the server country, need to be set according to the deposited value. However, the properties file does not allow spaces, quotations marks, apostrophes or special characters. This leads to country names having to be adjusted in undesired ways in order to allow importing of districts.

Steps to Reproduce

1. Add "Côte d'Ivoire" as a country in configurations
2. Set country.name="Côte d'Ivoire"
3. Import regions and districts. (Only the district import should through an error saying that the region is not in server country.)
4. Change the country name to "Cote_dIvoire" in configurations
5. Set country.name=Cote_dIvoire
6. Import districts again

Expected Behavior

Any deposited country should be able to be set as country.name and still maintain its proper caption

Screenshots

System Details

• Device:
• SORMAS version: 1.58.3/1.60
• Android version/Browser: firefox
• Server URL: https://test-inhp-ci.sormas.netzlink.com/
• User Role: admin, national user

Additional Information

tested country.name options

country.name	result
"Côte d'Ivoire"	fail
Côte d'Ivoire	fail
Ivory Coast	fail
Ivory_Coast	success
Côte_d'Ivoire	fail
Côte_dIvoire	fail
Cote_dIvoire	success
"Ivory Coast"	fail

error report:
sormas_import_error_ivorycoast(1).txt

district import template:
sormas_import_district_template_ivoryCoast(1).txt

Possible solutions

[These should merely be suggestions with the limited knowledge I have about the technical details]

• Allow text in between "" to be read in the properties file
• Add country references to lower hierarchy infrastructure (might also have the advantages that these hierarchies can be imported at all for other countries then set in country.name)
• restructure infrastructure so that each line has its own value independent of the name which then will be used as a caption. E.g. when creating an entry for Côte d'Ivoire, the entry produces a value that conforms to the needed format COTE_DIVOIRE and would then be referenced in the properties file.

[kwa20]

@bernardsilenou @Juliane-D here the bug report for the ivory coast issue.

@MartinWahnschaffe @Candice-Louw @T-Hundt for your information.

[bernardsilenou]

@kwa20 I would vote for the option to not use the name of server country but a some form of a code; So when server is initialize, there is a code assign to the server country + country created as a country in the system. If user then change the name of the country, then code is not changed.
When importing data or using other logics, we then compare with the uuid of the server country rather than the name.

[MartinWahnschaffe]

Looks like we have to make sure the country name also supports white spaces and other signs and/or is validated if there are good reasons to not allow this.

[bernardsilenou]

@MartinWahnschaffe @kwa20
The same error also occur when importing facilities. Invalid value ABIDJAN 2 in column region; Region is not in server country even after importing region as a region in server country.
What about adding a colum on infrastructure data import template to specify the uuid of the country and use this uuid in all logics. The logic to map the facilities to their country seam not to be working well.

[bernardsilenou]

@kwa20 A short term solution can be to set the country name to just a string "CoteDivoir", and rename the country name in the regions data, maybe this may help to allow us import facilities.

[StefanKock]

country.name is used for technical selection in code but also shown to the user, it's likely we need to seperate this. Needs to be looked into and specified

[kwa20]

@MartinWahnschaffe I guess another problem that occurs at the same time is that no infrastructure data below the region hierarchy can be imported for any other countries besides the deposited server country right? I'm not sure if we can account for that by allowing the country.name to allow additional formats.

[MateStrysewske]

@kwa20 No, that shouldn't be a problem. If you have a region associated to a different country, you should be able to import districts for that region.

@MartinWahnschaffe @StefanKock Is it difficult to "simply" allow whitespaces for the country.name property? That would indeed be (by far) the best solution in my eyes. Also, this is a critical bug that needs to be resolved in the upcoming release.

[kwa20]

@MateStrysewske when I try importing data for a region in a different country I get this error message. Do you know what the reason for this could be?

test_import_districts_otherCountries_Error.txt

[MateStrysewske]

@kwa20 Ooh, okay, I see - looks like we disabled it then. Can't tell why though... @MartinWahnschaffe Do you recall anything related to this?

[StefanKock]

I defined country.name=Bundesrepublik Deutschland in sormas.properties, that was correctly read into SORMAS:

When taking the given in example Côte d'Ivoire in UTF-8 encoding it fails:

When making sure that sormas.properties is written in ISO-8859-1/ANSI it works:

For me it looks more like an encoding problem when setting the country.name. Some research showed that from JDK 9 on, the encoding is UTF-8 per default, but has a fallback. Here is a statement, that encoding special characters as \uHHHH helps encode characters in an ISO-8859-1 encoded file.

Naming the country C\u00F4te d'Ivoire in sormas.properties avoids encoding problems (see unicode search).

[leventegal-she]

Implemented a hot fix + opened #5821