${...} substitution

[HolQue]

Reference: #69

I tried some use cases:

"str1" : "ABC",
"str2" : "DEF",
"str3" : ${str1},
"list" : ["A", "B", "C"],

// "str4" : ${str1}_test_${str2}
// -> Expecting ',' delimiter

// "str5" : "${str1}_test_${str2}"
// -> Expecting ',' delimiter

// ${str1}_test_${str2} : "MNO",
// -> Expecting property name enclosed in double quotes

// "${str2}_test_${str3}" : "SPQR"
// -> Expecting ':' delimiter

// "${str1}_test_${list}" : "XYZ"
// -> Expecting ':' delimiter

• str1 - list are successful.
• str4 and str5 cause an error.

What shall be the desired implementation in case of multiple substitutions on the right hand side of the colon: expression encapsulated in quotes or not?

Same question in case of multiple substitutions on the left hand side of the colon: expression encapsulated in quotes or not?

Both causes errors.

The last example contains a composite data type ${list} on the left hand side of the colon. As per
#69 (comment)
this should not be allowed. But the issue comment is related to the Testsuites Management. For me the question now is: Shall the JsonPreprocessor react in the same way or shall the JsonPreprocessor allow composite data types on the left hand side of the colon?

Nevertheless: The error message should not tell about a missing delimiter. This is not helpful.

[HolQue]

Addendum:

While playing around I overlooked something. list is a data type. It is not possible to use names of data types as parameter name. This does not work:

   "str"  : ${str1},
   "list" : ["A", "B", "C"]

Must be something different instead:

   "str1"  : ${str1},
   "list2" : ["A", "B", "C"]

I understand that the first version is not possible, but the error message is not nice:

Traceback (most recent call last):
  File "D:\ROBFW\Dev\JsonPreprocessorTest\JsonPreprocessorTest.py", line 27, in <module>
    dictValues = oJsonPreprocessor.jsonLoad(r"./config/test1.jsonp")
  File "D:\RobotFramework\python39\lib\site-packages\JsonPreprocessor\CJsonPreprocessor.py", line 740, in jsonLoad
    oJson, bNested = self.__updateAndReplaceNestedParam(oJson)
  File "D:\RobotFramework\python39\lib\site-packages\JsonPreprocessor\CJsonPreprocessor.py", line 520, in __updateAndReplaceNestedParam
    elif isinstance(v, list):
TypeError: isinstance() arg 2 must be a type or tuple of types

Maybe this can be improved.

Addendum:

It can be improved by:

import keyword
key_name = "import"
if key_name in keyword.kwlist:
   print(f"Invalid keyname '{key_name}' (name is reserved for Python)")

[HolQue]

Another use case: composite data type as part of expression on left hand side of colon; expression not encapsulated in quotes:

   "str2" : "DEF",
   "values" : ["A", "B", "C"],
   ${values}_test_${str2} : "XYZ"

Outcome: values_test_DEF = 'XYZ'

Now I am not sure about the expectation any more. Shall ${values} be resolved or not? I do not believe that simply using the parameter name as string is really helpful.

Addendum 1:

Regarding ${values}_test_${str2} : "XYZ" I learned in between: Because the key name is a mix of parameters and hard coded parts, the entire expression must be enclosed in quotes:

"${values}_test_${str2}" : "XYZ"

The version without quotes on the left hand side of the colon is therefore a syntax violation and should cause a corresponding error message.

Rule: If expression contains one or more ${...} together with parts outside ${...}, but is not encapsulated in quotes, then syntax violation error message.

The version with quotes must also not be accepted because ${values} refers to a composite data type that is not allowed as part of a key name.

Addendum 2:

Same with (mentioned above):

// "str4" : ${str1}_test_${str2}
// -> Expecting ',' delimiter

This is a valid error! Because the value is a mix of parameters and hard coded parts, the entire expression must be enclosed in quotes. But the error message Expecting ',' delimiter makes no sense.

The same rule like mentioned above should be applied: If expression contains one or more ${...} together with parts outside ${...}, but is not encapsulated in quotes, then syntax violation error message.

[HolQue]

Regarding this error:

Expecting ':' delimiter

caused by lines e.g. like

"ABC_${str3}_test_${str1}_XYZ" : "MNOT",

In CJsonPreprocessor.py I added a print (print(f"{sJsonDataUpdated}")):

try:
    oJson = json.loads(sJsonDataUpdated,
                       cls=CJSONDecoder,
                       object_pairs_hook=self.__processImportFiles)
except Exception as error:
    print(f"{sJsonDataUpdated}")

Outcome:

"ABC_"${str3}"_test_"${str1}"_XYZ" : "MNOT",

Oh, this is not nice. Nested double quotes are not possible here. This does not work.

But I wonder why inside sJsonDataUpdated the parameters are not already resolved, and I also wonder why there are double quotes around "${str3}" and "${str1}"? It makes no sense to add them.

[test-fullautomation]

Hi Son,
please look into this.
Expectation is what Holger writes.
Thank you,
Thomas

[namsonx]

Hello Holger,

The scenario:
// "str5" : "${str1}test${str2}"
// -> Expecting ',' delimiter
This case is working fine now with pull-request #81

Thank you,
Son

[HolQue]

Hello Son,

the situation is unclear to me. You have updated an already merged pull request? Is this a good idea?

Git tells me that I am already up to date. Therefore I downloaded the Python file directly from this web page. But I got a file that does not contain your recent changes (e.g. the reset function).

For me it makes no sense to continue in this way. You should create a fresh new branch, and as long we are in summer vacation phase you should upload all changes in this single branch only. Please no different versions located in different branches or pull requests in parallel. Otherwise no one will know what he can expect.

[namsonx]

Hello Holger,

I'm still implement the scenarios you mentioned in this tickets, when completed I will create new branch for testing purpose.

Regarding to the scenarios below:
// ${str1}test${str2} : "MNO",
// -> Expecting property name enclosed in double quotes

// "${str2}test${str3}" : "SPQR"
// -> Expecting ':' delimiter

// "${str1}test${list}" : "XYZ"
// -> Expecting ':' delimiter
A key dose not allow containing more than 2 nested parameters, because when a nested parameter is defined as a key, it means user want to overwrite new value for existing parameter.
For example:
I have Json config file like below:

{
    "nested_param" : 2017,
    "test_param" : "Just for testing",
    "var" : {
        "var_1": {
            "var_2" : {
                "var_3" : "initialized"
            }
        }
    },
    ${var}['var_1']['var_2']['var_3']: "new value",      
    "val08" : null
 }

The line ${var}['var_1']['var_2']['var_3']: "new value", will update the value of var['var_1']['var_2']['var_3'] to "new value" not set "initialized" : "new value".

Thank you,
Son

Addendum:

Hi Son,

yes! And when you add quotes like:

"${var}['var_1']['var_2']['var_3']" : "new value",

then the JsonPreprocessor should provide:

"initialized" : "new value"