Relative path to JSON file does not work

[HolQue]

I called the JsonPreprocessor with a relative path to a JSON file:

oJsonPreprocessor.jsonLoad("./file.jsonp")

Outcome:

Traceback (most recent call last):
  File "D:\ROBFW\Dev\JsonPreprocessor_examples\jpp_examples.py", line 61, in <module>
    dictReturned = oJsonPreprocessor.jsonLoad("./file.jsonp")
  File "D:\RobotFramework\python39\lib\site-packages\JsonPreprocessor\CJsonPreprocessor.py", line 716, in jsonLoad
    os.chdir(jsonPath)
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: ''

Reason is that 'jsonPath' is empty; therefore os.chdir(jsonPath) does not work.

This is the code of CJsonPreprocessor.py:

currentDir = os.getcwd()
os.chdir(jsonPath)

CJSONDecoder = None
if self.syntax != CSyntaxType.json:
    if self.syntax == CSyntaxType.python:
        CJSONDecoder = CPythonJSONDecoder
    else:
        raise Exception(f"Provided syntax '{self.syntax}' is not supported.")

try:
    oJson = json.loads(sJsonDataUpdated,
                       cls=CJSONDecoder,
                       object_pairs_hook=self.__processImportFiles)
except Exception as error:
    raise Exception(f"json file '{jFile}': '{error}'")

oJson = __handleStrNoneTrueFalse(oJson)
os.chdir(currentDir)

Why is it necessary to change the current working directory to jsonPath? In my opinion this should not be necessary.

Why is the content of jsonPath not checked before usage?

The current code is a very confusing mixture of several different path computations at different positions. This way of path handling is dangerous and urgently should be avoided.

Example:

self.lImportedFiles.append(os.path.abspath(jFile))
(jsonPath,tail)=os.path.split(jFile)

This does not work. In case of jFile is relative, then jsonPath is empty. The first line works with a path modified by os.path.abspath(), the second line does not. This is so much confusing. Why do you handle both in a different way?

Alternative:

jFile = os.path.abspath(jFile)
self.lImportedFiles.append(jFile)
jsonPath = os.path.dirname(jFile)

Relative paths to JSON files are working now.

In general:

Whenever you receive a path, you have to normalize this path as early as possible (CString.NormalizePath). This includes not to work with any relative path inside the code any more. You have to check if the path exists before you use it. In case of it's needed to know the position of a file or folder use: os.path.dirname(...) instead of manipulating the current working directory with os.chdir(). In case you implement in this way and work with absolute (normalized) paths only, you do not need os.chdir(jsonPath) any more.

Another aspect:

At the beginning of the code listed above, you get the current working directory:
currentDir = os.getcwd()

At the end you restore:
os.chdir(currentDir)

But in between at two positions you raise an exception (without restoring the working directory also at those positions). But you have to restore:

except Exception as error:
    os.chdir(currentDir)
    raise Exception(f"json file '{jFile}': '{error}'")

[test-fullautomation]

Hi Son,
please check and implement Holger's proposals. Absolute (inclusiv UNC paths) and relative paths must work in any case properly.
Thank you,
Thomas

[test-fullautomation]

@namsonx : What is the status of this issue?

[namsonx]

Hello Thomas,

I checked and could not reproduce this issue, the scenario mentioned in this ticket is still working fine.
For this ticket, if require, I will use CString.NormalizePath in PythonExtensionsCollection.String.CString. And when install JsonPreprocessor package, the PythonExtensionsCollection is a dependent package.

Thank you,
Son

[HolQue]

Hi Son, Hi Thomas,

I do not understand what is going on here.

• The issue is still reproducible
• The scenario is still not working fine
• The path computation inside the code is still improper
• The PythonExtensionsCollection is already a dependent package

[namsonx]

Hello Holger,

Currently, I have test.py file with content:

from JsonPreprocessor import CJsonPreprocessor
import re
oJsonPreprocessor = CJsonPreprocessor(syntax="python")

oJsonCfgData = oJsonPreprocessor.jsonLoad('./json_test.json')

for k, v in oJsonCfgData.items():  
    print(f"{k} : {v} - {type(v)}")

and it's working fine.

Why is it necessary to change the current working directory to jsonPath?
In JsonPreprocessor code, I changed the current directory to jsonPath because imported json files are aligned with Json configuration file.

Thank you,
Son

[namsonx]

Hello Holger,

Thank you very much due to the short call regarding to this issue.
The issue was not occurred at my end because of the difference between my development environment and normal environment.
I will revise the code based on your hints with NormalizePath function of PythonExtensionsCollection package.

Thank you,
Son

[namsonx]

Hello Holger, Hello Thomas,

I pushed the fix to stabi branch and created new pull-request #149 for this ticket.

Thank you,
Son