Pickle file path issue in Viya 3.5

[andysas]

Describe the issue
The python model is not able to locate the pickle file post registration in SAS Model Manager in SAS Viya 3.5. As a result of that, the model is not able to score data.

To Reproduce
Just use the example listed pzmmModelImportExample and register the final models in a Viya 3.5 image.

Expected behavior
I checked the *score.py file in model manager to check the path of the pickle file on the server and post that logged into the server to check the path there as well. On the server I was able to see a path(symlink) for astore files but nothing for pickle files. I am assuming a similar path should exist for pickle files as well. Refer screenshots for details.

Stack Trace
If you're experiencing an exception, include the full stack trace and error message.

Version
What version of sasctl are you using? 1.7.1

pzmm_error.docx

[smlindauer]

Hey @andysas,

That looks like a configuration issue within Viya 3.5. You'll need an admin to properly set up the python resources path.

With sudo access, running the following calls in bash will create the appropriate resources directory:
cd /
mkdir models
cd models
mkdir resources
cd resources
ln -s /opt/sas/viya/config/data/modelsvr/resources /models/resources/viya

[smlindauer]

This process is similar to the documentation on setting up the astore resource environment. See here.

[andysas]

Thanks for the prompt reply. It solved the pickle file related error. Getting a different error now. I think this might be related to the github example.
dtree_model_log.txt

[smlindauer]

It looks like the relevant Python error was truncated (pasted below). Can you score the model locally from the pickle file? Also, what dataset are you using?

I just reran the notebook myself aimed at a Viya 3.5 server and didn't have any errors with my testing dataset.

ERROR: Traceback (most recent call last):
File "/tmp/tmpw742p_bj/model_exec_83377731-2b29-484f-83a2-412b3a4f0116.py", line 23, in scoreDecisionTreeClassifier
prediction = _thisModelFit.predict(inputArray)
File "/opt/anaconda3/lib/python3.7/site-packages/sklearn/tree/_classes.py", line 419, in predict
X = self._validate_X_predict(X, check_input)
File "/opt/anaconda3/lib/python3.7/site-packages/sklearn/tree/_classes.py", line 380, in _validate_X_predict
X = check_array(X, dtype=DTYPE, accept_spa..

[andysas]

Hi
I am using HMEQ for scoring the data and the pickle file is able to score the data in my local python environment. I created a smaller table and ran a score test on that in Model Manager and I am seeing the above log messages in that as well (not sure why it is happening).

model_log.txt

[smlindauer]

Have you checked your python environment on Viya to make sure it has all of the correct versions of packages installed? Otherwise, we'd need to check the logs for more detailed error messages. Typically on Viya 3.5, you can find the logs at /opt/sas/viya/config/var/log/.

[andysas]

My desktop's Python version is 3.6 and the Python version on the RACE image is 3.7.3 and I am assuming my model should be upward compatible. I did not find anything in the above logs. Are there any other logs I should check?

[smlindauer]

I would agree with the upwards compatibility of Python versions. Do the packages you are utilizing with your model match between your local environment and the RACE image?

I'll try and find a more specific log location for checking the logs.

[andysas]

Hi
I identified an issue with the pickle file I created earlier in my dekstop. I used the Python instance on the RACE image to test out the pickle file( just to see if it is working on a different Python version), the scoring code threw some errors in the sklearn package (seems like Python is not fully upward compatible). Now, I am thinking of building the model on the Python instance on the RACE image and see if that works for me correctly.
My earlier query remains, is there an easy way to check logs and debug these issues? From the log messages I was getting earlier, it is difficult to exactly pin point what the issue is.

[smlindauer]

@andysas,
Other than access to the Viya 3.5 server, there is not an easy way to check on the truncated logs for Python models. You can create diagnostic models, which spit out the full Python log to a separate file or to the UI outputs, but otherwise, you will need to check the logs internal to the server.