Can't find PythonVersion on self hosted ubuntu image

[syonekura]

Hi,
I've built an image using Packer, cloning and following the instructions from azure pipelines image generation. The build succedeed and then I deployed a VM with that. I then assigned the VM as a build agent, so far there's no problems, but when I run a UsePythonVersion task the tool doesn't find the installed versions.

Following overview, the env variable AGENT_TOOLSDIRECTORY should overwrite the default _work/tools path, but it seems that this is not happening.

In the VM I can run echo $AGENT_TOOLSDIRECTORY and the result is /opt/hostedtoolcache (as expected), but when I run the pipeline with debug info, the variable Agent.ToolsDirectory keeps reading .../_work/tools

##[debug]Evaluating condition for step: 'UsePythonVersion'
##[debug]Evaluating: SucceededNode()
##[debug]Evaluating SucceededNode:
##[debug]=> True
##[debug]Result: True
##[section]Starting: UsePythonVersion
==============================================================================
Task         : Use Python version
Description  : Use the specified version of Python from the tool cache, optionally adding it to the PATH
Version      : 0.151.1
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/tool/use-python-version
==============================================================================
##[debug]agent.TempDirectory=~/myagent/_work/_temp
##[debug]loading inputs and endpoints
##[debug]loading INPUT_VERSIONSPEC
##[debug]loading INPUT_ADDTOPATH
##[debug]loading INPUT_ARCHITECTURE
##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
##[debug]loading SECRET_SYSTEM_ACCESSTOKEN
##[debug]loaded 7
##[debug]Agent.ProxyUrl=undefined
##[debug]Agent.CAInfo=undefined
##[debug]Agent.ClientCert=undefined
##[debug]Agent.SkipCertValidation=undefined
##[debug]Agent.ProxyUrl=undefined
##[debug]Agent.CAInfo=undefined
##[debug]Agent.ClientCert=undefined
##[debug]check path : ~/myagent/_work/_tasks/UsePythonVersion_33c63b11-352b-45a2-ba1b-54cb568a29ca/0.151.1/node_modules/azure-pipelines-tool-lib/lib.json
##[debug]adding resource file: ~/myagent/_work/_tasks/UsePythonVersion_33c63b11-352b-45a2-ba1b-54cb568a29ca/0.151.1/node_modules/azure-pipelines-tool-lib/lib.json
##[debug]system.culture=en-US
##[debug]check path : ~/myagent/_work/_tasks/UsePythonVersion_33c63b11-352b-45a2-ba1b-54cb568a29ca/0.151.1/task.json
##[debug]adding resource file: ~/myagent/_work/_tasks/UsePythonVersion_33c63b11-352b-45a2-ba1b-54cb568a29ca/0.151.1/task.json
##[debug]system.culture=en-US
##[debug]versionSpec=3.6
##[debug]addToPath=true
##[debug]architecture=x64
##[debug]Semantic version spec of 3.6 is 3.6
##[debug]isExplicit: null
##[debug]explicit? false
##[debug]Agent.Version=2.153.2
##[debug]Agent.ToolsDirectory=~/myagent/_work/_tool
##[debug]evaluating 0 versions
##[debug]match not found
##[debug]Agent.Version=2.153.2
##[debug]Agent.ToolsDirectory=~/myagent/_work/_tool
##[debug]Agent.Version=2.153.2
##[debug]Agent.ToolsDirectory=~/myagent/_work/_tool
##[debug]Agent.ToolsDirectory=~/myagent/_work/_tool
##[debug]task result: Failed
##[error]Version spec 3.6 for architecture x64 did not match any version in Agent.ToolsDirectory.
Versions in ~/myagent/_work/_tool:


If this is a Microsoft-hosted agent, check that this image supports side-by-side versions of Python at https://aka.ms/hosted-agent-software.
If this is a self-hosted agent, see how to configure side-by-side Python versions at https://go.microsoft.com/fwlink/?linkid=871498.

I guess if there's some other configuration step required to make this work

[damccorm]

Ok, so to be precise, the issue here is that AGENT_TOOLSDIRECTORY is set to one thing, but Agent.ToolsDirectory is getting set to something else?

In theory that shouldn't happen. To help diagnose - in your pipeline, before your usePythonVersion step, could you add a script to echo AGENT_TOOLSDIRECTORY? That should help us determine where exactly this issue is coming from. I believe that .../_work/tools is the default tools directory when none is found so my guess is that its being lost before then

[stephenmichaelf]

Closing due to inactivity. Please reopen with additional information.

[majorvin]

When I do ls im getting these fiiles instead.

/var/lib/waagent/custom-script/download/1/agent/_work/_tool
helm
kubectl

but doing it on the server is pointing to the right folders where I can see python, ruby and pypy

[dave-ops]

Not all python versions are available on all images. If you go to this link, click on the Available Software for the Image you are using, it will list all the software versions on your image.

For example: if you are running Ubuntu 16.04 or Ubuntu 18.04 the only 3.6.x version of python available to you is 3.6.9

https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops#software

[majorvin]

Hi @dave-ops we are talking about self hosted. and definitely I can confirm that 3.7.* and 3.8.* are all in toolscache folder.

[ionutleca]

Hi. If your agent is started as a service, it will not be aware of the AGENT_TOOLSDIRECTORY environment variable you set. https://github.com/microsoft/azure-pipelines-agent/blob/master/docs/start/nixsvc.md#setting-the-environment

Adding it to the .env file should solve your issue.

[AlexChongMicrosoft]

Hi. If your agent is started as a service, it will not be aware of the AGENT_TOOLSDIRECTORY environment variable you set. https://github.com/microsoft/azure-pipelines-agent/blob/master/docs/start/nixsvc.md#setting-the-environment

Adding it to the .env file should solve your issue.

Hi, from the pipeline log, I find that the env is already be recognized. But I still meet this issue. The log is below.

##[debug]versionSpec=3.7
##[debug]addToPath=true
##[debug]architecture=x64
##[debug]Semantic version spec of 3.7 is 3.7
##[debug]isExplicit: null
##[debug]explicit? false
##[debug]Agent.Version=2.165.2
##[debug]Agent.ToolsDirectory=/home/vmasuser/myagent/_work/_tool
##[debug]isExplicit: 3.7.7
##[debug]explicit? true
##[debug]evaluating 0 versions
##[debug]match not found
##[debug]Agent.Version=2.165.2
##[debug]Agent.ToolsDirectory=/home/vmasuser/myagent/_work/_tool
##[debug]isExplicit: 3.7.7
##[debug]explicit? true
##[debug]Agent.Version=2.165.2
##[debug]Agent.ToolsDirectory=/home/vmasuser/myagent/_work/_tool
##[debug]isExplicit: 3.7.7
##[debug]explicit? true
##[debug]Agent.ToolsDirectory=/home/vmasuser/myagent/_work/_tool
##[debug]task result: Failed
##[error]Version spec 3.7 for architecture x64 did not match any version in Agent.ToolsDirectory.
Versions in /home/vmasuser/myagent/_work/_tool:

If this is a Microsoft-hosted agent, check that this image supports side-by-side versions of Python at https://aka.ms/hosted-agent-software.
If this is a self-hosted agent, see how to configure side-by-side Python versions at https://go.microsoft.com/fwlink/?linkid=871498.