Skip to content

ld: cannot find -lpython3.10 #275

Closed
@joaodlf

Description

@joaodlf

Description:
Getting ld: cannot find -lpython3.10when installing some dependencies.

Platform:

  • Ubuntu
    macOS
    Windows

Runner type:

  • Hosted
    Self-hosted

Tools version:

3.10.0

We are trying to move our action to a self hosted environment, this is how the file looks:

name: Deploy

on:
  push:
    branches:
      - master

jobs:
  build:
    runs-on: self-hosted
    container: ubuntu:20.04
    services:
      postgres:
        ...

    steps:
      - uses: actions/checkout@v1

      ...

      - name: Setup Python
        uses: actions/setup-python@v2
        with:
          python-version: 3.10.0

      - name: Install poetry
        run: |
          python -m pip install poetry==1.1.11

      - name: Configure poetry
        run: |
          python -m poetry config virtualenvs.in-project true

      - name: Upgrade pip
        run: |
          python -m poetry run pip install --upgrade pip

      - name: Cache the virtualenv
        uses: actions/cache@v2
        with:
          path: ./.venv
          key: ${{ runner.os }}-venv1-${{ hashFiles('**/poetry.lock') }}

      - name: Install dependencies
        run: |
          python -m poetry install

     ...

We've managed to get to the Install dependencies step, most of the dependencies are installing correctly, but when we get to uwsgi, we are getting the following:

2021-11-17T14:24:25.7315863Z       gcc -pthread -o /__w/project/project/.venv/bin/uwsgi -L/opt/hostedtoolcache/Python/3.10.0/x64/lib -Wl,-rpath,/opt/hostedtoolcache/Python/3.10.0/x64/lib core/utils.o core/protocol.o core/socket.o core/logging.o core/master.o core/master_utils.o core/emperor.o core/notify.o core/mule.o core/subscription.o core/stats.o core/sendfile.o core/async.o core/master_checks.o core/fifo.o core/offload.o core/io.o core/static.o core/websockets.o core/spooler.o core/snmp.o core/exceptions.o core/config.o core/setup_utils.o core/clock.o core/init.o core/buffer.o core/reader.o core/writer.o core/alarm.o core/cron.o core/hooks.o core/plugins.o core/lock.o core/cache.o core/daemons.o core/errors.o core/hash.o core/master_events.o core/chunked.o core/queue.o core/event.o core/signal.o core/strings.o core/progress.o core/timebomb.o core/ini.o core/fsmon.o core/mount.o core/metrics.o core/plugins_builder.o core/sharedarea.o core/rpc.o core/gateway.o core/loop.o core/cookie.o core/querystring.o core/rb_timers.o core/transformations.o core/uwsgi.o proto/base.o proto/uwsgi.o proto/http.o proto/fastcgi.o proto/scgi.o proto/puwsgi.o lib/linux_ns.o core/yaml.o core/dot_h.o core/config_py.o plugins/python/python_plugin.o plugins/python/pyutils.o plugins/python/pyloader.o plugins/python/wsgi_handlers.o plugins/python/wsgi_headers.o plugins/python/wsgi_subhandler.o plugins/python/web3_subhandler.o plugins/python/pump_subhandler.o plugins/python/gil.o plugins/python/uwsgi_pymodule.o plugins/python/profiler.o plugins/python/symimporter.o plugins/python/tracebacker.o plugins/python/raw.o plugins/gevent/gevent.o plugins/gevent/hooks.o plugins/ping/ping_plugin.o plugins/cache/cache.o plugins/nagios/nagios.o plugins/rrdtool/rrdtool.o plugins/carbon/carbon.o plugins/rpc/rpc_plugin.o plugins/corerouter/cr_common.o plugins/corerouter/cr_map.o plugins/corerouter/corerouter.o plugins/fastrouter/fastrouter.o plugins/http/http.o plugins/http/keepalive.o plugins/http/https.o plugins/http/spdy3.o plugins/ugreen/ugreen.o plugins/signal/signal_plugin.o plugins/syslog/syslog_plugin.o plugins/rsyslog/rsyslog_plugin.o plugins/logsocket/logsocket_plugin.o plugins/router_uwsgi/router_uwsgi.o plugins/router_redirect/router_redirect.o plugins/router_basicauth/router_basicauth.o plugins/zergpool/zergpool.o plugins/redislog/redislog_plugin.o plugins/mongodblog/mongodblog_plugin.o plugins/router_rewrite/router_rewrite.o plugins/router_http/router_http.o plugins/logfile/logfile.o plugins/router_cache/router_cache.o plugins/rawrouter/rawrouter.o plugins/router_static/router_static.o plugins/sslrouter/sslrouter.o plugins/spooler/spooler_plugin.o plugins/cheaper_busyness/cheaper_busyness.o plugins/symcall/symcall_plugin.o plugins/transformation_tofile/tofile.o plugins/transformation_gzip/gzip.o plugins/transformation_chunked/chunked.o plugins/transformation_offload/offload.o plugins/router_memcached/router_memcached.o plugins/router_redis/router_redis.o plugins/router_hash/router_hash.o plugins/router_expires/expires.o plugins/router_metrics/plugin.o plugins/transformation_template/tt.o plugins/stats_pusher_socket/plugin.o -lpthread -lm -rdynamic -ldl -lcrypt -lpthread -ldl -lutil -lm -lm -lpython3.10 -lcrypt
2021-11-17T14:24:25.7330403Z       /usr/bin/ld: cannot find -lpython3.10
2021-11-17T14:24:25.7331040Z       collect2: error: ld returned 1 exit status
2021-11-17T14:24:25.7331619Z       *** error linking uWSGI ***
2021-11-17T14:24:25.7332292Z       ----------------------------------------```

The step has the following log output (before it is run):

python -m poetry install
  shell: sh -e {0}
  env:
    pythonLocation: /__w/_tool/Python/3.10.0/x64
    LD_LIBRARY_PATH: /__w/_tool/Python/3.10.0/x64/lib

Any help would be appreciated! Have not been able to find much about this issue...

edit:
just tried downgrading python version to 3.8, but same situation with -lpython3.8.

Activity

joaodlf

joaodlf commented on Nov 18, 2021

@joaodlf
Author

Further investigation, I can see the following files in our system:

/opt/hostedtoolcache/Python/3.10.0/x64/lib/libpython3.10.so
/opt/hostedtoolcache/Python/3.10.0/x64/lib/libpython3.10.so.1.0

We are even running the runner as root (RUNNER_ALLOW_RUNASROOT=1 ./run.sh) to avoid any possible permissions issue.

Not sure if this is important (my knowledge over docker and containers is very limited), but the host machine is a Centos 8 box, which I know isn't supported, but I was hoping to circumvent that issue by setting the container to ubuntu:20.04.

nikita-bykov

nikita-bykov commented on Nov 18, 2021

@nikita-bykov
Contributor

Hello @joaodlf, thanks for reporting! We will take a look.

dsame

dsame commented on Apr 15, 2022

@dsame
Contributor

Hello @joaodlf,

According to README it is necessary to set environment variable AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache in order the python to be installed in /opt/hostedtoolcache where your build expects to find the library

      - name: Setup Python
        uses: actions/setup-python@v2
        env:
          AGENT_TOOLSDIRECTORY: /opt/hostedtoolcache
        with:
          python-version: 3.10.0

According to your note

I can see the following files in our system:

/opt/hostedtoolcache/Python/3.10.0/x64/lib/libpython3.10.so
/opt/hostedtoolcache/Python/3.10.0/x64/lib/libpython3.10.so.1.0

I assume you meant the host system while your actual build is executed within the docker container

jobs:
  build:
    runs-on: self-hosted
    container: ubuntu:20.04

This is why the setup-python installs the python binaries into /__w/_tool/Python/3.10.0/x64/lib/ within the docker filesystem

Please confirm AGENT_TOOLSDIRECTORY: /opt/hostedtoolcache solves your problem.
NOTE: you may face an error during the setup-pyton task the binary not found - in this case please wait the PR #338 is merged

marko-zivic-93

marko-zivic-93 commented on Apr 27, 2022

@marko-zivic-93
Contributor

Hello @joaodlf
Due to the lack of response, I am going to close this issue :)
If you have any additional problems or concerns, please feel free to reopen it or to create a new one.

gchamon

gchamon commented on Sep 5, 2023

@gchamon

@dsame I had this issue with a similar setup as OP and I can confirm, at least on my end, setting AGENT_TOOLSDIRECTORY to /opt/hostedtoolcache solved the issue for me. In order for me not to modify every single workflow file deployed in my organization, I decided to set the variable at the runner level, in the file /opt/actions-runner/.env.

tyrken

tyrken commented on Jul 12, 2024

@tyrken

With our self-hosted runners (pods on EKS helped by https://github.com/actions/actions-runner-controller ) changing AGENT_TOOLSDIRECTORY didn't work for us - still getting gcc compilation errors about missing lpython3.9 library.

Instead we see this action already sets LD_LIBRARY_PATH to a custom value, e.g. /home/runner/_work/_tool/Python/3.9.19/x64/lib for an old app we're testing. While that works for running applications, that env-var isn't used for compilation. Setting LIBRARY_PATH to the same value as LD_LIBRARY_PATH does work.

It would be nice if this action was updated to set both natively...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      Participants

      @dsame@tyrken@joaodlf@gchamon@nikita-bykov

      Issue actions

        ld: cannot find -lpython3.10 · Issue #275 · actions/setup-python