Using CML behind a proxy

[rene-rex]

Hi

Everything running on my GitLab runner must go through a proxy to get anything from the outside world. So, I set the corresponding variables as usual:

stages: 
    - cml_run
    
cml:
    stage: cml_run
    image: dvcorg/cml-py3:latest
    
    script:
        - export http_proxy=proxy:8080
        - export https_proxy=proxy:8080
        - export no_proxy=localhost,127.0.0.1

        - pip3 install -r requirements.txt
        - python train.py > report.md
        
        - cml-publish roc_curve.png --md >> report.md
        - cml-send-comment report.md

However, this results in an error at cml-publish:

$ cml-publish roc_curve.png --md >> report.md
internal/url.js:257
  throw new ERR_INVALID_URL(input);
  ^
TypeError [ERR_INVALID_URL]: Invalid URL: proxy:8080
    at onParseError (internal/url.js:257:9)
    at new URL (internal/url.js:333:5)
    at Object.getProxyUrl (/cml/node_modules/@actions/http-client/proxy.js:17:20)
    at HttpClient._getAgent (/cml/node_modules/@actions/http-client/index.js:404:27)
    at HttpClient.getAgent (/cml/node_modules/@actions/http-client/index.js:358:21)
    at Object.getProxyAgent (/cml/node_modules/@actions/github/lib/internal/utils.js:36:15)
    at Object.<anonymous> (/cml/node_modules/@actions/github/lib/utils.js:34:22)
    at Module._compile (internal/modules/cjs/loader.js:1015:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1035:10)
    at Module.load (internal/modules/cjs/loader.js:879:32) {
  input: 'proxy:8080',
  code: 'ERR_INVALID_URL'

I tried to prefix the variables with http/https since it is complaining about an invalid URL but this just gave me another error:

$ cml-publish roc_curve.png --md >> report.md
FetchError: request to https://asset.cml.dev/ failed, reason: connect ECONNREFUSED 123.45.67.890:443
    at ClientRequest.<anonymous> (/cml/node_modules/node-fetch/lib/index.js:1461:11)
    at ClientRequest.emit (events.js:326:22)
    at TLSSocket.socketErrorListener (_http_client.js:428:9)
    at TLSSocket.emit (events.js:314:20)
    at emitErrorNT (internal/streams/destroy.js:92:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:60:3)
    at processTicksAndRejections (internal/process/task_queues.js:84:21) {
  type: 'system',
  errno: 'ECONNREFUSED',
  code: 'ECONNREFUSED'

The IP is anonymized above. Any idea how to make CML play nicely with the proxy?

[DavidGOrtega]

Hi @rene-rex, thanks for the feedback!
Calls to GL are done via node fetch. And it's not supported.
Im willing to extend it to support http_proxy.

[rene-rex]

That would be great, thanks! I happy to test a fix whenever you have one.

[DavidGOrtega]

I think that tomorrow we will have the PR 🚀

[pirnerjonas]

Are there any updates on this? I am currently also facing this problem with cml-runner I think..

[rene-rex]

Since I am unable to use CML due to this bug, I moved on. So I can't help you here @pirnerjonas

[pirnerjonas]

Thanks for writing @rene-rex. Yeah, I think I will also be unable to use CML then..

[DavidGOrtega]

@pirnerjonas @rene-rex priority 0 now ;)

[DavidGOrtega]

@pirnerjonas @rene-rex @0x2b3bfa0

while we had a solution for GL the problem is fundamentally GH. Basically they have the same blocker:

• Proxy Support octokit/rest.js#43