Feature Request: Private Links for cml publish

[shortcipher3]

I'm still learning cml, I noticed when I run:

cml publish dvc_plots/index.html

It creates a url at https://asset.cml.dev/ with a token. I'm using a private repo, so it was a little alarming to see that my data is being uploaded to a publicly available site, previously I thought it must be using some gitlab/github internals to upload the image to the mr/pr that I didn't yet understand.

In my case I'm using gcs as a dvc remote, so I think my workaround will be to instead upload the results to gcs then only people logged into a google account with access to the gcp project will have access.

It would be great if it was made clearer in the documentation that cml publish is uploading to cml.dev and if my use case with a private dvc remote is pretty typical it would be nice to provide a flag to cml publish that let's me instead upload to my private infrastructure.

[dacbd]

if you are using GitLab there is a --native flag you can use, additionally there is a --url component to do exactly that. It is lacking some documentation as we haven't really put together a guide for its setup.

example

Here is an internal test using this feature which you can use/snoop around with:

jobs:
  setup:
    outputs:
      url: ${{ steps.deploy.outputs.url }}
    runs-on: ubuntu-latest
    environment: test
    steps:
      - uses: actions/checkout@v3
        with:
          repository: 0x2b3bfa0/minroud
      - name: Install Deps
        run: npm install --global serverless
      - uses: aws-actions/configure-aws-credentials@v1
        with:
          role-to-assume: arn:aws:iam::342840881361:role/SandboxUser
          aws-region: us-east-1
      - name: Deploy
        id: deploy
        run: |
          npx serverless deploy
          url=$(npx serverless info | awk 'match($0, "endpoint: "){ print substr($0, RSTART + 10)}')
          echo "$url"
          echo "::set-output name=url::$url"
      - uses: actions/upload-artifact@v3
        with:
          name: state
          path: .serverless
  test:
    runs-on: ubuntu-latest
    needs: setup
    steps:
      - name: Test a CML Branch
        if: ${{ env.TEST_CML_BRANCH }}
        run: |
          npm install -g canvas@2 vega@5 vega-cli@5 vega-lite@5 github:iterative/cml#${{ env.TEST_CML_BRANCH }}
      - name: Standard CML setup
        if: ${{ !env.TEST_CML_BRANCH }}
        uses: iterative/setup-cml@v1
        with:
          version: ${{ env.TEST_CML_VERSION || 'latest' }}
      - name: Test
        env:
          CML_PUBLISH_URL: ${{ needs.setup.outputs.url }}
        run: |
          echo "hello" >> file.md
          url=$(cml publish file.md)
          echo "$url"
          test "hello" == $(curl --silent $url)
  clean-up:
    runs-on: ubuntu-latest
    if: ${{ always() }}
    needs: test
    environment: test
    steps:
      - uses: actions/checkout@v3
        with:
          repository: 0x2b3bfa0/minroud
      - name: Install Deps
        run: npm install --global serverless
      - uses: aws-actions/configure-aws-credentials@v1
        with:
          role-to-assume: arn:aws:iam::342840881361:role/SandboxUser
          aws-region: us-east-1
      - uses: actions/download-artifact@v3
        with:
          name: state
          path: .serverless
      - name: Clean up
        run: |
          bucket_name=$(aws s3 ls | awk 'match($0, /minroud/){ print $3}')
          aws s3 rm --recursive "s3://$bucket_name"
          npx serverless remove

cml/bin/cml/publish.js

Lines 35 to 39 in e87a23b

	url: {
	type: 'string',
	description: 'Self-Hosted URL',
	hidden: true
	},

[0x2b3bfa0]

Related to iterative/cml.dev#266

[dacbd]

• Amazon Web Services: https://github.com/iterative/minroud-s3
• Google Cloud
• Microsoft Azure

/?

[casperdcl]

@0x2b3bfa0 we have a (doubly hidden) publish --url now, but we're missing a public API.

i.e. we still need cml comment {create,update} --publish-url

[casperdcl]

fixed by iterative/cml.dev#324? Or was there some other reason you re-opened @0x2b3bfa0? If so please do re-open (again)

[0x2b3bfa0]

The reason why I reopened this issue is because Minroud doesn't produce “private” links in the sense that knowing the URL is enough.