Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try out v2-preview #62

Closed
konradpabjan opened this issue Mar 13, 2020 · 29 comments
Closed

Try out v2-preview #62

konradpabjan opened this issue Mar 13, 2020 · 29 comments
Assignees
Labels
Announcement help wanted Extra attention is needed

Comments

@konradpabjan
Copy link
Collaborator

konradpabjan commented Mar 13, 2020

The v2-preview of upload-artifact is out and we need your help!

You can try it out by using actions/upload-artifact@v2-preview
Any associated code and documentation can be found here: https://github.com/actions/upload-artifact/tree/v2-preview

This issue is for general feedback and to report an bugs/issues during the preview

There is also a v2-preview for download-artifact 😀 see: actions/download-artifact#23

Warning: At any time during the preview, there may be unannounced changes that can cause things to break. It is recommended not to use the preview of this action in critical workflows


Historical Context around v2-preview and v1

The v1 versions of upload-artifact (and download-artifact ) are plugins that are executed by the runner. The code for v1 can be found here: https://github.com/actions/runner/tree/master/src/Runner.Plugins/Artifact

The v1 code is written in C# and is tightly coupled to the runner, it also uses special APIs that only the runner can use to interact with artifacts. If any changes or updates had to be made related to artifacts, they had to done on the runner and a new release had to roll out that would take a significant amount of time. With v2, there is no dependency on the runner so it will be much easier and faster to make changes and accept community contributions (right now it was pretty much impossible).

The v2-preview of upload-artifact has been rewritten from scratch using Typescript with a new set of APIs that allow it interact with artifacts (previously only the runner could do this). There is a new NPM package called @actions/artifact that contains the core functionality to interact with artifacts (which this action uses for the most part). This NPM package is hosted in the actions/toolkit repo so anyone can use this to interact with artifacts when developing actions. You can find the package here (lots of documentation and extra info):
https://www.npmjs.com/package/@actions/artifact
https://github.com/actions/toolkit/tree/master/packages/artifact

Further work

Since v2-preview is effectively a total rewrite from v1, There is a huge potential for bugs so it needs to be tested thoroughly before creating an actual v2 release. We need help testing the core functionality which includes:

  • Uploading an individual file
  • Uploading a directory
  • Uploading using wildcards

There will be no new features added as part of the v2-preview, we need to test the core functionality first and once a v2 release is out, then we can start chipping away at issues/features that we have been unable to address with v1🙂 For the moment, please don't submit PRs for any new features to the v2-preview branch.

Some initial observations from some internal testing show that v2-preview is slightly slower than v1 (around 10-15%). A large part of this discrepancy can be attributed to porting over from C# to Node. In C#, the runner would upload artifacts with Async Tasks and true multi-threading. With Node, the HTTP pipeline is not as efficient (we would love some help over in the toolkit repo for @actions/artifact if anyone is really good with TS, Aysnc/Await and HTTP pipelining to hopefully make this better)

@Margen67
Copy link

Any chance of this being added? #55

@konradpabjan
Copy link
Collaborator Author

@Margen67 I left a comment in the issue. It is a feature that we will be able to add in the future once this is out of preview. Not adding support for this now though.

I'm trying to keep the amount of new features in v2-preview to a minimum to make sure the TS port of the action is stable before releasing v2.

@glvnst
Copy link

glvnst commented Mar 19, 2020

Working great for me. I did have one minor thing; I wanted to use this config to hoover up a bunch of text files in the current working directory:

      - uses: actions/upload-artifact@v2-preview
        with:
          name: receipts
          path: *-rcpt.txt

...but that wildcard expression conflicts with yaml aliases (and the web editor indicated as much by underlining the path line in red and giving the hover text unidentified alias "-rcpt.txt"). In my case was simple to just prefix with ./:

          path: ./*-rcpt.txt

...but it might be worth a line in the docs like:

Relative and absolute file paths are both allowed. Relative paths are rooted against the current working directory. Paths that begin with a wildcard character should be quoted to avoid being interpreted as YAML aliases.

@konradpabjan
Copy link
Collaborator Author

Thanks for the notice @glvnst!

I've updated the README with your suggestion

@testautomation
Copy link

testautomation commented Mar 20, 2020

Hi, is it possible to upload multiple files w/ their original name - w/o being zipped?

something like this:

- name: UPLOAD TEST RESULTS
      uses: actions/upload-artifact@master
      if: always()
      with:
        paths: |
          tests/results/output.xml
          tests/results/log.html
          tests/results/report.html

The reason is I would like to access log.html and report.html directly in the browser w/o having to download .zip archive + unzip it locally + open it in browser to view it

Similar to how these guys do (you know who I mean 😉 )
image

@testautomation
Copy link

besides my request above I tried w/

 uses: actions/upload-artifact@v2-preview
      with:
        name: robot_results
        path: tests/results/*

works fine, creates a robot_results.zip w/ (in my case) three files in it as you can see here

@konradpabjan
Copy link
Collaborator Author

Hi @testautomation

See my comment regarding uploading individual files. All files get uploaded individually without getting zipped. The problem is that in our UI we don't currently support downloading files individually (or browsing them) so we zip everything together at the end. We have plans to significantly enhance the UI experience for artifacts though😊

Also, multiple paths will come after the preview is over, see my comment here for more info

@krzkaczor
Copy link

@konradpabjan are all artifacts scoped to the current commit? Can I download an artifact from another branch?

I totally lack the exact definition of what artifact is somewhere in toolkit's docs and how is it different from cache API. Normally I would thought that cache can be accessed between commits (but it seems like it's scoped by branch) and artifacts are scoped by the current workflow execution.

@konradpabjan
Copy link
Collaborator Author

@krzkaczor generated artifacts are scoped to an individual workflow run (a run can be triggered by a variety of events: https://help.github.com/en/actions/reference/events-that-trigger-workflows).

Both caching and artifacts can be used to share data between jobs, though the main difference is that artifacts can be downloaded after the run has finished so data remains persistent. See: https://help.github.com/en/actions/configuring-and-managing-workflows/persisting-workflow-data-using-artifacts

It is currently not possible to download an artifact that was generated in another run using either of the artifact actions, see: actions/download-artifact#3

There is an API though that you can use to download artifacts that is independent of any of the artifact actions: https://developer.github.com/v3/actions/artifacts/#download-an-artifact

@espipj
Copy link

espipj commented Mar 24, 2020

Hey @konradpabjan is there any chance that we could get the URL of the artifact so that we can use it to drop a comment in the PR for example? I think that would be amazingly useful.

@domenkozar
Copy link

I'd really like to reference artifact url once it has been uploaded. The reason is that one artifact depends on another

@konradpabjan
Copy link
Collaborator Author

@espipj @domenkozar Right now it's not possible to return the URL. The URL gets dynamically generated after the run get completed. We currently have some limitations in our backend. Some extra info

@gengjiawen
Copy link

Is it possible we add a feature not compress files ? My problem is that my file is already a compress file. Related code: https://github.com/gengjiawen/node-github-workflow/pull/1/files.

@holytshirt
Copy link

holytshirt commented Mar 25, 2020

Tested this out with

    - name: Upload artifact
      uses: actions/upload-artifact@v2-preview
      with:
        # Artifact name
        name: NugetPackages
        # Directory containing files to upload
        path: "**/Paramore.Brighter.*.nupkg"

And it works
I noticed it add the files including their path.
Would like to add all files without the paths so all files in the root of the zip.
Having a setting or something like => . to add to root or foldername

@konradpabjan
Copy link
Collaborator Author

konradpabjan commented Mar 25, 2020

@gengjiawen See my earlier comment regarding zipping. It's a limitation with our UI currently so they will still appear compressed: #62 (comment)

@holytshirt if the provided path has a wildcard, everything past the first wildcard character will be used in the path for the artifact (I should probably document this behavior). Basically if you have something like this:

- uses: actions/upload-artifact@v2-preview
  with:
    name: my-artifact
    path: path/to/*/directory/something??-complex[abc].txt

and there are 3 files found, the upload structure would look like this

    path/to/*/directory/something??-complex[abc].txt =>
        ∟ path/to/some/directory/something12-complexa.txt
        ∟ path/to/some/directory/something12-complexb.txt
        ∟ path/to/other/directory/something12-complexa.txt

    would be flattened to =>
        ∟ some/directory/something12-complexa.txt
        ∟ some/directory/something12-complexb.txt
        ∟ other/directory/something12-complexa.txt

Throwing everything into a single folder with an optional flag could be an enhancement/addition post v2-preview.

@esamattis
Copy link

Hi! Tried to use this for Cypress screenshots with:

            - uses: actions/upload-artifact@v2-preview
              if: failure()
              with:
                  name: cypress-screenshots
                  path: "cypress/screenshots/*/*.png"

But the screenshot filenames are like

cypress/screenshots/basic.spec.ts/describe block -- test name (failed).png 

which fails with

##[error]Artifact path is not valid: /basic.spec.ts/describe block -- test name (failed).png Contains character: " ". Invalid characters include: ",:,<,>,|,*,?,

Got the same error even without the glob:

            - uses: actions/upload-artifact@v2-preview
              if: failure()
              with:
                  name: cypress-screenshots
                  path: "cypress/screenshots/"

@rumzledz
Copy link

Greetings all,

I seem to be getting a 403 error when uploading an artifact. Here are the logs:

2020-03-31T10:42:27.4294570Z ##[group]Run actions/upload-artifact@v2-preview
2020-03-31T10:42:27.4294960Z with:
2020-03-31T10:42:27.4295140Z   name: my-artifact
2020-03-31T10:42:27.4295340Z   path: path/to/artifact/world.txt
2020-03-31T10:42:27.4295530Z env:
2020-03-31T10:42:27.4295730Z   IS_CI: true
2020-03-31T10:42:27.4295930Z   DEVELOPER_DIR: /Applications/Xcode_11.4.app
2020-03-31T10:42:27.4296110Z ##[endgroup]
2020-03-31T10:42:27.5159110Z With the provided path, there will be 1 files uploaded
2020-03-31T10:42:27.5970080Z HttpClientResponse {
2020-03-31T10:42:27.5970710Z   message: IncomingMessage {
2020-03-31T10:42:27.5972760Z     _readableState: ReadableState {
2020-03-31T10:42:27.5973940Z       objectMode: false,
2020-03-31T10:42:27.5974590Z       highWaterMark: 16384,
2020-03-31T10:42:27.5975310Z       buffer: BufferList { head: null, tail: null, length: 0 },
2020-03-31T10:42:27.5975690Z       length: 0,
2020-03-31T10:42:27.5976120Z       pipes: null,
2020-03-31T10:42:27.5976450Z       pipesCount: 0,
2020-03-31T10:42:27.5976850Z       flowing: true,
2020-03-31T10:42:27.5977280Z       ended: true,
2020-03-31T10:42:27.5977650Z       endEmitted: true,
2020-03-31T10:42:27.5977990Z       reading: false,
2020-03-31T10:42:27.5978430Z       sync: true,
2020-03-31T10:42:27.5978790Z       needReadable: false,
2020-03-31T10:42:27.5979240Z       emittedReadable: false,
2020-03-31T10:42:27.5979620Z       readableListening: false,
2020-03-31T10:42:27.5980060Z       resumeScheduled: false,
2020-03-31T10:42:27.5980450Z       paused: false,
2020-03-31T10:42:27.5980890Z       emitClose: true,
2020-03-31T10:42:27.5981280Z       autoDestroy: false,
2020-03-31T10:42:27.5981730Z       destroyed: false,
2020-03-31T10:42:27.5983570Z       defaultEncoding: 'utf8',
2020-03-31T10:42:27.5983950Z       awaitDrain: 0,
2020-03-31T10:42:27.5984340Z       readingMore: true,
2020-03-31T10:42:27.5984710Z       decoder: null,
2020-03-31T10:42:27.5985150Z       encoding: null
2020-03-31T10:42:27.5985520Z     },
2020-03-31T10:42:27.5985910Z     readable: false,
2020-03-31T10:42:27.5986360Z     _events: [Object: null prototype] { end: [Array], data: [Function] },
2020-03-31T10:42:27.5986820Z     _eventsCount: 2,
2020-03-31T10:42:27.5987190Z     _maxListeners: undefined,
2020-03-31T10:42:27.5987640Z     socket: TLSSocket {
2020-03-31T10:42:27.5988080Z       _tlsOptions: [Object],
2020-03-31T10:42:27.5988470Z       _secureEstablished: true,
2020-03-31T10:42:27.5988920Z       _securePending: false,
2020-03-31T10:42:27.5989310Z       _newSessionPending: false,
2020-03-31T10:42:27.5989740Z       _controlReleased: true,
2020-03-31T10:42:27.5990370Z       _SNICallback: null,
2020-03-31T10:42:27.5991570Z       servername: 'pipelines.actions.githubusercontent.com',
2020-03-31T10:42:27.5991980Z       alpnProtocol: false,
2020-03-31T10:42:27.5992440Z       authorized: true,
2020-03-31T10:42:27.5992880Z       authorizationError: null,
2020-03-31T10:42:27.5993260Z       encrypted: true,
2020-03-31T10:42:27.5993730Z       _events: [Object: null prototype],
2020-03-31T10:42:27.5994110Z       _eventsCount: 9,
2020-03-31T10:42:27.5994560Z       connecting: false,
2020-03-31T10:42:27.5994930Z       _hadError: false,
2020-03-31T10:42:27.5995360Z       _parent: null,
2020-03-31T10:42:27.5996600Z       _host: 'pipelines.actions.githubusercontent.com',
2020-03-31T10:42:27.5997060Z       _readableState: [ReadableState],
2020-03-31T10:42:27.5997450Z       readable: true,
2020-03-31T10:42:27.5997950Z       _maxListeners: undefined,
2020-03-31T10:42:27.5998740Z       _writableState: [WritableState],
2020-03-31T10:42:27.5999100Z       writable: false,
2020-03-31T10:42:27.5999470Z       allowHalfOpen: false,
2020-03-31T10:42:27.5999790Z       _sockname: null,
2020-03-31T10:42:27.6000150Z       _pendingData: null,
2020-03-31T10:42:27.6001000Z       _pendingEncoding: '',
2020-03-31T10:42:27.6001400Z       server: undefined,
2020-03-31T10:42:27.6001720Z       _server: null,
2020-03-31T10:42:27.6002070Z       ssl: [TLSWrap],
2020-03-31T10:42:27.6002420Z       _requestCert: true,
2020-03-31T10:42:27.6002750Z       _rejectUnauthorized: true,
2020-03-31T10:42:27.6005740Z       parser: null,
2020-03-31T10:42:27.6006190Z       _httpMessage: [ClientRequest],
2020-03-31T10:42:27.6006650Z       timeout: 180000,
2020-03-31T10:42:27.6007010Z       [Symbol(res)]: [TLSWrap],
2020-03-31T10:42:27.6007400Z       [Symbol(asyncId)]: 18,
2020-03-31T10:42:27.6007760Z       [Symbol(kHandle)]: [TLSWrap],
2020-03-31T10:42:27.6008170Z       [Symbol(lastWriteQueueSize)]: 0,
2020-03-31T10:42:27.6008520Z       [Symbol(timeout)]: Timeout {
2020-03-31T10:42:27.6008930Z         _idleTimeout: 180000,
2020-03-31T10:42:27.6009790Z         _idlePrev: [TimersList],
2020-03-31T10:42:27.6010140Z         _idleNext: [TimersList],
2020-03-31T10:42:27.6010560Z         _idleStart: 146,
2020-03-31T10:42:27.6010910Z         _onTimeout: [Function: bound ],
2020-03-31T10:42:27.6011440Z         _timerArgs: undefined,
2020-03-31T10:42:27.6011800Z         _repeat: null,
2020-03-31T10:42:27.6012200Z         _destroyed: false,
2020-03-31T10:42:27.6012550Z         [Symbol(refed)]: false,
2020-03-31T10:42:27.6012970Z         [Symbol(asyncId)]: 27,
2020-03-31T10:42:27.6013370Z         [Symbol(triggerId)]: 25
2020-03-31T10:42:27.6013720Z       },
2020-03-31T10:42:27.6014080Z       [Symbol(kBuffer)]: null,
2020-03-31T10:42:27.6014430Z       [Symbol(kBufferCb)]: null,
2020-03-31T10:42:27.6014810Z       [Symbol(kBufferGen)]: null,
2020-03-31T10:42:27.6015160Z       [Symbol(kBytesRead)]: 0,
2020-03-31T10:42:27.6015540Z       [Symbol(kBytesWritten)]: 0,
2020-03-31T10:42:27.6016580Z       [Symbol(connect-options)]: [Object]
2020-03-31T10:42:27.6017050Z     },
2020-03-31T10:42:27.6017400Z     connection: TLSSocket {
2020-03-31T10:42:27.6017800Z       _tlsOptions: [Object],
2020-03-31T10:42:27.6018200Z       _secureEstablished: true,
2020-03-31T10:42:27.6018570Z       _securePending: false,
2020-03-31T10:42:27.6018990Z       _newSessionPending: false,
2020-03-31T10:42:27.6019360Z       _controlReleased: true,
2020-03-31T10:42:27.6019790Z       _SNICallback: null,
2020-03-31T10:42:27.6021350Z       servername: 'pipelines.actions.githubusercontent.com',
2020-03-31T10:42:27.6021840Z       alpnProtocol: false,
2020-03-31T10:42:27.6022190Z       authorized: true,
2020-03-31T10:42:27.6022570Z       authorizationError: null,
2020-03-31T10:42:27.6022970Z       encrypted: true,
2020-03-31T10:42:27.6023510Z       _events: [Object: null prototype],
2020-03-31T10:42:27.6023930Z       _eventsCount: 9,
2020-03-31T10:42:27.6024530Z       connecting: false,
2020-03-31T10:42:27.6025170Z       _hadError: false,
2020-03-31T10:42:27.6025710Z       _parent: null,
2020-03-31T10:42:27.6026700Z       _host: 'pipelines.actions.githubusercontent.com',
2020-03-31T10:42:27.6027070Z       _readableState: [ReadableState],
2020-03-31T10:42:27.6027430Z       readable: true,
2020-03-31T10:42:27.6027760Z       _maxListeners: undefined,
2020-03-31T10:42:27.6028130Z       _writableState: [WritableState],
2020-03-31T10:42:27.6028500Z       writable: false,
2020-03-31T10:42:27.6028820Z       allowHalfOpen: false,
2020-03-31T10:42:27.6029180Z       _sockname: null,
2020-03-31T10:42:27.6029500Z       _pendingData: null,
2020-03-31T10:42:27.6030320Z       _pendingEncoding: '',
2020-03-31T10:42:27.6030660Z       server: undefined,
2020-03-31T10:42:27.6031020Z       _server: null,
2020-03-31T10:42:27.6031340Z       ssl: [TLSWrap],
2020-03-31T10:42:27.6031690Z       _requestCert: true,
2020-03-31T10:42:27.6032010Z       _rejectUnauthorized: true,
2020-03-31T10:42:27.6032380Z       parser: null,
2020-03-31T10:42:27.6032750Z       _httpMessage: [ClientRequest],
2020-03-31T10:42:27.6033090Z       timeout: 180000,
2020-03-31T10:42:27.6033450Z       [Symbol(res)]: [TLSWrap],
2020-03-31T10:42:27.6033770Z       [Symbol(asyncId)]: 18,
2020-03-31T10:42:27.6034130Z       [Symbol(kHandle)]: [TLSWrap],
2020-03-31T10:42:27.6034460Z       [Symbol(lastWriteQueueSize)]: 0,
2020-03-31T10:42:27.6034820Z       [Symbol(timeout)]: Timeout {
2020-03-31T10:42:27.6035140Z         _idleTimeout: 180000,
2020-03-31T10:42:27.6035510Z         _idlePrev: [TimersList],
2020-03-31T10:42:27.6036320Z         _idleNext: [TimersList],
2020-03-31T10:42:27.6036650Z         _idleStart: 146,
2020-03-31T10:42:27.6037000Z         _onTimeout: [Function: bound ],
2020-03-31T10:42:27.6037330Z         _timerArgs: undefined,
2020-03-31T10:42:27.6037700Z         _repeat: null,
2020-03-31T10:42:27.6038010Z         _destroyed: false,
2020-03-31T10:42:27.6038370Z         [Symbol(refed)]: false,
2020-03-31T10:42:27.6038690Z         [Symbol(asyncId)]: 27,
2020-03-31T10:42:27.6039050Z         [Symbol(triggerId)]: 25
2020-03-31T10:42:27.6039720Z       },
2020-03-31T10:42:27.6040070Z       [Symbol(kBuffer)]: null,
2020-03-31T10:42:27.6040430Z       [Symbol(kBufferCb)]: null,
2020-03-31T10:42:27.6040750Z       [Symbol(kBufferGen)]: null,
2020-03-31T10:42:27.6041250Z       [Symbol(kBytesRead)]: 0,
2020-03-31T10:42:27.6042020Z       [Symbol(kBytesWritten)]: 0,
2020-03-31T10:42:27.6043030Z       [Symbol(connect-options)]: [Object]
2020-03-31T10:42:27.6043380Z     },
2020-03-31T10:42:27.6043780Z     httpVersionMajor: 1,
2020-03-31T10:42:27.6044130Z     httpVersionMinor: 1,
2020-03-31T10:42:27.6045240Z     httpVersion: '1.1',
2020-03-31T10:42:27.6045610Z     complete: true,
2020-03-31T10:42:27.6045990Z     headers: {
2020-03-31T10:42:27.6046790Z       'cache-control': 'no-store,no-cache',
2020-03-31T10:42:27.6047650Z       pragma: 'no-cache',
2020-03-31T10:42:27.6048560Z       'content-length': '338',
2020-03-31T10:42:27.6049440Z       'content-type': 'application/json; charset=utf-8',
2020-03-31T10:42:27.6050360Z       'strict-transport-security': 'max-age=2592000',
2020-03-31T10:42:27.6051270Z       'x-tfs-processid': 'd4957f3b-d469-4538-b992-626bf5274a1f',
2020-03-31T10:42:27.6052220Z       activityid: '7c8e262a-604a-4a69-913e-fd4fafa5daa0',
2020-03-31T10:42:27.6053090Z       'x-tfs-session': '7c8e262a-604a-4a69-913e-fd4fafa5daa0',
2020-03-31T10:42:27.6054150Z       'x-vss-e2eid': '7c8e262a-604a-4a69-913e-fd4fafa5daa0',
2020-03-31T10:42:27.6055060Z       'x-vss-senderdeploymentid': 'd624195d-30e0-1768-06a5-b10a7879c7db',
2020-03-31T10:42:27.6055980Z       'x-frame-options': 'SAMEORIGIN',
2020-03-31T10:42:27.6057030Z       'x-msedge-ref': 'Ref A: 28F0EE1BF5D749EA92B5CF729142B244 Ref B: ATAEDGE1010 Ref C: 2020-03-31T10:42:27Z',
2020-03-31T10:42:27.6057880Z       date: 'Tue, 31 Mar 2020 10:42:27 GMT',
2020-03-31T10:42:27.6058850Z       connection: 'close'
2020-03-31T10:42:27.6059190Z     },
2020-03-31T10:42:27.6059570Z     rawHeaders: [
2020-03-31T10:42:27.6060300Z       'Cache-Control',
2020-03-31T10:42:27.6061140Z       'no-store,no-cache',
2020-03-31T10:42:27.6062070Z       'Pragma',
2020-03-31T10:42:27.6063720Z       'no-cache',
2020-03-31T10:42:27.6064370Z       'Content-Length',
2020-03-31T10:42:27.6081880Z       '338',
2020-03-31T10:42:27.6082710Z       'Content-Type',
2020-03-31T10:42:27.6083260Z       'application/json; charset=utf-8',
2020-03-31T10:42:27.6083810Z       'Strict-Transport-Security',
2020-03-31T10:42:27.6084310Z       'max-age=2592000',
2020-03-31T10:42:27.6084860Z       'X-TFS-ProcessId',
2020-03-31T10:42:27.6085420Z       'd4957f3b-d469-4538-b992-626bf5274a1f',
2020-03-31T10:42:27.6085950Z       'ActivityId',
2020-03-31T10:42:27.6086460Z       '7c8e262a-604a-4a69-913e-fd4fafa5daa0',
2020-03-31T10:42:27.6087010Z       'X-TFS-Session',
2020-03-31T10:42:27.6087560Z       '7c8e262a-604a-4a69-913e-fd4fafa5daa0',
2020-03-31T10:42:27.6088060Z       'X-VSS-E2EID',
2020-03-31T10:42:27.6088620Z       '7c8e262a-604a-4a69-913e-fd4fafa5daa0',
2020-03-31T10:42:27.6089180Z       'X-VSS-SenderDeploymentId',
2020-03-31T10:42:27.6089730Z       'd624195d-30e0-1768-06a5-b10a7879c7db',
2020-03-31T10:42:27.6090240Z       'X-FRAME-OPTIONS',
2020-03-31T10:42:27.6090770Z       'SAMEORIGIN',
2020-03-31T10:42:27.6091240Z       'X-MSEdge-Ref',
2020-03-31T10:42:27.6091880Z       'Ref A: 28F0EE1BF5D749EA92B5CF729142B244 Ref B: ATAEDGE1010 Ref C: 2020-03-31T10:42:27Z',
2020-03-31T10:42:27.6092390Z       'Date',
2020-03-31T10:42:27.6092890Z       'Tue, 31 Mar 2020 10:42:27 GMT',
2020-03-31T10:42:27.6093400Z       'Connection',
2020-03-31T10:42:27.6094320Z       'close'
2020-03-31T10:42:27.6094540Z     ],
2020-03-31T10:42:27.6094700Z     trailers: {},
2020-03-31T10:42:27.6094910Z     rawTrailers: [],
2020-03-31T10:42:27.6095090Z     aborted: false,
2020-03-31T10:42:27.6095300Z     upgrade: false,
2020-03-31T10:42:27.6095750Z     url: '',
2020-03-31T10:42:27.6096000Z     method: null,
2020-03-31T10:42:27.6096200Z     statusCode: 403,
2020-03-31T10:42:27.6096690Z     statusMessage: 'Forbidden',
2020-03-31T10:42:27.6096920Z     client: TLSSocket {
2020-03-31T10:42:27.6097450Z       _tlsOptions: [Object],
2020-03-31T10:42:27.6097680Z       _secureEstablished: true,
2020-03-31T10:42:27.6097870Z       _securePending: false,
2020-03-31T10:42:27.6098070Z       _newSessionPending: false,
2020-03-31T10:42:27.6098250Z       _controlReleased: true,
2020-03-31T10:42:27.6098450Z       _SNICallback: null,
2020-03-31T10:42:27.6099050Z       servername: 'pipelines.actions.githubusercontent.com',
2020-03-31T10:42:27.6099260Z       alpnProtocol: false,
2020-03-31T10:42:27.6099500Z       authorized: true,
2020-03-31T10:42:27.6099680Z       authorizationError: null,
2020-03-31T10:42:27.6099890Z       encrypted: true,
2020-03-31T10:42:27.6100080Z       _events: [Object: null prototype],
2020-03-31T10:42:27.6100280Z       _eventsCount: 9,
2020-03-31T10:42:27.6100460Z       connecting: false,
2020-03-31T10:42:27.6100650Z       _hadError: false,
2020-03-31T10:42:27.6100830Z       _parent: null,
2020-03-31T10:42:27.6101520Z       _host: 'pipelines.actions.githubusercontent.com',
2020-03-31T10:42:27.6101770Z       _readableState: [ReadableState],
2020-03-31T10:42:27.6101960Z       readable: true,
2020-03-31T10:42:27.6102150Z       _maxListeners: undefined,
2020-03-31T10:42:27.6102310Z       _writableState: [WritableState],
2020-03-31T10:42:27.6102520Z       writable: false,
2020-03-31T10:42:27.6102700Z       allowHalfOpen: false,
2020-03-31T10:42:27.6102880Z       _sockname: null,
2020-03-31T10:42:27.6103080Z       _pendingData: null,
2020-03-31T10:42:27.6103590Z       _pendingEncoding: '',
2020-03-31T10:42:27.6103850Z       server: undefined,
2020-03-31T10:42:27.6104040Z       _server: null,
2020-03-31T10:42:27.6104250Z       ssl: [TLSWrap],
2020-03-31T10:42:27.6104440Z       _requestCert: true,
2020-03-31T10:42:27.6104650Z       _rejectUnauthorized: true,
2020-03-31T10:42:27.6104880Z       parser: null,
2020-03-31T10:42:27.6105070Z       _httpMessage: [ClientRequest],
2020-03-31T10:42:27.6105300Z       timeout: 180000,
2020-03-31T10:42:27.6105480Z       [Symbol(res)]: [TLSWrap],
2020-03-31T10:42:27.6105710Z       [Symbol(asyncId)]: 18,
2020-03-31T10:42:27.6105900Z       [Symbol(kHandle)]: [TLSWrap],
2020-03-31T10:42:27.6106120Z       [Symbol(lastWriteQueueSize)]: 0,
2020-03-31T10:42:27.6106310Z       [Symbol(timeout)]: Timeout {
2020-03-31T10:42:27.6106530Z         _idleTimeout: 180000,
2020-03-31T10:42:27.6106720Z         _idlePrev: [TimersList],
2020-03-31T10:42:27.6106940Z         _idleNext: [TimersList],
2020-03-31T10:42:27.6107160Z         _idleStart: 146,
2020-03-31T10:42:27.6107360Z         _onTimeout: [Function: bound ],
2020-03-31T10:42:27.6107580Z         _timerArgs: undefined,
2020-03-31T10:42:27.6107760Z         _repeat: null,
2020-03-31T10:42:27.6107970Z         _destroyed: false,
2020-03-31T10:42:27.6108160Z         [Symbol(refed)]: false,
2020-03-31T10:42:27.6108380Z         [Symbol(asyncId)]: 27,
2020-03-31T10:42:27.6108560Z         [Symbol(triggerId)]: 25
2020-03-31T10:42:27.6108780Z       },
2020-03-31T10:42:27.6108990Z       [Symbol(kBuffer)]: null,
2020-03-31T10:42:27.6109170Z       [Symbol(kBufferCb)]: null,
2020-03-31T10:42:27.6109400Z       [Symbol(kBufferGen)]: null,
2020-03-31T10:42:27.6109580Z       [Symbol(kBytesRead)]: 0,
2020-03-31T10:42:27.6109800Z       [Symbol(kBytesWritten)]: 0,
2020-03-31T10:42:27.6110310Z       [Symbol(connect-options)]: [Object]
2020-03-31T10:42:27.6110570Z     },
2020-03-31T10:42:27.6110740Z     _consuming: false,
2020-03-31T10:42:27.6110960Z     _dumped: false,
2020-03-31T10:42:27.6111190Z     req: ClientRequest {
2020-03-31T10:42:27.6111730Z       _events: [Object: null prototype],
2020-03-31T10:42:27.6111950Z       _eventsCount: 4,
2020-03-31T10:42:27.6112130Z       _maxListeners: undefined,
2020-03-31T10:42:27.6112350Z       outputData: [],
2020-03-31T10:42:27.6112530Z       outputSize: 0,
2020-03-31T10:42:27.6112740Z       writable: true,
2020-03-31T10:42:27.6112920Z       _last: true,
2020-03-31T10:42:27.6113130Z       chunkedEncoding: false,
2020-03-31T10:42:27.6113320Z       shouldKeepAlive: false,
2020-03-31T10:42:27.6113540Z       useChunkedEncodingByDefault: true,
2020-03-31T10:42:27.6113990Z       sendDate: false,
2020-03-31T10:42:27.6114210Z       _removedConnection: false,
2020-03-31T10:42:27.6114440Z       _removedContLen: false,
2020-03-31T10:42:27.6114620Z       _removedTE: false,
2020-03-31T10:42:27.6114830Z       _contentLength: null,
2020-03-31T10:42:27.6115010Z       _hasBody: true,
2020-03-31T10:42:27.6115590Z       _trailer: '',
2020-03-31T10:42:27.6115790Z       finished: true,
2020-03-31T10:42:27.6116020Z       _headerSent: true,
2020-03-31T10:42:27.6116200Z       socket: [TLSSocket],
2020-03-31T10:42:27.6116410Z       connection: [TLSSocket],
2020-03-31T10:42:27.6117060Z       _header: 'POST /oFE7PsVTZmSAigR6O4q5M6JEC77S2Cp0NSB7OLzEAOZsLi7mFv/_apis/pipelines/workflows/67304459/artifacts?api-version=6.0-preview HTTP/1.1\r\n' +
2020-03-31T10:42:27.6117710Z         'accept: application/json;api-version=6.0-preview\r\n' +
2020-03-31T10:42:27.6118280Z         'content-type: application/json\r\n' +
2020-03-31T10:42:27.6118790Z         'user-agent: action/artifact\r\n' +
2020-03-31T10:42:27.6122670Z         'Authorization: Bearer ***\r\n' +
2020-03-31T10:42:27.6123270Z         'Content-Length: 56\r\n' +
2020-03-31T10:42:27.6123780Z         'Host: pipelines.actions.githubusercontent.com\r\n' +
2020-03-31T10:42:27.6124270Z         'Connection: close\r\n' +
2020-03-31T10:42:27.6124710Z         '\r\n',
2020-03-31T10:42:27.6124910Z       _onPendingData: [Function: noopPendingOutput],
2020-03-31T10:42:27.6125090Z       agent: [Agent],
2020-03-31T10:42:27.6125250Z       socketPath: undefined,
2020-03-31T10:42:27.6125710Z       method: 'POST',
2020-03-31T10:42:27.6126310Z       path: '/oFE7PsVTZmSAigR6O4q5M6JEC77S2Cp0NSB7OLzEAOZsLi7mFv/_apis/pipelines/workflows/67304459/artifacts?api-version=6.0-preview',
2020-03-31T10:42:27.6126520Z       _ended: true,
2020-03-31T10:42:27.6126670Z       res: [Circular],
2020-03-31T10:42:27.6126830Z       aborted: false,
2020-03-31T10:42:27.6127030Z       timeoutCb: [Function: emitRequestTimeout],
2020-03-31T10:42:27.6127270Z       upgradeOrConnect: false,
2020-03-31T10:42:27.6127450Z       parser: null,
2020-03-31T10:42:27.6127670Z       maxHeadersCount: null,
2020-03-31T10:42:27.6127890Z       [Symbol(kNeedDrain)]: false,
2020-03-31T10:42:27.6128070Z       [Symbol(isCorked)]: false,
2020-03-31T10:42:27.6128300Z       [Symbol(kOutHeaders)]: [Object: null prototype]
2020-03-31T10:42:27.6128500Z     }
2020-03-31T10:42:27.6128690Z   }
2020-03-31T10:42:27.6128860Z }
2020-03-31T10:42:27.6132130Z ##[error]Unable to create a container for the artifact my-artifact

Basically I'm just trying to test an upload using the code provided in this Repo's Guide.

steps:
  - uses: actions/checkout@v2
     with:
       lfs: true
  - run: mkdir -p path/to/artifact
  - run: echo hello > path/to/artifact/world.txt
  - uses: actions/upload-artifact@v2-preview
     with:
       name: my-artifact
       path: path/to/artifact/world.txt

Has anyone else experienced this?

Any help would be greatly appreciated.

Thank you.

@konradpabjan
Copy link
Collaborator Author

@esamattis it looks like you have whitespace in your files. Information about forbidden characters: https://github.com/actions/toolkit/blob/master/packages/artifact/docs/additional-information.md#non-supported-characters
Whitespace was technically allowed in v1 but to maintain platform agnostic behavior it is now checked for and it won't upload. This was done because what if that artifact gets downloaded on linux where spaces are not supported in file paths. There is a lot of undefined behavior that can start to happen.

@ledesmaromeo Do you have a link to the failing run or a repository name? I can take a look at our telemetry. If it's a private repo you can email me at konradpabjan at github.com What happens if you retry the upload? I've occasionally seen 500s and I think I've seen this 403 come up once though but retrying the upload always worked. What's interesting though is that this 403 came up at the very beginning of the upload (during the first http call), so I'm curious if there is something else going on here.

@mathieudevos
Copy link

mathieudevos commented Apr 3, 2020

@konradpabjan - same issue here.
Exact same issue here as @ledesmaromeo - some of the workflows are working perfectly fine, one of the more recent ones is just throwing this error.

Which servers is the artifact upload/download trying to reach?

Retrying the upload gives the same error.

@konradpabjan
Copy link
Collaborator Author

@ledesmaromeo Telemetry is showing that you've exceeded the amount of storage that is available. There are limits based on the type of account that you have: https://help.github.com/en/github/setting-up-and-managing-billing-and-payments-on-github/about-billing-for-github-packages

@mathieudevos I'm guessing you're hitting the same error.

Artifacts get auto-deleted after 90 days. You can also manually delete artifacts through the UI though it usually takes a couple of hours for the changes to propagate in billing (there is a lot of work that we would like to do around retention since it's a pretty big customer pain point right now)

I'll add a clearer warning when a 403 is hit since it's completely ambiguous right now what is going on. The server/endpoints that are being hit are pretty much identical to v1 so if you try it out using those actions you should see some similar behavior.

@mathieudevos
Copy link

Alright, adding steps so that we can clear them faster would be easier.

E.g. on dev/feat branches they only need to store the artifacts for an hour, but on my master branch I'd like to keep them much longer.

Is there also an option to constantly remove the artifacts? For example: download & remove in one operation?

Thanks for the quick debugging!

@anshuman852
Copy link

  • name: Upload APK
    uses: actions/upload-artifact@v2-preview
    with:
    name: app.apk
    path: app/build/outputs/apk/debug/app-debug.apk

It still uploads a zip instead of the single file. am i doing anything wrong here?

You can see it Here

@konradpabjan
Copy link
Collaborator Author

@mathieudevos we currently don't have a first class experience to constantly remove artifacts. There are some issues open to track these scenarios: https://github.com/actions/upload-artifact/issues?q=is%3Aissue+is%3Aopen+label%3Aretention
There are some semi-hacky scripts that users have documented that use our APIs (you can find these in the linked issues), but longterm we would like to make this considerably easier.

@anshuman852 The zip that you download from the UI is created independently of how the artifact is uploaded. It's more of a limitation of our UI and we currently don't support downloading individual files. See my comment here for more information.

@konradpabjan
Copy link
Collaborator Author

An update has rolled out to the @actions/artifact package that this action uses for the core upload logic. Detailed changelog here: https://github.com/actions/toolkit/blob/master/packages/artifact/RELEASES.md

The package update mostly contains fixes related to artifact download but some things related to upload are also in.

I've updated the v2-preview branch to user this latest package (version 0.3.0). I ran some tests and everything appears to be working correctly, but if there are any problems that come up please let me know.

@Margen67
Copy link

If it can't find any files, it only gives a warning: https://github.com/Margen67/Test/runs/576666010?check_suite_focus=true#step:2:4

This should probably be an error.

@Calinou
Copy link

Calinou commented Apr 22, 2020

How do I upload the root directory of a repository as an artifact? I need to do this to upload a repository's files as they originially exist, but with some compiled native libraries added. With the upload path set to "." or ".*", I get:

No files were found for the provided path: .. No artifacts will be uploaded.

YAML used

@konradpabjan
Copy link
Collaborator Author

There has been a small update to the @action/artifact package with some minor fixes and the v2-preview branch has been updated: https://github.com/actions/toolkit/blob/master/packages/artifact/RELEASES.md#031

Everything is looking 👍 and next week if we are planning on officially releasing v2 (the v2-preview branch will be merged into master and tags will be updated) Expect the v2-preview branch to be deleted shortly after so heads up and this issue will be closed. You can also expect some minor changelog to show up: https://github.blog/changelog/

@Margen67 If no files are found then it's usually an indication that the user provided path is wrong rather than something being busted with artifact upload (such as a failed connection or http call). The warning should be sufficient to give users a heads up they should check their path. Of course we could add an extra option or something in the future.

@Calinou Make sure you're using the checkout action and then you should be able to use ${{ github.workspace }} as that is where your repository gets checked out to. You can also just use ./ or ./* as the path: https://help.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#github-context

@konradpabjan
Copy link
Collaborator Author

@Calinou 🤔 It does appear that . works for v1. I'll go ahead and work on a fix to get that working so there are no compat issues between v1 and v2

pst added a commit to kbst/terraform-kubestack that referenced this issue Apr 27, 2020
As suggested in similar issues whith large uploads failing, try
using the rewrite.

More info here:

 * actions/upload-artifact#62
 * actions/download-artifact#23
 * actions/upload-artifact#29 (comment)
@konradpabjan
Copy link
Collaborator Author

v2 upload-artifact has been merged into master! Thanks everyone for the feedback and testing!
https://github.blog/changelog/2020-04-28-github-actions-v2-artifact-actions/

You can now use it by doing actions/upload-artifact@v2. I'll be deleting the v2-preview branch soon (probably in a week or so), so please switch over so your builds don't break.

Moving forward it should be a lot easier now to accept contributions and add features since this effectively decouples artifact upload from the runner

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Announcement help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests