Can not write files to a bring-your-own azure storage

[duynguyen]

I would like to use a BYO azure storage with the Files SDK:

const files = await filesLib.init({ azure: { storageAccount, storageAccessKey, containerName } })

However files.write gave a bad creds error even though all creds are correct:

2020-09-22T14:46:05.873Z       stdout: 2020-09-22T14:46:05.872Z @adobe/aio-lib-files:error {
  "sdk": "FilesLib",
  "sdkDetails": {
    "filePath": "my-folder/test.csv",
    "contentType": "Readable"
  },
  "code": "ERROR_BAD_CREDENTIALS",
  "message": "[FilesLib:ERROR_BAD_CREDENTIALS] cannot access `cloud storage provider`, make sure your credentials are valid",
  "stacktrace": "FilesLibError: [FilesLib:ERROR_BAD_CREDENTIALS] cannot access `cloud storage provider`, make sure your credentials are valid\n    at new <anonymous> (/nodejsAction/9KGzgxBe/index.js:41347:9)\n    at /nodejsAction/9KGzgxBe/index.js:23658:39\n    at processTicksAndRejections (internal/process/task_queues.js:97:5)\n    at async AzureBlobFiles._writeStream (/nodejsAction/9KGzgxBe/index.js:118225:5)\n    at async NodeActionRunner.main [as userScriptMain] (/nodejsAction/9KGzgxBe/index.js:41148:25)"
}

Reference: reported by @UrsBoller in the Firefly Forum: https://experienceleaguecommunities.adobe.com/t5/project-firefly-questions/access-to-own-storage-using-aio-lib-state-and-aio-lib-action/qaq-p/378249/comment-id/202#M202

[aiojbot]

JIRA issue created: https://jira.corp.adobe.com/browse/ACNA-937

[meryllblanchet]

Thanks for reporting the issue, @duynguyen ! We'll look into that.
As a temporary workaround, I'd recommend using the out-of-the-box storage.

[meryllblanchet]

Relates to #52.

[UrsBoller]

@moritzraho I just tested the new version 1.2.3 and I still have an issue with writing files (read works as expected). Anything I missed to get the fix up and running?
@duynguyen FYI
Edit: I still get the following error message on "get": [FilesLib:ERROR_BAD_CREDENTIALS] cannot access cloud storage provider, make sure your credentials are valid

[sandeep-paliwal]

@UrsBoller I am checking this.

[meryllblanchet]

Reopening the issue while we are investigating further.

[sandeep-paliwal]

We are debugging this, and looks like a very odd issue with azure write operation when file lib is imported a certain way.
@UrsBoller as a workaround can you try importing the file lib directly instead of using the aio-sdk? We found that when the lib is imported directly the azure didn't throw the bad credentials error. Here is sample code for import

const  {init, FilePermissions }  = require('@adobe/aio-lib-files')
const files = await init({
  azure: {
    storageAccount: '',
    storageAccessKey: '',
    containerName: '',
  }})
... //any file operations here

[meryllblanchet]

Hi @UrsBoller , we'll wait on your feedback related to @sandeep-paliwal 's proposal. This should do the job for your use-case, while we will work on the root cause, which pretty seems to be a dependency conflict.

[UrsBoller]

@sandeep-paliwal / @meryllblanchet sorry for delay, was on a outdoor trip the whole day. but just tested the fix with changing the init and it does not change anything, still the same error message ...

[meryllblanchet]

@UrsBoller do you have any remaining require('@adobe/aio-sdk') in your action? For example in order to use another library. Then I'd suggest replacing it in favor of a direct require to the library you want to use.

[UrsBoller]

@meryllblanchet yes, I'm using the "CORE" for the logger ...
EDIT: will try to rebuid the action without the logger, just a min ...

[meryllblanchet]

@UrsBoller or try const logger = require('@adobe/aio-lib-core-logging')('Your App')

[UrsBoller]

@meryllblanchet you are awesome! without the logger (aio-sdk) it works now! will try to change the logger later on, but this is no big issue (since I have my own logger anyway). thanks a lot, this fix helps!

[meryllblanchet]

Fantastic @UrsBoller , we're glad to know that you're unblocked!
I will close this issue now, and we will work on the dependency fix at aio-sdk level.

You can follow the resolution of this one by subscribing to this PR: adobe/aio-sdk#13.

[UrsBoller]

@meryllblanchet / @sandeep-paliwal : just FYI - I'm still having issues even with the workaround and after the fix with the aio-sdk. after hours of testing what is the cause of the failure I narrowed down the root cause to "aio-lib-analytics". As soon as I include analytics, the aio-lib-files failes again. I couldn't figure out what exactly happens, I just included the library and executed the function "getCurrentUser()". I found another way to solve my issue (without using aio-lib-analytics), so fixed for me.
Therefore, I won't raise a bug and wait if the issue happens again - waiting for more aio actions to build where I will use the aio-lib-analytics heavely - if I see the error again, I will raise a bug...

[meryllblanchet]

@UrsBoller thank you for confirming this issue to us, which is ultimately the same than the one that you observed when using aio-sdk and a custom storage for the aio-lib-files.

To fix it, we will have to update the swagger client dependency in every SDK library and make sure it doesn't fail when the "bring your own storage" feature of the aio-lib-files is being used.

cc @sandeep-paliwal @moritzraho for more details.