Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

while i am running the template spec to upload the data to ADME instance getting the access denied error. #37

Closed
sreedharguda opened this issue Jul 6, 2023 · 5 comments · Fixed by #38

Comments

@sreedharguda
Copy link

While i am running the template spec to upload the data to ADME instance getting the access denied error. Please help me how to mitigate this issue.

2023-07-05 06:06:20,558 [Dataload ] [INFO ] args: Namespace(dir='/app/open-test-data/datasets/documents', output='output/loaded-documents-datasets.json', subparser='datasets')
2023-07-05 06:06:20,614 [Dataload ] [INFO ] Amount of data to transfer: 26.22MB
2023-07-05 06:06:20,614 [Dataload ] [INFO ] Request worker count: 50
2023-07-05 06:06:20,614 [Dataload ] [INFO ] Max Chunk size: 32 MB
2023-07-05 06:06:20,654 [Dataload ] [INFO ] Total number of files to upload: 9
2023-07-05 06:06:20,870 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/FP.pdf with response <Response [403]>
2023-07-05 06:06:20,884 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/osduka_demo_texts.txt with response <Response [403]>
2023-07-05 06:06:20,977 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/prov22.pdf with response <Response [403]>
2023-07-05 06:06:21,108 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/FS12-3031.pdf with response <Response [403]>
2023-07-05 06:06:21,119 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/FI.pdf with response <Response [403]>
2023-07-05 06:06:21,121 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/69_D_CH_11.pdf with response <Response [403]>
2023-07-05 06:06:21,124 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/prov33.pdf with response <Response [403]>
2023-07-05 06:06:21,149 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/FS12-3003.pdf with response <Response [403]>
2023-07-05 06:06:21,539 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/69_D_CH_13.pdf with response <Response [403]>
2023-07-05 06:06:21,674 [Dataload ] [INFO ] File location map is saved to output/loaded-documents-datasets.json
2023-07-05 06:06:21,690 [Dataload ] [INFO ] Files that could not be uploaded: 9
2023-07-05 06:06:21,690 [Dataload ] [INFO ] ['/app/open-test-data/datasets/documents/FP.pdf',
'/app/open-test-data/datasets/documents/osduka_demo_texts.txt',
'/app/open-test-data/datasets/documents/prov22.pdf',
'/app/open-test-data/datasets/documents/FS12-3031.pdf',
'/app/open-test-data/datasets/documents/FI.pdf',

Originally posted by @sreedharguda in #34 (comment)

@Miller-Kyle
Copy link
Collaborator

This is likely due to the recent data partition name changes in ADME. I'll need to update Experience Lab to not prepend the instance name to the data partition id.

@sreedharguda
Copy link
Author

Hi @Miller-Kyle , Have used your feature branch and created the resources by using bicep template, its created experince lab resource group and ran experience-lab Container instance, its created the "experience-lab" resource group. I ran the Open-test-Data template spec its asked only "Instance Identifier", "Loglevel", didnt asked other details the way it ask when we use TNO data load repo template spec. I am suspecting this could be the reason to get access denied error.

2023-07-07 03:30:53,060 [Dataload ] [INFO ] Max Chunk size: 32 MB
2023-07-07 03:30:53,088 [Dataload ] [INFO ] Total number of files to upload: 9
2023-07-07 03:30:55,422 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/FS12-3003.pdf with response <Response [403]>
2023-07-07 03:30:55,435 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/FI.pdf with response <Response [403]>
2023-07-07 03:30:55,436 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/prov22.pdf with response <Response [403]>
2023-07-07 03:30:55,437 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/69_D_CH_13.pdf with response <Response [403]>
2023-07-07 03:30:55,548 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/69_D_CH_11.pdf with response <Response [403]>
2023-07-07 03:30:55,550 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/FS12-3031.pdf with response <Response [403]>
2023-07-07 03:30:55,550 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/FP.pdf with response <Response [403]>
2023-07-07 03:30:55,558 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/osduka_demo_texts.txt with response <Response [403]>
2023-07-07 03:30:55,559 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/prov33.pdf with response <Response [403]>
2023-07-07 03:30:55,702 [Dataload ] [INFO ] File location map is saved to output/loaded-documents-datasets.json
2023-07-07 03:30:55,713 [Dataload ] [INFO ] F

@sreedharguda
Copy link
Author

If i use this Repo https://github.com/Azure/osdu-data-load-tno and use for data load template spec file it ask all the details of ADME instance name, Client ID, partition name... once we run the container instance the data load works on ADME instance.

@Miller-Kyle
Copy link
Collaborator

Fixed in release v0.9

@sreedharguda
Copy link
Author

Thank you for your help @Miller-Kyle , have tested and its working fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants