AWS integration

[mathrb]

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 16.04.6 LTS
• Node version: node v10.19.0

Describe the problem

I'm trying to connect Gecko to AWS services, as defined in the docker-compose file.
I filled a .env file with these information:

GECKO_SERVER_PORT=4000
GECKO_SERVER_CONTAINER_PORT=4000
AWS_BUCKET=gecko-annotation
AWS_REGION=eu-west-3
AWS_ACCESS_KEY_ID=<<<access_key_id>>>
AWS_SECRET_ACCESS_KEY=<<<secret_key>>>
#AWS_COGNITO_POOL=<<<pool_id>>>
AWS_FOLDER=audio
GECKO_APP_HOST=localhost

After build with npm run build and running as a server npm run server, I don't see any interactions with S3 storage. It seems that it only support client mode.

I tried to setup an aws cognito pool, but I didn't succeed to get it work since the aws fails with this error:

/home/myuser/gecko/node_modules/aws-sdk/lib/request.js:31
            throw err;
            ^

CredentialsError: Missing credentials in config
    at Request.ENOTFOUND_ERROR (/home/myuser/gecko/node_modules/aws-sdk/lib/event_listeners.js:495:46)
    at Request.callListeners (/home/myuser/gecko/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (/home/myuser/gecko/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (/home/myuser/gecko/node_modules/aws-sdk/lib/request.js:683:14)
    at ClientRequest.error (/home/myuser/gecko/node_modules/aws-sdk/lib/event_listeners.js:333:22)
    at ClientRequest.<anonymous> (/home/myuser/gecko/node_modules/aws-sdk/lib/http/node.js:96:19)
    at ClientRequest.emit (events.js:198:13)
    at ClientRequest.EventEmitter.emit (domain.js:448:20)
    at TLSSocket.socketErrorListener (_http_client.js:401:9)
    at TLSSocket.emit (events.js:198:13)

I had a look at the cognito configuration, it seems to be only using region & pool id from configuration.

Could you please provide some documentation or integration state of AWS services?

Kind regards

[GolanLevy]

Hi @mathrb,

In order to use S3, provide URL parameters as follows:
http://localhost:8080/?save_mode=server&audio=http://localhost:8080/s3_files/[path_to_audio_file_without_bucket_name]&json=http://localhost:8080/s3_files/[path_to_transcript_file_without_bucket_name]

save_mode: on save, directs Gecko to override the file in S3 (assuming one was loaded through S3) instead of downloading it locally. Observe that it will save another file with a timestamp, to maintain some kind of primitive history mechanism.

audio/json: on load, download the files from S3 as audio/transcript files.

Sorry about the fact that we do not have proper documentations to that yet.
We would appreciate if you could add some into the README file.

Thanks,
Golan

[mathrb]

Hello,

Thanks for your response.
I have now file read/write with S3 Storage. I'll create a PR regarding the S3 integration.
About the cognito mode, any idea why it's not working?
Also, in the video and screenshots, we can add multiple annotation files when using the client mode. Is it supported in server mode with S3 storage?

Kind regards

[GolanLevy]

Hi,

Thanks for the PR.
I'm happy it is working for you now and I see that you figured how to load multiple annotation files in the sever mode using ";" between the links.
Can you explain what do you mean by cognito mode and what does not work?

[mathrb]

Sorry for the late response.
I saw in the code and environment variables: AWS_COGNITO_POOL
I was thinking that maybe it is something related to user management, where multiple users could connect to the gecko interface with credentials. Am I correct?
If I am, i'm experiencing issues doing it. Any idea?

[GolanLevy]

I'm less familiar with that concept.
@strelok2012 can you help out here?
If there are still problems I will try to check it out myself in the near future.

[strelok2012]

Hi, @mathrb!

Sorry for long answer

GECKO_SERVER_PORT=4000 GECKO_SERVER_CONTAINER_PORT=4000 AWS_BUCKET=gecko-annotation AWS_REGION=eu-west-3 AWS_ACCESS_KEY_ID=<<<access_key_id>>> AWS_SECRET_ACCESS_KEY=<<<secret_key>>> #AWS_COGNITO_POOL=<<<pool_id>>> AWS_FOLDER=audio GECKO_APP_HOST=localhost

Do you use exact this config? AWS_COGNITO_POOL is commented, and server doesn't see that param.

If you use uncommented config, can you please add a console.log before this line and check, will be shown or not? Thanks