Problem: Unable to fetch the gateway binary (privately) to deploy its server

[outerlook]

My suggestion: private S3 that we can control permissions, so we can permit our server to download it.

[brennanjl]

I have shared it here: https://drive.google.com/file/d/1l5yTZ5oAeeQzd7wRLUDRbi-BzI_tqhDa/view?usp=drive_link

[brennanjl]

Seems to be some issues sharing with Drive. I tried with Dropbox: https://www.dropbox.com/scl/fi/2zzp2xq5p439tmef3nb2i/kgw_v0.1.2.zip?rlkey=sqwf6tdzgqjuqv6d226wsrdq8&dl=0

[VladislavSpassov]

Hey @outerlook, is part of the problem's scope to run the kwil gateway binary or only the logic for fetching the binary from s3?

[outerlook]

Hi @VladislavSpassov!
IMO, this could handle only storing the binary in an S3, and setting up how to control resource access.

Also, I guess would be more appropriate to aggregate running the binary at this issue: #111. And then also to appropriately implement run on staging server at this #110

Look that we currently deploy a staging server with CDK, and here's the code for it: https://github.com/truflation/tsn-db/blob/main/truflation/infra/cdk_main.go#L222

so, being able to fetch from S3 means also to add permissions for the EC2 role on this CDK stack for reading this resource from S3.

Of course, if you initially thought of another workflow, feel free to add suggestions

[outerlook]

Another open question would be if, at staging, we deploy the gateway and kwil node at the same instance, and IMO, that would be OK.

[VladislavSpassov]

@outerlook Thanks a lot, s3 seems to me as the best approach.

[VladislavSpassov]

@outerlook could you help me with placing the binary into s3 and give me the bucket name?

[markholdex]

@VladislavSpassov are you able to download the binary from Dropbox? What exactly you are missing here to resolve the problem? Provide an ETA when the solution will be ready.

Reminding that: if you assign yourself to a problem, take responsibility for resolving it. If you are blocked by anything, feel free to report in a separate problem statement. Please follow our Engineering Guidelines. Let me know if there is anything unclear.

[VladislavSpassov]

@markholdex I think we are rather discussing what is the best approach, because in my vision I was thinking that we can place the kwil-gateway binary into s3 and from there we could fetch it in cdk stack initialization phase, but I might be completely wrong, so this is why I was asking Raphael for help. And as I said, the reason why I haven't raised an issue is because for me there is no issue until we crystalize the approach.

[outerlook]

path is
s3://kwil-binaries/gateway/kgw_v0.1.2.zip

the zip content is the same as received on dropbox

[markholdex]

@VladislavSpassov got it. For me, it seemed that you guys already agreed on the solution. Please aim to work async. Raffael begins his day when yours ends. If you have any questions, try to address them all together. If you believe you are blocked here, feel free to work on a different problem where you are not blocked.