R&D Setup ASW S3 buckets so the users can pay for the reading from it

[khorolets]

Given that we can setup S3 in a way that the users will pay for their read usage

#6 (comment)

We want to enable it to anyone who runs or will need to run their own indexer in order to remove the pain from the process.

We want to check how to set it up and how to use it so we can document it and recommend to use NEAR Lake Framework

[khorolets]

• Using Requester Pays bucket for storage transfer and usage

• Configuring Requester Pays on a bucket

• In a Bucket Permission settings we need to unblock public access (Block public access (bucket settings))

• Create or edit bucket policy (allows to read objects and list objects in the specified bucket

  {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicRead",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::near-lake-data-tesnet/*"
        },
        {
            "Sid": "PublicList",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::near-lake-data-testnet"
        }
    ]
  }
  

• Access control list (ACL):
  Grant Authenticated users group (anyone with an AWS account) List and Read

---

After these changes, all requests to our S3 have to be done by authenticated users (any AWS user) with request_payer provided. The requester will be charged for the queries to AWS by AWS

I'm going to apply the changes listed above to buckets:

• near-lake-data-testnet
• near-lake-data-mainnet

I need to update the README about it and close this after that.