Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Scope down dataset sharing requester IAM role managed IAM policy S3 permissions #1280

Merged
merged 9 commits into from
Jun 3, 2024
Merged

Scope down dataset sharing requester IAM role managed IAM policy S3 permissions #1280

merged 9 commits into from
Jun 3, 2024

Conversation

mourya-33
Copy link
Contributor

Feature or Bugfix

  • Bugfix

Detail

This PR will address the issue - 1226

Relates

Security

Please answer the questions below briefly where applicable, or write N/A. Based on
OWASP 10.

  • Does this PR introduce or modify any input fields or queries - this includes
    fetching data from storage outside the application (e.g. a database, an S3 bucket)? N/A
    • Is the input sanitized? N/A
    • What precautions are you taking before deserializing the data you consume? N/A
    • Is injection prevented by parametrizing queries? N/A
    • Have you ensured no eval or similar functions are used? N/A
  • Does this PR introduce any functionality or component that requires authorization? N/A
    • How have you ensured it respects the existing AuthN/AuthZ mechanisms? N/A
    • Are you logging failed auth attempts? N/A
  • Are you using or adding any cryptographic features? N/A
    • Do you use a standard proven implementations? N/A
    • Are the used keys controlled by the customer? Where are they stored? N/A
  • Are you introducing any new policies/roles/users? No
    • Have you used the least-privilege principle? How?Yes, restricted the s3:* permissions to s3 readonly permissions

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@mourya-33
Copy link
Contributor Author

This is an extension of #1262 to add s3:GetObject permissions and remove s3:* permissions

@SofiaSazonova
Copy link
Contributor

@mourya-33 while teting I found the following issues:

  1. We still have kms* policy
  2. We have s3* for share point access. May be we should revisit it as well? Now sure
  3. You corrected bucket policy, but there is still s3* in iam policy (which is overruled by resource policy, but still not good). This is granted in add_missing_resources_to_policy_statement and related both to kms and s3 policies. Please, fix it as well

@mourya-33
Copy link
Contributor Author

@SofiaSazonova , this also seem to require a change in the test assertions. I am testing this locally and will push the changes early next week.

@SofiaSazonova
Copy link
Contributor

@mourya-33 Sorry for not writing all at once, but I just realised:
we really should be changing the existing policies as well to patch the problem. Could you write migration script for that?

@noah-paige noah-paige linked an issue May 21, 2024 that may be closed by this pull request
Unrestricted S3 permissions for shares with consumer role #1226
Closed
@SofiaSazonova
Copy link
Contributor

@dlpzx @noah-paige I need your advice here: is it ok to put the backfilling of iam policies in migration scripts?

@dlpzx dlpzx changed the title Ds sharing consumptionrole s3 policy Dataset sharing requester IAM role s3 policy May 22, 2024
@dlpzx dlpzx changed the title Dataset sharing requester IAM role s3 policy Dataset sharing requester IAM role managed IAM policy S3 permissions May 22, 2024
@dlpzx dlpzx changed the title Dataset sharing requester IAM role managed IAM policy S3 permissions Scope down dataset sharing requester IAM role managed IAM policy S3 permissions May 22, 2024
@SofiaSazonova SofiaSazonova merged commit 3557b98 into data-dot-all:main Jun 3, 2024
9 checks passed
@noah-paige noah-paige mentioned this pull request Jun 11, 2024
Closed
@SofiaSazonova SofiaSazonova mentioned this pull request Jun 28, 2024
Merged
SofiaSazonova added a commit that referenced this pull request Jul 9, 2024
@SofiaSazonova
Dataall code/infrastructure AWS migrations (#1374)
bd8e1e8 
### Feature or Bugfix
- Feature


### Detail
During the work on #1280 we encountered a problem: we need to revise all
share-policies in already existed AWS deployments (remove s3:*
permissions and add appropriate ones). It sounds like "migration", but
has nothing to do with database, so it doesn't belong to alembic
migrations module.

As a general solution to such problems, we introduced 'dataall
migrations' -- a module, that is called in lambda function, triggered by
update of backend stack, after DB-migrations. This module has the same
logic of migrations, which are executed one by one. Current revision uid
is stored in SSM Parameter store.

Module contains:|
 - BaseDataAllMigration -- python class  -- a base for future migrations
 - Herder -- python class, that handles migrations
- folder "versions", where python files are stored with subclasses of
BaseDataAllMigration.

Each subclass has parameters
   - uid of the revision `key = '51132fed-c36d-470c-9946-5164581856cb'
   - name ` name = 'Remove Wildcard from Sharing Policy'
- short description `description = 'Remove Wildcard from Sharing Policy'
- uid of previous migration, to specify the order `previous_migration =
'0' # initial migration`
- possible methods `up` and `down` for upgrade and downgrade
respectively
   
In order to add new migration, developer needs to create a python class
in folder "versions" (folder name is hardcoded by now) and specify the
parameters described above. Right now it can be done only manually.


### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).

- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
  - Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
  - Is injection prevented by parametrizing queries?
  - Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
  - Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
  - Do you use a standard proven implementations?
  - Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
  - Have you used the least-privilege principle? How?


By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.

---------

Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
@dlpzx dlpzx mentioned this pull request Jul 11, 2024
Closed
@SofiaSazonova SofiaSazonova mentioned this pull request Jul 17, 2024
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SofiaSazonova SofiaSazonova SofiaSazonova approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Unrestricted S3 permissions for shares with consumer role
2 participants
@mourya-33 @SofiaSazonova