New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS S3 logs: import Requester field as User ID #86

Closed
mattab opened this Issue Aug 14, 2015 · 1 comment

Comments

Projects
None yet
1 participant
@mattab
Member

mattab commented Aug 14, 2015

Amazon S3 file format (http://docs.aws.amazon.com/AmazonS3/latest/dev/LogFormat.html) includes a few additional fields that are currently ignored by log importer.

Example S3 log line: xyzxyz hello.mobile [15/May/2015:00:52:49 +0000] 23.1.61.13 arn:aws:iam::179580289999:user/phillip.boss 82303D395D REST.GET.VERSIONING - "GET /?versioning HTTP/1.1" 200 - 162 - 9 - "-" "aws-sdk-java/1.9.8 Linux/3.9.5-301.fc19.x86_64 Java_HotSpot(TM)_64-Bit_Server_VM/20.13-b02/1.6.0_38-ea" -

In particular, arn:aws:iam::179580289999:user/phillip.boss, is the Requester field which contains some interesting information:

The canonical user ID of the requester, or the string "Anonymous" for unauthenticated requests. If the requester was an IAM user, this field will return the requester's IAM user name along with the AWS root account that the IAM user belongs to. This identifier is the same one used for access control purposes.

Suggestion:

  • Maybe we could map this Requester field to the "User ID"?
@mattab

This comment has been minimized.

Show comment
Hide comment
@mattab

mattab Aug 19, 2015

Member

Note: this can be done already with a custom regex via: --log-format-regex but it can be very tricky to write custom regex. (see --dump-log-regex for more info)

Member

mattab commented Aug 19, 2015

Note: this can be done already with a custom regex via: --log-format-regex but it can be very tricky to write custom regex. (see --dump-log-regex for more info)

@mattab mattab added this to the Backlog milestone Aug 26, 2015

@mattab mattab added enhancement and removed question labels Jun 20, 2017

@sgiehl sgiehl closed this in #181 Nov 2, 2017

@mattab mattab changed the title from S3 logs: import more fields (eg. Requester) to AWS S3 logs: import Requester field as User ID Dec 5, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment