In this tutorial, we will learn about AWS S3 Buckets and create one. We will also explore S3 versioning and S3 encryption and in the last, we will generate an AWS S3 Bucket policy and apply it to our S3 Bucket. But before getting started let's see what is Amazon s3
Create AWS S3 Bucket and Objects - Hands-On
First, we will log in to our AWS console then under the Services tab type S3. Currently, we don't have any S3 Buckets available. In order to create an S3 bucket, we will click on Create bucket
Here we will enter a bucket name that should be globally unique. Now we will enter a new name for our bucket that is globa After that, we will choose a Region where our bucket will reside. I have chosen it to be US East (Ohio). You can choose a region that is near to you. Note that the S3 console is Global.
Here we will Block all public access to our S3 bucket that are the default settings. This will be unchecked if we require public access to our buckets like in the case of hosting a website
Currently, we will keep the Bucket Versioning disabled. We will play with it in the later sections of this tutorial. We can also add tags to our bucket. We will leave the Server-side encryption disabled for now and will see it in the later sections. Now we will click on Create bucket.
Here we can see our bucket in the list of available buckets.
Now we will upload objects in our S3 bucket. Note that objects are fundamental entities stored in Amazon s3. Here we will click on Upload.
After that, we will click on Add files. We can also add a folder. Here we can see our uploaded object named aws.png.
We will leave the remaining settings to default and then click on Upload.
Here we can see the details of the object uploaded which include Properties, Permissions, and Versions. Note that we can see an Object URL over here. If we copy this URL and paste it into the browser let's see what happens.
The access to the file is denied and the reason is that we blocked public access while creating our S3 bucket. Now let's see what happens if we click on the Open button present in the top right corner.
Now we will explore the topic of S3 versioning. Note that with versioning we can keep multiple variants of an object in the same bucket. Versioning is used to restore, retrieve and preserve every version of every object and through it, we can also recover from both unintended user actions and application failures.
Now we will click on Edit and Enable Bucket Versioning and then click on Save Changes
Now we will upload the aws.png file again.
Now when we will turn on the Show versions option, we can clearly see both the objects one which we uploaded previously without versioning enabled has Version ID equal to null and the other which we uploaded now has a Version ID.
Here for deleting an object permanently we will explicitly select an object and then click on Delete. To confirm deletion we will write permanently delete and then click on Delete objects.
Now we will explore one more thing. After uploading the aws.png object again and disabling the Show versions option we will select the object and then delete it.
Here we can see that deleting specified objects adds delete markers to them. Now we will type delete and then click on Delete objects.
Here we will enable Show versions again to view the delete marker. Note that a delete marker is a placeholder for a versioned object named in a simple DELETE request. When we delete an object with versioning enabled the object is not deleted. The delete marker makes Amazon S3 behave as if the object has been deleted. To restore the object here, we can delete the delete marker.
Now we will explore encryption in S3 Buckets. When we will get in the details of an object, we can see that Default Encryption is Disabled that is set at the bucket level. When it is enabled the new objects that are uploaded to the S3 bucket will be encrypted by default.
When we will get into the Server-side encryption settings of the object we can specify an Encryption key. It can be either Amazon s3 key (SSE-S3) that is an encryption key created, managed, and used for us by Amazon S3, or an AWS Key Management Service key (SSE-KMS) that is protected by AWS Key Management Service. Note that to upload an object with SSE-C that is a customer-provided encryption key we need to use AWS CLI, AWS SDK, or Amazon S3 REST API.
Now when we will go to the bucket level encryption it's disabled by default. We will enable it through which new objects that are stored in the bucket are automatically encrypted. Here also either we can do encryption through SSE-S3 or SSE-KMS. If we want to use SSE-C we will have to use AWS CLI, AWS SDK, or Amazon S3 REST API as stated above.
Now when we will upload the object again in the bucket, we can see that the Default encryption is enabled.
According to what is mentioned on the official website of AWS,
A bucket policy is a resource-based AWS Identity and Access Management (IAM) policy. You add a bucket policy to a bucket to grant other AWS accounts or IAM users access permissions for the bucket and the objects in it. Object permissions apply only to the objects that the bucket owner creates.
In the Permissions section of the bucket we created earlier, we can see that there is no policy written for our bucket. We will click on Edit.
This will take us to a new screen where we can see Bucket ARN which we will use when we will generate our policy through the Policy generator. Now we will click on Policy generator.
This will lead us to this new screen where we can easily define policy for our bucket through this user-friendly UI. Here we can define policies that control access to AWS products and resources. First we will select the Policy Type that in our case is S3 Bucket Policy. We can also create different types of policies like IAM Policy, an S3 Bucket Policy, an SNS Topic Policy, a VPC Endpoint Policy, and an SQS Policy. Then we will add a statement that is a formal description of single permission. Here for Effect we will select Deny. In Principal we will write . In Actions, we will select PutObject and in Amazon Resource Name we will enter the value we found on the previous screen with the name Bucket ARN. Note that we will add a / after the Bucket ARN value. After that, we will click on Add Conditions.
Now here we will define that at what condition we don't want a specific object to be uploaded. We will select the Condition to Null, Key to s3:x-amz-server-side-encryption and Value to true. This means we are denying any object to be uploaded with Key s3:x-amz-server-side-encryption set to Null. Now we click on Add Condition.
Here we can see the condition we just defined.
Now we will click on Add Statement. Now we will add another statement in which all the other settings will remain the same except for the condition. In Condition we will select StringNotEquals, for Key, we will select s3:amz-server-side-encryption and the Value will be set to AES256. Now again we will click on Add Condition. Here it means that for Key s3:amz-server-side-encryption if the value is not equal to AES256 then we will not allow the object to get uploaded into our bucket.
Again we will click on Add Statement.
Now here we can see both our statements listed. Now we will click on Generate Policy.
It returns us a JSON Document which we will use for our S3 Bucket Policy. Note that how easy it is to create policies with Policy Generator.
Now we will copy the above JSON document and paste it into our Bucket Policy and after that, we will click on Save changes.
Now we will upload aws.png again into our S3 bucket but this time without specifying any encryption key. Let's see what happens.
We will receive an error message Upload Failed and we can clearly see that the access is denied since our object was not encrypted with AES256 i.e SSE-S3 while we uploaded it.
Now we will upload the object again but this time we will specify an encryption key by overriding default encryption bucket settings and then select SSE-S3 and click on Upload.
Here we can see that object has been uploaded successfully as it satisfies conditions set in our bucket policy.
With this, we have come to the end of our tutorial. In this tutorial, we learned about
S3 Buckets and Objects
S3 Versioning
S3 Encryption
S3 Bucket Policy
LAB 2
Lab Details
This lab walks you through Amazon Simple Storage Service. Amazon S3 has a simple web interface that you can use to store and retrieve any amount of data(at any time) from anywhere on the web. In this lab we will demonstrate AWS S3 by creating a sample S3 bucket, uploading an object to the S3 bucket and setting up the bucket permission and policy.
Tasks Log into the AWS Management Console.
Create an S3 bucket.
Upload an object to S3 Bucket.
Access the object on the browser.
Change S3 object permissions.
Setup the bucket policy and permission and test the object accessibility.
Set up the bucket name. S3 bucket name are globally unique, choose a name which is available. Click on Next
Leave other settings as default and click on Next
Leave other settings as default and click on Next
Leave other settings as default and click on Create bucket
Click on your bucket name.
Click on Upload
Click on Add files, and choose a file from your computer.
After choosing your file, click on Next
Click on Next
Click on Next
Click on Upload.
You could see the uploading procession below.
Now you have a private S3 bucket with a private object uploaded, which means you cannot visit it through Internet.
Change Bucket Permission
Change the permission of the bucket to make the image available publicly.
Click on your object in the bucket, You will see the object details like owner, size, link, etc.
A URL will be listed under Object URL
https://mys3bucketwhizlabs.s3.amazonaws.com/smiley.jpg
Open image Link in a new tab. You will see an AccessDenied message, which means the object is not publicly accessible.
Go back to your bucket and click on Permissions
Click on Everyone under the Public access, and click on Read object on the right of pop-up window. Then click on Save
Now its state switches to Read Object - Yes
Click on Overview, and click on your Object URL again
Create a Bucket Policy
In the previous step, you granted read access only to a specific object.
If you wish to make all objects inside a bucket available publicly, you can achieve this by creating a bucket policy.
Go to the bucket list and click on your bucket name
Click the Permissions tab, then configure the following
Click on Bucket Policy
A blank Bucket policy editor is displayed.
Copy the ARN of your bucket to the clipboard.e.g, arn:aws:s3:::s3bucket180
Replace your bucket ARN with the ARN listed in the JSON below, then copy the entire policy.
Click on Save
Test Public Access
Go back to your bucket list, then click on your bucket.
Upload another file.
Leave all of the settings as default
Click on your new object, and click on the Object URL
Success
Completion and Conclusion
You have successfully created a new AWS S3 Bucket.
You have successfully uploaded an image to the S3 bucket.
You have learned to change S3 object permissions.
You have learned how to create an S3 bucket policy.
How to enable versioning Amazon S3
https://play.whizlabs.com/site/task_details?lab_type=1&task_id=21&quest_id=35
Lab Details
This lab walks you through the steps on how to enables versioning on an AWS S3 Bucket.
Versioning allows you to keep multiple versions of an object in one bucket.
Task Details
Log into the AWS Management Console.
Create an S3 bucket.
Enable object versioning on the bucket.
Upload a text file to the S3 Bucket.
Test object versioning by changing the text file and re-uploading it.
Architecture Diagram
S3 Configuration
Service -> S3
Create an S3 Bucket
On the S3 dashboard, click on Create bucket and fill in the bucket details.
Bucket name: Enter yourBucketName
Note: S3 bucket names are globally unique, choose a name which is available.
Region: Select US East (N. Virginia)
Leave other settings as default.
Click on Create
Close the pop up window if it’s still open.
Enable Versioning on the S3 bucket
Go to the bucket list and click on your bucket name whizlabs234
Click on Properties.
Choose Versioning.
Choose Enable versioning and click on Save.
Now versioning on the S3 bucket is enabled.
Upload an object
Upload any file from your local machine.
On the S3 Buck list page, Click on youBucketName
Click on Overview tab.
Click on the Upload button.
Click on the Add files button.
Browse for the file you want to upload.
Click on Next and Grant public read access to this project(s)
Click on the Upload button.
You can watch the progress of the upload from within the transfer panel at the bottom of the screen.
If it’s a small file, you might not see the transfer.
Once your file has been uploaded, it will be displayed in the bucket.
Make the bucket public with a Bucket Policy
Go to the bucket list and click on your bucket name.
Click the Permissions tab to configure your bucket:
In the Permissions tab, click on Bucket Polity
A blank Bucket policy editor is displayed.
Before creating the policy, you will need to copy the ARN (Amazon Resource Name) of your bucket.
Copy the ARN of your bucket to the clipboard. It is displayed at the top of the policy editor. its look like ARN: arn:aws:s3:::your-bucket-name
In the policy below, update your bucket ARN in the Resource key value and copy the policy code.
Paste the bucket policy into the Bucket policy editor.
Click on Save
Click on Overview
Now open the text file link in your browser and you should see the text in file.
Upload the same object with different content
Upload the same file with different content from your local machine.
On the S3 Buck list page, Click on youBucketName
Click on Overview tab.
Click on the Upload button.
Click on the Add files button.
Browse for the file you want to upload.
Click on Next and Grant public read access to this project(s)
Click on the Upload button.
You should see the latest version of the file you uploaded.
Testing a previous version of the file
To enable the old version of the file, we need to delete the latest version of the file.
Click on object URL
Click on the file name.
On the top section (next to the object name) you can find a drop down of all file versions called
Latest version
Click on the drop down and delete the latest version of your file.
Completion and Conclusion
You have successfully created an S3 Bucket.
You have successfully enabled Object Versioning on the Bucket.
You have successfully uploaded a test file into the Bucket and tested its versioning.
Creating an S3 Lifecycle Policy
https://play.whizlabs.com/site/task_details?lab_type=1&task_id=45&quest_id=35
Lab Details
This Lab walks you through the steps on how to create a Lifecycle Rule for an object in an S3 Bucket.
Lab Tasks
Log into the AWS Management Console.
Create S3 Bucket and upload and object into the bucket.
Create a Lifecycle Rule on the object.
Create Transition types.
Create Transition Expiration.
Test the Lifecycle Rule on the uploaded object.
S3 Configuration
Create an S3 Bucket
On the S3 dashboard, click on Create bucket and fill in the bucket details.
Bucket name: Enter inputYourBucketName
Note: S3 bucket names are globally unique, choose a name which is available.
Region: Select US East (N. Virginia)
Leave other settings as default.
Click on Create
Close the pop up window if it’s still open.
Upload an object
Upload any file from your local machine.
On the S3 Buck list page, Click on youBucketName
Click on Overview tab.
Click on the Upload button.
Click on the Add files button.
Browse for the file you want to upload.
Click on Next and Grant public read access to this project(s)
Click on the Upload button.
You can watch the progress of the upload from within the transfer panel at the bottom of the screen.
If it’s a small file, you might not see the transfer.
Once your file has been uploaded, it will be displayed in the bucket.
Creating a Lifecycle Rule
Click on the Management tab.
Click on Add lifecycle rule to create a lifecycle rule for the uploaded object.
Enter the rule name: tmpLifecycle
Add a filter to limit the scope to prefix/tags: tmp
Choose prefix tmp from the drop-down and click Next.
Storage class transition: Select Current version
For current versions of objects: Click Add transition
Note: currently versioning is disabled so we can’t access previous versions of the object. Object Creation: Select Transition to One Zone-IA after from the drop-down list.
Days after creation: Enter 35. Click on Add transition.
Select Transition to Glacier after from the drop-down list. When asked for a number of days before moving to Glacier, enter 90.
Click on the checkbox saying I acknowledge that this lifecycle rule will increase the one-time lifecycle request cost if it transitions to small objects.
Click on Next.
Note:
Initially, when the object is uploaded, it will be in the standard storage class.
When we create a LifeCycle Policy, the object uploaded using the Lifecycle rule will be migrated to One Zone-IA after 35 days. This means the object will be available only in a Single Availability Zone after 35 days.
After 90 days, our object will be migrated to Glacier. This means our object will be in an archived state. You would have to retrieve the object from Glacier before accessing it.
Create Expiration: Select Current version
Expire the current version of an object: Enter 120.
Select the Clean up incomplete multipart uploads check-box.
Note: Leave the number of days at 7 (Default value). This means that objects which are not properly uploaded will be deleted after 7 days.
Before saving, verify the configurations (you can go back and edit later if you need to change anything). Click Edit.
Click on Save.
The Lifecycle Rule for our object will now be created and enabled.
Completion and Conclusion
You have successfully used the AWS management console to create a Lifecycle Rule for the object in the S3 bucket.
You have configured the details of a Lifecycle Rule.