Amazon S3 buckets protection is a critical security measure when using AWS. There have been numerous bad stories about unprotected S3 buckets, from established contractors like Accenture and Booz Allen Hamilton to huge companies like Verizon Wireless and Time Warner Cable. In this article, you will learn how to enhance the security of S3 buckets with pre-signed S3 URLs, and how to generate them for various cases.
Table of Contents
Pre-signed URLs are used to provide short-term access to a private object in your S3 bucket. They work by appending an AWS Access Key, expiration time, and Sigv4 signature as query parameters to the S3 object.
There are two common use cases when you may want to use them:
Let's explore each of these use cases below.
Imagine you may want to share a confidential presentation with a business partner, or you want to allow a friend to download a video file you're storing in your S3 bucket. In both situations, you could generate a URL in CloudBerry Explorer for Amazon S3, and share it to allow the recipient short-term access.
There are a couple of different approaches for generating these URLs in an ad-hoc, one-off fashion, including:
See below for examples of each of these approaches.
If you use the AWS Tools for Powershell, you can use the Get-S3PreSignedURLcmdlet to generate a pre-signed URL in your Powershell.
The syntax is:
Get-S3PreSignedURL -Bucket cloudberry-examples -Key presentation.ppt -Expires 2018-07-13
The cmdlet will return the URL that you can copy and use as needed.
To generate it with the AWS CLI, you can simply use the aws s3 presign command.
On a Windows system, the command is:
"C:\Program Files\Amazon\AWSCLI\aws.exe" s3 presign s3://cloudberry-examples/presentation.ppt
On a Mac or Linux system, the command is:
$ aws s3 presign s3://cloudberry-examples/presentation.ppt
The console will return a URL that you can copy and paste to the desired user.
MSP360 Explorer is the easiest way to generate a one-off URL. Simply follow the steps below.
First, choose the object for which you want to generate a URL, then click the "Web URL" button, as shown in the image below.
Second, choose whether you want an HTTP or HTTPS URL. You should prefer an HTTPS URL as the query string parameters, including the Access Key and signature, will be sent over a secure connection. Select the "Expire URL at a certain date"option to configure the expiration date. Finally, click "Generate", and copy the URL.
The examples shown above are useful for generating a single pre-signed URL that you need for an ad hoc use case. More commonly, you may have an application that needs to generate short-term access to an S3 bucket. For instance, you need this URL when you store files on S3, and users need to download them from your bucket, or, alternatively, upload something.
You can perform both of these operations with the AWS SDKs for any language. Below are examples of how to use Boto 3, the AWS SDK for Python, to generate these URLs in your application code.
As mentioned above, you may want to provide temporary read access to an S3 object to a user of your application. The code snippet below shows how you would do it in your application code.
First, we import the boto3 library and construct a client to interact with S3. Then, we generate a URL that will allow the GetObject API call on the object we specify:
import boto3 s3 = boto3.client('s3') url = s3.generate_presigned_url( ClientMethod='get_object', Params={ 'Bucket': 'cloudberry-examples', 'Key': 'invoices/user1234/july2018.pdf' } ) print(url) # https://cloudberry-examples.s3.amazonaws.com/invoices/user1234/july2018.pdf?AWSAccessKeyId=AKIALGKOKBY37F5FZF4I&Signature=bPSs8Kcak%2FgjEq
This URL could be sent to a user to view in their browser and receive temporary access.
You can generate a pre-signed URL that can be used for POST requests. This can be useful for allowing clients to upload large files directly from the browser with limited permissions.
Imagine I want to allow a user to upload a file to my cloudberry-examples bucket with the key name of uploads/image.jpg. In the example below, I use the generate_presigned_post method to construct the Amazon S3 URL and return it to the client. I can even add conditions onto the request, such as ensuring the file size is no larger than 1 MB:
import boto3 s3 = boto3.client('s3') response = s3.generate_presigned_post( Bucket='cloudberry-examples', Key='uploads/image.jpg', Conditions=[ ['content-length-range', 1, 1048579] ] ) print(response) {'url': 'https://cloudberry-examples.s3.amazonaws.com/', 'fields': {'key': 'uploads/image.jpg', 'AWSAccessKeyId': 'AKIALGKOKBY37F5FZF4I', 'policy': 'eyJleHBpcmF0aW9uIjogIjIwMTgtMDctMTNUMDI6Mzg6MTBaIiwgImNvbmRpdGlvb nMiOiBbWyJjb250ZW50LWxlbmd0aC1yYW5nZSIsIDEsIDEwNDg1NzldLCB7Im J1Y2tldCI6ICJjbG91ZGJlcnJ5LWV4YW1wbGVzIn0sIHsia2V5IjogInVwbG9 hZHMvaW1hZ2UuanBnIn1dfQ==', 'signature': 'ZY7Orehfdzg+ToJJXhYuV/XyK5o='}}
The response will include a URL property, as well as a fields property with a set of key-value pairs. The fields key-value pairs must be sent with the file as part of a multipart/form-data request.
In this article, we learned that pre-signed S3 URLs are an easy way to provide secure access to an object in your S3 buckets. We discussed a few scenarios where you would want to use them, and how to generate these URLs in a variety of methods.
Like any system, the MSP360 platform offers many features that are used every day to…
Managing your data and ensuring compliance with the ever-evolving regulatory landscape is not just a…
We’re excited to introduce MSP360 Managed Backup for Microsoft 365 and Google Workspace 4.0 with…
Join us for an in-depth exploration of how businesses can leverage MSP360's innovative backup solutions to secure their data…
In today's digital landscape, ensuring the security and accessibility of your data is paramount. Join…
For managed service providers (MSPs), clients are much more than transactions. By approaching each client…