Using Amazon S3 Pre-Signed URLs for Temporary Object Access

Protecting your S3 buckets is a critical security measure when using AWS. There have been numerous bad stories about unprotected S3 buckets, from established contractors like Accenture and Booz Allen Hamilton, to huge companies like Verizon Wireless and Time Warner Cable.

In this article, we'll learn how and why to use pre-signed S3 URLs to provide secure, temporary access to objects in your S3 buckets. We will discuss generating pre-signed S3 URLs for occasional, one-off use cases as well as programmatically generating them for use in yourapplication code. There are multiple code examples for each use case.

What are pre-signed S3 URLs and why use them?

Pre-signed S3 URLs are a way to securely provide short-term access to a private object in your S3 bucket. They work by appending an AWS Access Key, expiration time, and Sigv4 signature as query parameters to the S3 object. The Access Key and signature are configured to provide scoped, time-limited access to the particular object.

Pre-signed S3 URLs are useful whenever you want to easily provide temporary access to a protected asset. There are two common use cases when you may want to use pre-signed S3 URLs:

  • Simple, occasional sharing of private files.
  • Frequent, programmatic access to view or upload a file in an application.

Let's explore each of these use cases below.

Using pre-signed S3 URLs for simple sharing of a private file

Imagine you may want to share a confidential presentation with a business partner, or you want to allow a friend to download a video file you're storing in your S3 bucket. In both situations, you could generate a pre-signed S3 URL in CloudBerry Explorer, then email or message them the URL which would allow the recipient short-term access.

There are a couple different approaches for generating these pre-signed S3 URLs in an ad-hoc, one-off fashion, including:

See below for examples of each of these approaches.

Generating a pre-signed S3 URL with AWS Tools for Powershell

If you use the AWS Tools for Powershell, you can use the Get-S3PreSignedURLcmdlet to generate a pre-signed S3 URL in your Powershell.

The syntax is:

Get-S3PreSignedURL -Bucket cloudberry-examples -Key presentation.ppt -Expires 2018-07-13

The cmdlet will return the URL that you can copy and use as needed.

Generating a pre-signed S3 URL with the AWS CLI

To generate a pre-signed S3 URL with the AWS CLI, you can simply use the aws s3 presign command.

On a Windows system, the command is:

"C:\Program Files\Amazon\AWSCLI\aws.exe" s3 presign s3://cloudberry-examples/presentation.ppt

On a Mac or Linux system, the command is:

$ aws s3 presign s3://cloudberry-examples/presentation.ppt

The console will return a pre-signed S3 URL that you can copy and paste to the desired user.

Generating a pre-signed S3 URL with CloudBerry Explorer

CloudBerry Explorer is the easiest way to generate a one-off pre-signed S3 URL. Simply follow the steps below.

First, choose the object for which you want to generate a pre-signed S3 URL, then click the "Web URL" button, as shown in the image below.

Second, choose whether you want an HTTP or HTTPS URL. You should prefer an HTTPS URL as the querystring parameters, including the Access Key and signature, will be sent over a secure connection. Then click the box to "Expire URL at certain date", and choose when you want it to expire. Finally, click "Generate", then copy the URL that is shown in the box, as shown in the image below.

Using pre-signed S3 URLs for temporary, automated access in your application code

The examples shown above are useful for generating a single pre-signed S3 URL that you need for an ad hoc use case. More commonly, you may have an application that needs to programmatically generate short-term access to an S3 bucket.

Some examples of this programmatic usage include:

  • Your application generates invoice PDFs at the end of a billing cycle and stores the PDFs on S3. You need to provide a link for your users to download the PDF of their invoice.
  • Your application allows users to upload videos to your S3 bucket. You would like users to upload directly from their browser, rather than sending the video to your servers, without leaking credentials to the browser.

You can perform both of these operations with the AWS SDKs for any language. Below are examples of how to use Boto 3, the AWS SDK for Python, to generate pre-signed S3 URLs in your application code.

Generating a pre-signed S3 URL for reading an object in your application code with Python and Boto3

As mentioned above, you may want to provide temporary read access to an S3 object to a user of your application, such as downloading a PDF of an invoice. The code snippet below shows how you would do it in your application code.

First, we import the boto3 library and construct a client to interact with S3. Then, we generate a pre-signed S3 URL that will allow the GetObject API call on the object we specify:

import boto3

s3 = boto3.client('s3')

url = s3.generate_presigned_url(
    ClientMethod='get_object',
    Params={
        'Bucket': 'cloudberry-examples',
        'Key': 'invoices/user1234/july2018.pdf'
    }
)
print(url)
# https://cloudberry-examples.s3.amazonaws.com/invoices/user1234/july2018.pdf?AWSAccessKeyId=AKIALGKOKBY37F5FZF4I&Signature=bPSs8Kcak%2FgjEqqjOO5cFS022x0%3D&Expires=1531446995

The resulting URL could be sent to our user to view in their browser and receive temporary access to the invoice.

Generating a pre-signed S3 URL for uploading an object in your application code with Python and Boto3

You can generate a pre-signed S3 URL that can be used for POST requests. This can be useful for allowing clients to upload large files. Rather than sending the large file through your application's servers, the client can upload the file directly from the browser via tightly-scoped permissions.

Imagine I want to allow a user to upload a file to my cloudberry-examples bucket with the key name of uploads/image.jpg. In the example below, I use the generate_presigned_post method to construct the URL and return it to the client. I can even add conditions onto the request, such as ensuring the file size is no larger than 1 MB:

import boto3

s3 = boto3.client('s3')

response = s3.generate_presigned_post(
    Bucket='cloudberry-examples',
    Key='uploads/image.jpg',
    Conditions=[
        ['content-length-range', 1, 1048579]
    ]
)

print(response)
{'url': 'https://cloudberry-examples.s3.amazonaws.com/', 'fields': {'key': 'uploads/image.jpg', 'AWSAccessKeyId': 'AKIALGKOKBY37F5FZF4I', 'policy': 'eyJleHBpcmF0aW9uIjogIjIwMTgtMDctMTNUMDI6Mzg6MTBaIiwgImNvbmRpdGlvbnMiOiBbWyJjb250ZW50LWxlbmd0aC1yYW5nZSIsIDEsIDEwNDg1NzldLCB7ImJ1Y2tldCI6ICJjbG91ZGJlcnJ5LWV4YW1wbGVzIn0sIHsia2V5IjogInVwbG9hZHMvaW1hZ2UuanBnIn1dfQ==', 'signature': 'ZY7Orehfdzg+ToJJXhYuV/XyK5o='}}

The response will include a URL property, as well as a fields property with a set of key-value pairs. The fields key-value pairs must be sent with the file as part of a multipart/form-data request.

Conclusion

In this article, we learned that pre-signed S3 URLs are an easy way to provide secure access to an object in your S3 buckets. We discussed a few scenarios where you would want to use pre-signed S3 URLs. Finally, we learned how to generate pre-signed S3 URLs in a variety of methods.

 

1 Star2 Stars3 Stars4 Stars5 Stars (3 votes, average: 5.00 out of 5)
Loading...

About the author

Alex DeBrie is a software developer with a background in data engineering and cloud infrastructure management. He has deep experience with AWS and is a proponent of infrastructure as code. In his free time, he likes running and spending time with his wife and four kids. Check out his twitter @alexbdebrie