Implementing User Uploads with Presigned URLs

Alex Yakubovsky

When you build a web app, you'll often need to support to support file uploads, whether for setting user profile pictures or implementing file sharing. A common solution is to upload files directly to the server and from there upload to blob storage like S3.

Challenges with direct server uploads

Direct server uploads come with challenges:

  • Supporting multipart/form-data is tricky: Handling multipart/form-data, necessary for file uploads requires:

    • Implementing streaming to prevent large files from being fully loaded into memory.

    • Additional dependencies to manage multipart data parsing.

    • If including fields other than the file in the request payload, additional parsing for non-string data types (all non-file field values are interpreted as string)

  • Order of Fields Matters: In multipart requests, if your server attempts to read a text field that comes after a file, before the file itself is stream processed, the file ends up fully buffered in memory.

Presigned URLs

Presigned URLs are temporary links generated by your server that allow users to upload files directly to cloud storage services like AWS S3 and Cloudflare R2, bypassing the server entirely.

Building a file upload feature

In this guide, you'll create a basic file-sharing application using Node.js and React where users can upload files and receive a shareable download link. The complete code sample and installation instructions are available on GitHub

Set up Cloudflare's R2

You'll use Cloudflare's R2 for this walkthrough (any S3 compatible blob storage will work with minimal configuration changes). Generating API keys takes under a minute by following the documentation: https://developers.cloudflare.com/r2/api/s3/tokens/

Next, configure your storage space in Cloudflare's R2:

  1. Create a Bucket: Log into your Cloudflare dashboard, click the R2 tab and create a new bucket. Choose any name you prefer; this bucket will serve as the namespace for your uploads.

  2. Set Access Permissions: By default, a new bucket has restricted access. To enable file uploads and downloads:

    • Go to the R2.dev subdomain section within the R2 dashboard.

    • Click on Allow Access to open the bucket for uploads and downloads. For production applications, consider setting up a custom subdomain to control access more securely.

A new bucket has a restrictive CORS policy. To allow reading and writing from the React application, click the Add CORS policy button and copy over the JSON.

The React app uses Vite, which by default runs on http://localhost:5173. By setting AllowedOrigins as http://localhost:5173 with AllowedMethods of "GET" and "PUT", the React app can both read from and write to the bucket.

Add an endpoint

Install the @aws-sdk/client-s3 and @aws-sdk/s3-request-presigner packages on your server. Since Cloudflare's R2 and other blob storage solutions are S3-compatible, you can use the AWS S3 SDK to interact with them.

Set up your configuration variables:

  • ACCESS_KEY and SECRET_KEY: Use the API keys you generated earlier.

  • BUCKET_PUBLIC_URL: This is the R2 subdomain created earlier, following the pattern https://pub-1e...5a.r2.dev.

  • BUCKET_ENDPOINT: Set this to your Cloudflare storage endpoint, which looks like https://<cloudflare account id>.r2.cloudflarestorage.com. You can find your Cloudflare account ID on the R2 page.

  • BUCKET_NAME: Enter the name you assigned to your bucket.

Initialize the s3 SDK, passing the required configuration.

Initialize the server and create a POST /presigned-urls endpoint. This endpoint will generate the presigned URLs needed for file uploads. Your React application will use this endpoint to obtain a URL for uploading files.

The React app sends the intended upload file size to the server, which then verifies that the size is within acceptable limits (e.g., under 5MB). When generating the presigned URL, the server explicitly sets the expected file size. This prevents an attacker from creating a presigned URL for a small file and then uploading a much larger file.

For each file upload, generate a 16 byte random bucket key. This key becomes part of the shareable URL, making the URL impossible to guess.

Generate the presigned URL, which allows for a PUT request to upload a file to the specified bucketKey. The React app uses this presigned URL to upload the file provided by the user.

Set the presigned URL to expire after a short duration, such as 30 seconds, to minimize the risk of abuse if the URL is leaked.

Prepare the shareable URL for the uploaded file in advance. The React app will display this URL only after the file upload is complete. This URL provides direct access to the uploaded file, allowing anyone with the link to download it.

Finally, send both the signedURL and the publicURL back to the React app.

Add the upload file page

Use a form to handle user submissions.

Include in the form a single input element with type="file", rendering the browser's native file picker. Assign a name="file" attribute to this input to reference the selected file in your code.

When the user submits the form, the file upload is divided into two steps:

Step 1: Requesting a presigned URL

On form submission, the handler makes a POST request to /signed-urls, passing along the size of the file the user chose, to generate a new presigned URL. The server responds with the signedUrl and publicUrl in the JSON body.

Step 2: Upload the user selected file

The file injected in the body of the request made to PUT <signedUrl>. The file is then downloadable via the publicUrl once the request finishes.

Note, There's no need to set the Content-Type header during the file upload; the browser will automatically apply Content-Type: multipart/form-data upon detecting a file in the request body. For more details on multipart/form-data, refer to the MDN documentation.

Lastly, show the user the public URL from which they can share and download the file they have uploaded.

Recap

You've implemented user uploads using presigned URLs by:

  1. Generating a presigned URL with R2 to upload files.

  2. Having the client upload files directly to R2 via the presigned URL.

You've removed the need to manage the complexities of multipart/form-data on your server. If your server needs to associate the uploaded file with another entity (like assigning a profile picture to a user), your API endpoint can accept a URL for the file and use the publicUrl the the link.

Enjoyed this walkthrough? Follow us on Twitter for more similar content.

Craft your own walkthroughs!

Want to create educational walkthroughs like this one? Head over https://annotate.dev and create an account. Free for individuals and an affordable plan for teams!