16 December 2022 /

Amazon S3 Object URLs

When we push a file to your Amazon S3 bucket, Amazon will generate a (private) https link (named Object URL) for the file. The link is available in the AWS Management Console for every file hosted on your S3 bucket.

The Amazon S3 Management Console shows the Object URL for every object in the bucket.

This does not mean the link immediately works in the browser for everyone.

Whether or not the object or file is publicly accessible directly through the link is governed by the following:

For buckets with ACL enabled: ACL permissions active on the file (initially set through the ACL setting in the Pipe account dashboard's S3 credentials section) and Block public access bucket settings regarding ACL permissions.
for buckets with ACL disabled: the bucket policy and Block public access bucket settings regarding bucket policies

ACL is now disabled by default when you create a new bucket.

The Block public access S3 options need even more explanations.

Until now, we saved this Object URL but only made it available when the chosen canned ACL was public-read or public-read-write. These were the two canned ACLs that made the uploaded files immediately accessible through the link. The link was available through the Pipe account dashboard's s3_logs page and was sent through the video_copied_s3 webhook.

Since ACLs on a file can change and newer buckets don't use ACLs by default but bucket policies to govern the availability of the files, we will now make the link available regardless of the ACL used.

The links to files pushed to S3 services are now shown regardless of the chosen ACL.

The links will now always be visible in the Pipe account dashboard's s3_logs page and will always be sent through the video_copied_s3 webhook.

Previously the s3_logs page would show the message "Private ACL used, no links available", and the video_copied_s3 webhook would be delivered with "Video is not public" as the value of url, rawRecordingUrl, snapshotUrl, and filmstripUrl.

Previously we would show the message "Private ACL used, no links available."

Accessing those links for an object hosted on Amazon S3 that's not configured to be publicly accessible will result in a 403 Forbidden HTTP response status from the S3 server accompanied by an XML body with an Access Denied error message.

The XML response with Access Denied error message returned by Amazon S3.

These object URLs should be available with other storage providers compatible with the S3 API. We tested with DigitalOcean Spaces and Scaleway Object Storage.

However, these providers will have different controls for each object's public/private availability. Scaleway implements a simple "object visibility" control with two options: public and private. DigitalOcean implements a similar "Permissions" control.