Incomplete S3 Multipart Uploads: The Parts That Keep Billing Until You Abort

Today we are examining incomplete Amazon S3 multipart uploads. We are basing this discussion entirely on the AWS S3 user guide and pricing page sources. Right, we are focusing strictly on those source documents. A multipart upload splits a large object into parts. Yes, the object is divided before transmission.

What happens when one of these uploads is never finished? After you start one, S3 keeps all the parts until you either complete the upload or stop it. So it just waits. Yes, an upload that is started and never completed leaves its parts sitting in the bucket, and billing.

I want to ask exactly how this mechanism is billed while it sits in the bucket. Well, throughout the life of the upload, you are billed for the storage, bandwidth and requests for the upload and its parts. Let's break that down. Okay, you are billed for the storage of the parts. Yes, the storage is billed, and you are billed for the bandwidth and requests. Right, the requests for the upload and its parts are billed.

How does the storage class affect this? Parts are billed according to the storage class specified when the parts are uploaded, not when the upload was initiated. I need to confirm that specific detail. The billing is tied to the parts themselves being uploaded rather than the initiation of the overall object. Yes, that is exactly how it works. So the storage class specified for those specific parts determines the billing rate. Correct, it is based on the storage class specified when the parts are uploaded.

How do these incomplete uploads end up sitting in the bucket? Failed or abandoned uploads are the usual way this happens. So a failed upload leaves the parts behind. Yes, the system retains the parts from a failed upload, and an abandoned upload leaves the parts behind. Right, abandoned uploads also leave the parts sitting in the bucket.

What is the actual signal if someone is looking at their data? Incomplete parts are not ordinary objects. They are not ordinary objects, correct. They are not listed as regular objects, so a standard list operation does not show them. Right, they're not listed as regular objects.

AWS recommends a specific remediation for this. Yes, AWS recommends a lifecycle rule with the abort incomplete multipart upload action. Let's discuss that lifecycle rule, then. What does it do? It aborts incomplete uploads a set number of days after they were initiated. The guide's example uses seven days. Yes, the example in the documentation uses seven days.

What does this lifecycle rule actually apply to, and what might it break? That lifecycle action applies only to incomplete multipart uploads. So it only targets those specific incomplete parts. Right, it does not apply to objects, and it deletes no objects. Yes, because it deletes no completed objects, it is a low-risk rule compared with expiring objects.

What happens when the abort occurs? If you stop the upload, S3 deletes the upload artifacts and all the parts you uploaded, and you are not billed for them. Are there any penalties for this? No, there are no early delete charges for deleting incomplete multipart uploads, whatever the storage class. Whatever the storage class, yes, there are no early delete charges.

I need to add a caution here. You should pick a number of days longer than your slowest legitimate upload, because an upload still in progress when the rule's day count passes would be aborted. That is correct, an active upload would be aborted if it exceeds the day count.

There is a free, read-only, source-available command line tool, pip install cloudcostwise, that runs on your machine against about 20 AWS services and sends nothing to CloudWise.