How to delete files from S3 bucket periodically (AWS)

This article will help setup lifecycle policy for S3 buckets.

Take a use case where you want to clean up logs every month from a bucket automatically. In this case, you will require to setup a lifecycle policy for that bucket.

Step-by-step guide

Follow below steps to setup a life cycle which will delete all files from a directory of a bucket after every 30days interval permanently.

1. Navigate to https://console.aws.amazon.com/s3/home
2. Select the bucket you want to setup lifecycle policy for.
   Click on “Add lifecycle rule” to create a policy.
3. 1. Define name and scope: In scope, you can define a specific folder or directory you would like to apply this policy. If you want to apply this to the whole bucket then don’t select any prefix or tag.
      
   2. Define Transition: You only need to configure transition if you’ve enabled versioning on your bucket. If you haven’t then just click next.
      
   3. Define Expiration: Here you will have options to delete normal content and the incomplete multi part uploads1. Setup deletion of all the files in S3 bucket after 30 days. Select Current Version and set the days you would like to delete objects after.
      Note: You will require Previous version option only when you have versioning enable on S3 bucket, or else just skip that option.

      

      2. Setup deletion for failed/incomplete multipart uploads. AWS S3 gives an option to delete files which were left incomplete while multi part uploads.

      

   4. Review, Save, and exit
      Your policy should look something like this at the end.
      

   Once you’ve configured the policy you will be able to see it on your lifecycle console.

You can also choose to enable and disable policy by going onto More option.

The above Policy will delete files from abc-directory of XYZ bucket every 30days, and also it will delete incomplete uploads every 7days.