Hello Every One, in this blog I would like to explain what is versioning in AWS S3 and how to enable it using CLI.

Versioning

Versioning is used when you want to :

1. Track Changes → Maintain different versions of a file, so you never lose important data.

2. Preventing Accidental Deletion → If an object is deleted, you can restore the previous version.

3. Disaster Recovery → If an object is corrupted or overwritten, you can roll back to an earlier version.

So when you want to maintain any one of these you have to enable versioning.

When versioning is enabled, each update to an object creates a new version. The old versions are not deleted, but stored as previous versions, new version id will be added to the object meta data.

If you enable versioning after uploading some objects version id to the previous objects will be null.

Enabling Versioning

After creating bucket let’s say korla-goutham

 aws s3api put-bucket-versioning --bucket korla-goutham\
--versioning-configuration Status=Enabled

Versioning will be enabled by this command .

To verify whether the versioning applied or not

aws s3api get-bucket-versioning --bucket korla-goutham

You will get result like this:

Upload a object you will get version ID of the object as a response.

aws s3api put-object --bucket korla-goutham\
--key ~/KGF.txt
--body KGF.txt

As versioning is enabled we can again upload the same KGF.txt file by modifying the inner text.

Again after uploading recent upload becomes the newer object.

How to get all the versions of the object?

aws s3api list-object-versions --bucket korla-goutham

You can download the previous version of the object by

aws s3api get-object --bucket korla-goutham --key KGF.txt --version-id  12345 KGF

Restoring files

When you delete the file without mentioning the version id all the versions of the file can be restored, but if you mention the version it cannot be restored.

When you simply delete a file without version id , The object is not actually deleted but becomes hidden due to the creation of a delete marker file . If you delete the delete marker file all the versions of that file will be restored.

MFA Enabled

Like you protect your account with MFA, in the same way you can protect your objects with the same MFA when some one try to delete the object.

This setting can only be applied using CLI / REST API

You can enable using :

 aws s3api put-bucket-versioning --bucket korla-goutham\
--versioning-configuration Status=Enabled,MFADelete=Enabled \
--mfa "SERIAL_NUMBER MFA_CODE"

You can get the serial number of your MFA by simply navigating to security credentials, using CLI:

aws iam list-mfa-devices --user-name KGF

This will list all the mfa devices attached to KGF user select the one you like and enter the MFA code.

MFA delete will be set. From now when you delete the object you need the MFA code.

Real World Task

Now there can be 1000s of versions of a file. Do you delete it manually if you want to permanently delete the files ? Now lets permanently delete the files considering there are 1000s of versions of a file.

Lets write a script to do this.

To delete a version of a object you’ll need version id of a object.

aws s3api list-object-versions --bucket korla-goutham\
--query '{Versions: Versions[].{Key:Key, Versionid:VersionId}}' > versions.yaml

Now Output looks something like this.

Disable the mfa delete setting as you need to frequently pass the mfa code while script is running.

Now lets write script to delete these objects.

#!/bin/bash

for key_info in $(yq -o=json '.Versions[]' versions.yaml | jq -c '.'); do
  # Extract Key and VersionId from each object
  key=$(echo "$key_info" | jq '.Key')
  version_id=$(echo "$key_info" | jq  '.Versionid')

  # Print or use the values
  echo "Key: $key, VersionId: $version_id"

  # Perform an action, e.g., delete the object
  aws s3api delete-object --bucket korla-goutham --key "$key" --version-id "$version_id"
done

This deletes all the versions of object.

Thanks for reading my article . Have a great day 🎉🎉🎉