Hello Machas ❗❗❗ 👋, In this blog I would like to cover every thing on , how S3 Encryption works and what are different types of encryption does S3 allow.
When you upload an object to an S3 bucket, S3 automatically encrypts it before storing it in the bucket. While downloading the object, S3 decrypts it and returns it to us.
I have learned this topic from knowledge india-cloud youtube channel.
Why Encryption Matters Even After the Object Reaches AWS
You might be thinking, 'Hey, I uploaded my file securely using the AWS CLI or SDK, so why bother with encryption once it’s sitting safely in an S3 bucket?' Well, here’s the deal:
When you upload an object to Amazon S3, AWS ensures that it is securely transmitted from local to AWS using HTTPS called as encryption at transit.
Although AWS provides robust security controls, there is always a possibility of unauthorized access due to misconfigurations, insider threats, or rare security vulnerabilities.
So, machas, encrypting your object before storing it in the bucket is super important.
How many ways we can encrypt the Object in S3 ?
Client Side Encryption.
Server Side Encryption (Encryption at rest)
1. SSE - S3 ( S3 managed Keys )
2. SSE - KMS ( KMS managed Keys )
3. SSE - C ( Client Managed Keys )
DSSE ( Dual Layer SSE )
So what are these keys ?
Keys are responsible for encrypting the data using some encryption algorithm. AWS uses AES-256 as encryption algorithm and to decrypt the data, if it is symmetric key encryption, then same key is used to decrypt and if it is asymmetric key encryption, different key is used to decrypt the data.
Before diving deep to the various S3 encryption methods. I will explain these encryption methods using AWS KMS ( Key Management Service ) , its important to learn basics of KMS.
Basics of KMS
AWS Key Management Service (KMS) is a fully managed service that helps you create, manage, and control, encryption keys used to secure your data.
Services like S3, EBS, RDS, and Lambda use KMS by default for encryption at rest.
Envelope Encryption
KMS uses Envelope Encryption method to encrypt the data. When you/AWS creates a Key in KMS it generates a 2 keys , plain data key and encrypted data key, this plain data key is responsible for encrypting the actual data and the encrypted data key, is stored along with the encrypted data, during decryption this keys gets utilized.
Once encryption is done, plain data key gets deleted. KMS Key never leaves KMS Service unencrypted.
Encryption
When encrypted data is requested, as it contains an encrypted data key, S3 makes a Decrypt API call to AWS KMS. KMS decrypts the encrypted data key and returns the plain data key. Using this plain data key, S3 decrypts the encrypted data and sends it to the user.
This is how encryption works when we use KMS.
Now lets learn basic kms commands to encrypt and decrypt data before moving on to S3 encryption.
Create-Key
There are 2 main options while creating a key in KMS:
—key-usage ( This defines the purpose of the key )
which means whether it is used for encryption & decryption or used for signing and verifying data.
—key-spec ( Specifies the Type of Key )
which means symmetric or asymmetric key.
Lets create a asymmetric key for encryption and decryption.
aws kms create-key --key-usage ENCRYPT_DECRYPT --key-spec RSA_2048
available options for —key-usage & —key-spec are :
If you select SYMMETRIC_DEFAULT it creates symmetric key
Generate Data Key
This command is going to generate a data key ,which is used to encrypt the data right, as we discussed above. This command is only be used if the key is symmetric, if it asymmetric this command won’t work as in asymmetric key , public key is used to encrypt and private key is used to decrypt the data.
aws kms generate-data-key --key-id 1234567 --key-spec AES_256 --output yaml > generate.yaml
This generates a data key file and this is used to encrypt the data.
Encrypt and Decrypt
aws kms encrypt --key-id 1234 --plaintext fileb://cloud.txt --output text \
--query CiphertextBlob > encrypt.txt
This command is used to encrypt the local text file
fileb is to convert simple text to base64.
aws kms decrypt --ciphertext-blob file://encrypt.yaml --output text > decrypt.yaml
This command is used to decrypt the encrypted text file to base 64 .
Now that we have understood the basics of KMS lets dive in to the S3 Encryption .
In all these Server Side Encryption methods, AWS uses Envelope Encryption method , AWS uses AES-256 algorithm , this is symmetric key encryption encryption algorithm , SO if you create a asymmetric key and wanted use SSE it won’t work.
SSE - S3 Managed Keys
Encryption is handled automatically by S3, and no key management is required.
By default S3 uses AES-256 encryption algorithm.
aws s3api put-object --bucket my-bucket --key ~/KGF.txt --body KGF.txt \
--server-side-encryption AES256
If you select AES 256 while uploading object , it means you are using S3 managed keys. S3 in back ground uses KMS only to encrypt but it is AWS owned Key, not visible to us .
SSE- KMS Managed Keys
Uses a KMS key (AWS-managed or customer-managed) for encryption.
aws s3api put-object --bucket my-bucket --key ~/KGF.txt --body KGF.txt \
--server-side-encryption aws:kms
While uploading object if you use aws:kms it means you are using AWS managed KMS key, it is there by default.
How to give our own key which we have created in KMS service ?
aws s3api put-object --bucket my-bucket --key ~/KGF.txt --body KGF.txt \
--server-side-encryption aws:kms --sse-kms-id 12345
We have to give our KMS key Id.
SSE-C
Client provides the key for encryption and manage the encryption key. S3 does not store the key, and AWS does not have access to it. When uploading the data, you must provide the key for encryption. On retrieval, you must provide the same key for decryption.
I will use KMS itself for creating a key here as well , here you can openssl or any other alternatives as well.
I will create a KMS key using aws kms create key and I will generate a data key using generate-data-key command as it will output encrypted data key as well so to filter only plain data key I use —query.
aws kms generate-data-key \
--key-id <kms-key-id> \
--key-spec AES_256 \
--query Plaintext\
--output text > Key.txt
Now using this key I ask AWS S3 to encrypt and decrypt the object.
aws s3api put-object \
--bucket my-bucket \
--key ~/KGF.sh\
--body KGF.sh \
--sse-customer-key fileb://Key.txt\
--sse-customer-algorithm AES256 # Bydefault it uses AES 256 only
While retrieving, you will need to again send this key.
aws s3api get-object \
--bucket my-bucket \
--key ~/KGF.sh \
--sse-customer-key fileb://Key.txt \
--sse-customer-algorithm AES256 \
outfile.sh
Key you have provided will be removed from S3 memory after each request.
S3 will store a randomly hash based message authentication for your encryption key to validate future request.
With bucket versioning, different object versions can be encrypted with different keys.
DSSE-KMS ( Dual Layer SSE)
In DSSE, there are two separate layers of encryption applied to the data.
•First Layer: Data is encrypted using a data key (AES-256). Data key can be generated by either S3 managed Key or KMS Key.
Second Layer: The data key from the first layer is further encrypted using a separate KMS key
aws s3api put-object --bucket my-bucket --key ~/KGF.sh\
--server-side-encryption aws:kms:dsse --ssekms-key-id 1234
If you don’t mention key id s3 uses s3 managed keys .
Extra cost will be applied.
So in all these SSE, AWS takes care of the encryption and decryption process.
Client Side Encryption
In this you will encrypt the object and send to S3 and again S3 don’t use S3 managed keys to encrypt . While downloading , S3 sends you the encrypted file you need to decrypt with the key you have .
Bucket Key
You can choose S3 bucket key option to lower down the KMS charges.
When we try to encrypt the object using KMS, S3 makes a encrypt decrypt calls to KMS service . If there are several objects and several retrieve object calls are made then this could ramp up the bill.
Using Bucket key bill gets reduced by 90%. When we enable bucket key
Encryption
During encryption : KMS Key generates bucket key and encrypted bucket key , this bucket key generates actual data key and encrypted data key. This data key is used to encrypt the data. S3 stores both encrypted data key and encrypted bucket key. S3 deletes the data keys.
During Decryption: S3 contains encrypted data key , this must be decrypted with bucket key , S3 also contains encrypted bucket key, S3 requests KMS service for the bucket key for the corresponding encrypted bucket key. Now bucket key decrypts the encrypted data key and data key decrypts the data and returns to user.
This is useful only frequent retrieve calls are being made to S3 object if not, this doesn’t make sense right? S3 stores the bucket key temporarily if in this time , getobject calls are made to S3 then S3 doesn’t request KMS service for the bucket key then bill wont grow much. If frequent calls are not being made then enabling bucket key has no use.
This bucket key generates one more data key to encrypt one more object and stores its encrypted data key and so on…
How to Set Bucket Encryption using CLI
To Set SSE-S3
aws s3api put-bucket-encryption \
--bucket my-bucket \
--server-side-encryption-configuration '{
"Rules": [
{
"ApplyServerSideEncryptionByDefault": {
"SSEAlgorithm": "AES256"
}
}
]
}'
To Set KMS Key
aws s3api put-bucket-encryption \
--bucket my-bucket \
--server-side-encryption-configuration '{
"Rules": [
{
"ApplyServerSideEncryptionByDefault": {
"SSEAlgorithm": "aws:kms",
"KMSMasterKeyID": 12345 // Optional
}
}
]
}'
You can even write write rules in a json file and use in a command.
// encrypt.json
{
"ApplyServerSideEncryptionByDefault": {
"SSEAlgorithm": "aws:kms",
"KMSMasterKeyID": 12345 // Optional
}
}
aws s3api put-bucket-encryption \
--bucket my-bucket \
--server-side-encryption-configuration file://encrypt.json
Types of KMS Keys
Keys live in AWS account within region.
AWS Managed Keys
AWS manages the key , you cannot enable , disable , delete , control etc…
AWS creates one key per region per service
One key will be used for all the resources of a particular service
Example : All S3 buckets will use one key.
Usage charges will be there and no monthly charges.
Customer Managed Keys
You manage keys , you can enable , disable , delete etc..
You can create multiple keys and one key can be used for all the resources for a particular service.
Monthly charges will be there and also usage charges .
AWS Owned Keys
These keys are created & used by AWS for internal encryption across different AWS services.
Customers do not have visibility to these keys .
Example: SSE - S3 option work with these keys.
I hope, I have explained all about S3 encryption. If not let me know what I have missed.
Thanks for reading my article. Have a great day 🚀🚀🚀 .