AWS S3 Best Practices: Security, Performance, and Cost

S3 is one of the first AWS services most developers use, and one of the easiest to misconfigure. A public bucket leaks sensitive data. Missing lifecycle rules accumulate storage costs for years. Incorrect access patterns create performance bottlenecks that are invisible until traffic scales. These problems are avoidable when you apply AWS S3 best practices from the start rather than fixing them retroactively under pressure.

This tutorial covers production-grade S3 configuration across three areas: security (keeping data private and encrypted), performance (optimizing uploads, downloads, and access patterns), and cost (controlling storage expenses as data grows). Each section includes the specific configuration you need with working code examples.

Security: Locking Down Your Buckets

S3 security misconfigurations are among the most common causes of data breaches in cloud applications. The defaults have improved over the years, but understanding each layer is essential for meeting AWS S3 best practices in production.

Block Public Access (Non-Negotiable)

Every production bucket should have Block Public Access enabled at both the account level and the bucket level. This setting overrides any bucket policy or ACL that would make objects publicly accessible.

import boto3

s3 = boto3.client('s3')

# Enable block public access on a specific bucket
s3.put_public_access_block(
    Bucket='my-app-data',
    PublicAccessBlockConfiguration={
        'BlockPublicAcls': True,
        'IgnorePublicAcls': True,
        'BlockPublicPolicy': True,
        'RestrictPublicBuckets': True,
    }
)

Enable this at the AWS account level too, so new buckets are protected by default:

s3_control = boto3.client('s3control')

s3_control.put_public_access_block(
    AccountId='123456789012',
    PublicAccessBlockConfiguration={
        'BlockPublicAcls': True,
        'IgnorePublicAcls': True,
        'BlockPublicPolicy': True,
        'RestrictPublicBuckets': True,
    }
)

If you need to serve files publicly (static website assets, public downloads), use CloudFront with an Origin Access Identity instead of making the S3 bucket public. This keeps the bucket private while CloudFront serves content at the edge.

Bucket Policies: Least Privilege

Bucket policies define who can access which objects and what actions they can perform. Apply the principle of least privilege — grant only the permissions each principal actually needs.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowAppServerReadWrite",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::123456789012:role/app-server-role"
      },
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Resource": "arn:aws:s3:::my-app-data/uploads/*"
    },
    {
      "Sid": "DenyUnencryptedUploads",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::my-app-data/*",
      "Condition": {
        "StringNotEquals": {
          "s3:x-amz-server-side-encryption": "aws:kms"
        }
      }
    },
    {
      "Sid": "DenyInsecureTransport",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::my-app-data",
        "arn:aws:s3:::my-app-data/*"
      ],
      "Condition": {
        "Bool": {
          "aws:SecureTransport": "false"
        }
      }
    }
  ]
}

This policy does three things: it grants the application role access to only the uploads/ prefix, it denies any upload that is not KMS-encrypted, and it denies any request over plain HTTP. These three statements cover the most critical security controls for production buckets.

Server-Side Encryption

Enable default encryption so every object stored in the bucket is automatically encrypted at rest. S3 offers three encryption options:

• SSE-S3: AWS manages the keys. Simplest option, no additional cost
• SSE-KMS: AWS KMS manages the keys. Provides audit trail via CloudTrail and fine-grained key policies
• SSE-C: You manage the keys. Maximum control but you are responsible for key storage and rotation

# Enable default SSE-KMS encryption on a bucket
s3.put_bucket_encryption(
    Bucket='my-app-data',
    ServerSideEncryptionConfiguration={
        'Rules': [{
            'ApplyServerSideEncryptionByDefault': {
                'SSEAlgorithm': 'aws:kms',
                'KMSMasterKeyID': 'arn:aws:kms:us-east-1:123456789012:key/my-key-id'
            },
            'BucketKeyEnabled': True  # Reduces KMS API costs
        }]
    }
)

BucketKeyEnabled is important for cost control. Without it, every S3 operation makes a separate KMS API call. With bucket keys, S3 generates a bucket-level key that handles encryption locally, reducing KMS costs by up to 99% for high-volume buckets.

Pre-Signed URLs for Temporary Access

When users need to upload or download files, generate pre-signed URLs instead of exposing bucket credentials. A pre-signed URL grants temporary access to a specific object with a configurable expiration.

# Generate a pre-signed URL for uploading (PUT)
upload_url = s3.generate_presigned_url(
    'put_object',
    Params={
        'Bucket': 'my-app-data',
        'Key': f'uploads/{user_id}/{filename}',
        'ContentType': 'image/jpeg',
    },
    ExpiresIn=300  # 5 minutes
)

# Generate a pre-signed URL for downloading (GET)
download_url = s3.generate_presigned_url(
    'get_object',
    Params={
        'Bucket': 'my-app-data',
        'Key': f'uploads/{user_id}/{filename}',
    },
    ExpiresIn=3600  # 1 hour
)

Pre-signed URLs keep the bucket private while giving clients direct access to S3 for uploads and downloads. This avoids routing file data through your application server, which saves compute costs and reduces latency. For applications with Lambda-based architectures, pre-signed URLs also avoid Lambda’s 6 MB payload limit for synchronous invocations.

Versioning and Object Lock

Enable versioning to protect against accidental deletions and overwrites. When versioning is enabled, S3 keeps every version of an object — deleting an object creates a delete marker, and the previous version remains recoverable.

s3.put_bucket_versioning(
    Bucket='my-app-data',
    VersioningConfiguration={'Status': 'Enabled'}
)

For compliance-sensitive data (financial records, healthcare data), Object Lock prevents any user — including the root account — from deleting or modifying objects for a specified retention period.

Performance: Optimizing Throughput and Latency

S3 handles enormous throughput by default (3,500 PUT/COPY/POST/DELETE and 5,500 GET/HEAD requests per second per prefix), but suboptimal access patterns can create bottlenecks that are hard to diagnose.

Key Naming and Prefix Distribution

S3 partitions data by key prefix for parallel access. If all your objects share the same prefix, requests concentrate on the same partition, limiting throughput. Distribute objects across multiple prefixes for better parallelism.

# Poor: all objects under a single prefix
# s3://my-bucket/uploads/2026-04-10-file1.jpg
# s3://my-bucket/uploads/2026-04-10-file2.jpg

# Better: distribute by user ID or hash prefix
# s3://my-bucket/uploads/user-1234/2026-04-10-file1.jpg
# s3://my-bucket/uploads/user-5678/2026-04-10-file2.jpg

# Best for high-throughput: add a hash prefix
import hashlib

def generate_s3_key(user_id: str, filename: str) -> str:
    hash_prefix = hashlib.md5(user_id.encode()).hexdigest()[:4]
    return f"uploads/{hash_prefix}/{user_id}/{filename}"

S3 automatically handles prefix scaling for most workloads, but adding a hash prefix ensures even distribution from day one. This matters most for write-heavy workloads with thousands of objects per second.

Multipart Uploads for Large Files

For files larger than 100 MB, use multipart uploads. S3 splits the file into parts, uploads them in parallel, and assembles the final object. This improves throughput and allows resuming failed uploads without restarting from scratch.

from boto3.s3.transfer import TransferConfig

# Configure multipart upload thresholds
config = TransferConfig(
    multipart_threshold=100 * 1024 * 1024,  # 100 MB
    max_concurrency=10,
    multipart_chunksize=50 * 1024 * 1024,   # 50 MB chunks
)

s3.upload_file(
    'large-backup.tar.gz',
    'my-app-data',
    'backups/2026-04-10/large-backup.tar.gz',
    Config=config
)

The boto3 transfer manager handles multipart uploads automatically when the file exceeds the threshold. Adjust max_concurrency based on your available bandwidth — more concurrent parts mean faster uploads on high-bandwidth connections.

Transfer Acceleration

S3 Transfer Acceleration routes uploads through CloudFront edge locations, using AWS’s backbone network to reach the S3 bucket faster. This benefits users uploading from locations far from your bucket’s region.

# Enable transfer acceleration on the bucket
s3.put_bucket_accelerate_configuration(
    Bucket='my-app-data',
    AccelerateConfiguration={'Status': 'Enabled'}
)

# Use the accelerated endpoint for uploads
s3_accelerated = boto3.client(
    's3',
    endpoint_url='https://my-app-data.s3-accelerate.amazonaws.com'
)

s3_accelerated.upload_file('video.mp4', 'my-app-data', 'videos/upload.mp4')

Transfer Acceleration adds roughly $0.04/GB on top of standard transfer costs. Test whether it actually improves upload speed for your users before enabling it — for users close to the bucket’s region, the improvement may not justify the additional cost.

CloudFront for Read-Heavy Workloads

If your application serves the same objects to many users (product images, static assets, public documents), put CloudFront in front of S3. CloudFront caches objects at edge locations worldwide, reducing both latency and S3 request costs.

# CDK example: S3 bucket with CloudFront distribution
import * as cdk from 'aws-cdk-lib';
import * as s3 from 'aws-cdk-lib/aws-s3';
import * as cloudfront from 'aws-cdk-lib/aws-cloudfront';
import * as origins from 'aws-cdk-lib/aws-cloudfront-origins';

const bucket = new s3.Bucket(this, 'AssetsBucket', {
  blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
  encryption: s3.BucketEncryption.S3_MANAGED,
});

const distribution = new cloudfront.Distribution(this, 'AssetsDistribution', {
  defaultBehavior: {
    origin: origins.S3BucketOrigin.withOriginAccessControl(bucket),
    viewerProtocolPolicy: cloudfront.ViewerProtocolPolicy.REDIRECT_TO_HTTPS,
    cachePolicy: cloudfront.CachePolicy.CACHING_OPTIMIZED,
  },
});

For teams managing infrastructure with AWS CDK, the S3 + CloudFront pattern is a common construct that can be defined once and reused across projects.

Cost: Controlling Storage Expenses

S3 storage costs are low individually ($0.023/GB/month for Standard), but they accumulate relentlessly. Without lifecycle policies, data stored “temporarily” five years ago is still costing you money today.

Storage Classes

S3 offers multiple storage classes at different price points. Choose the class based on how frequently you access the data:

Storage Class	Cost (GB/month)	Access Pattern	Retrieval Cost
Standard	$0.023	Frequent access	None
Intelligent-Tiering	$0.023 + monitoring fee	Unpredictable	None
Standard-IA	$0.0125	Infrequent (monthly)	$0.01/GB
One Zone-IA	$0.01	Infrequent, non-critical	$0.01/GB
Glacier Instant	$0.004	Rare (quarterly)	$0.03/GB
Glacier Flexible	$0.0036	Archive (hours to retrieve)	$0.01-0.03/GB
Glacier Deep Archive	$0.00099	Long-term archive (12+ hours)	$0.02/GB

Intelligent-Tiering is the low-effort choice — S3 automatically moves objects between access tiers based on usage patterns. You pay a small monitoring fee ($0.0025/1,000 objects/month), but you never overpay for storage class selection.

Lifecycle Rules

Lifecycle rules automatically transition objects between storage classes or delete them after a specified period. Define these rules based on your data’s access patterns.

s3.put_bucket_lifecycle_configuration(
    Bucket='my-app-data',
    LifecycleConfiguration={
        'Rules': [
            {
                'ID': 'archive-old-logs',
                'Prefix': 'logs/',
                'Status': 'Enabled',
                'Transitions': [
                    {'Days': 30, 'StorageClass': 'STANDARD_IA'},
                    {'Days': 90, 'StorageClass': 'GLACIER_IR'},
                    {'Days': 365, 'StorageClass': 'DEEP_ARCHIVE'},
                ],
            },
            {
                'ID': 'delete-temp-uploads',
                'Prefix': 'temp/',
                'Status': 'Enabled',
                'Expiration': {'Days': 7},
            },
            {
                'ID': 'cleanup-old-versions',
                'Prefix': '',
                'Status': 'Enabled',
                'NoncurrentVersionExpiration': {'NoncurrentDays': 30},
            },
        ]
    }
)

This configuration does three things: it transitions logs through progressively cheaper storage classes over time, it deletes temporary uploads after 7 days, and it removes old object versions after 30 days. Without these rules, logs and temp files accumulate indefinitely at Standard storage prices.

Monitoring Storage Costs

Use S3 Storage Lens for a dashboard view of your storage usage, cost, and activity patterns across all buckets. Storage Lens identifies buckets with no lifecycle policies, objects that have not been accessed in over a year, and buckets where versioning creates excessive storage.

Set up CloudWatch alarms on the BucketSizeBytes metric to catch unexpected storage growth before it impacts your bill. A sudden spike in storage often indicates a logging misconfiguration or a retry loop that is writing duplicate objects.

For broader strategies on managing your AWS bill, cost optimization fundamentals cover the patterns that apply across all AWS services, not just S3.

Requester Pays

If you host datasets or files that external parties download heavily, enable Requester Pays on the bucket. The requester’s AWS account pays for the data transfer instead of yours. This is common for public datasets and shared resources.

Real-World Scenario: Optimizing S3 for a Media-Heavy SaaS Platform

A project management SaaS allows users to attach files to tasks — documents, images, and videos. After two years of operation with around 3,000 active accounts, the S3 bill reaches $1,200/month. The team investigates and discovers several issues.

First, temporary files from failed uploads sit in a temp/ prefix indefinitely. These account for 800 GB of Standard storage that nobody accesses. Adding a lifecycle rule to delete temp files after 48 hours immediately recovers that storage.

Second, file attachments from completed projects (older than 6 months) are rarely accessed but stored in Standard. Transitioning these to Standard-IA after 90 days and Glacier Instant Retrieval after 180 days reduces their storage cost by roughly 75%. The occasional access to old project files incurs a small retrieval fee, but the storage savings far exceed it.

Third, versioning is enabled but old versions are never cleaned up. The bucket contains 3 TB of current objects and 5 TB of non-current versions. Adding a lifecycle rule to expire non-current versions after 30 days recovers the 5 TB over the following month.

After applying these three changes, the monthly S3 bill drops from $1,200 to approximately $400 — a 67% reduction with no impact on user experience. The team also enables Intelligent-Tiering for the main uploads prefix so that future storage automatically optimizes without manual intervention.

When to Apply These AWS S3 Best Practices

• Any production application storing user data, backups, or logs in S3
• Applications handling file uploads where security (public access prevention, encryption) is non-negotiable
• Media-heavy applications where storage costs grow linearly with user activity
• Applications serving static assets to geographically distributed users

When to Reconsider S3

• Your storage needs are under 10 GB and a simpler solution (local disk, managed hosting) handles it without the AWS overhead
• All file access is within a single server and does not need the durability, availability, or scale that S3 provides
• You need a file system interface (directory operations, file locks) — EFS or EBS fits better than S3’s object storage model

Common Mistakes with AWS S3

• Leaving Block Public Access disabled because “we might need public access later” — enable it always and use CloudFront or pre-signed URLs for public content
• Not enabling default encryption, assuming “nobody will access the raw storage” — encryption at rest is a baseline security requirement, not an optional enhancement
• Skipping lifecycle rules because storage is cheap — it is cheap per GB, but unmanaged growth over years creates surprising costs
• Using the same storage class for all data regardless of access frequency — logs accessed monthly should not sit in Standard alongside actively served user uploads
• Generating pre-signed URLs with excessively long expiration times (days or weeks) — keep expirations as short as practically possible to limit exposure if a URL leaks
• Not enabling versioning on buckets that store data you cannot afford to lose — accidental deletion without versioning means permanent data loss
• Routing all file uploads through your application server instead of using pre-signed URLs for direct client-to-S3 uploads, wasting compute and adding unnecessary latency

Building on AWS S3 Best Practices

AWS S3 best practices fall into three categories that reinforce each other. Security (block public access, encryption, least-privilege policies, pre-signed URLs) protects your data. Performance (prefix distribution, multipart uploads, transfer acceleration, CloudFront) keeps access fast. Cost optimization (storage classes, lifecycle rules, version cleanup, monitoring) prevents storage expenses from growing unchecked.

Apply these configurations when you create each bucket, not after problems appear. The cost of setting up lifecycle rules and encryption on day one is a few minutes of configuration. The cost of retroactively cleaning up years of accumulated data, investigating a public bucket exposure, or debugging performance issues under load is orders of magnitude higher.