AWS EFS & FSx¶

1. Where These Fit — Full AWS Storage Picture¶

EBS   → Block storage  → 1 EC2 only (mostly)     → hard disk attached to one server
S3    → Object storage → HTTP API, any client    → unlimited cloud bucket
EFS   → File storage   → many Linux EC2s at once → NFS shared network drive
FSx   → File storage   → specialist file systems → NFS/SMB/Lustre/ZFS for specific stacks

The single question that determines which to use:

Do multiple servers need to READ AND WRITE the same files simultaneously?
  NO  → EBS (block) or S3 (objects)
  YES → EFS or FSx (shared file system)

  YES + Linux workloads, simple NFS → EFS
  YES + Windows / HPC / enterprise NAS → FSx

Part 1 — Amazon EFS (Elastic File System)¶

---All exact performance specs pulled directly from official AWS documentation. Writing the complete file now.

````markdown

AWS EFS & FSx — Complete Reference¶

1. Where These Fit — Full AWS Storage Picture¶

EBS → Block storage → 1 EC2 only (mostly) → hard disk attached to one server S3 → Object storage → HTTP API, any client → unlimited cloud bucket EFS → File storage → many Linux EC2s at once → NFS shared network drive FSx → File storage → specialist file systems → NFS/SMB/Lustre/ZFS for specific stacks

The single question that determines which to use:

``` Do multiple servers need to READ AND WRITE the same files simultaneously? NO → EBS (block) or S3 (objects) YES → EFS or FSx (shared file system)

YES + Linux workloads, simple NFS → EFS YES + Windows / HPC / enterprise NAS → FSx ```

Part 1 — Amazon EFS (Elastic File System)¶

2. What is EFS?¶

Amazon EFS is a fully managed, serverless, elastic NFS (Network File System) for Linux. It scales capacity automatically — you never provision storage size. Multiple EC2 instances, containers (ECS/EKS), and Lambda functions across multiple AZs can mount and access the same file system simultaneously.

AZ-1 AZ-2 AZ-3 ┌──────────┐ ┌──────────┐ ┌──────────┐ │ EC2 #1 │ │ EC2 #2 │ │ EC2 #3 │ └────┬─────┘ └────┬─────┘ └────┬─────┘ │ │ │ ┌────▼─────────────────▼─────────────────▼─────┐ │ EFS File System │ │ (Mount Targets in each AZ's subnet) │ └───────────────────────────────────────────────┘

3. EFS File System Types¶

Regional (Multi-AZ) — Default¶

Data stored redundantly across 3+ AZs Mount target created in each AZ's subnet If one AZ fails → instances in other AZs continue unaffected Use: production workloads requiring high availability

One Zone (Single-AZ)¶

Data stored in a single AZ Lower cost (~47% cheaper than Regional Standard) Mount target in one subnet only Use: dev/test, non-critical data, data you can recreate Risk: AZ failure = data unavailable (or lost if AZ is permanently destroyed)

4. EFS Storage Classes ⭐¶

EFS automatically moves files between storage classes based on access patterns:

Intelligent Tiering — Lifecycle Management¶

``` Enable lifecycle policy → EFS automatically transitions files:

After 30 days no access → Standard → Standard-IA After 90 days no access → Standard-IA → Archive

First access after transition → file moves back to Standard (configurable)

Similar to S3 Intelligent-Tiering but for file systems. ```

A retrieval fee applies when reading from IA/Archive classes. For files accessed frequently, keep them in Standard to avoid per-read charges.

5. Performance Modes¶

General Purpose (Default — Always Use This)¶

``` Lowest per-operation latency Supports all throughput modes One Zone file systems always use General Purpose

Recommended for: 99%+ of workloads ```

Max I/O (Legacy — Avoid)¶

``` Designed for highly parallelized workloads BUT: higher per-operation latency than General Purpose NOT supported for: One Zone file systems or Elastic throughput

AWS Recommendation: "Due to higher per-operation latencies with Max I/O, we recommend using General Purpose performance mode for all file systems."

Monitor PercentIOLimit CloudWatch metric — if consistently near 100%, switch to Elastic throughput instead of Max I/O mode. ```

6. Throughput Modes ⭐¶

Throughput mode controls how much throughput your file system can drive:

Elastic Throughput (Recommended — Default)¶

``` Automatically scales throughput up and down with your workload No capacity planning needed — you pay per GB read/written

Best for: Spiky or unpredictable workloads Average-to-peak ratio of 5% or less New file systems where patterns are unknown

Performance (Regional + Elastic + General Purpose): Read latency: ~1 ms Write latency: ~2.7 ms Max read IOPS: 900,000–2,500,000 Max write IOPS: 500,000 Max read throughput (per file system): 20–60 GiBps Max write throughput (per file system): 1–5 GiBps Max per-client: 1,500 MiBps (with amazon-efs-utils v2.0+) ```

Provisioned Throughput¶

``` You specify a fixed throughput level regardless of file system size You pay for provisioned amount above baseline

Best for: Known, steady workloads Average-to-peak ratio of 5% or more

Performance (Regional + Provisioned): Max read IOPS: 55,000 Max write IOPS: 25,000 Max read throughput: 3–10 GiBps Max write throughput: 1–3.33 GiBps Max per-client: 500 MiBps

Note: after switching to Provisioned or changing Provisioned amount, must wait 24 hours before switching back to Elastic/Bursting. ```

Bursting Throughput¶

``` Throughput scales proportionally to storage size in Standard class Accumulates burst credits when idle → spends credits when busy

Baseline: 50 KiBps per GiB of Standard storage Burst: 100 MiBps per TiB of Standard storage

Example (100 GiB Standard storage): Baseline: 5 MiBps continuous write Burst: 100 MiBps write for 72 minutes/day (on full credit balance)

Example (1 TiB Standard storage): Baseline: 50 MiBps write Burst: 100 MiBps write for 12 hours/day

Performance (Regional + Bursting): Max read IOPS: 35,000 Max write IOPS: 7,000 Max read throughput: 3–5 GiBps Max write throughput: 1–3 GiBps

Best for: workloads with long quiet periods followed by bursts Avoid: if throughput is consistently high (credits will be exhausted) ```

Throughput Mode Comparison¶

7. Mounting EFS on Linux¶

```bash

Install EFS mount helper (amazon-efs-utils)¶

sudo yum install -y amazon-efs-utils # Amazon Linux / RHEL sudo apt-get install -y amazon-efs-utils # Ubuntu / Debian

Mount using EFS mount helper (recommended — handles TLS + retries)¶

sudo mount -t efs fs-12345678:/ /mnt/efs

Mount with TLS encryption in transit¶

sudo mount -t efs -o tls fs-12345678:/ /mnt/efs

Mount using NFS directly (alternative)¶

sudo mount -t nfs4 \ -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 \ fs-12345678.efs.us-east-1.amazonaws.com:/ /mnt/efs

Auto-mount on boot — add to /etc/fstab:¶

fs-12345678:/ /mnt/efs efs defaults,_netdev 0 0 ```

For EKS (Kubernetes): Use the aws-efs-csi-driver — creates PersistentVolume backed by EFS; multiple pods read/write simultaneously using ReadWriteMany access mode (not possible with EBS).

8. EFS Access Points¶

Access points enforce a specific directory, POSIX user/group, and file permissions for application access:

``` EFS Root: / ├── /app1 ← Access Point A (uid:1001, gid:1001, root path /app1) ├── /app2 ← Access Point B (uid:1002, gid:1002, root path /app2) └── /logs ← Access Point C (uid:1000, gid:1000, root path /logs)

App1 mounts via Access Point A → sees only /app1, cannot access /app2 App2 mounts via Access Point B → sees only /app2

Benefit: multi-tenant isolation on one EFS file system Use case: Lambda functions, containerized apps needing scoped access ```

9. EFS Security¶

10. EFS Use Cases¶

Part 2 — Amazon FSx¶

11. What is FSx?¶

Amazon FSx provides fully managed, third-party file systems — you get the exact file system you're familiar with (Lustre, ONTAP, ZFS, Windows), managed by AWS. Choose FSx when your workload requires a specific file system that EFS (NFS-only, Linux-only) cannot serve.

Four FSx file systems: FSx for Windows File Server → Windows SMB workloads FSx for Lustre → HPC, ML, high-throughput Linux FSx for NetApp ONTAP → Enterprise multi-protocol NAS FSx for OpenZFS → ZFS Linux workloads, low latency

12. FSx for Windows File Server ⭐¶

What It Is¶

Fully managed Windows file system backed by Windows Server with full SMB (Server Message Block) protocol support and Active Directory integration.

Protocol: SMB 2.0, 2.1, 3.0, 3.1.1 Clients: Windows, Linux (via CIFS), macOS Auth: Microsoft Active Directory (AWS Managed AD or self-managed)

Key Features¶

When to Use¶

• Lift-and-shift Windows applications to AWS
• .NET apps needing Windows file shares
• SQL Server home directory, user profiles
• Any workload requiring Windows ACLs or Active Directory

13. FSx for Lustre ⭐¶

What It Is¶

Fully managed Lustre — the world's most popular high-performance parallel file system, used in the largest supercomputers and ML clusters. Linux-only, extremely high throughput.

Protocol: Custom POSIX-compliant (Lustre protocol) Clients: Linux only Auth: POSIX permissions

Performance¶

FSx for Lustre throughput (1,000 GB/s) is the highest of any FSx file system — 10–70× higher than the others. Built specifically for data-intensive workloads.

Deployment Types¶

``` Scratch (Temporary): No replication within AZ Data NOT preserved if file server fails Higher burst throughput Use: short-term processing, cost-sensitive HPC

Persistent (Long-term): Data replicated within single AZ File server failures are auto-recovered Use: long-running workloads, ML training runs ```

S3 Integration ⭐¶

``` FSx for Lustre can be linked to an S3 bucket: Import: data in S3 automatically imported to Lustre on first access (lazy loading) Export: results written back to S3 automatically

Pattern for ML training: Training data in S3 (cheap, durable) → Link to FSx for Lustre (high-speed scratch during training) → Model output exported back to S3 → Delete FSx after training (pay only during training job) ```

When to Use¶

• Machine learning training on large datasets
• High-performance computing (genomics, financial simulations, weather modeling)
• Video rendering and transcoding
• Seismic data processing

14. FSx for NetApp ONTAP ⭐¶

What It Is¶

Fully managed NetApp ONTAP — the most feature-rich FSx option, supporting multiple protocols simultaneously.

Protocols: NFS (3, 4.0, 4.1, 4.2) + SMB (2.0–3.1.1) + iSCSI (block storage) Clients: Windows, Linux, macOS — simultaneously

Performance¶

Unique Capabilities¶

When to Use¶

• Lift-and-shift existing NetApp ONTAP NAS to AWS
• Multi-protocol workloads (Windows + Linux accessing same files)
• Enterprise NAS migration
• Complex data management (cloning, replication, tiering)
• Any workload where you're already using NetApp on-premises

15. FSx for OpenZFS ⭐¶

What It Is¶

Fully managed OpenZFS — a Linux-native file system known for data integrity, inline compression, and the lowest latency of any FSx option.

Protocol: NFS (3, 4.0, 4.1, 4.2) Clients: Windows, Linux, macOS

Performance¶

Key Features¶

When to Use¶

• Lift-and-shift ZFS workloads to AWS
• Linux-based file servers needing low latency
• Development/test environments needing instant cloning
• Any workload needing sub-millisecond NFS latency

16. FSx — Full Comparison Table¶

17. EFS vs FSx — When to Use Which ⭐¶

18. EFS vs EBS vs S3 — Complete Storage Comparison¶

19. Common Mistakes ✅¶

2. What is EFS?¶

Amazon EFS is a fully managed, serverless, elastic NFS (Network File System) for Linux. It scales capacity automatically — you never provision storage size. Multiple EC2 instances, containers (ECS/EKS), and Lambda functions across multiple AZs can mount and access the same file system simultaneously.

AZ-1 AZ-2 AZ-3 ┌──────────┐ ┌──────────┐ ┌──────────┐ │ EC2 #1 │ │ EC2 #2 │ │ EC2 #3 │ └────┬─────┘ └────┬─────┘ └────┬─────┘ │ │ │ ┌────▼─────────────────▼─────────────────▼─────┐ │ EFS File System │ │ (Mount Targets in each AZ's subnet) │ └───────────────────────────────────────────────┘

3. EFS File System Types¶

Regional (Multi-AZ) — Default¶

Data stored redundantly across 3+ AZs Mount target created in each AZ's subnet If one AZ fails → instances in other AZs continue unaffected Use: production workloads requiring high availability

One Zone (Single-AZ)¶

Data stored in a single AZ Lower cost (~47% cheaper than Regional Standard) Mount target in one subnet only Use: dev/test, non-critical data, data you can recreate Risk: AZ failure = data unavailable (or lost if AZ is permanently destroyed)

4. EFS Storage Classes ⭐¶

EFS automatically moves files between storage classes based on access patterns:

Intelligent Tiering — Lifecycle Management¶

``` Enable lifecycle policy → EFS automatically transitions files:

After 30 days no access → Standard → Standard-IA After 90 days no access → Standard-IA → Archive

First access after transition → file moves back to Standard (configurable)

Similar to S3 Intelligent-Tiering but for file systems. ```

A retrieval fee applies when reading from IA/Archive classes. For files accessed frequently, keep them in Standard to avoid per-read charges.

5. Performance Modes¶

General Purpose (Default — Always Use This)¶

``` Lowest per-operation latency Supports all throughput modes One Zone file systems always use General Purpose

Recommended for: 99%+ of workloads ```

Max I/O (Legacy — Avoid)¶

``` Designed for highly parallelized workloads BUT: higher per-operation latency than General Purpose NOT supported for: One Zone file systems or Elastic throughput

AWS Recommendation: "Due to higher per-operation latencies with Max I/O, we recommend using General Purpose performance mode for all file systems."

Monitor PercentIOLimit CloudWatch metric — if consistently near 100%, switch to Elastic throughput instead of Max I/O mode. ```

6. Throughput Modes ⭐¶

Throughput mode controls how much throughput your file system can drive:

Elastic Throughput (Recommended — Default)¶

``` Automatically scales throughput up and down with your workload No capacity planning needed — you pay per GB read/written

Best for: Spiky or unpredictable workloads Average-to-peak ratio of 5% or less New file systems where patterns are unknown

Performance (Regional + Elastic + General Purpose): Read latency: ~1 ms Write latency: ~2.7 ms Max read IOPS: 900,000–2,500,000 Max write IOPS: 500,000 Max read throughput (per file system): 20–60 GiBps Max write throughput (per file system): 1–5 GiBps Max per-client: 1,500 MiBps (with amazon-efs-utils v2.0+) ```

Provisioned Throughput¶

``` You specify a fixed throughput level regardless of file system size You pay for provisioned amount above baseline

Best for: Known, steady workloads Average-to-peak ratio of 5% or more

Performance (Regional + Provisioned): Max read IOPS: 55,000 Max write IOPS: 25,000 Max read throughput: 3–10 GiBps Max write throughput: 1–3.33 GiBps Max per-client: 500 MiBps

Note: after switching to Provisioned or changing Provisioned amount, must wait 24 hours before switching back to Elastic/Bursting. ```

Bursting Throughput¶

``` Throughput scales proportionally to storage size in Standard class Accumulates burst credits when idle → spends credits when busy

Baseline: 50 KiBps per GiB of Standard storage Burst: 100 MiBps per TiB of Standard storage

Example (100 GiB Standard storage): Baseline: 5 MiBps continuous write Burst: 100 MiBps write for 72 minutes/day (on full credit balance)

Example (1 TiB Standard storage): Baseline: 50 MiBps write Burst: 100 MiBps write for 12 hours/day

Performance (Regional + Bursting): Max read IOPS: 35,000 Max write IOPS: 7,000 Max read throughput: 3–5 GiBps Max write throughput: 1–3 GiBps

Best for: workloads with long quiet periods followed by bursts Avoid: if throughput is consistently high (credits will be exhausted) ```

Throughput Mode Comparison¶

7. Mounting EFS on Linux¶

```bash

Install EFS mount helper (amazon-efs-utils)¶

sudo yum install -y amazon-efs-utils # Amazon Linux / RHEL sudo apt-get install -y amazon-efs-utils # Ubuntu / Debian

Mount using EFS mount helper (recommended — handles TLS + retries)¶

sudo mount -t efs fs-12345678:/ /mnt/efs

Mount with TLS encryption in transit¶

sudo mount -t efs -o tls fs-12345678:/ /mnt/efs

Mount using NFS directly (alternative)¶

sudo mount -t nfs4 \ -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 \ fs-12345678.efs.us-east-1.amazonaws.com:/ /mnt/efs

Auto-mount on boot — add to /etc/fstab:¶

fs-12345678:/ /mnt/efs efs defaults,_netdev 0 0 ```

For EKS (Kubernetes): Use the aws-efs-csi-driver — creates PersistentVolume backed by EFS; multiple pods read/write simultaneously using ReadWriteMany access mode (not possible with EBS).

8. EFS Access Points¶

Access points enforce a specific directory, POSIX user/group, and file permissions for application access:

``` EFS Root: / ├── /app1 ← Access Point A (uid:1001, gid:1001, root path /app1) ├── /app2 ← Access Point B (uid:1002, gid:1002, root path /app2) └── /logs ← Access Point C (uid:1000, gid:1000, root path /logs)

App1 mounts via Access Point A → sees only /app1, cannot access /app2 App2 mounts via Access Point B → sees only /app2

Benefit: multi-tenant isolation on one EFS file system Use case: Lambda functions, containerized apps needing scoped access ```

9. EFS Security¶

10. EFS Use Cases¶

Part 2 — Amazon FSx¶

11. What is FSx?¶

Amazon FSx provides fully managed, third-party file systems — you get the exact file system you're familiar with (Lustre, ONTAP, ZFS, Windows), managed by AWS. Choose FSx when your workload requires a specific file system that EFS (NFS-only, Linux-only) cannot serve.

Four FSx file systems: FSx for Windows File Server → Windows SMB workloads FSx for Lustre → HPC, ML, high-throughput Linux FSx for NetApp ONTAP → Enterprise multi-protocol NAS FSx for OpenZFS → ZFS Linux workloads, low latency

12. FSx for Windows File Server ⭐¶

What It Is¶

Fully managed Windows file system backed by Windows Server with full SMB (Server Message Block) protocol support and Active Directory integration.

Protocol: SMB 2.0, 2.1, 3.0, 3.1.1 Clients: Windows, Linux (via CIFS), macOS Auth: Microsoft Active Directory (AWS Managed AD or self-managed)

Key Features¶

When to Use¶

• Lift-and-shift Windows applications to AWS
• .NET apps needing Windows file shares
• SQL Server home directory, user profiles
• Any workload requiring Windows ACLs or Active Directory

13. FSx for Lustre ⭐¶

What It Is¶

Fully managed Lustre — the world's most popular high-performance parallel file system, used in the largest supercomputers and ML clusters. Linux-only, extremely high throughput.

Protocol: Custom POSIX-compliant (Lustre protocol) Clients: Linux only Auth: POSIX permissions

Performance¶

FSx for Lustre throughput (1,000 GB/s) is the highest of any FSx file system — 10–70× higher than the others. Built specifically for data-intensive workloads.

Deployment Types¶

``` Scratch (Temporary): No replication within AZ Data NOT preserved if file server fails Higher burst throughput Use: short-term processing, cost-sensitive HPC

Persistent (Long-term): Data replicated within single AZ File server failures are auto-recovered Use: long-running workloads, ML training runs ```

S3 Integration ⭐¶

``` FSx for Lustre can be linked to an S3 bucket: Import: data in S3 automatically imported to Lustre on first access (lazy loading) Export: results written back to S3 automatically

Pattern for ML training: Training data in S3 (cheap, durable) → Link to FSx for Lustre (high-speed scratch during training) → Model output exported back to S3 → Delete FSx after training (pay only during training job) ```

When to Use¶

• Machine learning training on large datasets
• High-performance computing (genomics, financial simulations, weather modeling)
• Video rendering and transcoding
• Seismic data processing

14. FSx for NetApp ONTAP ⭐¶

What It Is¶

Fully managed NetApp ONTAP — the most feature-rich FSx option, supporting multiple protocols simultaneously.

Protocols: NFS (3, 4.0, 4.1, 4.2) + SMB (2.0–3.1.1) + iSCSI (block storage) Clients: Windows, Linux, macOS — simultaneously

Performance¶

Unique Capabilities¶

When to Use¶

• Lift-and-shift existing NetApp ONTAP NAS to AWS
• Multi-protocol workloads (Windows + Linux accessing same files)
• Enterprise NAS migration
• Complex data management (cloning, replication, tiering)
• Any workload where you're already using NetApp on-premises

15. FSx for OpenZFS ⭐¶

What It Is¶

Fully managed OpenZFS — a Linux-native file system known for data integrity, inline compression, and the lowest latency of any FSx option.

Protocol: NFS (3, 4.0, 4.1, 4.2) Clients: Windows, Linux, macOS

Performance¶

Key Features¶

When to Use¶

• Lift-and-shift ZFS workloads to AWS
• Linux-based file servers needing low latency
• Development/test environments needing instant cloning
• Any workload needing sub-millisecond NFS latency

16. FSx — Full Comparison Table¶

17. EFS vs FSx — When to Use Which ⭐¶

18. EFS vs EBS vs S3 — Complete Storage Comparison¶

19. Common Mistakes¶