Elastic File System (EFS) : Shared Storage Space

AWS currently offers EBS which is block storage as a service, which has been very widely used over years. EBS, though highly durable, does not support automatic scaling and sharing across multiple EC2 instance. Amazon has recently released AWS EFS, which is a scalable (in both size and throughput) and can be used as shared block storage. It is a fully managed shared file systems which is an implementation of NFSv4.

Prior to EFS the need for common storage services was mostly handled by combining different EBS volumes with services such as Glusterfs and then using the gluster share to access the common data required by many machines for different purposes. Glusterfs is in the market from a long time and is a tested and proven solution with many benchmark tests carried out.

Here

I will be analyzing how EFS can help us overcome the burden of configuring and administrating the Glusterfs and use EFS service .
Perform the File system Benchmark Test on the EFS and compare this with other storage services.
I will also be looking into pricing of different storage services
Use case for EFS.

Amazon Elastic File System (Amazon EFS)

Is a file storage service for Amazon Elastic Compute Cloud (Amazon EC2) instances. Amazon EFS has the following benefits .

Highly Available and Durable:

Amazon EFS is designed to be durable and highly available. Each Amazon EFS file system object (i.e. directory, file, and link) is redundantly stored across multiple Availability Zones

Scale Up and Down Seamlessly:

Amazon EFS automatically scales your file system storage capacity up or down as you add or remove files without disrupting your applications

Share File storage Across Instances:

Multiple Amazon EC2 instances can access an Amazon EFS file system, so applications that scale beyond a single instance can access a file system.Amazon EC2 instances running in multiple Availability Zones (AZs) within the same region can access the file system, so that many users can access and share a common data source.

Consistence, Scalable Performance :

Amazon EFS is SSD-based, and is designed to provide the throughput, IOPS, and low latency needed for a broad range of workloads. With Amazon EFS, throughput and I OPS scale as a file system grows.

File System Benchmark Test

NOTE: I will not be using provisioned IOPS enabled EBS volumes in these tests as the Amazon does not mention the IOPS that the EFS has to offer. I believe it would not be a fair comparison if provisioned IOPS volumes are used, hence for the test purpose I will be using Magnetic and General purpose SSD which do not have provisioned IOPS

The simplest read and write performance test can be done with the help of dd command. This command is used to write or read from any block device in Linux.

dd if=/dev/zero of=speedtest bs=32k count=33555 conv=fdatasync

Where,

bs= sets the blocksize, for example bs=1M would be 1MiB blocksize

count = copy only N input blocks

I am using bs=32k and 33555 input blocks so that when both are multiplied I get the data of 1GB size.

Sequential Write Speed

Instance Store’s Sequential write speed is more because instance store is the volume that is attached to the host server physically. Hence it is obvious that the read speed of the instance store will be more when compared to the other storage volumes as they are accessed via the network . Write speed of the EFS is more compare to EBS and Magnetic Storage and hence we can conclude that the EFS will be better performing when compared to the other storage services.

Command for testing sequential write speed

dd if=/dev/zero of=speedtest bs=32k count=33555 conv=fdatasync

Where,

bs = sets the blocksize, for example bs=1M would be 1MiB blocksize

count = copy only N input blocks

I am using bs=32k and 33555 input blocks so that when both are multiplied I get the data of 1GB size.

twoInstance Store’s Sequential write speed is more because instance store is the volume that is attached to the host server physically . Hence it is obvious that the read speed of the instance store will be more when compared to the other storage volumes as they are accessed via the nerk . Write speed of the EFS is more compare to EBS and Magnetic Storage and hence we can conclude that the EFS will be better performing when compared to the other storage services.

Sequential Read Speed

Command for testing sequential Read speed

dd if=speedtest of=/dev/null bs=32k count=33555 conv=fdatasync

Where,

bs= sets the blocksize, for example bs=1M would be 1MiB blocksize

count = copy only N input blocks

I am using bs=32k and 33555 input blocks so that when both are multiplied I get the data of 1GB size

Pricing comparison between EFS, EBS and Instance Store

Pricing of the Elastic file system (EFS) is based on the amount of file system storage you use in GB.

It is chareged $0.30/GB-month

Comparison with other storage system is as follows

Pricing of the Elastic file system (EFS) is based on the amount of file system storage you use in GB. It is chareged $0.30/GB-month

Performance Factor for EFS,EBS and Instance Store

Use Case of Elastic File system (EFS)

EFS can be used for many purposes to handle huge amount of data that is verying by nature. Some of the use cases are mentioned below

Employee Home Directories

Amazon EFS can offer storage for organisation that have many users that need to access and share the common data. In Amazon EFS we can create a file system accessible to the entire organiztion and set the permissions for users at directory or file level.

Big Data

Amazon EFS provides the scale and performance required for big data applications that require high throughput and low latency.

Development Environments

Amazon EFS can be used as storage for development and test environment that can scale up and down as the data grows and shrinks and can store common set of source files, binaries and other resource.

Content Repositories

Amazon EFS can be used as durable ,high throughput file system for content management system that can store and serve the data to variety of applications.