Updating Disk Properties in AWS
- Virgil Fritz Taneza
Updating the disk size, type and performance characteristics of Amazon Elastic Block Store (EBS) disks in AWS involves modifying existing EBS volumes. You can resize EBS volumes to increase their capacity, or change the volume type to adjust performance characteristics.
Warning: Before proceeding, ensure that you have appropriate permissions to modify EBS volumes, and make sure to backup critical data to avoid data loss during the process. Please check also the table below for the different supported EBS volume types and its brief description as guidance in deciding which volume type to use depending on your use case, performance and budget requirements.
Storage Type | Description |
|---|---|
Next Generation General Purpose SSD (gp3) | gp3 volumes are the latest generation of general-purpose SSD-based EBS volumes that enable customers to provision performance independent of storage capacity, while providing up to 20% lower pricing per GB than existing gp2 volumes. |
General Purpose SSD (gp2) | gp2 is the default EBS volume type for Amazon EC2 instances. These volumes are backed by solid-state drives (SSDs) and are suitable for a broad range of transactional workloads, including dev/test environments, low-latency interactive applications, and boot volumes. |
Provisioned IOPS SSD (io1) | io1 is backed by solid-state drives (SSDs) and is a high performance EBS storage option designed for critical, I/O intensive database and application workloads, as well as throughput-intensive database and data warehouse workloads, such as HBase, Vertica, and Cassandra. |
Throughput Optimized HDD (st1) | st1 is backed by hard disk drives (HDDs) and is ideal for frequently accessed, throughput-intensive workloads with large datasets and large I/O sizes, such as MapReduce, Kafka, log processing, data warehouse, and ETL workloads. |
Cold HDD (sc1) | sc1 is backed by hard disk drives (HDDs) and provides the lowest cost per GB of all EBS volume types. It is ideal for less frequently accessed workloads with large, cold datasets. |
Magnetic (standard) | Magnetic volumes are previous generation volumes that are backed by magnetic drives. They are suited for workloads with small datasets where data is accessed infrequently and performance is not of primary importance. |
AWS may have released new types of EBS volumes which may not be supported by SoftNAS as of this writing. Rest assured we are constantly monitoring new updates from AWS and test them accordingly but If you want a new type to be supported in SoftNAS, please let us know so we can prioritize it by contacting our Support.
For updated pricing and performance characteristics of each of the EBS volume types, please check AWS documentations directly such as Amazon EBS Volume Types - Amazon Web Services and High-Performance Block Storage– Amazon EBS Pricing – Amazon Web Services
When using SoftNAS, you can add EBS disks and attach them to the instance and create a Storage pool out of them. Storage pools are used to aggregate disk storage into a large pool of storage that can be conveniently allocated and shared by volumes.
Warning: To achieve optimal performance of a Storage Pool, SoftNAS recommends the use of disks of same types, sizes and performance characteristics. In existing Storage pools with varying disk types and sizes, SoftNAS recommends updating all the disk types to same type and performance characteristics and updating the disk sizes to a value greater than or equal to the largest disk size used by the pool. These configurations can be updated by changing the disk size, type and performance characteristics via the Amazon Console.
Here's a step-by-step guide on how to do this:
1. Login to AWS Console
Sign in to your AWS Management Console using your AWS account credentials.
2. Navigate to EC2
Go to the EC2 Dashboard by clicking on "Services" in the upper left corner and selecting "EC2" under "Compute."
3. Identify the EBS Volume
In the EC2 Dashboard, navigate to "Volumes" in the left sidebar to view a list of your EBS volumes. Identify the volume you want to resize.
3.1 Get the EBS Volume ID of the Desired Disk from SoftNAS® StorageCenter
In SoftNAS StorageCenter, navigate to “Disk Devices” in the left side bar to view a list of your disk devices. Identify the corresponding EBS volume ID of the disk you want to resize.
3.2 Search Desired Volume from the EBS Volumes List
In the EBS Volumes list, search for the desired volume using the volume id retrieved from SoftNAS StorageCenter’s Disk Devices list.
4. Select the Volume
Click on the volume's ID to select it.
5. Click Modify
In the top navigation bar, click on the "Modify" button to modify the desired EBS volume
6. Modify Volume Size, Type and Performance Characteristics
In the "Modify Volume" dialog, do the following:
Set to new desired EBS volume type
Adjust the volume size to your desired capacity.
Adjust the IOPS and Throughput performance characteristics.
7. Confirm and Modify
Review the changes and click the "Modify" button to initiate the volume size change.
8. Check Status
After modifying, the volume's status will change to "modifying." It may take a while to complete, depending on the size increase.
9. Update File System
If you are resizing/expanding the EBS volume/disk, you may need to expand the filesystem using appropriate commands for your system.
9.1 Expand the Storage Pool Quota
If you are resizing/expanding the EBS volume/disk which is part of an existing SoftNAS Storage Pool, you need to expand the Storage Pool’s size and set new quota to 80% of the new Storage Pool’s size.
Here are the steps on how to update the Storage Pool’s quota:
9.1.1 Expand the device to use all available space
Expand the device to use all available space. If the device is part of a mirror or raidz then all devices must be expanded before the new space will become available to the Storage pool. Execute the command zpool online -e <pool name> <device name> as below:
Before expanding the device used by the pool, lets examine first the pool’s current size
root@ip-172-31-38-63:~# zpool get -p size nas_pool # get the pool's current size
NAME PROPERTY VALUE SOURCE
nas_pool size 102005473280 - # size is around 95GB
as you can see, the pool’s current size is 95GB which is the usable space of the pool composed of disks of sizes 64GB and 32GB.
Now, we can expand the device used by the pool as below:
root@ip-172-31-38-63:~# zpool online -e nas_pool nvme1n1 # expand the pool's disk
root@ip-172-31-38-63:~# echo $? # check if command is successfull
0
root@ip-172-31-38-63:~# zpool get -p size nas_pool # check pool's new size
NAME PROPERTY VALUE SOURCE
nas_pool size 136365211648 - # size is now around 127GB
root@ip-172-31-38-63:~#Get 80% of the pool’s new size and set it as the pool’s new quota.
# new_quota = 127GB * 0.80 = 101.6GB
root@ip-172-31-38-63:~# zfs set quota=101.6G nas_pool # set pool's new quota
root@ip-172-31-38-63:~# zfs get quota nas_pool # get pool's new quota
NAME PROPERTY VALUE SOURCE
nas_pool quota 102G local # 102GB is the new quota rounding up 101.6GB
root@ip-172-31-38-63:~#10. Monitor and Test
After the modification is complete, monitor your system's performance to ensure it meets your expectations.