/
SmartTiers™ Configuration

SmartTiers™ Configuration

Oct 05, 2021

SoftNAS' SmartTiers™ is a multi-tiered storage environment providing automated data movement based on managed policies, ensuring that your organization can create for itself the best solution possible for managing the data explosion their organizations are currently experiencing.

SoftNAS’ SmartTiers™ provides the ability to seamlessly move blocks of data from high performance, low latency storage to more economical, larger capacity storage based on the value of the data. Data migration occurs automatically based on frequency of access. A given block of data that is not accessed regularly is relegated to the larger, economical storage provided. If a read request is submitted, no matter which tier the block of data resides, it will be satisfied and then that block will “reverse” migrate back up the tier hierarchy. 

SmartTiers™ scans all allocated blocks in the storage pool every N seconds (default 14400, or 4 hours), inspecting the last access time of each block and migrating as needed.

A configurable migration policy allows setting a maximum block age and a reverse block age for each tier. When a block has aged beyond the defined maximum, the migration scan will migrate it from tier X to tier X+1 (as long as tier X+1 is not full) If a migrated block on tier X+1 is later accessed, it will be migrated to tier X only if the reverse migration age has been met. To maximize performance and ensure sufficient space remains available on the hot tier, a special policy directive allows the administrator to configure an alternate block age and hot tier % threshold. A task runs every 5 minutes to check on the hot tier capacity and if it has consumed more than the specified percentage, a pressure-initiated migration task runs immediately to migrate all blocks on every tier in the pool using the "alternate block age". If a migration schedule is configured, neither automatic or pressure-initiated migration will run outside of the window. Migration begins immediately at the start of the scheduled window.

SoftNAS®'s SmartTiers™ provides the ability to position data based on its value to the customer.  Several tiers can be maintained.  Each tier contained within a specific SmartTiers™ hierarchy can have its own unique migration policy while allowing the IT organization to satisfy existing Service Level Agreements.

 

Currently, SoftNAS does not support S3-IA or S3-RRS storage for use as a storage tier in a SmartTiers™ deployment.

If planning to use Azure Cool Blob Storage or AWS Infrequent Access for your SmartTiers™ solution, it is important to understand their policies on early deletion of data. For more information on these policies, see Cool or Infrequently Accessed Storage, linked in the table of contents below.

Note: 

For version 4.1, SmartTiers™ only supports storage tiers residing in the same cloud.  Configuration of tiers that span multiple clouds is not supported. 

Preliminary Steps

To begin creating your SmartTiers™ solution, several preliminary tasks should be completed:

  1. First, create your SoftNAS® instance/virtual machine, according to the instructions for your platform (AWS, Azure, VMware). See Creating Your Instance for more details.

  2. Next, plan for and create the storage you will need for each tier. The storage to be leveraged in the SmartTiers™ configuration must be created before moving to storage pools, where SmartTiers™ will be configured. For information on creating storage, see Adding Storage. 

    1. For frequently accessed storage, we recommend high-performance local SSD storage. 

    2. For infrequently accessed storage, lower cost HDD storage can be leveraged.

  3. You will also need to keep in mind the desired RAID configuration, and create enough disks to support your desired configuration. For more information about RAID configuration, see Software RAID Considerations.

 

If leveraging object storage (such as S3 or Azure Blob) for your infrequently accessed tiers, it is best practice not to use software RAID configuration for those tiers, as you will see limited performance gains, if any at all. If leveraging HDD block devices, RAID can be leveraged to improve performance.

Configuring SmartTiers™

As mentioned above, if SmartTiers™ is to be leveraged, it is configured when creating the storage pool. To begin creating a SmartTiers™ Configuration, select Storage Pools in the Storage Administration pane. 
Click Create, and then select the SmartTiers™  radio button.
Click Next to begin the process.

 

First, we will be configuring the primary tier, also referred to as the Hot tier. This is the tier which will be most frequently accessed, and should provide a high performance index. 

 

SmartTiers™ requires a minimum of 2 tiers to be configured. The second tier is referred to as the "Warm" tier to differentiate it from later "Cold" tiers in more complex configurations. As you can see, the second tier operates the same as the primary.

When provisioning tiers, remember that each tier will result in approximately 10% less useable storage after formatting for tier usage. In other words, you may wish to provision  at least 10% additional storage for each tier you provision (ie: for 2 tiers, an additional 20% storage, for 3 tiers, an additional 30%).

For simplicity's sake, we will stop here at two tiers. The process continues in the same manner. In other words, if mirroring the described configuration, click 'Next' to proceed, rather than Add Another Tier. If you wish to add more tiers, select Add Another Tier. You will now assign a name to the SmartTiers™ configuration. This will be the name used when creating the Volume to be used. 
Select Finish to proceed.  If you need to make changes, you can select Back to return to the previous panel.
The final step in creating your SmartTiers™ configuration is to establish the migration policy that will be used to govern each of the tiers. 

Both Maximum Block Age and Reverse Migration Grace Period are calculated to the second. The default time period for Maximum Block Age is 24 hours which, counted in seconds, equals 86400. The default for Reverse Migration Grace Period is by the same token 12 hours. To determine your settings, use these default values to calculate the time frame for each tier.

 

During the 100TB data grow test performed by QA on a SmartTier pool the following issue was noticed:

  • During a large data transfer to the SmartTier pool a high number of s3backer timeouts were noticed when an aged blocked migration occurred. These timeouts resulted in an impact to the data transfer speed and stability.

Based on this behavior it was determined that the Maximum block age for all tiers should be set to 2592000 (30 days) for the Tiered device, to avoid an automatic migration from occurring while the initial large data transfer is in progress. Once the initial data has been transferred to the SmartTier pool, this value can be change to one more appropriate for the environment.

Managing an Existing SmartTiers™ Migration Policy

Now that SmartTiers™ has been configured, let's look at how to view and change settings for the Migration Policy on an existing SmartTiers™.

To open the Migration Policy, return to Disk Devices from the Storage Administration pane where you will see the disk listed as /dev/sdtiera, with the make and model of SoftNAS SmartTiers™ Storage Device.