Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 32 Next »

SoftNAS Cloud®’s FlexTiers™ is a multi-tiered storage environment providing automated data movement based on managed policies, ensuring that your organization can create for itself the best solution possible for managing the data explosion their organizations are currently experiencing.

SoftNAS’ FlexTiers™ provides the ability to seamlessly move blocks of data from high performance, low latency storage to more economical, larger capacity storage based on the value of the data. Data migration occurs automatically based on frequency of access. A given block of data that is not accessed regularly is relegated to the larger, economical storage provided. If a read request is submitted, no matter which tier the block of data resides, it will be satisfied and then that block will “reverse” migrate back up the tier hierarchy. 

FlexTiers™ scans all allocated blocks in the storage pool every N seconds (default 14400, or 4 hours), inspecting the last access time of each block and migrating as needed.

A configurable migration policy allows setting a maximum block age and a reverse block age for each tier. When a block has aged beyond the defined maximum, the migration scan will migrate it from tier X to tier X+1 (as long as tier X+1 is not full) If a migrated block on tier X+1 is later accessed, it will be migrated to tier X only if the reverse migration age has been met. To maximize performance and ensure sufficient space remains available on the hot tier, a special policy directive allows the administrator to configure an alternate block age and hot tier % threshold. A task runs every 5 minutes to check on the hot tier capacity and if it has consumed more than the specified percentage, a pressure-initiated migration task runs immediately to migrate all blocks on every tier in the pool using the "alternate block age". If a migration schedule is configured, neither automatic or pressure-initiated migration will run outside of the window. Migration begins immediately at the start of the scheduled window.



SoftNAS Cloud®'s FlexTiers™ provides the ability to position data based on its value to the customer.  Several tiers can be maintained.  Each tier contained within a specific FlexTiers™ hierarchy can have its own unique migration policy while allowing the IT organization to satisfy existing Service Level Agreements.

Note: For version 3.7, FlexTiers™ does not support iSCSI deployments.

Note: If planning to use Azure Cool Blob Storage or AWS Infrequent Access for your FlexTiers™ solution, it is important to understand their policies on early deletion of data. For more information on these policies, see Cool or Infrequently Accessed Storage, linked in the table of contents below.

Preliminary Steps

To begin creating your FlexTiers™ solution, several preliminary tasks should be completed:

  1. First, create your SoftNAS Cloud® instance/virtual machine, according to the instructions for your platform (AWS, Azure, VMware). See Creating Your Instance for more details.

  2. Next, plan for and create the storage you will need for each tier. The storage to be leveraged in the FlexTiers™ configuration must be created before moving to storage pools, where FlexTiers™ will be configured. For information on creating storage, see Adding Storage. 
    1. For frequently accessed storage, we recommend high-performance local SSD storage. 
    2. For infrequently accessed storage, lower cost HDD storage or S3/blob storage can be leveraged.

  3. You will also need to keep in mind the desired RAID configuration, and create enough disks to support your desired configuration. For more information about RAID configuration, see Software RAID Considerations.

Note: If leveraging object storage (such as S3 or Azure Blob) for your infrequently accessed tiers, it is best practice not to use software RAID configuration for those tiers, as you will see limited performance gains, if any at all. If leveraging HDD block devices, RAID can be leveraged to improve performance.

Configuring FlexTiers™

  1. As mentioned above, if FlexTiers™ is to be leveraged, it is configured when creating the storage pool. To begin creating a FlexTiers™ Configuration, select Storage Pools in the Storage Administration pane. 




  2. Click Create, and then select the FlexTiers™  radio button. Click Next to begin the process.



  3. First, we will be configuring the primary tier, also referred to as the Hot tier. This is the tier which will be most frequently accessed, and should provide a high performance index. 
    1. Assign a name to your Hot tier. 
    2. Select the RAID configuration that you planned for. In the below example software RAID is not leveraged (No RAID,JBOD). If leveraging RAID, additional disks would be selected, based on the RAID selection. 
    3. Select the desired Disks for your configuration. To be leveraged in Software RAID, disks must be of the same type and size. Ensure that the disks selected are those with the best performance index.
    4. Click Next. 



  4. FlexTiers™ requires a minimum of 2 tiers to be configured. The second tier is referred to as the "Warm" tier to differentiate it from later "Cold" tiers in more complex configurations. As you can see, the second tier operates the same as the primary.
    1. Provide the second tier a name.
    2. Determine the RAID level.
    3. Select disks of the storage type designated for your secondary tier (for example S3 or Blob object storage).
    4. If you plan to only have 2 tiers, select Next to proceed to the next step of the FlexTiers™ creation. To add additional tiers, select Add another tier.



  5. For the purposes of documentation, we will stop at two tiers, as the process continues in the same manner. In other words, if mirroring this configuration, click 'Next' to proceed, rather than Add Another Tier. You will now assign a name to the FlexTiers™ configuration. This will be the name used when creating the Volume to be used.  Select Finish to proceed.  If you need to make changes, you can select Back to return to the previous panel.




  6. The final step in creating your FlexTiers™ configuration is to establish the migration policy that will be used to govern each of the tiers. 
    1. Maximum Block Age refers to the amount of time a block of data has not been accessed, and is therefore eligible to be migrated to the next tier.
    2. Reverse Migration grace period refers to when blocks of data that have been accessed by a read inquiry will be moved back up the hierarchy.

    Note: Maximum Block Age does not apply to the last tier in the hierarchy. Nor does the Reverse Migration Grace Period apply to the Hot Tier.




    Both Maximum Block Age and Reverse Migration Grace Period are calculated to the second. The default time period for Maximum Block Age is 24 hours which, counted in seconds, equals 86400. The default for Reverse Migration Grace Period is by the same token 12 hours. To determine your settings, use these default values to calculate the time frame for each tier.

Managing an Existing FlexTiers™ Migration Policy

Now that FlexTiers™ has been configured, let's look at how to view and change settings for the Migration Policy on an existing FlexTiers™.

  1. To open the Migration Policy, return to Disk Devices from the Storage Administration pane where you will see the disk listed as /dev/sdtiera, with the make and model of SoftNAS FlexTiers™ Storage Device.




  2. When this disk is selected from the Disk Device list,  the Configure option will become available. Select Configure to open the Tiered Storage Configuration Wizard.




  3. In the Tiered Storage Configuration Wizard you can alter the migration policy as well as set specific parameters for the Hot tier.



    1. The Migration interval in Device options controls the frequency that FlexTiers™ will check for blocks needing to be migrated, based on the specific tier policies.

    2. Hot Tier Storage Threshold allows you to set a high-water mark on the tier which will force a migration to be done to move blocks to the secondary tier. In other words, if the data on the Hot Tier reaches the specified percentage, the blocks accessed the least frequently will be moved to ensure the Hot tier does not fill its available space.

    3. The Alternate Block Age is used when the “Hot Tier Threshold” has been met.  This will override the Maximum Block Age value specific to determine which blocks will be moved.

    4. You can also set a specific daily schedule for migrations to be allowed.  This is in a 24hour format.

  4. Select Save once all the changes have been applied.


Increase Capacity of FlexTiers™ 

Increasing the capacity of your FlexTiers™ configuration is simple:

    1. First, select the FlexTiers™, and expand it to display the individual tiers contained within it.
    2. Choose the tier you wish to expand. If expanding more than one tier, you will have to repeat the process for each tier.
    3. With the tier selected, click Expand from the menu bar above.

  2. When the wizard opens you can select the appropriate disk devices to be used to increase that tier’s capacity. Note that the disks in question must already be added in Disk Devices to be made available for selection.

Cool or Infrequently Accessed Storage

When using Azure Cool Blob of AWS S3 Infrequent Access storage, it is important to understand the policies by which each are managed. For each platform, there is a delete cost associated with any data that resides on this storage and is deleted within 30 days. This means that if customers are planning to leverage Azure Cool Blob or AWS Infrequent Access Storage, they must understand that it should be used as more of an Archive level storage, where data will be resident for over 30 days, and possibly not for general storage that is commonly read, written and deleted. This is particularly important for FlexTiers™ users, as FlexTiers™ routinely deletes data from a storage tier when it rehydrates the data to a higher tier.

  • https://azure.microsoft.com/en-us/pricing/details/storage/blobs/
    Cool and Archive early deletion (effective February 1, 2018)
    In addition to the per-GB, per-month charge, any blob that is moved to Archive is subject to an Archive early deletion period of 180 days. Additionally, for General Purpose v2 accounts, any blob that is moved to Cool is subject to a Cool early deletion period of 30 days. This charge is prorated. For example, if a blob is moved to Archive and then deleted or moved to the Hot tier after 45 days, the customer is charged an early deletion fee equivalent to 135 (180 minus 45) days of storing that blob in Archive.

  • https://aws.amazon.com/s3/pricing/
    Standard - Infrequent Access Storage has a minimum billable object size of 128KB. Smaller objects may be stored but will be charged for 128KB of storage.  Standard – Infrequent Access Storage is charged for a minimum storage duration of 30 days. Objects that are deleted, overwritten, or transitioned to a different storage class before 30 days will incur the normal usage charge plus a pro-rated request charge for the remainder of the 30 day minimum. Objects stored 30 days or longer will not incur a 30-day minimum request charge.

 

  • No labels