Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

You have a pool created and ready, but you have a need to either migrate this pool to another SoftNAS instance, for example to break up the load, or you have a need to shift data (at the pool level) from one pool to another, such as when changing the underlying EBS volume to improve throughput or even migrate from EBS to S3 or vice versa.

Purpose: 

This document will highlight the different methods available to migrate a pool data and offer the pros and cons of each method at a high level. Then we will show you how these tasks are performed.

Migrating a Pool within the same instance

The following are the two primary methods for migrating a pool within the same instance:

...

NOTE: For large amount of data ZFS Send and Receive might not be efficient and will cause a huge performance degradation, so you may consider migrating to another instance then importing the resulted pool to the current instance 

Migrating a Pool to another SoftNAS instance

The following methods can be used if migrating a pool to another SoftNAS ® Instance: 

...

  • EBS Volume Snapshots
  • Using SnapReplicate™


EBS Volume Snapshots Method (migrating pools on the same instance or different instance).

      Note: If you are in a RAID array setup it is highly recommended to use our Backup & Restore applet from UI to backup your pool(s) which will make it easier for future pool restoration

...

  1. From the UI go to Storage Pools>Import> and choose the pools and import the volumes.


  2. If it is a different instance, Perform final housekeeping tasks, such as modifying or moving the required sharing config files, ( “/etc/exports /etc/target/saveconfig.json /etc/samba/smb.conf”) starting the needed services, and directing your clients or applications to use the IP address of the new share.


ZFS send & receive (Migrating pools on the same instance)

  1. From the SoftNAS UI create the desired pool with the desired backend volumes. 

    https://www.softnas.com/docs/softnas/v3/html/create_a_storage_pool.html
  2. SSH into your instance. 
    1. For guidance on how to connect to your Linux instance, click here.
  3. Create the first pool snapshot using the below command. This step will not require downtime. 

    zfs snapshot –r source-pool@Full


  4. Send this snapshot to the destination pool, using the below command.  This step will cause some performance degradation while transferring the data, so it is recommended to be performed outside of peak business hours.

    zfs send -R source-pool@Full | zfs receive -s -F destination-pool

  5. Turn off write activity on the source volume, by bringing any applications using NFS, CIFS, and/or iSCSI offline, or from SoftNAS itself by using the following commands:

    service monit stop
    /var/www/softnas/scripts/stop-nasservices.sh

  6. Take the final snapshot.

    zfs snapshot –r source-pool@Final

  7. Incrementally send the volume using the final snapshot. This will send only the data modified between the time of the full snapshot and the final snapshot, incurring very little overhead.

    zfs send -R -i source-pool@Full source-pool@Final | zfs receive destination-pool

  8. Now you can export source-pool and remove its disks,then rename the destination-pool as the source pools as below:

    zpool export source-pool

    zpool import source-pool destination-pool

  9. Now you can bring back your clients/applications up and remount your volumes, OR start SoftNAS storage services again, if they were stopped in step 5.

    service monit start
    /var/www/softnas/scripts/start-nasservices.sh

...

zfs send  -t  $(zfs get -H -o value receive_resume_token destination-pool)  | zfs receive  -s  destination-pool

ZFS export (and import from a new SoftNAS® Instance)

  1. SSH into your instance. 
    1. For guidance on how to connect to your Linux instance, click here..
  2. From CLI run the following command:

    zpool export POOL-NAME 

  3. From the AWS console,  detach the EBS volumes that the pool is based on.

  4. From AWS console, attach the EBS volumes to the new instance using their exact mappings i.e (/dev/sdx). As mapping the wrong volume name to the wrong /dev/sdx may lead to data corruption during pool import 

  5. Log into the new SoftNAS instance using your browser. In the new instance's UI,  go to Storage Pools, and click  Import, then choose the pool and import it.

  6. Perform final housekeeping tasks, such as modifying or moving the required sharing config files, ( “/etc/exports /etc/target/saveconfig.json /etc/samba/smb.conf”) starting the needed services, and directing your clients or applications to use the IP address of the new share.


SnapReplicate™ (Migrating to a new SoftNAS® Instance)

SnapReplicate™ is the easiest way to migrate your pool to a new instance, whether in the same availability zone, or a different availability zone, or in the same one. With SnapReplicate, you don't have to worry about cleaning up config files or services. 

...