Configuring VM Settings
Required Settings
After installing the OVF to create the SoftNAS Virtual Storage Appliance VM, configure the VM settings in accordance with best practices and network needs. The boot disk (Hard Disk 1) should be set to 30 GB, thin-provisioned. This is the default configuration, and should not be changed without reason.
...
RAM Note: The operating system and SoftNAS consume up to 1 GB of RAM, using most of the remaining RAM for cache memory and metadata. The more RAM assigned to the VM, the better read cache performance will be, as SoftNAS will keep as much data in RAM cache as possible. Consider this resource allocation for deduplication: at least 1 GB of RAM per terabyte of deduplicated storage, to keep the deduplication tables in memory (or supplement the RAM cache with a read cache device).
Optional Settings
Paravirtual SCSI Disk Controller Support
For maximum throughput and IOPS on VMware, choose the Paravirtual SCSI Controller for the SoftNAS VM (instead of using the default LSI Logic Parallel SCSI controller).
VM Snapshot Mode
Before applying software updates to SoftNAS after it is in production, and to support online backups in popular backup programs, VM snapshots are useful as part of the backup and recovery process. Depending on the plan to manage backups of VM data, choose which mode snapshots will operate in.
- Independent Mode - to enable smaller VM snapshots, configure the boot disk, Hard Disk 1, in the "Independent" mode. This causes VM snapshots to apply only to this first hard disk by default (and not include all added data disks, which could be prohibitively large). The advantage of using Independent mode is VM snapshots will be faster and smaller.
- Dependent Mode - by default, VM snapshots include all hard disks attached to the VM. When used with SoftNAS and a VM backup process, this setting causes all SoftNAS VM disks to be backed up together as a set. This results in much larger backup sets, but may be preferable as a means of achieving additional protection and recoverability in the event of a disaster or need to restore the entire storage system to a different computer or location. If there are only a few terabytes to back up, this may be the prudent choice.
Network Adapter
On a typical 1 gigabit network, the default E1000 network adapter is sufficient; however, for a 10 gigabit or higher-performance network card, the VMXNET 3 network adapter should be used for best results and higher throughput. Note that installation of the VMXNET 3 requires installation of the proper VMware Tools in the guest operating system (in this case, CentOS 64-bit Linux).
Memory / CPU Hot Plug
It is recommended to allow CPU Hot Plug and disable Memory Hot Add, which will make it more convenient to add CPU later to use a lot of data compression or other features that consume additional CPU. Linux seems to do fine when additional CPU are added at run-time.
Note: Add Add memory with the system powered down and disable hot add of memory at run time.
Performance Tuning for VMware vSphere
Achieving peak storage performance in the VMware environment involves tuning the VMware configuration beyond default values. The following are recommended best practices for tuning VMware for use with SoftNAS.
VMDirectPath
VMDirectPath provides a means of passing a disk controller device directly through to the guest operating system (i.e., CentOS Linux).
...
Note that Intel VT-d (or equivalent) processor feature is required for support of VMDirectPath.
VM SCSI Controller - Set to Paravirtual
In VMware, change the SCSI controller type to "Paravirtual", which provides more efficient access to storage.
Physical NIC Settings
A host physical NIC can have settings, which can provide better utilization and performance improvement:
...
Also, check the host for SR-IOV support, which provides additional performance and throughput in virtualized systems like VMware.
Adjust Network Heap Size for high network traffic
By default, the ESX server network stack allocates 64MB of buffers to handle network data. Increase buffer allocation from 64MB to 128MB memory to handle more network data.
To change Heap Size ESX Host:
Go to the Configuration tab for the ESX Server host, and select Advanced Settings-VMkernel - Boot- VMkernel.Boot.netPktHeapMaxSize
Virtual NIC Settings
VM’s virtual adapter has many tuning options, which can also provide much better throughput:
...
An alternative way to disable virtual interrupt coalescing for all virtual NICs on the host which affects all VMs, not just the latency-sensitive ones, is by setting the advanced networking performance option (Configuration -Advanced Settings - Net) CoalesceDefaultOn to 0 (disabled).
Disable LRO
Reload the vmxnet3 driver in the SoftNAS CentOS guest operating system. Log into the SoftNAS Cloud® VM SoftNAS VM using SSH (or the Desktop Console) and su root:
...
Then reload the driver using:
# modprobe vmxnet3
...
Physical Host BIOS Settings
On most servers, these BIOS Settings can improve the overall performance of the host:
- Turn on Hyper-threading in BIOS
- Confirm that the BIOS is set to enable all populated sockets for all cores
- Enable “Turbo Mode” for processors that support it
- Confirm that hardware-assisted virtualization features are enabled in the BIOS
- Disable any other power-saving mode in the BIOS
- Disable any unneeded devices from the BIOS, such as serial and USB ports
- In order to allow ESXi to control CPU power-saving features, set power management in the BIOS to “OS Controlled Mode” or equivalent. Even without planning to use these power-saving features, ESXi provides a convenient way to manage them.
- C-states deeper than C1/C1E (i.e., C3, C6) allow further power savings, though with an increased chance of performance impacts. We recommend, however, enabling all C-states in BIOS, then use ESXi host power management to control their use.
...
NUMA Settings
NUMA systems are advanced server platforms with more than one system bus. They can harness large numbers of processors in a single system image with superior price to performance ratios. The high latency of accessing remote memory in NUMA (Non-Uniform Memory Access) architecture servers can add a non-trivial amount of latency to application performance.
...
Processor affinity for vCPUs to be scheduled on specific NUMA nodes, as well as memory affinity for all VM memory to be allocated from those NUMA nodes, can be set using the vSphere Client under VM Settings – Options tab – Advanced General – Configuration Parameters and adding entries for “numa.nodeAffinity=0, 1, …,” where 0, 1, etc. are the processor socket numbers.
Networking Tips
10 Gigabit Network Configurations on VMware vSphere
By default, the SoftNAS Cloud® VMSoftNAS VM (on VMware vSphere) ships with the default E1000 virtual NIC adapter and VMware defaults to MTU 1500.
...
- Replace the E1000 virtual NIC adapter with a vmxnet3 on the SoftNAS Cloud® VM.
- Use MTU 9000 instead of MTU 1500 for vSwitch, vmKernel and physical switch configurations. Be sure to configure the network interface in SoftNAS for MTU 9000 also.
- Refer to the MTU 9000 section for more information.
A dedicated VLAN for storage traffic is recommended. For VMware, refer to the Performance Tuning for VMware vSphere section for details.
iSCSI Multi-pathing
To increase performance throughput and resiliency, use of iSCSI multipathing is recommended by VMware and other vendors.
...
- On the VMware host where the SoftNAS Cloud® VM runs, install and use multiple physical NIC adapters.
- Assign a dedicated vSwitch for each incoming iSCSI target path (one per physical NIC).
- Assign the SoftNAS Cloud® VM a dedicated virtual NIC adapter for each incoming iSCSI target path (per vSwitch physical NIC).
- Assign a unique IP address to each corresponding Linux network interface (for each virtual NIC attached to the SoftNAS Cloud® VM).
- Restart the SoftNAS iSCSI service and verify connectivity from the iSCSI initiator client(s) to each iSCSI target path.
A dedicated VLAN for storage traffic is recommended.