Modern Monitoring Tool for SoftNAS

Symptoms

As system usage become more complex and resource intensive in these modern times; the need to have a modern monitoring tool is consequential for every production workload to see resource utilization at nearly real time at a glance. Having this tool integrated with SoftNAS is vital for the reasons below:

Proactive monitoring for Customers: Not only does it provide a powerful and intuitive graphical Dashboard that gives customers an in depth resource utilization of their SoftNAS system(s) at a glance; it would also help them proactively take initiative/action to rectify potential problems before they become major
Helps harness historical Data: Often times customers are using production workloads that are not really suited for the instance types they are running on, and since there is no historical data to reference in order to fully understand how their systems are being utilizes to map out a pattern that can help advise the best instance type to use. We are often left guessing or helping to fix the immediate problem while neglecting the root cause.
Better than the SAR tool : Our SAR monitoring tool (which is the current monitoring tool on SoftNAS) records system events every 10 minutes. Anything can happen between the 1 and 10 minute time interval before the next event is recorded which is clearly not visible to customers or support to really understand what was going with the system which led to an outage
Helps Buurst Support : Since the modern monitoring tool is a time series database, support can easily go back in time (this can be days, weeks or even months) and compare datapoints to see patterns in which certain events have been happening that lead to a failover or system down scenarios. It will also help support to quickly get a head start on what the root cause of issues are by just browsing the Dashboard with customers instead of relying on logs alone which often times can take days to analyze and share findings.

Purpose

In the following guide we'll delve into some of the capacities of the monitoring tool:

Quick CPU /Mem/Disk overview

A.
A properly configured DNS is another very important prerequisite to the implementation of a kerberos authentication back-end. On that note, before continuing with guide, please make sure that a properly configured DNS is in place.The example below will demonstrate how a properly configured DNS looks like. From SoftNAS, we used the 'nslookup' command:
#. nslookup <hostname>
#. nslookup <IP>
Next, let's configure NTP to make sure that our time is in sync with SoftNAS/AD using the link here.
Edit the SoftNAS smb.conf file to include the 2 lines below. The "SOFTNAS" portion must be changed to your workgroup name configured in your smb.conf file after joining AD.
1. idmap config SOFTNAS:backend = rid
  idmap config SOFTNAS:range = 10000-99999
b. Our example /etc/samba/smb.conf has added the two lines above:
Restart the SoftNAS samba services and flush the cache for the changes to take effect:
#. service sernet-samba-winbindd restart
#. service sernet-samba-smbd restart
#. service sernet-samba-nmbd restart
The commands below will help to verify the RPC calls we make to AD moving forward.
# wbinfo -t
# wbinfo -m
#. wbinfo -u | tail
#. wbinfo -g | tail
#. getent passwd | tail
1. In order to enable NFS kerberos authentication from AD we need to make some changes to the NFS /etc/export file to include the line below.
  #. vim /etc/exports
  #. copy and paste this line as shown below sec=sys:krb5:krb5i:krb5p
2. By default, the NFS security feature is not enabled by SoftNAS, so we would have to switch it on manually, and the example below will show you exactly how.
  #. vim /etc/sysconfig/nfs
  #. Search for SECURE_NFS="yes" and uncomment it.
3. In this step, we are going to add an NFS upn-principle to allow us to turn on the "rpcsvcgssd,rpcgssd"
  # net ads join -U buba createupn=nfs/NAS1.softnas.local@SOFTNAS.LOCAL
Next, let's install the necessary packages if they are not already installed on our Linux clients:
1. RHEL 7/CENTOS 7: # yum -y samba-winbind-clients samba-winbind krb5-workstation krb5-libs cifs-utils nfs4-acl-tools
2. UBUNTU 16.04: # apt update && apt upgrade
Note: The steps above are important to avoid dependency conflicts, or obsolete packages which might hinder the installation of the new packages below.
#. apt install -y samba winbind krb5-user libpam-winbind libnss-winbind cifs-utils nfs4-acl-tools
In order to establish a communication relationship between AD and our Linux Client(s) a proper DNS record must be in place; and since we already have one on our SoftNAS system we can just copy it over using the command below:
#. vim /etc/resolv.conf
To have a unified GID/UID mappings with SoftNAS/AD for the interest of seamless integration; copy and paste the "[ global ]" section of the smb.conf file ONLY, from SoftNAS ( This step applies to all Linux Distributions ). The end result should look like the output of step #2. b
#. less /etc/samba/smb.conf
#. mv /etc/samba/smb.conf /etc/samba/smb.conf.BAK && vim /etc/samba/smb.conf
The /etc/krb5.conf file is responsible for controlling the SoftNAS/AD kerberos authentication backend in order to establish a secure trust relationship necessary to hand out kerberos ticket which will allow us to join AD in step #12. And since there is one readily available on SoftNAS; copy it over using the commands below:
#. cat /etc/krb5.conf from SoftNAS
#. mv /etc/krb5.conf /etc/krb5.conf.BAK & vim /etc/krb5.conf to your Linux Client(s).
Next, let's restart our samba winbind services , and flush the cache:
1. RHEL 7/CENTOS 7: # service smb restart && service nmb restart && service winbind restart && net cache flush
2. UBUNTU 16.04: # service smb restart && service nmb restart && service winbind restart && net cache flush
To ensure that our CIFS/NFS services would come back up on system reboot, lets enable the services involved, by running the commands below:
1. RHEL 7/CENTOS 7: #. systemctl enable winbind smb nmb
2. UBUNTU 16.04: #. systemctl enable winbind smbd nmbd winbind
  Note: The commands above are the only commands we borrowed from RHEL 7/CENTOS 7 & UBUNTU 16.04 because they are native to those versions, therefore if you are on older versions use this command instead chkconfig <service> on
The Linux nsswitch.conf configuration file controls how name resolution works when looking up various types of objects, such as host addresses and passwords, and AD Users & Groups. Without this, we cannot be able to resolve our AD Users & Groups. Therefore, edit the config file and add 'winbind' like the example below:
#: vim /etc/nsswitch
We are now ready to acquire our first Kerberos ticket to join AD by running the commands below on our systems.
#. kinit -p buba@SOFTNAS.LOCAL
Next, Lets Join AD while also creating an nfs-upn on the fly by using the command below on RHEL 7/CENTOS 7 & UBUNTU 16.04 respectively:
1. RHEL 7/CENTOS 7: # net ads join -U buba createupn=nfs/centos-7.softnas.local@SOFTNAS.LOCAL
2. UBUNTU 16.04: #. net ads join -U buba createupn=nfs/ubuntu-16.softnas.local@SOFTNAS.LOCAL
Please refer to step #4 of this guide to complete this step!
Next, lets create mount-points to mount our CIFS/NFS shares and verify:
1. RHEL/CentOS 7: #. mkdir /mnt/cifs /mnt/nfs-sec /mnt/nfs-krb5 /mnt/nfs-krb5p /mnt/nfs-krb5i
  UBUNTU 16.04: #. mkdir /mnt/cifs /mnt/nfs-sys /mnt/nfs-krb5 /mnt/nfs-krb5p /mnt/nfs-krb5i
2. CIFS mounts: for all Linux Distributions:
  #. mount -t cifs -o domain=SOFTNAS,username=buba,password=Thegladiator1 //na1/cifs1 /mnt/cifs
3. NFS mounts for all Linux Distributions :
  #. mount -t nfs nas1:/pool1/cifs1 /mnt/nfs-syc -o sec=sys
  #. mount -t nfs nas1:/pool1/cifs1 /mnt/krb5 -o sec=krb5
  #. mount -t nfs nas1:/pool1/cifs1 /mnt/krb5i -o sec=krb5i
  #.mount -t nfs nas1:/pool1/cifs1 /mnt/krb5p -o sec=krb5p
  #. df -h
Let's compare ACLs from AD,SoftNAS and Linux Clients:
1. From AD logged in as a "Domain Admin" User
2. ACLs from SoftNAS:
  #. cd /pool1/cifs1
  #. getfacl *
  2. ACLs from RHEL 7/CENTOS 7 "Client1"
    #. cd /mnt/cifs && ls -ltr && getfacl *
    #. cd /mnt/nfs && ls -ltr && getfacl *
  3. ACLS from UBUNTU 16.04 Client "ubuntu1"
    cd /mnt/cifs && ls -ltr && getfacl *
Our final step is to automate the mounting process on system reboot by utilizing the /etc/fstab file:
1. CIFS fstab mount Options: For security reasons, we can create a secure CIFS credentials file call 'creds' to hold our 'Domain Admin user' credentials for auto-mounting, instead of exposing it in plain text in /etc/fstab:
  1. Create the secure file for the /etc/fstab file:
    #. vim ~/.creds && chmod 600 .creds
  2. Paste the following:
    +
    domain=SOFTNAS
    username=buba
    password=Thegladiator1
    #. echo '//10.0.0.192/cifs1 /mnt/cifs cifs domain=SOFTNAS,credentials=/root/.creds 0 0' >> /etc/fstab
2. NFS fstab mount Options:
  #. echo 'nas1:/pool1/cifs1 /mnt/nfs nfs4 rsize=8192,wsize=8192,intr 0 0' >> /etc/fstab
3. NFS with Krb Authentication:
  #. vim /etc/idmap.conf
  domain = softnas.local
  #. vim /etc/sysconfig/nfs
  #. net ads join -U buba createupn=nfs/client1.softnas.local@SOFTNAS.LOCAL
  #net ads join -U buba createupn=nfs/ubuntu1.softnas.local@SOFTNAS.LOCAL
  net ads keytab add nfs -U buba
  
  nfs, rpcbind, rpcgssd, rpcidmapd, or rpcsvcgssd
Congratulations! With the right credentials, you can now administer all your SoftNAS shares on different protocols (CIFS/NFS ) from a single location, using multiple platforms.
Stay tuned for our Advanced CIFS/AD ACLs administration Guide.

Modern Monitoring Tool for SoftNAS

Purpose

Related articles