Configuring Solaris Zones in VCS Clusters

I have been thinking from quite a while for providing the High Availability to Non-Global zones Vertias Clusters.

Lets see , how we can configure the same.

In Vertias Cluster, Assuming that the non global zone will be on a shared storage and we are configuring a failover or a parallel Service Group.

Configure a parallel service group with required storage resources (DiskGroup,Volume, Mount, etc.) to mount the zone root on the nodes. Set the required dependency between storage resources (DiskGroup->Volume->Mount).
sysA# hagrp -add zone_grp
sysA# hagrp -modify zone_grp Parallel 1 (This step is not included while configuring failover)
sysA# hagrp -modify zone_grp SystemList sysA 0 sysB 1
sysA# hares -add zone_dg DiskGroup zone_grp
sysA# hares -add zone_vol Volume zone_grp
sysA# hares -add zone_mnt Mount zone_grp
sysA# hares -link zone_mnt zone_vol
sysA# hares -link zone_vol zone_dg

Bring the service group online on all the nodes. This command mounts the zone root on all the nodes.
sysA# hagrp -online zone_grp -any  (Parallel)
sysA# hagrp -online zone_grp -sysA  (Failover)

Boot the local zone on all the nodes outside VCS.
sysA# zoneadm -z local-zone1 boot
sysB# zoneadm -z local-zone2 boot

Run the hazonesetup utility with correct arguments on all the nodes successively.(Parallel)
sysA# hazonesetup -g zone_grp -r zone_res -z local-zone1 -p password -a -l -s sysA,sysB
sysB# hazonesetup -g zone_grp -r zone_res -z local-zone2 -p password -a -l -s sysA,sysB

Run the hazonesetup utility with correct arguments on all the nodes successively.(Failover)
sysA# hazonesetup -g zone_grp -r zone_res -z local-zone1 -p password -a  -s sysA,sysB
sysB# hazonesetup -g zone_grp -r zone_res -z local-zone2 -p password -a  -s sysA,sysB

Set the proper dependency between the Zone resource and other storage
resources. The Zone resource should depend on storage resource
(Mount->Zone).
sysA# hares -link zone_res zone_mnt

How to failover a VCS Service Group ?

For Example: You have a service group(sg1) currently running on nodeA and you want to fail it over to node B. Then you need to run the below command to perform the failover.

hagrp -switch sg1 to nodeB

VCS- Implementing Veritas Cluster File System (CFS)

Veritas Cluster File System (CFS)

CFS allows the same file system to be simultaneously mounted on multiple nodes in the cluster.

The CFS is designed with master/slave architecture. Though any node can initiate an operation to create, delete, or resize data, the master node carries out the actual operation. CFS caches the metadata in memory, typically in the memory buffer cache or the vnode cache. A distributed locking mechanism, called GLM, is used for metadata and cache coherency among the multiple nodes.

The examples here are :

1. Based on VCS 5.x but should also work on 4.x
2. A new 4 node cluster with no resources defined.
3. Diskgroups and volumes will be created and shared across all nodes.

Before you configure CFS

1. Make sure you have an established Cluster and running properly.
2. Make sure these packages are installed on all nodes:

VRTScavf Veritas cfs and cvm agents by Symantec
VRTSglm Veritas LOCK MGR by Symantec

3. Make sure you have a license installed for Veritas CFS on all nodes.
4. Make sure vxfencing driver is active on all nodes (even if it is in disabled mode).

Check the status of the cluster

Here are some ways to check the status of your cluster. On these examples, CVM/CFS are not configured yet.

# cfscluster status
 NODE         CLUSTER MANAGER STATE            CVM STATE
serverA        running                        not-running                   
serverB        running                        not-running                   
serverC        running                        not-running                   
serverD        running                        not-running                   

 Error: V-35-41: Cluster not configured for data sharing application

# vxdctl -c mode
mode: enabled: cluster inactive

# /etc/vx/bin/vxclustadm nidmap
Out of cluster: No mapping information available

# /etc/vx/bin/vxclustadm -v nodestate
state: out of cluster

# hastatus -sum

-- SYSTEM STATE
-- System               State                Frozen             

A  serverA             RUNNING              0                   
A  serverB             RUNNING              0                   
A  serverC             RUNNING              0                   
A  serverD             RUNNING              0

Configure the cluster for CFS

During
configuration, veritas will pick up all information that is set on your
cluster configuration. And will activate CVM on all the nodes.

# cfscluster config

       The cluster configuration information as read from cluster
       configuration file is as follows.
               Cluster : MyCluster
               Nodes   : serverA serverB serverC serverD


       You will now be prompted to enter the information pertaining
       to the cluster and the individual nodes.

       Specify whether you would like to use GAB messaging or TCP/UDP
       messaging. If you choose gab messaging then you will not have
       to configure IP addresses. Otherwise you will have to provide
       IP addresses for all the nodes in the cluster.
 
       ------- Following is the summary of the information: ------
               Cluster         : MyCluster
               Nodes           : serverA serverB serverC serverD
               Transport       : gab
       -----------------------------------------------------------


       Waiting for the new configuration to be added.

       ========================================================

       Cluster File System Configuration is in progress...
       cfscluster: CFS Cluster Configured Successfully

Check the status of the cluster

Now let's check the status of the cluster. And notice that there is now a new service group cvm. CVM is required to be online before we can bring up any clustered filesystem on the nodes.

# cfscluster status

 Node             :  serverA
 Cluster Manager  :  running
 CVM state        :  running
 No mount point registered with cluster configuration


 Node             :  serverB
 Cluster Manager  :  running
 CVM state        :  running
 No mount point registered with cluster configuration


 Node             :  serverC
 Cluster Manager  :  running
 CVM state        :  running
 No mount point registered with cluster configuration


 Node             :  serverD
 Cluster Manager  :  running
 CVM state        :  running
 No mount point registered with cluster configuration

# vxdctl -c mode
mode: enabled: cluster active - MASTER
master: serverA

# /etc/vx/bin/vxclustadm nidmap
Name                             CVM Nid    CM Nid     State
serverA                         0          0          Joined: Master
serverB                         1          1          Joined: Slave
serverC                         2          2          Joined: Slave
serverD                         3          3          Joined: Slave

# /etc/vx/bin/vxclustadm -v nodestate
state: cluster member
       nodeId=0
       masterId=1
       neighborId=1
       members=0xf
       joiners=0x0
       leavers=0x0
       reconfig_seqnum=0xf0a810
       vxfen=off

# hastatus -sum

-- SYSTEM STATE
-- System               State                Frozen             

A  serverA             RUNNING              0                   
A  serverB             RUNNING              0                   
A  serverC             RUNNING              0                   
A  serverD             RUNNING              0                   

-- GROUP STATE
-- Group           System               Probed     AutoDisabled    State         

B  cvm             serverA             Y          N               ONLINE        
B  cvm             serverB             Y          N               ONLINE        
B  cvm             serverC             Y          N               ONLINE        
B  cvm             serverD             Y          N               ONLINE

How to freeze and unfreeze the Service Group (SG) on Veritas Cluster Server (VCS)

Freeze the Service Group test_sg
#hagrp -freeze test_sg


Unfreeze the Service Group test_sg

#hagrp -unfreeze test_sg

Veritas Cluster Server (VCS) - Adding a user account

Many faced problems while adding a user account to the Veritas Cluster Server online.

Here is what you need to do.


Make the configuration read-write
#haconf -makerw

Add the user admin1
#hauser -add admin1

Add the user admin1 to the Administrators Group
#haclus -modify Administrators -add admin1

Then again make the VCS configuration read-only.(It is always important to make it readonly)
#haconf -makero