HP-UX Booting Process Overview.

System boot process:

-Power on
-PDC executes CPU, Memory and other tests
-PDC initialize the console to display messages
-Looks for Primary boot path
-It loads and executes Initial System Loader (ISL). Roll of boot ROM ends.
-ISL loads the secondary system loader called "hpux". It resides on boot area of the disk.
-"hpux" loads "/stand/vmunix" into memory and hands control over to kernel
-Kernel loads the device drivers
-Swapper process started. It has process ID 0. It manages memory for swapping in and out. It schedules all other process. Its first job is to startup the init process.
-/sbin/pre_init_rc getting executed. It checks and repairs root file system.
-/sbin/init started. Process ID is 1. It refers /etc/inittab. It does the following.

Read the /etc/inittab for default run level
-ioinitrc - It invokes insf command with -e option to find any new devices.
-bcheckrc - activates LVM and run fsck on file systems if required.
-/sbin/rc - It brings the system to the default run level. This script is run whenever the system moves from to a higher one or vice versa.
-/usr/bin/getty

HP-UX - How to check the boot disk name from which the server is booted?

# echo "boot_string/s" | adb /stand/vmunix /dev/kmem
boot_string:
boot_string: disk(0/1/1/0.0.0.0.0.0.0;0)/stand/vmunix

vxdiskadm, vxassist fails: Replacement of disk app01 in group app_dg with device c0t1d0 failed. ld.so.1: vxdg: fatal: relocation error: file /etc/vx/slib/libnsl.so.1: symbol_libc_register_forkhandler: referenced symbol not found .killed

Issue
Replacement of disk app01 in group app_dg with device c0t1d0 failed. ld.so.1: vxdg: fatal: relocation error: file /etc/vx/slib/libnsl.so.1: symbol_libc_register_forkhandler: referenced symbol not found .killed

Solution:
# ldd /usr/sbin/vxassist
libarray.so => /usr/lib/libarray.so
libsocket.so.1 => /etc/vx/slib/libsocket.so.1
libnsl.so.1 => /etc/vx/slib/libnsl.so.1
libdl.so.1 => /usr/lib/libdl.so.1
libvxdiscovery.so => /etc/vx/slib/libvxdiscovery.so
libdevinfo.so.1 => /etc/vx/slib/libdevinfo.so.1
libc.so.1 => /etc/vx/slib/libc.so.1
libmp.so.2 => /usr/lib/libmp.so.2
liba5k.so.2 => /etc/vx/slib/liba5k.so.2
libg_fc.so.2 => /etc/vx/slib/libg_fc.so.2
libdevice.so.1 => /etc/vx/slib/libdevice.so.1
libnvpair.so.1 => /usr/lib/libnvpair.so.1
/usr/platform/SUNW,Ultra-5_10/lib/libc_psr.so.1

When the above error is seen, it is usually because /etc/vx/slib/libnsl.so.1 has been overwritten. The workaround is to simply re-copy the library as follows:

Checked the sum:
#sum /usr/lib/libnsl.so.1
30572 1798 /usr/lib/libnsl.so.1

#sum /etc/vx/slib/libnsl.so.1
27966 1737 /etc/vx/slib/libnsl.so.1

#cp /etc/vx/slib/libnsl.so.1 /etc/vx/slib/libnsl.so.1.backup

#cp /usr/lib/libnsl.so.1 /etc/vx/slib/libnsl.so.1 --Copied the library as the sum was different.

THE ISSUE got fixed by copying the library /usr/lib/libnsl.so.1 and then Everything worked fine.

reset root password on a HP-UX Integrity Machine

Login to the console.

Enter the command menu:
MP> CM

Halt the system:
CM> RS

Go back to the MP> prompt by typing CTRL B then go back to the console:
MP> CO

When the system comes back with a series of boot options, highlight the HPUX boot option, click enter and click enter again to interrupt the boot process. Then boot in single user mode:

HPUX> hpux -is

Once the system has come up, you will have to mount the following file systems: /usr, /var, /var/tmp, /opt and /tmp. You will also have to fsck them.
# TERM=vt100
# export TERM

Find out which file systems need to be mounted:
# cat /etc/fstab

Mount the relevant file systems:
# mount /dev/vg00/lvol7 /usr
UX:vxfs mount: ERROR: V-3-21268: /dev/vg00/lvol7 is corrupted. needs checking

# fsck /dev/vg00/lvol7
log replay in progress
replay complete - marking super-block as CLEAN

# mount /dev/vg00/lvol7 /usr


# mount /dev/vg00/lvol9 /var
UX:vxfs mount: ERROR: V-3-21268: /dev/vg00/lvol9 is corrupted. needs checking

# fsck /dev/vg00/lvol9
log replay in progress
replay complete - marking super-block as CLEAN

# mount /dev/vg00/lvol9 /var


# mount /dev/vg00/lvol11 /var/tmp
UX:vxfs mount: ERROR: V-3-21268: /dev/vg00/lvol9 is corrupted. needs checking

# fsck /dev/vg00/lvol11
log replay in progress
replay complete - marking super-block as CLEAN

# mount /dev/vg00/lvol11 /var/tmp


Repeat with /opt and /tmp

Remove root password:
# cd /tcb/files/auth/r
# vi root
Remove the line beginning with :u_pwd
Save the file and exit

# passwd root
Type your chosen password. Make sure there is a numeric character.

Reboot the system:
# shutdown -r

NOTE: if your system isn’t a Trusted Computer Base system, there won’t be a /tcb directory. Therefore, you will have to:
# vi /etc/passwd
Remove the password field from the root entry and save the file.

Reboot the system:
# shutdown -r

When the system comes back up:
# passwd root
Type your chosen password. Make sure there is a numeric character.

VMware vSphere 4.0 Notes

VMOTION and Storage VMOTION

VMOTION
VMware vMotion enables the live migration of running virtual machines from one physical server to another with zero down time, continuous service availability, and complete transaction integrity.

Storage vMotion
Storage vMotion enables the migration of virtual machine files from one datastore to another without service interruption.You can choose to place the virtual machine and all its disks in a single location, or select separate locations for the virtual machine configuration file and each virtual disk. The virtual machine remains on the same host during Storage vMotion.


VMware Distributed Scheduler(DRS)
Feature that allocates and balances computing capacity dynamically across collections of hardware resources for virtual machines. This feature includes distributed power management (DPM) capabilities that enable a datacenter to significantly reduce its power consumption.

VMware DRS helps you manage a cluster of physical hosts as a single compute resource. You can assign a virtual machine to a cluster and DRS finds an appropriate host on which to run the virtual machine. DRS places virtual machines in such a way as to ensure that load across the cluster is balanced, and cluster-wide resource allocation policies (for example, reservations, priorities, and limits) are enforced. When a virtual machine is powered on, DRS performs an initial placement of the virtual machine on a host. As cluster conditions change (for example, load and available resources), DRS migrates (using vMotion) virtual machines to other hosts as necessary.

VMware Fault Tolerence (FT)
When Fault Tolerance is enabled for a virtual machine, a secondary copy of the original (or primary) virtual machine is created. All actions completed on the primary virtual machine are also applied to the secondary virtual machine. If the primary virtual machine becomes unavailable, the secondary machine
becomes active, providing continuous availability.

Using VMware vLockstep technology, VMware Fault Tolerance (FT) on the ESX/ESXi host platform provides continuous availability by protecting a virtual machine (the primary virtual machine) with a shadow copy (secondary virtual machine) that runs in virtual lockstep on a separate host. Inputs and events performed on the primary virtual machine are recorded and replayed on the Secondary virtual machine ensuring that the two remain in an identical state. For example, mouse-clicks and keystrokes are recorded on the primary virtual machine and replayed on the secondary virtual machine. Because the secondary virtual machine is in virtual lockstep with the primary virtual machine, it can take over execution at any point without service interruption or loss of data.

VMware High Availability (HA)
VMware HA enables quick automated restart of virtual machines on a different physical server within a cluster if a host fails. All applications within the virtual machines have the high availability benefit. HA monitors all physical hosts in a cluster and detects host failures. An agent placed on each physical host maintains a heartbeat with the other hosts in the resource pool. Loss of a heartbeat initiates the process of restarting all affected virtual machines on other hosts. See Figure 7 for an example of VMware HA. HA admission control ensures that sufficient resources are available in the cluster at all times to restart virtual machines on different physical hosts in the event of host failure.

RHEL -How to check the WWN and WWPN Numbers on Linux

#systool -c fc_host -v -----> Give Server FC WWN & WWPN numbers
#systool -c fc_transport -v --------> Give Server & Storage FC WWN & WWPN numbers

How to zone a Brocade SAN switch in command line base?

How to zone a Brocade SAN switch in command line base?
Here are the basics to add a new device to a Brocade SAN fabric. working with a Brocade DCX switch. In this scenario, there is a new server come with 2 HBA card that going to attach to the SAN switches.

Terminology
HBA - Host Bus Adapter, HBA card is the device that connects the servers to a Storage Area Networks (SAN) via Fibre Channel.

WWN - World Wide Name, a unique 8-byte number identifying the HBA. Each WWN is an 8-byte number derived from an IEEE OUI (for the first 3 bytes) and vendor-supplied information.

FC Zone - Fibre Channel Zone, a partitioned subset of the fabric. Members of a zone are allowed to communicate with each other, but devices are not allowed to communicate across zones.

Steps to Zone Brocade Switch

1.Plug in the FC Connector into an open port on the switch.
2.Login to the server and verify the HBA connection. It should see the switch but not the storage device.
3.Check the Brocade Switch Port. To do so, telnet to the brocade switches and login as user admin.
switch 1 - fabric A (SWITCH_1)
switch 2 - fabric B (SWITCH_2)

Create aliases for the hosts HBA's on each switch
Example:
switch_1> alicreate " HOSTNAME01_HBA_0","10:00:00:c9:69:3d:53"
switch_2> alicreate " HOSTNAME01_HBA_1","10:00:00:c9:69:ae:4e"

Tips: Do a switchshow to capture the WWN before start the aliases creation.

4.Create the storage zones using the aliases on each switch
switch_1> zonecreate "Z_HOSTNAME01_A", "HOSTNAME01_HBA_0"
switch_2> zonecreate "Z_HOSTNAME01_B", "HOSTNAME01_HBA_1"

Check the configuration
switch_1> zoneshow Z_HOSTNAME01_A
switch_2> zoneshow Z_HOST01_B

5.Add the zones to the fabrics on each switch
switch_1> cfgshow SWITCH_A
switch_1> cfgadd "SWITCH_A","Z_ HOSTNAME01_A"
switch_1> cfgsave
switch_1> cfgenable SWITCH_A

Check the configuration
switch_1> zoneshow
switch_2> cfgshow SWITCH_B
switch_2> cfgadd " SWITCH_B","Z_ HOSTNAME01_B"
switch_2> cfgsave
switch_2> cfgenable SWITCH_A

Check the configuration
switch_2> zoneshow

6.Login back in to the server to verify. It should now see the storage devices

Aix Security Files and commands

-----
SECURITY
-----

nulladm /var/adm/wtmp To recreate/clear down the wtmp file.

groups Lists out the groups that the user is a member of
setgroups Shows user and process groups

chmod abcd (filename) Changes files/directory permissions

Where a is (4 SUID) + (2 SGID) + (1 SVTX)
b is (4 read) + (2 write) + (1 execute) permissions for owner
c is (4 read) + (2 write) + (1 execute) permissions for group
d is (4 read) + (2 write) + (1 execute) permissions for others

-rwxrwxrwx -rwxrwxrwx -rwxrwxrwx

Owner Group Others

-rwSrwxrwx = SUID -rwxrwSrwx = SGID drwxrwxrwt = SVTX

chown (new owner) (filename) Changes file/directory owners
chgrp (new group) (filename) Changes file/directory groups

chown (new owner).(new group) (filename) Do both !!!

umask Displays umask settings
umask abc Changes users umask settings

where ( 7 - a = new file read permissions)
( 7 - b = new file write permissions)
( 7 - c = new file execute permissions)

eg umask 022 = new file permissions of 755 = read write and execute for owner
read ----- and execute for group
read ----- and execute for other

mrgpwd > file.txt Creates a standard password file in file.txt

passwd Change current user password

pwdadm (username) Change a users password

pwdck -t ALL Verifies the correctness of local authentication

lsgroup ALL Lists all groups on the system
mkgroup (new group) Creates a group
chgroup (attribute) (group) Change a group attribute
rmgroup (group) Removes a group

AIX Commands User Management

-----
USERS / ENVIRONMENT
-----

passwd -f Change current users gecos (user description)
passwd -s Change current users shell

chfn (username) Changes users gecos
chsh (username) (shell) Changes users shell

env Displays values of environment variables
printenv

id Displays current user's uid and gid details
id (user) Displays user uid and gid details

whoami Displays current user details
who am i (or who -m)

who Displays details of all users currently logged in.
w
who -b Displays system reboot time

uptime Displays number of users logged in, time since last
reboot, and the machine load averages.

lsuser ALL Lists all users details
lsuser (username) Lists details for user
lsuser -a(attribute) (username or ALL) Lists user attributes
lsuser -a home ALL

mkuser -a(attributes) (newuser) Add a new user

chuser (attributes) (user) Change a user
chuser login=false (user) Lock a user account

rmuser -p (user) Removes a user and all entries in security files

usrck -t ALL Checks all the user entires are okay.

fuser -u (logical volume) Displays processes using the files in that LV
fuser -k /dev/lv02 Will send a kill signal to all processes using
/dev/lv02

lsattr -D -l sys0 -a maxuproc Displays max number of processes per user
chdev -l sys0 -a maxuproc=(number) Changes max number of processes per user

chlang (language) Changes the language settings for system or user.
En_GB = PC850 code pages
en_GB = ISO8859 code pages (Great Britain)
C = posix

su - (user) Switch to new user and change to the new users environment.
su (user) Switch to new user, current environment is propated to the
new shell.

-----
REMOTE USERS
-----

ruser -a -f (user) Adds entry into /etc/ftpusers file
ruser -a -p (host) Adds entry into /etc/host.lpd file
ruser -a -r (host) Adds entry into /etc/hosts.equiv file

ruser -d -f (user) Deletes entry in /etc/ftpusers file
ruser -d -p (host) Deletes entry in /etc/host.lpd file
ruser -d -r (host) Deletes entry in /etc/hosts.equiv file

ruser -s -F Shows all entries in /etc/ftpusers file
ruser -s -P Shows all entries in /etc/host.lpd file
ruser -s -R Shows all entries in /etc/hosts.equiv file

ruser -X -F Deletes all entries in /etc/ftpusers file
ruser -X -P Deletes all entries in /etc/host.lpd file
ruser -X -R Deletes all entries in /etc/hosts.equiv file

Checking MP IP from the HP-UX OS

a) SFM (newer versions):
# /opt/sfm/bin/CIMUtil -e root/cimv2 HP_ManagementProcessor

b) SMH:
# /opt/hpsmh/data/htdocs/comppage/getMPInfo.cgi

HP-UX Commands and Tricks 11.11 11.23 11.31-Unlock User

In HP-UX 11.11/11.23
1. If the system is trusted then run modprpw command to unlock the account.
/usr/lbin/modprpw -k username

2. In HP-UX 11.31
userdbset -d -u username auth_failures

HP-UX Networking - NTP Losing Synchronization

When trying to synchronize an HP-UX 11.11 server to an external time source it repeatedly loses synchronization. The following excerpt is from the syslog :

Dec 4 10:01:09 rep2 xntpd[27557]: synchronized to 132.163.4.102, stratum=1
Dec 4 10:20:59 rep2 xntpd[27557]: time reset (step) -0.276097 s
Dec 4 10:20:59 rep2 xntpd[27557]: synchronization lost
Dec 4 10:26:19 rep2 xntpd[27557]: synchronized to 132.163.4.102, stratum=1
Dec 4 12:42:22 rep2 xntpd[27557]: time reset (step) -0.279892 s
Dec 4 12:42:22 rep2 xntpd[27557]: synchronization lost
Action Taken:
1. Ran ntpq -p . It does not have any good servers to synchronize.
2. It was discovered that none of the NTP servers in the /etc/ntp.conf file, could be successfully pinged.
3. It was also discovered that ntptrace timed out twice and occasionally worked on the third try.
4. It appears that the NTP servers listed in the /etc/ntp.conf file are not reliable.
5. Chose a Reliable time Source.
1. NTP servers are available on the Internet. If your system is connected to the Internet, you may use time from one or more of these servers as reference time.
Click here for a list of NTP servers http://www.ntp.org.
2. If your local network is not connected to the Internet, you may use one of the machines on your LAN as your timeserver and consider its clock as standard.
6. Chose multiple time servers and edited the /etc/ntp.conf to include the reliable NTP servers.
7. Used the following commands to restart the xntpd daemon
/etc/init.d/xntpd stop
/etc/init.d/xntpd start
8. The NTP daemon process is started at boot time. At start time, xntpd reads its configuration file /etc/ntp.conf and configures itself.
9. After choosing the reliable time source, NTP is synchronizing properly.

EMC TimeFinder Commands

The following are the Timefinder Procedural Commands

It outlines everything that needs to be done from start to finish. Realize that for routine operations, some of these steps won’t be needed; however, for the sake of completeness.

Prepare EMC structures

1. Create a Symmetrix disk group

symdg -t [ Regular | RDF1 | RDF2 ] create ${group}

2. Add devices to the disk group

symld -g ${group} add pd /dev/dsk/c#t#d#
symld -g ${group} add dev 01a

3. Associate BCV devices to the disk group

symbcv -g ${group} associate pd ${bcv_ctd}
symbcv -g ${group} associate dev ${bcv_dev}

Establish BCV mirrors

1. ID the logical device names: Timefinder defaults to using the logical device names. You can id the logical device names by:

symmir -g ${group} query

2. First time establish, execute a full establish:

symmir -g ${group} -full establish ${std_log_dev} bcv ${bcv_log_dev}

3. Use symmir query to monitor progress.

symmir -g ${group} query

Break BCV mirrors

1. Types of splits:

1. Instant split: Split is performed in the background after the completion of the split I/O request.
2. Force split: Splits the pair during establish or restore operations; invalid tracks may exist.
3. Reverse split: Resyncs the BCV with the full data copy from its local or remote mirror.
4. Reverse differential split: Enables a copy of only out-of-sync tracks to the BCV from its mirror.
5. Differential split: Enables a copy of only the updated tracks to the BCV’s mirror.

2. Commands:

symmir -g ${group} split
symmir -g ${group} split -instant
symmir -g ${group} split -differential
symmir -g ${group} reverse split -differential

Reestablish or restore BCV mirrors

1. Restore copies data from BCV back to standard pair. >Reestablish, on the other hand, does a differential update of the BCV from the standard device.

2. Commands:
symmir -g ${group} establish Differential reestablish from standard device to BCV
symmir -g ${group} -full restore Full restore of all tracks on BCV to standard device.
symmir -g ${group} restore Differential restore of BCV data to standard device.

The Timefinder Strategies are as follows

1. Maintain BCV mirrors with the standard device; break the mirrors when you want to backup, test, or develop on a copy of the original.

This is probably the most common way of running Timefinder. The advantage is that the split operation will happen almost instantly as the mirrors are fully synced all the time. The disadvantage is that anything towards that happens to the standard device will be reflected in the BCV mirror.

2. Maintain the BCV as a split device to keep an online backup of the original data.

How To Increase GFS and LVM Size

It is assumed that you have configured the storage so that all systems in the cluster can see it prior to continuing through these steps. If you have not, then please see either the iSCSI or SAN How To.

################################################################
LVM Configuration

1. Initialize LUNs with pvcreate..

iSCSI/non-multipath:

pvcreate /dev/sd SAN/multipath:

pvcreate /dev/dm-<0-9> 2. Add a device into the Volume group

vgextend vgcl_gfs__

3. Sync the LVM Metadata
clvmd -R

4. Extend the size of the Logical Volume

lvextend -L +100G /dev/vgcl_gfs__/gfslv

[root ~]# lvextend -L +73G /dev/vgcl_gfs_san_cus/cuslv

Extending logical volume cuslv to 85.00 GB
Logical volume cuslv successfully resized5. Sync the LVM Metadata

clvmd -R

#######################################################################
GFS Configuration

6. Grow the existing GFS file system into the additional space

gfs_grow -v (device|mount)

[root ~]# gfs_grow -v /dev/mapper/vgcl_gfs_san_cus/cuslv /gfs/cus
Preparing to write new FS information...
Done.
Device has grown by less than 100 blocks.... skipping

This TEST will not do anything, but tell you what device and mount point to use.

[root ~]# gfs_grow -Tv /dev/mapper/vgcl_gfs_san_cus/cuslv /gfs/cus

Redhat Cluster Commands -Linux

Checking status of the cluster
# clustat
Moving a service/package over to another node
# clusvcadm -r -m
Starting a service/package
# clusvcadm -e -m
Stopping/disabling a service/package
# clusvcadm -d

Resource Group Locking (for cluster Shutdown / Debugging):

clusvcadm -l --This prevents resource groups from starting on the local node.
clusvcadm -S --Show lock state
clusvcadm -u --Unlock local resource group manager.This allows resource groups to start on the local node.
clusvcadm -Z -Freeze group in place
clusvcadm -U -Unfreeze/thaw group

[root@host1 ~]# clustat -l
Cluster Status for host1cl @ Thu Apr 14 02:25:29 2011
Member Status: Quorate

Member Name ID Status
------ ---- ---- ------
host1.com 1 Online, Local, rgmanager
host2.com 2 Online, rgmanager

Service Information
------- -----------

Service Name : service:fibrbase
Current State : failed (114)
Flags : none (0)
Owner : none
Last Owner : host1.com
Last Transition : Thu Apr 14 01:39:21 2011

Service Name : service:fsgprod
Current State : failed (114)
Flags : none (0)
Owner : none
Last Owner : host1.com
Last Transition : Thu Apr 14 01:39:27 2011

Service Name : service:wcmnrocp
Current State : failed (114)
Flags : none (0)
Owner : none
Last Owner : host1.com
Last Transition : Thu Apr 14 01:30:20 2011

[root@host1~]#


Gracefully halting the cluster
# clusvcadm -d
**Do the following on each node:**
# umount
# service rgmanager stop
# service gfs stop
# service clvmd stop
# service fenced stop
# cman_tool status
# cman_tool leave
# service ccsd stop

Gracefully starting the cluster (Done on each node)
# service ccsd start
# service cman start
# service fenced start
# service clvmd start
# service gfs start
# service rgmanager start
# cman_tool nodes (shows status of nodes)

Setting up HP-UX iscsi configuration

Verify the iSCSI initiator is running
# iscsiutil -l

server1# /opt/iscsi/bin/iscsiutil -l
Initiator Name : iqn.1986-03.com.hp:server1.128931587
Initiator Alias :

Authentication Method :
CHAP Method : CHAP_UNI
Initiator CHAP Name :
CHAP Secret :
NAS Hostname :
NAS Secret :
Radius Server Hostname :
Header Digest : None,CRC32C (default)
Data Digest : None,CRC32C (default)
SLP Scope list for iSLPD :

Add the discovery target address
/opt/iscsi/bin/iscsiutil -a -I 10.6.8.199

Confirm connection to volume targets
iscsiutil –p –D



Discovery Target Information
----------------------------

Target # 1
-----------
IP Address : elomp1227dm2
iSCSI TCP Port : 3260
iSCSI Portal Group Tag : 1

User Configured:
----------------

Authenticaton Method :
CHAP Method : CHAP_UNI
Initiator CHAP Name :
CHAP Secret :
Header Digest : None,CRC32C (default)
Data Digest : None,CRC32C (default)

Target # 2
-----------
IP Address : inompd3fbno1
iSCSI TCP Port : 3260
iSCSI Portal Group Tag : 1

User Configured:
----------------

Authenticaton Method :
CHAP Method : CHAP_UNI
Initiator CHAP Name :
CHAP Secret :
Header Digest : None,CRC32C (default)
Data Digest : None,CRC32C (default)


A target is the same as a LUN on the SAN.

Confirm the Devices are Visible
ioscan -fnH 255

Class I H/W Path Driver S/W State H/W Type Description
===========================================================================
iscsi 0 255/0 iscsi CLAIMED VIRTBUS iSCSI Virtual Node
ext_bus 4 255/0/0.0 iscsial CLAIMED INTERFACE iSCSI-SCSI Protocol Interface
target 9 255/0/0.0.0 tgt CLAIMED DEVICE
disk 5 255/0/0.0.0.0 sdisk CLAIMED DEVICE EMC Celerra
/dev/dsk/c4t0d0 /dev/rdsk/c4t0d0
disk 6 255/0/0.0.0.1 sdisk CLAIMED DEVICE EMC Celerra
/dev/dsk/c4t0d1 /dev/rdsk/c4t0d1
disk 7 255/0/0.0.0.2 sdisk CLAIMED DEVICE EMC Celerra
ext_bus 6 255/0/2.0 iscsial CLAIMED INTERFACE iSCSI-SCSI Protocol Interface
target 25 255/0/2.0.0 tgt CLAIMED DEVICE
disk 25 255/0/2.0.0.0 sdisk CLAIMED DEVICE NETAPP LUN
/dev/dsk/c6t0d0 /dev/rdsk/c6t0d0
ext_bus 7 255/0/3.0 iscsial CLAIMED INTERFACE iSCSI-SCSI Protocol Interface
target 26 255/0/3.0.0 tgt CLAIMED DEVICE
disk 26 255/0/3.0.0.0 sdisk CLAIMED DEVICE NETAPP LUN
/dev/dsk/c7t0d0 /dev/rdsk/c7t0d0
disk 27 255/0/3.0.0.1 sdisk CLAIMED DEVICE NETAPP LUN
/dev/dsk/c7t0d1 /dev/rdsk/c7t0d1
disk 28 255/0/3.0.0.2 sdisk CLAIMED DEVICE NETAPP LUN
/dev/dsk/c7t0d2 /dev/rdsk/c7t0d2
disk 29 255/0/3.0.0.3 sdisk CLAIMED DEVICE NETAPP LUN


If a HW path does not have a device file use insf -e to generate one..

Troubleshooting iSCSI issues


A change in the IQN on the host will lead to LUNs not being available..
the first time a server is booted after the iscsi SW is installed an IQN is created as displayed in the iscsiutil -l command. the storage team uses the IQN much like a WWPN to allow a host to see the LUNs. If storage is nolonger visible ask the storage team what IQN associated witht he host are they looking for. and if it has changed on the server side use

/opt/iscsi/bin/iscsiutil -i -N 'iqn.1986-03.com.hp:server1.128931587'
where the oiption after the -N is the IQN that you are changing to/ then reboot the server to make sure the server properly connects to the iSCSI NAShead. The IQN is stored in the Kernel Registry Service with data stored in /stand/krs/* in a non text format.

Clear the Kernel Registry of iSCSI information..

HP-UX accommodates 256 iSCSI sessions and does not clear block device links to old targets including deleted snapshots. You must manually clear the registry of stale sessions once this maximum is reached in order to establish new iSCSI sessions. Note that clearing the kernel registry clears out all configurations, so after clearing you will need to recreate any old configurations you want to preserve.
Here is the procedure to clear out your iSCSI configuration from the kernel registry:

1.Before deleting all the iSCSI information in the registry and rebooting the system, preserve the original configuration as follows:
# ioscan -kfn > /tmp/ioscan.pre_shutdown.out
# iscsiutil -p > /tmp/iscsi_tgt.pre_shutdown.out

2.Delete all iSCSI information from the registry (all steps must be done as root):
# echo "iscsi_krs_delete/W 1" |adb -w /stand/vmunix /dev/kmem
(assuming system was booted from /stand/vmunix ...)
# iscsiutil –x

3.Reboot the system immediately.


4.Once the system is back online, you will have to recreate your iSCSI configuration

30 questions to ask before turning to the cloud

•What happens when the cloud goes offline? How will you get notified? Are services restarted automatically?
•If you want to terminate a contract before the term ends, what kinds of “lock-in” or “must pay” clauses exist?
•Can you only terminate the contract “for cause?” And if so, how difficult is it to prove a cause?
•How much does it cost to customize your apps once they’re in the cloud?
•How does your cloud provider supply QoS to your business?
•Do they provide load balancing or availability guarantees?
•How much does it cost to import your data? How much will it cost to export your data?
•How must does it cost to back up your data once it’s in the cloud?
•What are your expected RPO and RTO from your cloud provider?
•What are the cloud computing solutions providers’ annual revenues? How long has it been in business? How many customers?
•Is your cloud computing solutions provider providing you with SLAs that are currently better than what you have today?
•How will you transfer the software licenses you currently own?
•Will your apps be clustered in the cloud?
•How will you manage your apps for performance and/or outages?
•Will your cloud computing solutions provider use an underlying Iaas for their own infrastructure? Or will they build their own infrastructure?
•What is the SLA? What happens if your cloud provider doesn’t meet it?
•Do they have a disaster recovery plan? If so, can they send you a copy?
•Can they provide documentation of their uptime for this year and last year?
•How do they communicate problems, outages, and fixes to their customers?
•How does your cloud provider implement backups, and how long do they keep your data?
•Who has access to your data and/or the hardware on which it is running?
•How is redundancy implemented within your environment? Do you have spare disk drives, servers, power supplies, internet circuits?
•Do you have full ownership of my data?
•Does your cloud provider offer a standard annual termination for convenience?
•Do they allow for annual usage-level alignment (up or down) based on your business needs?
•Can you apply monthly rollover usage to address seasonal peaks?
•Can they provide long-term price protection?
How is your organization currently handling disaster recovery? Are you considering the cloud? We’d love to hear your thoughts.

Extending the root filesystem in HP-UX

Solution:
This is assumed that you already have a online JFS installed on the HP box.

1.Assume / is /dev/vg00/lvol3 and it is a vxfs file system.

lets assume that /dev/vg00/lvol4 is the next logical volume on the disk.

2. Move lvol4 to the other disk in VG00 ,extend the root volume and the filesystem.
pvmove -n /dev/vg00/lvol4 /dev/dsk/
lvextend -L /dev/vg00/lvol3
fsadm -F vxfs -b 100M /

Your / filesystem must be a contiguous logical volume so you need to move the logical volume after it to another disk.

3.After you extended /dev/vg00/lvol3 you can now move lvol4 back to the root disk
pvmove -n /dev/vg00/lvol4 /dev/ /dev/

If you do not have a ONLINE JFS installed or have a HFS root filesystem ignite may be your best choice.

Ignite Procedure:

After the initial Ignite interface is loaded from your tape or Ignite server, you will have an opportunity to interact with the Ignite tabbed interface. Use the tab key to access the FileSystem tab and resize file systems as you wish. After each change, be sure to click
[Modify].

After you have resized your file systems, click [Go!].
Ignite will rebuild your boot disk using the new file system sizes, and restore your data from the recovery archive. Using make_net_recovery, this oftentimes takes less than 20 minutes!

HP-UX Operating Systems - Error Messages in Syslog.log; Fibre Cable Connected to Server Is Bad

HP-UX Operating Systems - Error Messages in Syslog.log; Fiber Cable Connected to server Is Bad

ISSUE:
The following error messages are in the syslog.log :
Jun 5 15:09:25 vmunix: LVM: WARNING: VG 64 0x010000: LV 10: Some I/O requests to this LV are waiting
Jun 5 15:09:25 vmunix: indefinitely for an unavailable PV. These requests will be queued until
Jun 5 15:09:25 vmunix: the PV becomes available (or a timeout is specified for the LV).
Jun 5 15:09:39 vmunix: LVM: NOTICE: VG 64 0x010000: LV 10: All I/O requests to this LV that were
Jun 5 15:09:39 vmunix:waiting indefinitely for an unavailable PV have now completed.
Jun 5 15:14:55 vmunix: LVM: WARNING: VG 64 0x010000: LV 27: Some I/O requests to this LV are waiting
Jun 5 15:14:55 vmunix: indefinitely for an unavailable PV. These requests will be queued until
Jun 5 15:14:55 vmunix:the PV becomes available (or a timeout is specified for the LV).
Jun 5 15:15:11 vmunix: LVM: NOTICE: VG 64 0x010000: LV 27: All I/O requests to this LV that were
Jun 5 15:15:11 vmunix:waiting indefinitely for an unavailable PV have now completed.
Jun 5 15:15:54 vmunix: LVM: WARNING: VG 64 0x010000: LV 27: Some
I/O requests to this LV are waiting
SOLUTION:
The following troubleshooting steps are taken:
1. The syslogs are checked and there are many errors.
2. The ioscan is run and everything looks fine.
3. The vgdisplay output is checked and everything looks fine.
4. The LVM patches are checked to see if the latest ones are installed.
5. The sar output is checked to see the activity of the disks, using sar -d . There is no improper behavior.
6. It appears that there might be an issue with the fiber connectivity due to the many errors around the disks.
The fcmutil commands are run to check the status of fiber connectivity for both fiber channel ports.

fcmsutil /dev/fcd0

fcmutil /dev/fcd0 stat -s

fcmsutil /dev/fcd1

fcmutil /dev/fcd1 stat –s


Analysis
(/dev/xp_vgsapPRD) #fcmsutil /dev/fcd0 stat -s Thu Jun 5 16:06:31 2008 Channel Statistics

Link Statistics from firmware ...
Link Failure count 0 Loss of Sync count 0
Loss of Signal count 0 Prim seq proto err count 0
Invalid Word count 0 Invalid CRC count 0

(/dev/xp_vgsapPRD) # fcmsutil /dev/fcd1 stat -s Thu Jun 5 16:07:11 2008 Channel Statistics
Link Statistics from firmware ...
Link Failure count 0 Loss of Sync count 0
Loss of Signal count 0 Prim seq proto err count 0
Invalid Word count 0 Invalid CRC count 9469

The invalid CRC count for /dev/fcd1 is very high.
The fiber cable connected to /dev/fcd1 is changed, and the error disappears

What is Vault drive?

The Vault is where the OS for the array lives. In the case of Clariion and Celerra we are talking about the FLARE and DART code that enables the array to do its thing. Up until now this has always been the 1st 5 drives in the array – Drive Slots 0 – 4. These disks were setup as a 4+1 R5 Raid Group on which the FLARE/DART took up a small about of space on each drive. The remaining space could be used just not for any high I/O loads. Drive size was up to you but pretty much any drive size/type would work.

Zone Control Commands

The following control commands can be used to manage and monitor transitions between states:
• zlogin options zone-name
• zoneadm -z zone-name boot
• zoneadm -z zone-name halt
• zoneadm -z zone-name install
• zoneadm -z zone-name ready
• zoneadm -z zone-name reboot
• zoneadm -z zone-name uninstall
• zoneadm -z zone-name verify
• zonecfg -z zone-name: Interactive mode; can be used to remove properties of the following types: fs, device, rctl, net, attr

• zonecfg -z zone-name commit
• zonecfg -z zone-name create
• zonecfg -z zone-name delete
• zonecfg -z zone-name verify

APA - How to Resolve Error "get_start_dlpi_version: failed" While Booting the Server

Issue
ISSUE:
OS: HP-UX 11.23
ERROR:
Check for the error in the /etc/rc.log.
Configure Auto-Port Aggregation(LAN Monitor) interfaces
Output from "/sbin/rc2.d/S343hplm start":

get_start_dlpi_version: failed, errno is 22
ERROR: Incorrect version liblan.a on the system.
Please install DLPI patch for complete initialization of APA.
"/sbin/rc2.d/S343hplm start" FAILED
Solution
Checked for liblan.a which is contained in the "LAN cumulative patch". The latest version of that patch for 11.23 is PHNE_36839
Recommended to install the PHNE_36839 patch which resolved the issue.

HP-UX Operating Systems - "error: duplex mismatch port"

ISSUE:
An error is seen when starting Auto Port Aggregation (APA):

/sbin/init.d/hpapa start
error: duplex mismatch port 1
SOLUTION:
1. The settings are checked for both of the Network Interface Cards (NICs). It is found that one was configured as full-duplex and the other as half-duplex, while the network switch ports were configured as full-duplex.
lanadmin -x 1
lanadmin -x 2

2. The LAN settings were changed for the NIC that mismatched the network switch port at the other end of the connection.
lanadmin -X 100FD 1

3. APA is started without errors.
/sbin/init.d/hpapa start

4. Network services is started.
/sbin/init.d/net start

It is verified that ssh(1) functioned correctly (the originally reported issue had to do with ssh functionality).

PVG issue

HP-UX 11.x - lvsplit Gives an Error "Physical volume ... does not belong to volume group"
Issue

lvsplit is not working

ERROR:


# lvsplit -g backup /dev/fc01ora/lvol1
Physical volume "/dev/dsk/c89t5d6" does not belong to volume group "/dev/fc01ora".

Solution

Troubleshooting Done:

1. Checked the lvmtab file which shows the following disks without errors


/dev/fc01ora
/dev/dsk/c74t5d2
/dev/dsk/c74t5d6
/dev/dsk/c88t5d2
/dev/dsk/c88t5d6
/dev/dsk/c89t5d5
/dev/dsk/c89t5d6

2. Vgdisplay shows the disk "c89t5d6" without errors and this is a Alternate Link

3. lvdisplay works fine without errors on lvol1

4. Checked the PVG file


PVG Name backup
PV Name /dev/dsk/c85t5d5 Main Disk
PV Name /dev/dsk/c84t5d5 Alternate link
PV Name /dev/dsk/c87t5d5 Alternate link
PV Name /dev/dsk/c89t5d5 Alternate link
PV Name /dev/dsk/c89t5d6 Alternate link

PV Name /dev/dsk/c89t5d6 Alternate link
PV Name /dev/dsk/c87t5d6 Alternate link
PV Name /dev/dsk/c85t5d6 Main disk
PV Name /dev/dsk/c84t5d6 Alternate Disk

Solution:

Issue is with the PVG file which has a wrong order of disks.

In PVG file Main Disk should come first and then it should be followed with the alternate links.

Configuration worked and simulated the same on other servers.


PVG Name backup
PV Name /dev/dsk/c85t5d5 Main Disk
PV Name /dev/dsk/c84t5d5 Alternate link
PV Name /dev/dsk/c87t5d5 Alternate link
PV Name /dev/dsk/c89t5d5 Alternate link

PV Name /dev/dsk/c85t5d6 Main disk
PV Name /dev/dsk/c89t5d6 Alternate link
PV Name /dev/dsk/c87t5d6 Alternate link
PV Name /dev/dsk/c84t5d6 Alternate link

How do you trim the wtmp file so that it will contain only the data from the specified months?

ON All HP-UX operating Systems:

1. Run the following command to create an ASCII text file (/tmp/wtmp.txt ) from the binary wtmp file.

# /usr/sbin/acct/fwtmp < /var/adm/wtmp > /tmp/wtmp.txt

2. Search for the specified months, and convert the filtered data into the Binary file.
Example of February, March, and April:

# egrep "Feb|March|April" /tmp/wtmp.txt | /usr/sbin/acct/fwtmp -ic > /var/adm/wtmp
Now /var/adm/wtmp will contain data only for February, March, and April, so the last(1) command will display only records for those months.

How to access the console in HP Virtual Machines

Accessing the Virtual Console:
# hpvmconsole -P vm01


vMP MAIN MENU

CO: Console
CM: Command Menu
CL: Console Log
SL: Show Event Logs
VM: Virtual Machine Menu
HE: Main Help Menu
X: Exit Connection

[vm01] vMP>

To Exit Press-- Ctrl+b

Short Guide for Netapp commands

This post contains the list of commands that will be most used and will come handy when managing or monitoring or troubleshooting a Netapp filer.


•sysconfig -a : shows hardware configuration with more verbose information

•sysconfig -d : shows information of the disk attached to the filer

•version : shows the netapp Ontap OS version.

•uptime : shows the filer uptime

•dns info : this shows the dns resolvers, the no of hits and misses and other info

•nis info : this shows the nis domain name, yp servers etc.

•rdfile : Like "cat" in Linux, used to read contents of text files/

•wrfile : Creates/Overwrites a file. Similar to "cat > filename" in Linux

•aggr status : Shows the aggregate status

•aggr status -r : Shows the raid configuration, reconstruction information of the disks in filer
•aggr show_space : Shows the disk usage of the aggreate, WAFL reserve, overheads etc.

•vol status : Shows the volume information

•vol status -s : Displays the spare disks on the filer

•vol status -f : Displays the failed disks on the filer

•vol status -r : Shows the raid configuration, reconstruction information of the disks

•df -h : Displays volume disk usage

•df -i : Shows the inode counts of all the volumes

•df -Ah : Shows "df" information of the aggregate

•license : Displays/add/removes license on a netapp filer

•maxfiles : Displays and adds more inodes to a volume

•aggr create : Creates aggregate

•vol create : Creates volume in an aggregate

•vol offline : Offlines a volume

•vol online : Onlines a volume

•vol destroy : Destroys and removes an volume

•vol size [+|-] : Resize a volume in netapp filer

•vol options : Displays/Changes volume options in a netapp filer

•qtree create : Creates qtree

•qtree status : Displays the status of qtrees

•quota on : Enables quota on a netapp filer

•quota off : Disables quota

•quota resize : Resizes quota

•quota report : Reports the quota and usage

•snap list : Displays all snapshots on a volume

•snap create : Create snapshot

•snap sched : Schedule snapshot creation

•snap reserve : Display/set snapshot reserve space in volume

•/etc/exports : File that manages the NFS exports

•rdfile /etc/exports : Read the NFS exports file

•wrfile /etc/exports : Write to NFS exports file

•exportfs -a : Exports all the filesystems listed in /etc/exports

•cifs setup : Setup cifs

•cifs shares : Create/displays cifs shares

•cifs access : Changes access of cifs shares

•lun create : Creates iscsi or fcp luns on a netapp filer

•lun map : Maps lun to an igroup

•lun show : Show all the luns on a filer

•igroup create : Creates netapp igroup

•lun stats : Show lun I/O statistics

•disk show : Shows all the disk on the filer

•disk zero spares : Zeros the spare disks

•disk_fw_update : Upgrades the disk firmware on all disks

•options : Display/Set options on netapp filer

•options nfs : Display/Set NFS options

•options timed : Display/Set NTP options on netapp.

•options autosupport : Display/Set autosupport options
•options cifs : Display/Set cifs options
•options tcp : Display/Set TCP options
•options net : Display/Set network options
•ndmpcopy : Initiates ndmpcopy

•ndmpd status : Displays status of ndmpd

•ndmpd killall : Terminates all the ndmpd processes.

•ifconfig : Displays/Sets IP address on a network/vif interface

•vif create : Creates a VIF (bonding/trunking/teaming)

•vif status : Displays status of a vif

•netstat : Displays network statistics

•sysstat -us 1 : begins a 1 second sample of the filer's current utilization (crtl - c to end)
•nfsstat : Shows nfs statistics

•nfsstat -l : Displays nfs stats per client

•nfs_hist : Displays nfs historgram

•statit : beings/ends a performance workload sampling [-b starts / -e ends]
•stats : Displays stats for every counter on netapp. Read stats man page for more info

•ifstat : Displays Network interface stats

•qtree stats : displays I/O stats of qtree

•environment : display environment status on shelves and chassis of the filer
•storage show : Shows storage component details

•snapmirror intialize : Initialize a snapmirror relation

•snapmirror update : Manually Update snapmirror relation

•snapmirror resync : Resyns a broken snapmirror

•snapmirror quiesce : Quiesces a snapmirror bond

•snapmirror break : Breakes a snapmirror relation

•snapmirror abort : Abort a running snapmirror

•snapmirror status : Shows snapmirror status

•lock status -h : Displays locks held by filer

•sm_mon : Manage the locks

•storage download shelf : Installs the shelf firmware

•software get : Download the Netapp OS software

•software install : Installs OS

•download : Updates the installed OS

•cf status : Displays cluster status

•cf takeover : Takes over the cluster partner

•cf giveback : Gives back control to the cluster partner

•reboot : Reboots a filer

Aix General Commands and Tips

1. creation and deletion of user accounts
mkuser
rmuser

2. list all users
lsuser -f ALL

3. change user account
chuser -a

4. smit

5. hostid -For checking the hostid

6. Top,Sar,vmstat,iostat for performance troubleshooting
topas
sar 4 5 (4 seconds and 5 readings)
vmstat
iostat

9.How to Check the logs?
alog -o -t

10.How to see the size in MBs?
df -m

11.Display the swap size?
lsps -a

12.How to ativate the swap?
swapon -a

13. How to check the runlvel in AIX?
uname -r (runlevel)

14.For device listing
lsdev -C

15.network parameters
no -a

16.
bootinfo -s hdisk#

17. How to list the physical volumes?
lspv

18.How to create a Volume Group?
mkvg

19.How to list the logical volumes?
lslv

20.How to create a Logical Volume?
mklv

21.How to extend the logical volume?
extendlv

22.How to remove the logical volume?
rmlv

23.How to list the devices of particular Class (processor,tape)

lsdev -Cc processor
lsdev -Cc tape

24.How to check the model number?
bootinfo -m

25.How to find out the kernel parameter?
bootinfo -K

26.How to uninstall the software?
installp -u

27.How to install a software?
installp -a

28.How to verify the software is installed?
lppchk -v

29. How to list all softwares?
lslpp -L all

30.Where the software directory is located?
/sw/lpp

31.How to list the installed patches?
instfix -i

32.How to install devices for the attached peripherals?
cfgmgr -v

Setup ISCSI on Linux from Netapp

A quick guide to setup IP SAN using Linux-Netapp-iscsi.

On Linux server :
1. Install iSCSI initiator (iscsi-initiator-utils rpm) on your linux machine. This will create the necessary binaries and will create /etc/iscsi.conf and /etc/initiatorname.iscsi
2. Add iscsi-iname to /etc/initiatorname.iscsi .

[root@unixfoo ~]# iscsi-iname
iqn.1987-05.com.cisco:01.44c65d9587d9
[root@unixfoo ~]#

Add the output to /etc/initiatorname.iscsi
3. Add the below lines to /etc/iscsi.conf

Continuous=no
HeaderDigest=never
DataDigest=never
ImmediateData=yes
DiscoveryAddress=192.185.12.12

DiscoveryAddress should be the IP address of the storage.
On Netapp filer :
1. Make sure you have iscsi license enabled.
2. Create volume for holding the iscsi luns.

filer1> vol create iscsivol aggr01 100g
3. Create lun on the volume

filer1> lun create -s 50g -t linux /vol/iscsivol/lun1
4. Create an igroup and add the Linux iscsi-iname to it.

filer1> igroup create -i -t linux iscsigrp
filer1> igroup add iscsigrp iqn.1987-05.com.cisco:01.44c65d9587d9
filer1> igroup show
iscsigrp (iSCSI) (ostype: linux):
iqn.1987-05.com.cisco:01.44c65d9587d9 (logged in on: iswta)
filer1>
5. Map the lun to the iscsi-group.

filer1> lun map /vol/iscsivol/lun1 iscsigrp 0
6. Enable only one interface for iscsi use and disable others

filer1> iswt interface disable e7
filer1> iswt interface show
Interface e0 disabled
Interface e4 enabled
Interface e5 disabled
Interface e7 disabled
filer1>
7. Done on the Netapp side.

On Linux again :
1. Start iscsi initiator

[root@unixfoo ~]# /etc/init.d/iscsi start
Checking iscsi config: [ OK ]
Loading iscsi driver: [ OK ]
Starting iscsid: [ OK ]
[root@unixfoo ~]#
2. Set iscsi initiator to start automatically after reboot.

[root@unixfoo ~]# chkconfig iscsi on
3. Check whether the iscsi lun shows up on the linux machine.

[root@unixfoo ~]# iscsi-ls
*******************************************************************************
SFNet iSCSI Driver Version ...4:0.1.11-3(02-May-2006)
*******************************************************************************
TARGET NAME : iqn.1992-08.com.netapp:sn.50380528
TARGET ALIAS :
HOST ID : 2
BUS ID : 0
TARGET ID : 0
TARGET ADDRESS : 192.185.12.12:3260,2
SESSION STATUS : ESTABLISHED AT Sat Dec 29 21:55:37 PST 2007
SESSION ID : ISID 00023d000001 TSIH 501

DEVICE DETAILS:
---------------
LUN ID : 0
Vendor: NETAPP Model: LUN Rev: 0.2
Type: Direct-Access ANSI SCSI revision: 04
page83 type3: 60a980004f6444662653053584516d34
page80: 4f6444516d344305358066265a
Device: /dev/sdb

*******************************************************************************
[root@unixfoo ~]#
4. Now you have a new device in your linux box (/dev/sdb) - that is your iscsi device. You can create filesystem on it and use it.
5. Done!