RHCS – Cluster from scratch | Part 01

Leave a reply

In my previous post, I’ve just touch Red Hat High Availability Add-on for Red Hat Enterprise Linux and it’s eliminate single point of failures, so in case if the active cluster member on which a high availability service group is running become inoperative, the high availability service will start up again(fail over) in another cluster node without interruption.

Okay let’s get started with high availability clustering! but first of all, let’s understand some basic concepts. if you need clear and fully understand about all of these things, I highly recommend to read Red Hat Enterprise Linux 6 Cluster Administration Guide. It’s the best resources for RHEL6 HA clustering. and also you can use CentOS or Oracle Linux as alternatives to follow this article series without Using Red Hat Enterprise Linux.

Cluster Node
Cluster node is a server that is configured to be a cluster member. Normally shared storage (SAN,NAS) is available to all cluster members.

Cluster Resources
Cluster resources is the things you going to keep high available and all of these resources need to available for all cluster nodes. all or some of these resources can be associated with an application you plan to keep highly available.

Cluster Service Group
A collection of related cluster resources that defines actions to be taken during a fail-over operation of the access point of resilient resources. These resilient resources include applications, data, and devices.

Fencing
Fencing is the method that cuts off access to a cluster resource (shared storage, etc.) from a node in your cluster if it loses contact with the rest of the nodes in the cluster.

There are some more things related to clustering including this basic components and we can learn most of them when we deploying our high availability web service. So wait till the next post ;)

RHCS – Cluster from scratch

Leave a reply

According to the Red Hat, Red Hat Cluster Suite (RHCS) High Availability Add-On provides on-demand failover to make applications highly available. It delivers continuous availability of services by eliminating single points of failure. And clustering is a group of computers (called node or members) to work together as a team to provide continued service when system components fail.

Assume that we are running a critical database service on a standalone server, if a software or hardware component failed on that database server, administrative intervention is required and database service will be unavailable until the crashed server is fixed, but with clustering that database service get automatically restarted on another available node in the cluster without administrator invention and database service will be continuously available to the end-users. cluster can deploy as a active/active (one active node and one standby node) or active/passive (both nodes are active) to suite our clustering needs.

In this series of “RHCS – Cluster from scratch” articles, I’m planning to deeply explain how to deploy high availability web service as a active/passive cluster using Red Hat High Availability Add-On on a Red Hat Enterprise Linux 6.

Storage LUN Online Re-Scan on RHEL6

2 Replies

[A]lways storage maintenance and downtimes are comes together under production environment. but it’s not necessary at all the times because ISCSI(NAS) and FC(SAN) storage’s LUN re-scanning and re-sizing(expanding) is quiet easy under linux(not only Red Hat Enterprise Linux 6) environments. Okay then, assume that we have 10GB LUN on the ISCSI(NAS) storage and it has four paths to the server and also that four paths are muiltipathed. now we want to re-size this 10GB LUN to the 20GB on the storage and perform a online re-scan in the RHEL6 linux server to get that extra 10GB blocks to existing 10GB multipathed volume.
[cc lang="bash"][root@server ~]# multipath -ll
mpatha (1IET 00010001) dm-4 IET,VIRTUAL-DISK
size=10G features=’0′ hwhandler=’0′ wp=rw
|-+- policy=’round-robin 0′ prio=1 status=active
| `- 4:0:0:1 sda 8:0 active ready running
|-+- policy=’round-robin 0′ prio=1 status=enabled
| `- 5:0:0:1 sdb 8:16 active ready running
|-+- policy=’round-robin 0′ prio=1 status=enabled
| `- 3:0:0:1 sdc 8:32 active ready running
`-+- policy=’round-robin 0′ prio=1 status=enabled
`- 2:0:0:1 sdd 8:48 active ready running[/cc]
You can see there are four path to “mpatha” multipathed volume and sda, sdb, adc, sdd block devices are underling to it. so now we can expand the LUN in the ISCSI(NAS) and re-scan the block devices to add that extra expanded blocks to the “mpatha” volume.
[cc lang="bash"][root@server ~]# echo 1 > /sys/block/sda/device/rescan
[root@server ~]# echo 1 > /sys/block/sdb/device/rescan
[root@server ~]# echo 1 > /sys/block/sdc/device/rescan
[root@server ~]# echo 1 > /sys/block/sdd/device/rescan[/cc]
Using above commands, we re-scan the “sda”, “sdb”, “sdc” and “sdd” devices and that mean we are re-sized the all four path for “mpatha” volume. but device-mapper-multipath still using the old device blocks and we have to tell that multipathd to underlying devices to “mpatha” are re-sized, using
[cc lang="bash"][root@server ~]# multipathd -k”resize multipath mpatha”
ok[/cc] or simply reloading the multipathd.
[cc lang="bash"][root@server ~]# service multipathd reload
Reloading multipathd: [ OK ][/cc]
Now we are re-scanned the underling block devices for “mpatha” volume and let “device-mapper-multipath” know that “mpatha” was re-sized. that mean we are almost done with our ISCSI(NAS) storage re-scanning part. let’s check it out.
[cc lang="bash"][root@server ~]# multipath -ll
mpatha (1IET 00010001) dm-4 IET,VIRTUAL-DISK
size=20G features=’0′ hwhandler=’0′ wp=rw
|-+- policy=’round-robin 0′ prio=1 status=active
| `- 4:0:0:1 sda 8:0 active ready running
|-+- policy=’round-robin 0′ prio=1 status=enabled
| `- 5:0:0:1 sdb 8:16 active ready running
|-+- policy=’round-robin 0′ prio=1 status=enabled
| `- 2:0:0:1 sdd 8:48 active ready running
`-+- policy=’round-robin 0′ prio=1 status=enabled
`- 3:0:0:1 sdc 8:32 active ready running[/cc]
Oh yeah baby it’s working now. :P see the “multipath -ll” output and now “mpatha” volume is 20GB long.

In addition to ISCSI(NAS), if you are in a FC(SAN) environment, online re-scanning is quiet different but easy as previous we did in ISCSI. We can perform a FC(SAN) LUN online re-scan using the preferred command
[cc lang="bash"]echo “- – -” > /sys/class/scsi_host/hostX/scan[/cc]
where “hostX” is your HBA(Host Bus Adapter), That’s it.