May 2020
M T W T F S S
« Mar    
 123
45678910
11121314151617
18192021222324
25262728293031

Categories

WordPress Quotes

History is a relentless master. It has no present, only the past rushing into the future. To try to hold fast is to be swept aside.
John F. Kennedy
May 2020
M T W T F S S
« Mar    
 123
45678910
11121314151617
18192021222324
25262728293031

Short Cuts

2012 SERVER (64)
2016 windows (9)
AIX (13)
Amazon (40)
Ansibile (19)
Apache (135)
Asterisk (2)
cassandra (2)
Centos (211)
Centos RHEL 7 (270)
centos8 (3)
chef (3)
cloud (2)
cluster (3)
Coherence (1)
DB2 (5)
DISK (25)
DNS (9)
Docker (30)
Eassy (11)
ELKS (1)
EXCHANGE (3)
Fedora (6)
ftp (5)
GIT (3)
GOD (2)
Grub (1)
Hacking (10)
Hadoop (6)
health (2)
horoscope (23)
Hyper-V (10)
IIS (15)
IPTABLES (15)
JAVA (7)
JBOSS (32)
jenkins (1)
Kubernetes (7)
Ldap (5)
Linux (188)
Linux Commands (166)
Load balancer (5)
mariadb (14)
Mongodb (4)
MQ Server (24)
MYSQL (84)
Nagios (5)
NaturalOil (13)
Nginx (35)
Ngix (1)
openldap (1)
Openstack (6)
Oracle (35)
Perl (3)
Postfix (19)
Postgresql (1)
PowerShell (2)
Python (3)
qmail (36)
Redis (12)
RHCE (28)
SCALEIO (1)
Security on Centos (29)
SFTP (1)
Shell (64)
Solaris (58)
Sql Server 2012 (4)
squid (3)
SSH (10)
SSL (14)
Storage (1)
swap (3)
TIPS on Linux (28)
tomcat (62)
Ubuntu (1)
Uncategorized (30)
Veritas (2)
vfabric (1)
VMware (28)
Weblogic (38)
Websphere (71)
Windows (19)
Windows Software (2)
wordpress (1)
ZIMBRA (17)

WP Cumulus Flash tag cloud by Roy Tanck requires Flash Player 9 or better.

Who's Online

0 visitors online now
0 guests, 0 bots, 0 members

Hit Counter provided by dental implants orange county

RHCS(Red Hat Cluster Suite) quorum disk

Methods to deal with split-brain situation:
1. Redundant heartbeat path
network port communication plus serial port communication
2. I/O fencing
Remaining nodes separate failed node from its storage either by shutdown/reboot power port or storage port
3. Quorum disk
Quorum disk is a kind of I/O fencing, but the reboot action is executed by failed node’s own quorum daemon. It also has additional feature: contributing vote to cluster. if you want the last standing node to keep the multiple-nodes cluster  running, quorum disk appears to be the only solution.
RHCS (Red Hat Cluster Suite) Quorum disk facts
– A shared block device (SCSI/iSCSI/FC..), Device size requirement is approximately 10MiB
– Supports maximum 16 nodes, nodes id must be sequentially ordered
– Quorum disk can contribute  votes. In multiple nodes cluster, together with quorum vote, the last standing node can still keep the cluster running
– single node votes+1 <=Quorum’s disk vote < nodes total votes
– The failure of the shared quorum disk won’t result in cluster failure, as long as Quorum’s disk vote < nodes total votes
– each node write its own health information in its own region, the health is determined by external checking program such as “ping”
Setup Quorum disk

#initialise quorum disk once in any node 
mkqdisk -c /dev/sdx -l myqdisk 

Add quorum disk to cluster 
Use luci or system-config-cluster to add quorum disk, following is the result xml file

<clusternodes>
<clusternode name="station1.example.com" nodeid="1" votes="2">
<fence/>
</clusternode>
<clusternode name="station2.example.com" nodeid="2" votes="2">
<fence/>
</clusternode>
<clusternode name="station3.example.com" nodeid="3" votes="2">
<fence/>
</clusternode>
</clusternodes>
#expected votes =9=(nodes total votes + quorum disk votes) = (2+2+2+3)       
<cman expected_votes="9"/> 
#Health check result is writen to quorum disk every 2 secs
#if health check fails over 5 tko, 10 (2*5) secs, the node is rebooted by quorum daemon
#Each heuristic check is run very 2 secs and earn 1 score,if shell script return is 0
<quorumd interval="2" label="myqdisk" min_score="2" tko="5" votes="3">
<heuristic interval="2" program="ping -c1 -t1 192.168.1.60" score="1"/>
<heuristic interval="2" program="ping -c1 -t1 192.168.1.254" score="1"/>
</quorumd>

Start quorum disk daemon
The daemon is also one of daemons automatically started by cman
service qdiskd start
Check quorum disk information

$ mkqdisk -L -d
mkqdisk v0.6.0
/dev/disk/by-id/scsi-1IET_00010002:
/dev/disk/by-uuid/55fbf858-df75-493b-a764-5640be5a9b46:
/dev/sdc:
Magic:                eb7a62c2
Label:                myqdisk
Created:              Sat May  7 05:56:35 2011
Host:                 station2.example.com
Kernel Sector Size:   512
Recorded Sector Size: 512
Status block for node 1
Last updated by node 1
Last updated on Sat May  7 15:09:37 2011
State: Master
Flags: 0000
Score: 0/0
Average Cycle speed: 0.001500 seconds
Last Cycle speed: 0.000000 seconds
Incarnation: 4dc4d1764dc4d176
Status block for node 2
Last updated by node 2
Last updated on Sun May  8 01:09:38 2011
State: Running
Flags: 0000
Score: 0/0
Average Cycle speed: 0.001000 seconds
Last Cycle speed: 0.000000 seconds
Incarnation: 4dc55e164dc55e16
Status block for node 3
Last updated by node 3
Last updated on Sat May  7 15:09:38 2011
State: Running
Flags: 0000
Score: 0/0
Average Cycle speed: 0.001500 seconds
Last Cycle speed: 0.000000 seconds
Incarnation: 4dc4d2f04dc4d2f0

The cluster is still running with last node standing
Please note Total votes=quorum votes=5=2+3, if quorum disk vote is less than  (node votes+1), the cluster  wouldn’t have  survived

$cman_tool status
..
Nodes: 1
Expected votes: 9
Quorum device votes: 3
Total votes: 5
Quorum: 5  
..

3 comments to RHCS(Red Hat Cluster Suite) quorum disk

  • Shanti

    hi mohan,
    I got problem while using quorum disk, but i read somewhere that i can also configure quorum without quorum disk, and use network quorum. Can you help me to clear this confiusion for me?
    I am configuring in RHEL6.3, and i usually i have to update manually the cluster.conf, and the cluster hangs…
    2 kernel: dlm: closing connection to node 2

    Can you help me?

    Thanks!

    • Quorum with Disk is Most recommended as they work as split brain concept. 2 kernel: dlm: closing connection to node 2 check the /var/log/messages
      stop the network manager services.

      Check quorum disk information

      $ mkqdisk -L -d

  • cute makeup ideas

    After exploring a handful of the articles on your website, I really like your technique of writing a blog.
    I added it to my bookmark website list and will be checking back soon.
    Please visit my web site as well and let me know your opinion.

Leave a Reply

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

  

  

  

Blue Captcha Image
Refresh

*

Protected by WP Anti Spam