Table of Contents

Cluster LVM

sources:

https://www.suse.com/documentation/sle_ha/book_sleha/data/sec_ha_clvm_config.html

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Cluster_Administration/s1-admin-manage-nodes-delete-add-cli-CA.html

https://www.ibm.com/developerworks/community/blogs/mhhaque/resource/RHCS_IBM_POWER.pdf?lang=zh

Installation (Debian Jessie)

  1. install necessary packages:
    apt-get install clvm cman
  2. edit /etc/lvm/lmv.conf, set:
    mirror_logs_require_separate_pvs = 1
    locking_type = 3
  3. set no wait time for quorum:
    echo "CMAN_QUORUM_TIMEOUT=0" >> /etc/default/cman
  4. create config file /etc/cluster/cluster.conf and put it to all nodes:
    <cluster name="mycluster" config_version="1">
       <clusternodes>
         <clusternode name="10.0.0.23" nodeid="1">
             <fence>
             </fence>
         </clusternode>
         <clusternode name="10.0.0.112" nodeid="2">
             <fence>
             </fence>
         </clusternode>
       </clusternodes>
       <fencedevices>
       </fencedevices>
       <rm>
       </rm>
    </cluster>
  5. start cman service and clvm service on all nodes:
    service cman start
    service clvm start
  6. check that nodes are online in cluster:
    cman_tool nodes

    output should be:

    root@debian:~# cman_tool nodes
    Node  Sts   Inc   Joined               Name
       1   M      4   2015-10-30 09:34:43  10.0.0.23
       2   M      8   2015-10-30 09:36:04  10.0.0.112
  7. assuming that /dev/sdb and /dev/sdc are volumes exported from SANs
  8. create physical volumes:
    pvcreate /dev/sdb
    pvcreate /dev/sdc
  9. create clustered volume group:
    vgcreate --clustered y guma_cerna /dev/sdb /dev/sdc
  10. create mirrored lvm:
    lvcreate -n testovaci_lv -m1 -L 1.0g guma_cerna --nosync

    (–nosync means that data will not be synced upon creating - since the lv should not contain any data upon its creating its OK)

  11. (optional) check that the volume were created succesfully on both SANs:
    lvs -a -o +devices

Adding/deleting node from cluster

  1. If deleting stop all cluster services on node e.g.:
    service stop clvm
    service stop cman
  2. install cman clvm on new node(s) - don't forget to setup the /etc/lvm/lvm.conf file
  3. attach SANs on new node(s)
  4. edit the /etc/cluster/cluster.conf and increase serial:
    <cluster name="mycluster" config_version="2">
       <clusternodes>
         <clusternode name="10.0.0.23" nodeid="1">
             <fence>
             </fence>
         </clusternode>
         <clusternode name="10.0.0.112" nodeid="2">
             <fence>
             </fence>
         </clusternode>
         <clusternode name="10.0.0.110" nodeid="3">
             <fence>
             </fence>
         </clusternode>
      </clusternodes>
       <fencedevices>
       </fencedevices>
       <rm>
       </rm>
    </cluster>
  5. distribute the file to ALL nodes in cluster and issue following command from any node:
    cman_tool version -r -S
  6. to use previously created volumes those must be activated on new node:
    lvchange -a y /dev/guma_cerna/testovaci_lv

Removing node from the cluster

  1. stop all cluster services and then stop cman service with parameter leave
    service clvm stop
    service cman stop leave