Configuring Red Hat High Availability Add-On With the ccs Command - Red Hat Enterprise Linux Manual

Chapter 5. Configuring Red Hat High Availability Add-On With the ccs Command

As of the Red Hat Enterprise Linux 6.1 release and later, the Red Hat High Availability Add-On provides support for the ccs cluster configuration command. The ccs command allows an administrator to create, modify and view the cluster.conf cluster configuration file. You can use the ccs command to configure a cluster configuration file on a local file system or on a remote node. Using the ccs command, an administrator can also start and stop the cluster services on one or all of the nodes in a configured cluster.

This chapter describes how to configure the Red Hat High Availability Add-On cluster configuration file using the ccs command. For information on using the ccs command to manage a running cluster, see Chapter 6, Managing Red Hat High Availability Add-On With ccs.

This chapter consists of the following sections:

Important

Make sure that your deployment of High Availability Add-On meets your needs and can be supported. Consult with an authorized Red Hat representative to verify your configuration prior to deployment. In addition, allow time for a configuration burn-in period to test failure modes.

Important

This chapter references commonly used cluster.conf elements and attributes. For a comprehensive list and description of cluster.conf elements and attributes, refer to the cluster schema at /usr/share/cluster/cluster.rng, and the annotated schema at /usr/share/doc/cman-X.Y.ZZ/cluster_conf.html (for example /usr/share/doc/cman-3.0.12/cluster_conf.html).

5.1. Operational Overview

This section describes the following general operational aspects of using the ccs command to configure a cluster:

5.1.1. Creating the Cluster Configuration File on a Local System

Using the ccs command, you can create a cluster configuration file on a cluster node, or you can create a cluster configuration file on a local file system and then send that file to a host in a cluster. This allows you to work on a file from a local machine, where you can maintain it under version control or otherwise tag the file according to your needs. Using the ccs command does not require root privilege.

When you create and edit a cluster configuration file on a cluster node with the ccs command, you use the -h option to specify the name of the host. This creates and edits the cluster.conf file on the host:

ccs -h host [options]

To create and edit a cluster configuration file on a local system, use the -f option of the ccs command to specify the name of the configuration file when you perform a cluster operation. You can name this file anything you want.

ccs -f file [options]

After you have created the file locally you can send it to a cluster node using the --setconf option of the ccs command. On a host machine in a cluster, the file you send will be named cluster.conf and it will be placed in the /etc/cluster directory.

ccs -h host -f file --setconf

For information on using the --setconf option of the ccs command, see Section 5.15, "Propagating the Configuration File to the Cluster Nodes".

5.1.2. Viewing the Current Cluster Configuration

If at any time during the creation of a cluster configuration file you want to print the current file, use the following command, specifying a node in the cluster as the host:

ccs -h host --getconf

If you are creating your cluster configuration file on a local system you can specify the -f option instead of the -h option, as described in Section 5.1.1, "Creating the Cluster Configuration File on a Local System".

5.1.3. Specifying ricci Passwords with the ccs Command

Executing ccs commands that distribute copies of the cluster.conf file to the nodes of a cluster requires that ricci be installed and running on the cluster nodes, as described in Section 2.13, "Considerations for ricci". Using ricci requires a password the first time you interact with ricci from any specific machine.

If you have not entered a password for an instance of ricci on a particular machine from the machine you are using, you will be prompted for that password when the ccs command requires it. Alternately, you can use the -p option to specify a ricci password on the command line.

ccs -h host -p password --sync --activate

When you propagate the cluster.conf file to all of the nodes in the cluster with the --sync option of the ccs command and you specify a ricci password for the command, the ccs command will use that password for each node in the cluster. If you need to set different passwords for ricci on individual nodes, you can use the --setconf option with the -p option to distribute the configuration file to one node at a time.

5.1.4. Modifying Cluster Configuration Components

You use the ccs command to configure cluster components and their attributes in the cluster configuration file. After you have added a cluster component to the file, in order to modify the attributes of that component you must remove the component you have defined and add the component again, with the modified attributes. Information on how to do this with each component is provided in the individual sections of this chapter.

The attributes of the cman cluster component provide an exception to this procedure for modifying cluster components. To modify these attributes, you execute the --setcman option of the ccs command, specifying the new attributes. Note that specifying this option resets all values that you do not explicitly specify to their default values, as described in Section 5.1.5, "Commands that Overwrite Previous Settings".

5.1.5. Commands that Overwrite Previous Settings

There are several options of the ccs command that implement overwriting semantics when setting properties. This means that you can issue the ccs command with one of these options without specifying any settings, and it will reset all settings to their default values. These options are as follows:

--settotem
--setdlm
--setrm
--setcman
--setmulticast
--setaltmulticast
--setfencedaemon
--setlogging
--setquorumd

For example, to reset all of the fence deamon properties, you can run the following command.

# ccs -h hostname --setfencedaemon

Note, however, that if you use one of these commands to reset a property, then the other properties of the command will be reset to their default values. For example, you can use the following command to set the post_fail_delay property to 5:

# ccs -h hostname --setfencedaemon post_fail_delay=5

If, after running that command, you run the following command to reset the post_join_delay property to 10, the post_fail_delay property will be restored to its default value:

# ccs -h hostname --setfencedaemon post_join_delay=10

To reset both the post_fail_delay and the post_join_delay properties, you indicate them both on the same command, as in the following example:

# ccs -h hostname --setfencedaemon post_fail_delay=5 post_join_delay=10

For more information on configuring fence devices, refer to Section 5.5, "Configuring Fence Devices".

5.1.6. Configuration Validation

When you use the ccs command to create and edit the cluster configuration file, the configuration is automatically validated according to the cluster schema. As of the Red Hat Enterprise Linux 6.3 release, the ccs command validates the configuration according to the cluster schema at /usr/share/cluster/cluster.rng on the node that you specify with the -h option. Previously the ccs command always used the cluster schema that was packaged with the ccs command itself, /usr/share/ccs/cluster.rng on the local system. When you use the -f option to specify the local system, the ccs command still uses the cluster schema /usr/share/ccs/cluster.rng that was packaged with the ccs command itself on that system.

5.2. Configuration Tasks

Configuring Red Hat High Availability Add-On software with the ccs consists of the following steps:

Ensuring that ricci is running on all nodes in the cluster. Refer to Section 5.3, "Starting ricci".
Creating a cluster. Refer to Section 5.4, "Creating A Cluster".
Configuring fence devices. Refer to Section 5.5, "Configuring Fence Devices".
Configuring fencing for cluster members. Refer to Section 5.7, "Configuring Fencing for Cluster Members".
Creating failover domains. Refer to Section 5.8, "Configuring a Failover Domain".
Creating resources. Refer to Section 5.9, "Configuring Global Cluster Resources".
Creating cluster services. Refer to Section 5.10, "Adding a Cluster Service to the Cluster".
Configuring a quorum disk, if necessary. Refer to Section 5.13, "Configuring a Quorum Disk".
Configuring global cluster properties. Refer to Section 5.14, "Miscellaneous Cluster Configuration".
Propagating the cluster configuration file to all of the cluster nodes. Refer to Section 5.15, "Propagating the Configuration File to the Cluster Nodes".

5.3. Starting ricci

In order to create and distribute cluster configuration files on the nodes of the cluster, the ricci service must be running on each node. Before starting ricci, you should ensure that you have configured your system as follows:

The IP ports on your cluster nodes should be enabled for ricci. For information on enabling IP ports on cluster nodes, see Section 2.3.1, "Enabling IP Ports on Cluster Nodes".
The ricci service is installed on all nodes in the cluster and assigned a ricci password, as described in Section 2.13, "Considerations for ricci".

After ricci has been installed and configured on each node, start the ricci service on each node:

# service ricci startStarting ricci: [  OK  ]

5.4. Creating A Cluster

This section describes how to create, modify, and delete a skeleton cluster configuration with the ccs command without fencing, failover domains, and HA services. Subsequent sections describe how to configure those parts of the configuration.

To create a skeleton cluster configuration file, first create and name the cluster and then add the nodes to the cluster, as in the following procedure:

Create a cluster configuration file on one of the nodes in the cluster by executing the ccs command using the -h parameter to specify the node on which to create the file and the createcluster option to specify a name for the cluster:
```
ccs -h host --createcluster clustername
```
For example, the following command creates a configuration file on node-01.example.com named mycluster:
```
ccs -h node-01.example.com --createcluster mycluster
```
The cluster name cannot exceed 15 characters.
If a cluster.conf file already exists on the host that you specify, executing this command will replace that existing file.
If you want to create a cluster configuration file on your local system you can specify the -f option instead of the -h option. For information on creating the file locally, refer to Section 5.1.1, "Creating the Cluster Configuration File on a Local System".
To configure the nodes that the cluster contains, execute the following command for each node in the cluster:
```
ccs -h host --addnode node
```
For example, the following three commands add the nodes node-01.example.com, node-02.example.com, and node-03.example.com to the configuration file on node-01.example.com:
```
ccs -h node-01.example.com --addnode node-01.example.comccs -h node-01.example.com --addnode node-02.example.comccs -h node-01.example.com --addnode node-03.example.com
```
To view a list of the nodes that have been configured for a cluster, execute the following command:
```
ccs -h host --lsnodes
```
Example 5.1, "cluster.conf File After Adding Three Nodes" shows a cluster.conf configuration file after you have created the cluster mycluster that contains the nodes node-01.example.com, node-02.example.com, and node-03.example.com.
Example 5.1. cluster.conf File After Adding Three Nodes
```
<cluster name="mycluster" config_version="2">   <clusternodes> <clusternode name="node-01.example.com" nodeid="1"> <fence> </fence> </clusternode> <clusternode name="node-02.example.com" nodeid="2"> <fence> </fence> </clusternode> <clusternode name="node-03.example.com" nodeid="3"> <fence> </fence> </clusternode>   </clusternodes>   <fencedevices>   </fencedevices>   <rm>   </rm></cluster>
```
When you add a node to the cluster, you can specify the number of votes the node contributes to determine whether there is a quorum. To set the number of votes for a cluster node, use the following command:
```
ccs -h host --addnode host --votes votes
```
When you add a node, the ccs assigns the node a unique integer that is used as the node identifier. If you want to specify the node identifier manually when creating a node, use the following command:
```
ccs -h host --addnode host --nodeid nodeid
```
To remove a node from a cluster, execute the following command:
```
ccs -h host --rmnode node
```

When you have finished configuring all of the components of your cluster, you will need to sync the cluster configuration file to all of the nodes, as described in Section 5.15, "Propagating the Configuration File to the Cluster Nodes".

5.5. Configuring Fence Devices

Configuring fence devices consists of creating, updating, and deleting fence devices for the cluster. You must create and name the fence devices in a cluster before you can configure fencing for the nodes in the cluster. For information on configuring fencing for the individual nodes in the cluster, refer to Section 5.7, "Configuring Fencing for Cluster Members".

Before configuring your fence devices, you may want to modify some of the fence daemon properties for your system from the default values. The values you configure for the fence daemon are general values for the cluster. The general fencing properties for the cluster you may want to modify are summarized as follows:

The post_fail_delay attribute is the number of seconds the fence daemon (fenced) waits before fencing a node (a member of the fence domain) after the node has failed. The post_fail_delay default value is 0. Its value may be varied to suit cluster and network performance.
The post-join_delay attribute is the number of seconds the fence daemon (fenced) waits before fencing a node after the node joins the fence domain. The post_join_delay default value is 6. A typical setting for post_join_delay is between 20 and 30 seconds, but can vary according to cluster and network performance.

You reset the values of the post_fail_delay and post_join_delay attributes with the --setfencedaemon option of the ccs command. Note, however, that executing the ccs --setfencedaemon command overwrites all existing fence daemon properties that have been explicitly set and restores them to their default values.

For example, to configure a value for the post_fail_delay attribute, execute the following command. This command will overwrite the values of all other existing fence daemon properties that you have set with this command and restore them to their default values.

ccs -h host --setfencedaemon post_fail_delay=value

To configure a value for the post_join_delay attribute, execute the following command. This command will overwrite the values of all other existing fence daemon properties that you have set with this command and restore them to their default values.

ccs -h host --setfencedaemon post_join_delay=value

To configure a value for both the the post_join_delay attribute and the post_fail_delay attribute, execute the following command:

ccs -h host --setfencedaemon post_fail_delay=value post_join_delay=value

Note

For more information about the post_join_delay and post_fail_delay attributes as well as the additional fence daemon properties you can modify, refer to the fenced(8) man page and refer to the cluster schema at /usr/share/cluster/cluster.rng, and the annotated schema at /usr/share/doc/cman-X.Y.ZZ/cluster_conf.html.

To configure a fence device for a cluster, execute the following command:

ccs -h host --addfencedev devicename [fencedeviceoptions]

For example, to configure an APC fence device in the configuration file on the cluster node node1 named myfence with an IP address of apc_ip_example, a login of login_example, and a password of password_example, execute the following command:

ccs -h node1 --addfencedev myfence agent=fence_apc ipaddr=apc_ip_example login=login_example passwd=password_example

The following example shows the fencedevices section of the cluster.conf configuration file after you have added this APC fence device:

<fencedevices>  <fencedevice agent="fence_apc" ipaddr="apc_ip_example" login="login_example" name="myfence" passwd="password_example"/></fencedevices>

When configuring fence devices for a cluster, you may find it useful to see a listing of available devices for your cluster and the options available for each device. You may also find it useful to see a listing of fence devices currently configured for your cluster. For information on using the ccs command to print a list of available fence devices and options or to print a list of fence devices currently configured for your cluster, refer to Section 5.6, "Listing Fence Devices and Fence Device Options".

To remove a fence device from your cluster configuration, execute the following command:

ccs -h host --rmfencedev fence_device_name

For example, to remove a fence device that you have named myfence from the cluster configuration file on cluster node node1, execute the following command:

ccs -h node1 --rmfencedev myfence

If you need to modify the attributes of a fence device you have already configured, you must first remove that fence device then add it again with the modified attributes.

Note that when you have finished configuring all of the components of your cluster, you will need to sync the cluster configuration file to all of the nodes, as described in Section 5.15, "Propagating the Configuration File to the Cluster Nodes".

5.6. Listing Fence Devices and Fence Device Options

You can use the ccs command to print a list of available fence devices and to print a list of options for each available fence type. You can also use the ccs command to print a list of fence devices currently configured for your cluster.

To print a list of fence devices currently available for your cluster, execute the following command:

ccs -h host --lsfenceopts

For example, the following command lists the fence devices available on the cluster node node1, showing sample output.

[root@ask-03 ~]# ccs -h node1 --lsfenceoptsfence_rps10 - RPS10 Serial Switchfence_vixel - No description availablefence_egenera - No description availablefence_xcat - No description availablefence_na - Node Assassinfence_apc - Fence agent for APC over telnet/sshfence_apc_snmp - Fence agent for APC over SNMPfence_bladecenter - Fence agent for IBM BladeCenterfence_bladecenter_snmp - Fence agent for IBM BladeCenter over SNMPfence_cisco_mds - Fence agent for Cisco MDSfence_cisco_ucs - Fence agent for Cisco UCSfence_drac5 - Fence agent for Dell DRAC CMC/5fence_eps - Fence agent for ePowerSwitchfence_ibmblade - Fence agent for IBM BladeCenter over SNMPfence_ifmib - Fence agent for IF MIBfence_ilo - Fence agent for HP iLOfence_ilo_mp - Fence agent for HP iLO MPfence_intelmodular - Fence agent for Intel Modularfence_ipmilan - Fence agent for IPMI over LANfence_kdump - Fence agent for use with kdumpfence_rhevm - Fence agent for RHEV-M REST APIfence_rsa - Fence agent for IBM RSAfence_sanbox2 - Fence agent for QLogic SANBox2 FC switchesfence_scsi - fence agent for SCSI-3 persistent reservationsfence_virsh - Fence agent for virshfence_virt - Fence agent for virtual machinesfence_vmware - Fence agent for VMwarefence_vmware_soap - Fence agent for VMware over SOAP APIfence_wti - Fence agent for WTIfence_xvm - Fence agent for virtual machines

To print a list of the options you can specify for a particular fence type, execute the following command:

ccs -h host --lsfenceopts fence_type

For example, the following command lists the fence options for the fence_wti fence agent.

[root@ask-03 ~]# ccs -h node1 --lsfenceopts fence_wtifence_wti - Fence agent for WTI  Required Options:  Optional Options: option: No description available action: Fencing Action ipaddr: IP Address or Hostname login: Login Name passwd: Login password or passphrase passwd_script: Script to retrieve password cmd_prompt: Force command prompt secure: SSH connection identity_file: Identity file for ssh port: Physical plug number or name of virtual machine inet4_only: Forces agent to use IPv4 addresses only inet6_only: Forces agent to use IPv6 addresses only ipport: TCP port to use for connection with device verbose: Verbose mode debug: Write debug information to given file version: Display version information and exit help: Display help and exit separator: Separator for CSV created by operation list power_timeout: Test X seconds for status change after ON/OFF shell_timeout: Wait X seconds for cmd prompt after issuing command login_timeout: Wait X seconds for cmd prompt after login power_wait: Wait X seconds after issuing ON/OFF delay: Wait X seconds before fencing is started retry_on: Count of attempts to retry power on

To print a list of fence devices currently configured for your cluster, execute the following command:

ccs -h host --lsfencedev

5.7. Configuring Fencing for Cluster Members

Once you have completed the initial steps of creating a cluster and creating fence devices, you need to configure fencing for the cluster nodes. To configure fencing for the nodes after creating a new cluster and configuring the fencing devices for the cluster, follow the steps in this section. Note that you must configure fencing for each node in the cluster.

This section documents the following procedures:

5.7.1. Configuring a Single Power-Based Fence Device for a Node

Use the following procedure to configure a node with a single power-based fence device that uses a fence device named apc, which uses the fence_apc fencing agent.

Add a fence method for the node, providing a name for the fence method.
```
ccs -h host --addmethod method node
```
For example, to configure a fence method named APC for the node node-01.example.com in the configuration file on the cluster node node-01.example.com, execute the following command:
```
ccs -h node01.example.com --addmethod APC node01.example.com
```
Add a fence instance for the method. You must specify the fence device to use for the node, the node this instance applies to, the name of the method, and any options for this method that are specific to this node:
```
ccs -h host --addfenceinst fencedevicename node method [options]
```
For example, to configure a fence instance in the configuration file on the cluster node node-01.example.com that uses the APC switch power port 1 on the fence device named apc to fence cluster node node-01.example.com using the method named APC, execute the following command:
```
ccs -h node01.example.com --addfenceinst apc node01.example.com APC port=1
```

You will need to add a fence method for each node in the cluster. The following commands configure a fence method for each node with the method name APC. The device for the fence method specifies apc as the device name, which is a device previously configured with the --addfencedev option, as described in Section 5.5, "Configuring Fence Devices". Each node is configured with a unique APC switch power port number: The port number for node-01.example.com is 1, the port number for node-02.example.com is 2, and the port number for node-03.example.com is 3.

ccs -h node01.example.com --addmethod APC node01.example.comccs -h node01.example.com --addmethod APC node02.example.comccs -h node01.example.com --addmethod APC node03.example.comccs -h node01.example.com --addfenceinst apc node01.example.com APC port=1ccs -h node01.example.com --addfenceinst apc node02.example.com APC port=2ccs -h node01.example.com --addfenceinst apc node03.example.com APC port=3

Example 5.2, "cluster.conf After Adding Power-Based Fence Methods " shows a cluster.conf configuration file after you have added these fencing methods and instances to each node in the cluster.

Example 5.2. cluster.conf After Adding Power-Based Fence Methods

<cluster name="mycluster" config_version="3">   <clusternodes> <clusternode name="node-01.example.com" nodeid="1"> <fence> <method name="APC">  <device name="apc" port="1"/> </method> </fence> </clusternode> <clusternode name="node-02.example.com" nodeid="2"> <fence> <method name="APC">  <device name="apc" port="2"/> </method> </fence> </clusternode> <clusternode name="node-03.example.com" nodeid="3"> <fence> <method name="APC">  <device name="apc" port="3"/> </method> </fence> </clusternode>   </clusternodes>   <fencedevices> <fencedevice agent="fence_apc" ipaddr="apc_ip_example" login="login_example" name="apc" passwd="password_example"/>   </fencedevices>   <rm>   </rm></cluster>

5.7.2. Configuring a Single Storage-Based Fence Device for a Node

When using non-power fencing methods (that is, SAN/storage fencing) to fence a node, you must configure unfencing for the fence device. This ensures that a fenced node is not re-enabled until the node has been rebooted. When you configure unfencing for a node, you specify a device that mirrors the corresponding fence device you have configured for the node with the notable addition of the explicit action of on or enable.

For more information about unfencing a node, refer to the fence_node(8) man page.

Use the following procedure to configure a node with a single storage-based fence device that uses a fence device named sanswitch1, which uses the fence_sanbox2 fencing agent.

Add a fence method for the node, providing a name for the fence method.
```
ccs -h host --addmethod method node
```
For example, to configure a fence method named SAN for the node node-01.example.com in the configuration file on the cluster node node-01.example.com, execute the following command:
```
ccs -h node01.example.com --addmethod SAN  node01.example.com
```
Add a fence instance for the method. You must specify the fence device to use for the node, the node this instance applies to, the name of the method, and any options for this method that are specific to this node:
```
ccs -h host --addfenceinst fencedevicename node method [options]
```
For example, to configure a fence instance in the configuration file on the cluster node node-01.example.com that uses the SAN switch power port 11 on the fence device named sanswitch1 to fence cluster node node-01.example.com using the method named SAN, execute the following command:
```
ccs -h node01.example.com --addfenceinst sanswitch1 node01.example.com SAN port=11
```
To configure unfencing for the storage based fence device on this node, execute the following command:
```
ccs -h host --addunfence fencedevicename node action=on|off
```

You will need to add a fence method for each node in the cluster. The following commands configure a fence method for each node with the method name SAN. The device for the fence method specifies sanswitch as the device name, which is a device previously configured with the --addfencedev option, as described in Section 5.5, "Configuring Fence Devices". Each node is configured with a unique SAN physical port number: The port number for node-01.example.com is 11, the port number for node-02.example.com is 12, and the port number for node-03.example.com is 13.

ccs -h node01.example.com --addmethod SAN node01.example.comccs -h node01.example.com --addmethod SAN node02.example.comccs -h node01.example.com --addmethod SAN node03.example.comccs -h node01.example.com --addfenceinst sanswitch1 node01.example.com SAN port=11ccs -h node01.example.com --addfenceinst sanswitch1 node02.example.com SAN port=12ccs -h node01.example.com --addfenceinst sanswitch1 node03.example.com SAN port=13ccs -h node01.example.com --addunfence sanswitch1 node01.example.com port=11 action=onccs -h node01.example.com --addunfence sanswitch1 node02.example.com port=12 action=onccs -h node01.example.com --addunfence sanswitch1 node03.example.com port=13 action=on

Example 5.3, "cluster.conf After Adding Storage-Based Fence Methods " shows a cluster.conf configuration file after you have added fencing methods, fencing instances, and unfencing to each node in the cluster.

Example 5.3. cluster.conf After Adding Storage-Based Fence Methods

<cluster name="mycluster" config_version="3">   <clusternodes> <clusternode name="node-01.example.com" nodeid="1"> <fence> <method name="SAN">  <device name="sanswitch1" port="11"/> </method> </fence> <unfence> <device name="sanswitch1" port="11" action="on"/>  </unfence> </clusternode> <clusternode name="node-02.example.com" nodeid="2"> <fence> <method name="SAN">  <device name="sanswitch1" port="12"/> </method> </fence> <unfence> <device name="sanswitch1" port="12" action="on"/>  </unfence> </clusternode> <clusternode name="node-03.example.com" nodeid="3"> <fence> <method name="SAN">  <device name="sanswitch1" port="13"/> </method> </fence> <unfence> <device name="sanswitch1" port="13" action="on"/>  </unfence> </clusternode>   </clusternodes>   <fencedevices> <fencedevice agent="fence_sanbox2" ipaddr="san_ip_example"login="login_example" name="sanswitch1" passwd="password_example"/>   </fencedevices>   <rm>   </rm></cluster>

5.7.3. Configuring a Backup Fence Device

You can define multiple fencing methods for a node. If fencing fails using the first method, the system will attempt to fence the node using the second method, followed by any additional methods you have configured. To configure a backup fencing method for a node, you configure two methods for a node, configuring a fence instance for each method.

Note

The order in which the system will use the fencing methods you have configured follows their order in the cluster configuration file. The first method you configure with the ccs command is the primary fencing method, and the second method you configure is the backup fencing method. To change the order, you can remove the primary fencing method from the configuration file, then add that method back.

Note that at any time you can print a list of fence methods and instances currently configured for a node by executing the following command. If you do not specify a node, this command will list the fence methods and instances currently configured for all nodes.

ccs -h host --lsfenceinst [node]

Use the following procedure to configure a node with a primary fencing method that uses a fence device named apc, which uses the fence_apc fencing agent, and a backup fencing device that uses a fence device named sanswitch1, which uses the fence_sanbox2 fencing agent. Since the sanswitch1 device is a storage-based fencing agent, you will need to configure unfencing for that device as well.

Add a primary fence method for the node, providing a name for the fence method.
```
ccs -h host --addmethod method node
```
For example, to configure a fence method named APC as the primary method for the node node-01.example.com in the configuration file on the cluster node node-01.example.com, execute the following command:
```
ccs -h node01.example.com --addmethod APC node01.example.com
```
Add a fence instance for the primary method. You must specify the fence device to use for the node, the node this instance applies to, the name of the method, and any options for this method that are specific to this node:
```
ccs -h host --addfenceinst fencedevicename node method [options]
```
For example, to configure a fence instance in the configuration file on the cluster node node-01.example.com that uses the APC switch power port 1 on the fence device named apc to fence cluster node node-01.example.com using the method named APC, execute the following command:
```
ccs -h node01.example.com --addfenceinst apc node01.example.com APC port=1
```
Add a backup fence method for the node, providing a name for the fence method.
```
ccs -h host --addmethod method node
```
For example, to configure a backup fence method named SAN for the node node-01.example.com in the configuration file on the cluster node node-01.example.com, execute the following command:
```
ccs -h node01.example.com --addmethod SAN  node01.example.com
```
Add a fence instance for the backup method. You must specify the fence device to use for the node, the node this instance applies to, the name of the method, and any options for this method that are specific to this node:
```
ccs -h host --addfenceinst fencedevicename node method [options]
```
For example, to configure a fence instance in the configuration file on the cluster node node-01.example.com that uses the SAN switch power port 11 on the fence device named sanswitch1 to fence cluster node node-01.example.com using the method named SAN, execute the following command:
```
ccs -h node01.example.com --addfenceinst sanswitch1 node01.example.com SAN port=11
```
Since the sanswitch1 device is a storage-based device, you must configure unfencing for this device.
```
ccs -h node01.example.com --addunfence sanswitch1 node01.example.com port=11 action=on
```

You can continue to add fencing methods as needed.

This procedure configures a fence device and a backup fence device for one node in the cluster. You will need to configure fencing for the other nodes in the cluster as well.

Example 5.4, "cluster.conf After Adding Backup Fence Methods " shows a cluster.conf configuration file after you have added a power-based primary fencing method and a storage-based backup fencing method to each node in the cluster.

Example 5.4. cluster.conf After Adding Backup Fence Methods

<cluster name="mycluster" config_version="3">   <clusternodes> <clusternode name="node-01.example.com" nodeid="1"> <fence> <method name="APC">  <device name="apc" port="1"/> </method> <method name="SAN">  <device name="sanswitch1" port="11"/> </method> </fence> <unfence> <device name="sanswitch1" port="11" action="on"/>  </unfence </clusternode> <clusternode name="node-02.example.com" nodeid="2"> <fence> <method name="APC">  <device name="apc" port="2"/> </method> <method name="SAN">  <device name="sanswitch1" port="12"/> </method> </fence> <unfence> <device name="sanswitch1" port="12" action="on"/>  </unfence </clusternode> <clusternode name="node-03.example.com" nodeid="3"> <fence> <method name="APC">  <device name="apc" port="3"/> </method> <method name="SAN">  <device name="sanswitch1" port="13"/> </method> </fence> <unfence> <device name="sanswitch1" port="13" action="on"/>  </unfence </clusternode>   </clusternodes>   <fencedevices> <fencedevice agent="fence_apc" ipaddr="apc_ip_example" login="login_example" name="apc" passwd="password_example"/> <fencedevice agent="fence_sanbox2" ipaddr="san_ip_example" login="login_example" name="sanswitch1" passwd="password_example"/>   </fencedevices>   <rm>   </rm></cluster>

Note

The order in which the system will use the fencing methods you have configured follows their order in the cluster configuration file. The first method you configure is the primary fencing method, and the second method you configure is the backup fencing method. To change the order, you can remove the primary fencing method from the configuration file, then add that method back.

5.7.4. Configuring a Node with Redundant Power

If your cluster is configured with redundant power supplies for your nodes, you must be sure to configure fencing so that your nodes fully shut down when they need to be fenced. If you configure each power supply as a separate fence method, each power supply will be fenced separately; the second power supply will allow the system to continue running when the first power supply is fenced and the system will not be fenced at all. To configure a system with dual power supplies, you must configure your fence devices so that both power supplies are shut off and the system is taken completely down. This requires that you configure two instances within a single fencing method, and that for each instance you configure both fence devices with an action attribute of off before configuring each of the devices with an action attribute of on.

To configure fencing for a node with dual power supplies, follow the steps in this section.

Before you can configure fencing for a node with redundant power, you must configure each of the power switches as a fence device for the cluster. For information on configuring fence devices, see Section 5.5, "Configuring Fence Devices".
To print a list of fence devices currently configured for your cluster, execute the following command:
```
ccs -h host --lsfencedev
```
Add a fence method for the node, providing a name for the fence method.
```
ccs -h host --addmethod method node
```
For example, to configure a fence method named APC-dual for the node node-01.example.com in the configuration file on the cluster node node-01.example.com, execute the following command:
```
ccs -h node01.example.com --addmethod APC-dual node01.example.com
```
Add a fence instance for the first power supply to the fence method. You must specify the fence device to use for the node, the node this instance applies to, the name of the method, and any options for this method that are specific to this node. At this point you configure the action attribute as off.
```
ccs -h host --addfenceinst fencedevicename node method [options] action=off
```
For example, to configure a fence instance in the configuration file on the cluster node node-01.example.com that uses the APC switch power port 1 on the fence device named apc1 to fence cluster node node-01.example.com using the method named APC-dual, and setting the action attribute to off, execute the following command:
```
ccs -h node01.example.com --addfenceinst apc1 node01.example.com APC-dual port=1 action=off
```
Add a fence instance for the second power supply to the fence method. You must specify the fence device to use for the node, the node this instance applies to, the name of the method, and any options for this method that are specific to this node. At this point you configure the action attribute as off for this instance as well:
```
ccs -h host --addfenceinst fencedevicename node method [options] action=off
```
For example, to configure a second fence instance in the configuration file on the cluster node node-01.example.com that uses the APC switch power port 1 on the fence device named apc2 to fence cluster node node-01.example.com using the same method as you specified for the first instance named APC-dual, and setting the action attribute to off, execute the following command:
```
ccs -h node01.example.com --addfenceinst apc2 node01.example.com APC-dual port=1 action=off
```
At this point, add another fence instance for the first power supply to the fence method, configuring the action attribute as on. You must specify the fence device to use for the node, the node this instance applies to, the name of the method, and any options for this method that are specific to this node, and specifying the action attribute as on:
```
ccs -h host --addfenceinst fencedevicename node method [options] action=on
```
For example, to configure a fence instance in the configuration file on the cluster node node-01.example.com that uses the APC switch power port 1 on the fence device named apc1 to fence cluster node node-01.example.com using the method named APC-dual, and setting the action attribute to on, execute the following command:
```
ccs -h node01.example.com --addfenceinst apc1 node01.example.com APC-dual port=1 action=on
```
Add another fence instance for second power supply to the fence method, specifying the action attribute as on for this instance. You must specify the fence device to use for the node, the node this instance applies to, the name of the method, and any options for this method that are specific to this node as well as the action attribute of on.
```
ccs -h host --addfenceinst fencedevicename node method [options] action=on
```
For example, to configure a second fence instance in the configuration file on the cluster node node-01.example.com that uses the APC switch power port 1 on the fence device named apc2 to fence cluster node node-01.example.com using the same method as you specified for the first instance named APC-dual and setting the action attribute to on, execute the following command:
```
ccs -h node01.example.com --addfenceinst apc2 node01.example.com APC-dual port=1 action=on
```

Example 5.5, "cluster.conf After Adding Dual-Power Fencing " shows a cluster.conf configuration file after you have added fencing for two power supplies for each node in a cluster.

Example 5.5. cluster.conf After Adding Dual-Power Fencing

<cluster name="mycluster" config_version="3">   <clusternodes> <clusternode name="node-01.example.com" nodeid="1"> <fence> <method name="APC-dual">  <device name="apc1" port="1"action="off"/>  <device name="apc2" port="1"action="off"/>  <device name="apc1" port="1"action="on"/>  <device name="apc2" port="1"action="on"/> </method> </fence> </clusternode> <clusternode name="node-02.example.com" nodeid="2"> <fence> <method name="APC-dual">  <device name="apc1" port="2"action="off"/>  <device name="apc2" port="2"action="off"/>  <device name="apc1" port="2"action="on"/>  <device name="apc2" port="2"action="on"/> </method> </fence> </clusternode> <clusternode name="node-03.example.com" nodeid="3"> <fence> <method name="APC-dual">  <device name="apc1" port="3"action="off"/>  <device name="apc2" port="3"action="off"/>  <device name="apc1" port="3"action="on"/>  <device name="apc2" port="3"action="on"/> </method> </fence> </clusternode>   </clusternodes>   <fencedevices>   <fencedevice agent="fence_apc" ipaddr="apc_ip_example" login="login_example" name="apc1" passwd="password_example"/>   <fencedevice agent="fence_apc" ipaddr="apc_ip_example" login="login_example" name="apc2" passwd="password_example"/>   </fencedevices>   <rm>   </rm></cluster>

5.7.5. Removing Fence Methods and Fence Instances

To remove a fence method from your cluster configuration, execute the following command:

ccs -h host --rmmethod method node

For example, to remove a fence method that you have named APC that you have configured for node01.example.com from the cluster configuration file on cluster node node01.example.com, execute the following command:

ccs -h node01.example.com  --rmmethod APC node01.example.com

To remove all fence instances of a fence device from a fence method, execute the following command:

ccs -h host --rmfenceinst fencedevicename node method

For example, to remove all instances of the fence device named apc1 from the method named APC-dual configured for node01.example.com from the cluster configuration file on cluster node node01.example.com, execute the following command:

ccs -h node01.example.com --rmfenceinst apc1 node01.example.com APC-dual

5.8. Configuring a Failover Domain

A failover domain is a named subset of cluster nodes that are eligible to run a cluster service in the event of a node failure. A failover domain can have the following characteristics:

Unrestricted - Allows you to specify that a subset of members are preferred, but that a cluster service assigned to this domain can run on any available member.
Restricted - Allows you to restrict the members that can run a particular cluster service. If none of the members in a restricted failover domain are available, the cluster service cannot be started (either manually or by the cluster software).
Unordered - When a cluster service is assigned to an unordered failover domain, the member on which the cluster service runs is chosen from the available failover domain members with no priority ordering.
Ordered - Allows you to specify a preference order among the members of a failover domain. The member at the top of the list is the most preferred, followed by the second member in the list, and so on.
Failback - Allows you to specify whether a service in the failover domain should fail back to the node that it was originally running on before that node failed. Configuring this characteristic is useful in circumstances where a node repeatedly fails and is part of an ordered failover domain. In that circumstance, if a node is the preferred node in a failover domain, it is possible for a service to fail over and fail back repeatedly between the preferred node and another node, causing severe impact on performance.
Note
The failback characteristic is applicable only if ordered failover is configured.

Note

Changing a failover domain configuration has no effect on currently running services.

Note

Failover domains are not required for operation.

By default, failover domains are unrestricted and unordered.

In a cluster with several members, using a restricted failover domain can minimize the work to set up the cluster to run a cluster service (such as httpd), which requires you to set up the configuration identically on all members that run the cluster service. Instead of setting up the entire cluster to run the cluster service, you can set up only the members in the restricted failover domain that you associate with the cluster service.

Note

To configure a preferred member, you can create an unrestricted failover domain comprising only one cluster member. Doing that causes a cluster service to run on that cluster member primarily (the preferred member), but allows the cluster service to fail over to any of the other members.

To configure a failover domain, perform the following procedure:

To add a failover domain, execute the following command:
```
ccs -h host --addfailoverdomain name [restricted] [ordered] [nofailback]
```
Note
The name should be descriptive enough to distinguish its purpose relative to other names used in your cluster.
For example, the following command configures a failover domain named example_pri on node-01.example.com that is unrestricted, ordered, and allows failback:
```
ccs -h node-01.example.com --addfailoverdomain example_pri ordered
```
To add a node to a failover domain, execute the following command:
```
ccs -h host --addfailoverdomainnode failoverdomain node priority
```
For example, to configure the failover domain example_pri in the configuration file on node-01.example.com so that it contains node-01.example.com with a priority of 1, node-02.example.com with a priority of 2, and node-03.example.com with a priority of 3, execute the following commands:
```
ccs -h node-01.example.com --addfailoverdomainnode example_pri node-01.example.com 1ccs -h node-01.example.com --addfailoverdomainnode example_pri node-02.example.com 2ccs -h node-01.example.com --addfailoverdomainnode example_pri node-03.example.com 3
```

You can list all of the failover domains and failover domain nodes configured in a cluster with the following command:

ccs -h host --lsfailoverdomain

To remove a failover domain, execute the following command:

ccs -h host --rmfailoverdomain name

To remove a node from a failover domain, execute the following command:

ccs -h host --rmfailoverdomainnode failoverdomain node

5.9. Configuring Global Cluster Resources

You can configure two types of resources:

Global - Resources that are available to any service in the cluster.
Service-specific - Resources that are available to only one service.

To see a list of currently configured resources and services in the cluster, execute the following command:

ccs -h host --lsservices

To add a global cluster resource, execute the following command. You can add a resource that is local to a particular service when you configure the service, as described in Section 5.10, "Adding a Cluster Service to the Cluster".

ccs -h host --addresource resourcetype [resource options]

For example, the following command adds a global file system resource to the cluster configuration file on node01.example.com. The name of the resource is web_fs, the file system device is /dev/sdd2, the file system mountpoint is /var/www, and the file system type is ext3.

ccs -h node01.example.com --addresource fs name=web_fs device=/dev/sdd2 mountpoint=/var/www fstype=ext3

For information about the available resource types and resource options, see Appendix B, HA Resource Parameters.

To remove a global resource, execute the following command:

ccs -h host --rmresource resourcetype [resource options]

If you need to modify the parameters of an existing global resource, you can remove the resource and configure it again.

5.10. Adding a Cluster Service to the Cluster

To configure a cluster service in a cluster, perform the following steps:

Add a service to the cluster with the following command:
```
ccs -h host --addservice servicename [service options]
```
Note
Use a descriptive name that clearly distinguishes the service from other services in the cluster.
When you add a service to the cluster configuration, you configure the following attributes:
- autostart - Specifies whether to autostart the service when the cluster starts. Use "1" to enable and "0" to disable; the default is enabled.
- domain - Specifies a failover domain (if required).
- exclusive - Specifies a policy wherein the service only runs on nodes that have no other services running on them.
- recovery - Specifies a recovery policy for the service. The options are to relocate, restart, disable, or restart-disable the service. The restart recovery policy indicates that the system should attempt to restart the failed service before trying to relocate the service to another node. The relocate policy indicates that the system should try to restart the service in a different node. The disable policy indicates that the system should disable the resource group if any component fails. The restart-disable policy indicates that the system should attempt to restart the service in place if it fails, but if restarting the service fails the service will be disabled instead of being moved to another host in the cluster.
  If you select Restart or Restart-Disable as the recovery policy for the service, you can specify the maximum number of restart failures before relocating or disabling the service, and you can specify the length of time in seconds after which to forget a restart.
For example, to add a service to the configuration file on the cluster node node-01.example.com named example_apache that uses the failover domain example_pri, and that has recovery policy of relocate, execute the following command:
```
ccs -h node-01.example.com --addservice example_apache domain=example_pri recovery=relocate
```
When configuring services for a cluster, you may find it useful to see a listing of available services for your cluster and the options available for each service. For information on using the ccs command to print a list of available services and their options, refer to Section 5.11, "Listing Available Cluster Services".
Add resources to the service with the following command:
```
ccs -h host --addsubservice servicename subservice [service options]
```
Depending on the type of resources you want to use, populate the service with global or service-specific resources. To add a global resource, use the --addsubservice option of the ccs to add a resource. For example, to add the global file system resource named web_fs to the service named example_apache on the cluster configuration file on node-01.example.com, execute the following command:
```
ccs -h node01.example.com --addsubservice example_apache fs ref=web_fs
```
To add a service-specific resource to the service, you need to specify all of the service options. For example, if you had not previously defined web_fs as a global service, you could add it as a service-specific resource with the following command:
```
ccs -h node01.example.com --addsubservice example_apache fs name=web_fs device=/dev/sdd2 mountpoint=/var/www fstype=ext3
```
To add a child service to the service, you also use the --addsubservice option of the ccs command, specifying the service options.
If you need to add services within a tree structure of dependencies, use a colon (":") to separate elements and brackets to identify subservices of the same type. The following example adds a third nfsclient service as a subservice of an nfsclient service which is in itself a subservice of an nfsclient service which is a subservice of a service named service_a:
```
ccs -h node01.example.com --addsubservice service_a nfsclient[1]:nfsclient[2]:nfsclient
```
Note
If you are adding a Samba-service resource, add it directly to the service, not as a child of another resource.

Note

To verify the existence of the IP service resource used in a cluster service, you can use the /sbin/ip addr show command on a cluster node (rather than the obsoleted ifconfig command). The following output shows the /sbin/ip addr show command executed on a node running a cluster service:

1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP> mtu 1356 qdisc pfifo_fast qlen 1000 link/ether 00:05:5d:9a:d8:91 brd ff:ff:ff:ff:ff:ff inet 10.11.4.31/22 brd 10.11.7.255 scope global eth0 inet6 fe80::205:5dff:fe9a:d891/64 scope link inet 10.11.4.240/22 scope global secondary eth0   valid_lft forever preferred_lft forever

To remove a service and all of its subservices, execute the following command:

ccs -h host --rmservice servicename

To remove a subservice, execute the following command:

ccs -h host --rmsubservice servicename subservice [service options]

5.11. Listing Available Cluster Services

You can use the ccs command to print a list of services that are available for a cluster. You can also use the ccs command to print a list of the options you can specify for a particular service type.

To print a list of cluster services currently available for your cluster, execute the following command:

ccs -h host --lsserviceopts

For example, the following command lists the cluster services available on the cluster node node1, showing sample output.

[root@ask-03 ~]# ccs -h node1 --lsserviceoptsservice - Defines a service (resource group).ASEHAagent - Sybase ASE Failover InstanceSAPDatabase - SAP database resource agentSAPInstance - SAP instance resource agentapache - Defines an Apache web serverclusterfs - Defines a cluster file system mount.fs - Defines a file system mount.ip - This is an IP address.lvm - LVM Failover scriptmysql - Defines a MySQL database servernamed - Defines an instance of named servernetfs - Defines an NFS/CIFS file system mount.nfsclient - Defines an NFS client.nfsexport - This defines an NFS export.nfsserver - This defines an NFS server resource.openldap - Defines an Open LDAP serveroracledb - Oracle 10g Failover Instanceorainstance - Oracle 10g Failover Instanceoralistener - Oracle 10g Listener Instancepostgres-8 - Defines a PostgreSQL serversamba - Dynamic smbd/nmbd resource agentscript - LSB-compliant init script as a clustered resource.tomcat-6 - Defines a Tomcat servervm - Defines a Virtual Machineaction - Overrides resource action timings for a resource instance.

To print a list of the options you can specify for a particular service type, execute the following command:

ccs -h host --lsserviceopts service_type

For example, the following command lists the service options for the vm service.

[root@ask-03 ~]# ccs -f node1 --lsserviceopts vmvm - Defines a Virtual Machine  Required Options: name: Name  Optional Options: domain: Cluster failover Domain autostart: Automatic start after quorum formation exclusive: Exclusive resource group recovery: Failure recovery policy migration_mapping: memberhost:targethost,memberhost:targethost .. use_virsh: If set to 1, vm.sh will use the virsh command to manage virtual machines instead of xm. This is required when using non-Xen virtual machines (e.g. qemu / KVM). xmlfile: Full path to libvirt XML file describing the domain. migrate: Migration type (live or pause, default = live). path: Path to virtual machine configuration files. snapshot: Path to the snapshot directory where the virtual machine image will be stored. depend: Top-level service this depends on, in service:name format. depend_mode: Service dependency mode (soft or hard). max_restarts: Maximum restarts for this service. restart_expire_time: Restart expiration time; amount of time before a restart is forgotten. status_program: Additional status check program hypervisor: Hypervisor hypervisor_uri: Hypervisor URI (normally automatic). migration_uri: Migration URI (normally automatic). __independent_subtree: Treat this and all children as an independent subtree. __enforce_timeouts: Consider a timeout for operations as fatal. __max_failures: Maximum number of failures before returning a failure to a status check. __failure_expire_time: Amount of time before a failure is forgotten. __max_restarts: Maximum number restarts for an independent subtree before giving up. __restart_expire_time: Amount of time before a failure is forgotten for an independent subtree.

5.12. Virtual Machine Resources

Virtual machine resources are configured differently than other cluster resources. In particular, they are not grouped into service definitions. As of the Red Hat Enterprise Linux 6.2 release, when you configure a virtual machine in a cluster with the ccs command you can use the --addvm (rather than the addservice option). This ensures that the vm resource is defined directly under the rm configuration node in the cluster configuration file.

A virtual machine resource requires at least a name and a path attribute. The name attribute should match the name of the libvirt domain and the path attribute should specify the directory where the shared virtual machine definitions are stored.

Note

The path attribute in the cluster configuration file is a path specification or a directory name, not a path to an individual file.

If virtual machine definitions are stored on a shared directory named /mnt/vm_defs, the following command will define a virtual machine named guest1:

# ccs -h node1.example.com --addvm guest1 path=/mnt/vm_defs

Running this command adds the following line to the rm configuration node in the cluster.conf file:

<vm name="guest1" path="/mnt/vm_defs"/>

5.13. Configuring a Quorum Disk

Important

Quorum-disk parameters and heuristics depend on the site environment and the special requirements needed. To understand the use of quorum-disk parameters and heuristics, refer to the qdisk(5) man page. If you require assistance understanding and using quorum disk, contact an authorized Red Hat support representative.

Use the following command to configure your system for using a quorum disk:

ccs -h host --setquorumd [quorumd options]

Note that this command resets all other properties that you can set with the --setquorumd option to their default values, as described in Section 5.1.5, "Commands that Overwrite Previous Settings".

Table 5.1, "Quorum Disk Options" summarizes the meaning of the quorum disk options you may need to set. For a complete list of quorum disk parameters, refer to the cluster schema at /usr/share/cluster/cluster.rng, and the annotated schema at /usr/share/doc/cman-X.Y.ZZ/cluster_conf.html.

Table 5.1. Quorum Disk Options

Parameter	Description
interval	The frequency of read/write cycles, in seconds.
votes	The number of votes the quorum daemon advertises to `cman` when it has a high enough score.
tko	The number of cycles a node must miss to be declared dead.
min_score	The minimum score for a node to be considered "alive". If omitted or set to 0, the default function, `floor((n+1)/2)`, is used, where `n` is the sum of the heuristics scores. The Minimum Score value must never exceed the sum of the heuristic scores; otherwise, the quorum disk cannot be available.
device	The storage device the quorum daemon uses. The device must be the same on all nodes.
label	Specifies the quorum disk label created by the `mkqdisk` utility. If this field contains an entry, the label overrides the Device field. If this field is used, the quorum daemon reads `/proc/partitions` and checks for qdisk signatures on every block device found, comparing the label against the specified label. This is useful in configurations where the quorum device name differs among nodes.

Use the following command to configure the heuristics for a quorum disk:

ccs -h host --addheuristic [heuristic options]

Table 5.2, "Quorum Disk Heuristics" summarizes the meaning of the quorum disk heuristics you may need to set.

Table 5.2. Quorum Disk Heuristics

Parameter	Description
program	The path to the program used to determine if this heuristic is available. This can be anything that can be executed by `/bin/sh -c`. A return value of indicates success; anything else indicates failure. This parameter is required to use a quorum disk.
interval	The frequency (in seconds) at which the heuristic is polled. The default interval for every heuristic is 2 seconds.
score	The weight of this heuristic. Be careful when determining scores for heuristics. The default score for each heuristic is 1.
tko	The number of consecutive failures required before this heuristic is declared unavailable.

To see a list of the quorum disk options and heuristics that are configured on a system, you can execute the following command:

ccs -h host --lsquorum

To remove a heuristic specified by a heuristic option, you can execute the following command:

ccs -h host rmheuristic [heuristic options]

Note

Syncing and activating propagates and activates the updated cluster configuration file. However, for the quorum disk to operate, you must restart the cluster (refer to Section 6.2, "Starting and Stopping a Cluster"), ensuring that you have restarted the qdiskd daemon on each node.

5.14. Miscellaneous Cluster Configuration

This section describes using the ccs command to configure the following:

You can also use the ccs command to set advanced cluster configuration parameters, including totem options, dlm options, rm options and cman options. For information on setting these parameters see the ccs(8) man page and the annotated cluster configuration file schema at /usr/share/doc/cman-X.Y.ZZ/cluster_conf.html.

To view a list of the miscellaneous cluster attributes that have been configured for a cluster, execute the following command:

ccs -h host --lsmisc

5.14.1. Cluster Configuration Version

A cluster configuration file includes a cluster configuration version value. The configuration version value is set to 1 by default when you create a cluster configuration file and it is automatically incremented each time you modify your cluster configuration. However, if you need to set it to another value, you can specify it with the following command:

ccs -h host --setversion n

You can get the current configuration version value on an existing cluster configuration file with the following command:

ccs -h host --getversion

To increment the current configuration version value by 1 in the cluster configuration file on every node in the cluster, execute the following command:

ccs -h host --incversion

5.14.2. Multicast Configuration

If you do not specify a multicast address in the cluster configuration file, the Red Hat High Availability Add-On software creates one based on the cluster ID. It generates the lower 16 bits of the address and appends them to the upper portion of the address according to whether the IP protocol is IPv4 or IPv6:

For IPv4 - The address formed is 239.192. plus the lower 16 bits generated by Red Hat High Availability Add-On software.
For IPv6 - The address formed is FF15:: plus the lower 16 bits generated by Red Hat High Availability Add-On software.

Note

The cluster ID is a unique identifier that cman generates for each cluster. To view the cluster ID, run the cman_tool status command on a cluster node.

You can manually specify a multicast address in the cluster configuration file with the following command:

ccs -h host --setmulticast multicastaddress

Note that this command resets all other properties that you can set with the --setmulticast option to their default values, as described in Section 5.1.5, "Commands that Overwrite Previous Settings".

If you specify a multicast address, you should use the 239.192.x.x series (or FF15:: for IPv6) that cman uses. Otherwise, using a multicast address outside that range may cause unpredictable results. For example, using 224.0.0.x (which is "All hosts on the network") may not be routed correctly, or even routed at all by some hardware.

If you specify or modify a multicast address, you must restart the cluster for this to take effect. For information on starting and stopping a cluster with the ccs command, refer to Section 6.2, "Starting and Stopping a Cluster".

Note

If you specify a multicast address, make sure that you check the configuration of routers that cluster packets pass through. Some routers may take a long time to learn addresses, seriously impacting cluster performance.

To remove a multicast address from a configuration file, use the --setmulticast option of the ccs but do not specify a multicast address:

ccs -h host --setmulticast

5.14.3. Configuring a Two-Node Cluster

If you are configuring a two-node cluster, you can execute the following command to allow a single node to maintain quorum (for example, if one node fails):

ccs -h host --setcman two_node=1 expected_votes=1

Note that this command resets all other properties that you can set with the --setcman option to their default values, as described in Section 5.1.5, "Commands that Overwrite Previous Settings".

When you use the ccs --setcman command to add, remove, or modify the two_node option, you must restart the cluster for this change to take effect. For information on starting and stopping a cluster with the ccs command, refer to Section 6.2, "Starting and Stopping a Cluster".

5.14.4. Logging

You can enable debugging for all daemons in a cluster, or you can enable logging for specific cluster processing.

To enable debugging for all daemons, execute the following command. By default, logging is directed to the /var/log/cluster/daemon.log file.

ccs -h host --setlogging [logging options]

For example, the following command enables debugging for all daemons.

# ccs -h node1.example.com --setlogging debug=on

Note that this command resets all other properties that you can set with the --setlogging option to their default values, as described in Section 5.1.5, "Commands that Overwrite Previous Settings".

To enable debugging for an individual cluster process, execute the following command. Per-daemon logging configuration overrides the global settings.

ccs -h host --addlogging [logging daemon options]

For example, the following commands enable debugging for the corosync and fenced daemons.

# ccs -h node1.example.com --addlogging name=corosync debug=on# ccs -h node1.example.com --addlogging name=fenced debug=on

To remove the log settings for individual daemons, use the following command.

ccs -h host --rmlogging name=clusterprocess

For example, the following command removes the daemon-specific log settings for the fenced daemaon

ccs -h host --rmlogging name=fenced

For a list of the logging daemons for which you can enable logging as well as the additional logging options you can configure for both global and per-daemon logging, refer to the cluster.conf(5) man page.

5.14.5. Configuring Redundant Ring Protocol

As of Red Hat Enterprise Linux 6.4, the Red Hat High Availability Add-On supports the configuration of redundant ring protocol. When using redundant ring protocol, there are a variety of considerations you must take into account, as described in Section 7.6, "Configuring Redundant Ring Protocol".

To specify a second network interface to use for redundant ring protocol, you add an alternate name for the node using the --addalt option of the ccs command:

ccs -h host --addalt node_name alt_name

For example, the following command configures the alternate name clusternet-node1-eth2 for the cluster node clusternet-node1-eth1:

# ccs -h clusternet-node1-eth1 --addalt clusternet-node1-eth1 clusternet-node1-eth2

Optionally, you can manually specify a multicast address, a port, and a TTL for the second ring. If you specify a multicast for the second ring, either the alternate multicast address or the alternate port must be different from the multicast address for the first ring. If you specify an alternate port, the port numbers of the first ring and the second ring must differ by at least two, since the system itself uses port and port-1 to perform operations. If you do not specify an alternate multicast address, the system will automatically use a different multicast address for the second ring.

To specify an alternate multicast address, port, or TTL for the second ring, you use the --setaltmulticast option of the ccs command:

ccs -h host --setaltmulticast [alt_multicast_address] [alt_multicast_options].

For example, the following command sets an alternate multicast address of 239.192.99.88, a port of 888, and a TTL of 3 for the cluster defined in the cluster.conf file on node clusternet-node1-eth1:

ccs -h clusternet-node1-eth1 --setaltmulticast 239.192.99.88 port=888 ttl=3

To remove an alternate multicast address, specify the --setaltmulticast option of the ccs command but do not specify a multicast address. Note that executing this command resets all other properties that you can set with the --setaltmulticast option to their default values, as described in Section 5.1.5, "Commands that Overwrite Previous Settings".

5.15. Propagating the Configuration File to the Cluster Nodes

After you have created or edited a cluster configuration file on one of the nodes in the cluster, you need to propagate that same file to all of the cluster nodes and activate the configuration.

Use the following command to propagate and activate a cluster configuration file:

ccs -h host --sync --activate

To verify that all of the nodes specified in the hosts cluster configuration file have the identical cluster configuration file, execute the following command:

ccs -h host --checkconf

If you have created or edited a configuration file on a local node, use the following command to send that file to one of the nodes in the cluster:

ccs -f file -h host --setconf

To verify that all of the nodes specified in the local file have the identical cluster configuration file, execute the following command:

ccs -f file --checkconf

Cluster Administration

Chapter 5. Configuring Red Hat High Availability Add-On With the ccs Command

5.1. Operational Overview

5.1.1. Creating the Cluster Configuration File on a Local System

5.1.2. Viewing the Current Cluster Configuration

5.1.3. Specifying ricci Passwords with the ccs Command

5.1.4. Modifying Cluster Configuration Components

5.1.5. Commands that Overwrite Previous Settings

5.1.6. Configuration Validation

5.2. Configuration Tasks

5.3. Starting ricci

5.4. Creating A Cluster

5.5. Configuring Fence Devices

5.6. Listing Fence Devices and Fence Device Options

5.7. Configuring Fencing for Cluster Members

5.7.1. Configuring a Single Power-Based Fence Device for a Node

5.7.2. Configuring a Single Storage-Based Fence Device for a Node

5.7.3. Configuring a Backup Fence Device

5.7.4. Configuring a Node with Redundant Power

5.7.5. Removing Fence Methods and Fence Instances

5.8. Configuring a Failover Domain

5.9. Configuring Global Cluster Resources

5.10. Adding a Cluster Service to the Cluster

5.11. Listing Available Cluster Services

5.12. Virtual Machine Resources

5.13. Configuring a Quorum Disk

5.14. Miscellaneous Cluster Configuration

5.14.1. Cluster Configuration Version

5.14.2. Multicast Configuration

5.14.3. Configuring a Two-Node Cluster

5.14.4. Logging

5.14.5. Configuring Redundant Ring Protocol

5.15. Propagating the Configuration File to the Cluster Nodes

Chapter 6. Managing Red Hat High Availability Add-On With ccs

6.1. Managing Cluster Nodes

6.1.1. Causing a Node to Leave or Join a Cluster

6.1.2. Adding a Member to a Running Cluster

6.2. Starting and Stopping a Cluster

6.3. Diagnosing and Correcting Problems in a Cluster