Cari di RHE Linux 
     Red Hat Enterprise Linux Manual
Daftar Isi ☛
(Sebelumnya) 1 : Index - Installation Guide2 : Chapter 4. Configuring Fen ... (Berikutnya)

Fence Configuration Guide

Configuring and Managing Fence Devices for the High Availability Add-On

Edition 0

Legal Notice

Copyright © 2012 Red Hat, Inc. and others.
The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution-Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, JBoss, MetaMatrix, Fedora, the Infinity Logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
All other trademarks are the property of their respective owners.


1801 Varsity Drive
Raleigh, NC 27606-2072 USA
Phone: +1 919 754 3700
Phone: 888 733 4281
Fax: +1 919 754 3701

Daftar Isi

Abstract

Fencing is the disconnection of a node from the cluster's shared storage. Fencing cuts off I/O from shared storage, thus ensuring data integrity. This manual documents the configuration of fencing on clustered systems using High Availability Add-On and details the configuration of supported fence devices.
Preface
1. Document Conventions
1.1. Typographic Conventions
1.2. Pull-quote Conventions
1.3. Notes and Warnings
2. Getting Help and Giving Feedback
2.1. Do You Need Help?
2.2. We Need Feedback!
1. Fencing
2. Fencing Pre-Configuration
2.1. Configuring ACPI For Use with Integrated Fence Devices
2.1.1. Disabling ACPI Soft-Off with chkconfig Management
2.1.2. Disabling ACPI Soft-Off with the BIOS
2.1.3. Disabling ACPI Completely in the grub.conf File
3. Configuring Fencing with the ccs Command
3.1. Configuring Fence Devices
3.2. Listing Fence Devices and Fence Device Options
3.3. Configuring Fencing for Cluster Members
3.3.1. Configuring a Single Power-Based Fence Device for a Node
3.3.2. Configuring a Single Storage-Based Fence Device for a Node
3.3.3. Configuring a Backup Fence Device
3.3.4. Configuring a Node with Redundant Power
3.3.5. Removing Fence Methods and Fence Instances
4. Configuring Fencing with Conga
4.1. Configuring Fence Daemon Properties
4.2. Configuring Fence Devices
4.2.1. Creating a Fence Device
4.2.2. Modifying a Fence Device
4.2.3. Deleting a Fence Device
4.3. Configuring Fencing for Cluster Members
4.3.1. Configuring a Single Fence Device for a Node
4.3.2. Configuring a Backup Fence Device
4.3.3. Configuring a Node with Redundant Power
5. Fence Devices
5.1. APC Power Switch over SNMP
5.2. APC Power Switch over Telnet and SSH
5.3. Brocade Fabric Switch
5.4. Cisco MDS
5.5. Cisco UCS
5.6. Dell Drac 5
5.7. Eaton Network Power Switch
5.8. Egenera BladeFrame
5.9. ePowerSwitch
5.10. Fence Virt
5.11. Fujitsu-Siemens RemoteView Service Board (RSB)
5.12. Hewlett-Packard BladeSystem
5.13. Hewlett-Packard iLO
5.14. Hewlett-Packard iLO MP
5.15. IBM BladeCenter
5.16. IBM BladeCenter over SNMP
5.17. IBM iPDU
5.18. IF-MIB
5.19. Intel Modular
5.20. IPMI over LAN
5.21. RHEV-M REST API
5.22. SCSI Persistent Reservations
5.23. VMWare over SOAP API
5.24. WTI Power Switch
A. Revision History
Index

Preface

1. Document Conventions

This manual uses several conventions to highlight certain words and phrases and draw attention to specific pieces of information.
In PDF and paper editions, this manual uses typefaces drawn from the Liberation Fonts set. The Liberation Fonts set is also used in HTML editions if the set is installed on your system. If not, alternative but equivalent typefaces are displayed. Note: Red Hat Enterprise Linux 5 and later includes the Liberation Fonts set by default.

1.1. Typographic Conventions

Four typographic conventions are used to call attention to specific words and phrases. These conventions, and the circumstances they apply to, are as follows.
Mono-spaced Bold
Used to highlight system input, including shell commands, file names and paths. Also used to highlight keys and key combinations. For example:
To see the contents of the file my_next_bestselling_novel in your current working directory, enter the cat my_next_bestselling_novel command at the shell prompt and press Enter to execute the command.
The above includes a file name, a shell command and a key, all presented in mono-spaced bold and all distinguishable thanks to context.
Key combinations can be distinguished from an individual key by the plus sign that connects each part of a key combination. For example:
Press Enter to execute the command.
Press Ctrl+Alt+F2 to switch to a virtual terminal.
The first example highlights a particular key to press. The second example highlights a key combination: a set of three keys pressed simultaneously.
If source code is discussed, class names, methods, functions, variable names and returned values mentioned within a paragraph will be presented as above, in mono-spaced bold. For example:
File-related classes include filesystem for file systems, file for files, and dir for directories. Each class has its own associated set of permissions.
Proportional Bold
This denotes words or phrases encountered on a system, including application names; dialog box text; labeled buttons; check-box and radio button labels; menu titles and sub-menu titles. For example:
Choose System ⤍ Preferences ⤍ Mouse from the main menu bar to launch Mouse Preferences. In the Buttons tab, click the Left-handed mouse check box and click Close to switch the primary mouse button from the left to the right (making the mouse suitable for use in the left hand).
To insert a special character into a gedit file, choose Applications ⤍ Accessories ⤍ Character Map from the main menu bar. Next, choose Search ⤍ Find . . . . . . from the Character Map menu bar, type the name of the character in the Search field and click Next. The character you sought will be highlighted in the Character Table. Double-click this highlighted character to place it in the Text to copy field and then click the Copy button. Now switch back to your document and choose Edit ⤍ Paste from the gedit menu bar.
The above text includes application names; system-wide menu names and items; application-specific menu names; and buttons and text found within a GUI interface, all presented in proportional bold and all distinguishable by context.
Mono-spaced Bold Italic or Proportional Bold Italic
Whether mono-spaced bold or proportional bold, the addition of italics indicates replaceable or variable text. Italics denotes text you do not input literally or displayed text that changes depending on circumstance. For example:
To connect to a remote machine using ssh, type ssh username@domain.name at a shell prompt. If the remote machine is example.com and your username on that machine is john, type ssh [email protected].
The mount -o remount file-system command remounts the named file system. For example, to remount the /home file system, the command is mount -o remount /home.
To see the version of a currently installed package, use the rpm -q package command. It will return a result as follows: package-version-release.
Note the words in bold italics above - username, domain.name, file-system, package, version and release. Each word is a placeholder, either for text you enter when issuing a command or for text displayed by the system.
Aside from standard usage for presenting the title of a work, italics denotes the first use of a new and important term. For example:
Publican is a DocBook publishing system.

1.2. Pull-quote Conventions

Terminal output and source code listings are set off visually from the surrounding text.
Output sent to a terminal is set in mono-spaced roman and presented thus:
books Desktop   documentation  drafts  mss photos   stuff  svnbooks_tests  Desktop1  downloads  images  notes  scripts  svgs
Source-code listings are also set in mono-spaced roman but add syntax highlighting as follows:
package org.jboss.book.jca.ex1;import javax.naming.InitialContext;public class ExClient{   public static void main(String args[]) throws Exception   {  InitialContext iniCtx = new InitialContext();  Object ref = iniCtx.lookup("EchoBean");  EchoHome   home   = (EchoHome) ref;  Echo   echo   = home.create();  System.out.println("Created Echo");  System.out.println("Echo.echo('Hello') = " + echo.echo("Hello"));   }}

1.3. Notes and Warnings

Finally, we use three visual styles to draw attention to information that might otherwise be overlooked.

Note

Notes are tips, shortcuts or alternative approaches to the task at hand. Ignoring a note should have no negative consequences, but you might miss out on a trick that makes your life easier.

Important

Important boxes detail things that are easily missed: configuration changes that only apply to the current session, or services that need restarting before an update will apply. Ignoring a box labeled 'Important' will not cause data loss but may cause irritation and frustration.

Warning

Warnings should not be ignored. Ignoring warnings will most likely cause data loss.

2. Getting Help and Giving Feedback

2.1. Do You Need Help?

If you experience difficulty with a procedure described in this documentation, visit the Red Hat Customer Portal at http://access.redhat.com. Through the customer portal, you can:
  • search or browse through a knowledgebase of technical support articles about Red Hat products.
  • submit a support case to Red Hat Global Support Services (GSS).
  • access other product documentation.
Red Hat also hosts a large number of electronic mailing lists for discussion of Red Hat software and technology. You can find a list of publicly available mailing lists at https://www.redhat.com/mailman/listinfo. Click on the name of any mailing list to subscribe to that list or to access the list archives.

2.2. We Need Feedback!

If you find a typographical error in this manual, or if you have thought of a way to make this manual better, we would love to hear from you! Please submit a report in Bugzilla: http://bugzilla.redhat.com/ against the product Documentation.
When submitting a bug report, be sure to mention the manual's identifier: Fence_Configuration_Guide
If you have a suggestion for improving the documentation, try to be as specific as possible when describing it. If you have found an error, please include the section number and some of the surrounding text so we can find it easily.

Chapter 1. Fencing

Fencing is the disconnection of a node from the cluster's shared storage. Fencing cuts off I/O from shared storage, thus ensuring data integrity. The cluster infrastructure performs fencing through the fence daemon, fenced.
When CMAN determines that a node has failed, it communicates to other cluster-infrastructure components that the node has failed. fenced, when notified of the failure, fences the failed node. Other cluster-infrastructure components determine what actions to take - that is, they perform any recovery that needs to done. For example, DLM and GFS2, when notified of a node failure, suspend activity until they detect that fenced has completed fencing the failed node. Upon confirmation that the failed node is fenced, DLM and GFS2 perform recovery. DLM releases locks of the failed node; GFS2 recovers the journal of the failed node.
The fencing program determines from the cluster configuration file which fencing method to use. Two key elements in the cluster configuration file define a fencing method: fencing agent and fencing device. The fencing program makes a call to a fencing agent specified in the cluster configuration file. The fencing agent, in turn, fences the node via a fencing device. When fencing is complete, the fencing program notifies the cluster manager.
The High Availability Add-On provides a variety of fencing methods:
  • Power fencing - A fencing method that uses a power controller to power off an inoperable node.
  • storage fencing - A fencing method that disables the Fibre Channel port that connects storage to an inoperable node.
  • Other fencing - Several other fencing methods that disable I/O or power of an inoperable node, including IBM Bladecenters, PAP, DRAC/MC, HP ILO, IPMI, IBM RSA II, and others.
Figure 1.1, "Power Fencing Example" shows an example of power fencing. In the example, the fencing program in node A causes the power controller to power off node D. Figure 1.2, "Storage Fencing Example" shows an example of storage fencing. In the example, the fencing program in node A causes the Fibre Channel switch to disable the port for node D, disconnecting node D from storage.
Power Fencing Example
Power Fencing Example

Figure 1.1. Power Fencing Example


Storage Fencing Example
Storage Fencing Example

Figure 1.2. Storage Fencing Example


Specifying a fencing method consists of editing a cluster configuration file to assign a fencing-method name, the fencing agent, and the fencing device for each node in the cluster.
The way in which a fencing method is specified depends on if a node has either dual power supplies or multiple paths to storage. If a node has dual power supplies, then the fencing method for the node must specify at least two fencing devices - one fencing device for each power supply (refer to Figure 1.3, "Fencing a Node with Dual Power Supplies"). Similarly, if a node has multiple paths to Fibre Channel storage, then the fencing method for the node must specify one fencing device for each path to Fibre Channel storage. For example, if a node has two paths to Fibre Channel storage, the fencing method should specify two fencing devices - one for each path to Fibre Channel storage (refer to Figure 1.4, "Fencing a Node with Dual Fibre Channel Connections").
Fencing a Node with Dual Power Supplies
Fencing a Node with Dual Power Supplies

Figure 1.3. Fencing a Node with Dual Power Supplies


Fencing a Node with Dual Fibre Channel Connections
Fencing a Node with Dual Fibre Channel Connections

Figure 1.4. Fencing a Node with Dual Fibre Channel Connections


You can configure a node with one fencing method or multiple fencing methods. When you configure a node for one fencing method, that is the only fencing method available for fencing that node. When you configure a node for multiple fencing methods, the fencing methods are cascaded from one fencing method to another according to the order of the fencing methods specified in the cluster configuration file. If a node fails, it is fenced using the first fencing method specified in the cluster configuration file for that node. If the first fencing method is not successful, the next fencing method specified for that node is used. If none of the fencing methods is successful, then fencing starts again with the first fencing method specified, and continues looping through the fencing methods in the order specified in the cluster configuration file until the node has been fenced.

Chapter 2. Fencing Pre-Configuration

2.1. Configuring ACPI For Use with Integrated Fence Devices
2.1.1. Disabling ACPI Soft-Off with chkconfig Management
2.1.2. Disabling ACPI Soft-Off with the BIOS
2.1.3. Disabling ACPI Completely in the grub.conf File
This chapter describes tasks to perform and considerations to make before deploying fencing on clusters using Red Hat High Availability Add-On, and consists of the following sections.

2.1. Configuring ACPI For Use with Integrated Fence Devices

If your cluster uses integrated fence devices, you must configure ACPI (Advanced Configuration and Power Interface) to ensure immediate and complete fencing.

Note

For the most current information about integrated fence devices supported by Red Hat High Availability Add-On, refer to http://www.redhat.com/cluster_suite/hardware/.
If a cluster node is configured to be fenced by an integrated fence device, disable ACPI Soft-Off for that node. Disabling ACPI Soft-Off allows an integrated fence device to turn off a node immediately and completely rather than attempting a clean shutdown (for example, shutdown -h now). Otherwise, if ACPI Soft-Off is enabled, an integrated fence device can take four or more seconds to turn off a node (refer to note that follows). In addition, if ACPI Soft-Off is enabled and a node panics or freezes during shutdown, an integrated fence device may not be able to turn off the node. Under those circumstances, fencing is delayed or unsuccessful. Consequently, when a node is fenced with an integrated fence device and ACPI Soft-Off is enabled, a cluster recovers slowly or requires administrative intervention to recover.

Note

The amount of time required to fence a node depends on the integrated fence device used. Some integrated fence devices perform the equivalent of pressing and holding the power button; therefore, the fence device turns off the node in four to five seconds. Other integrated fence devices perform the equivalent of pressing the power button momentarily, relying on the operating system to turn off the node; therefore, the fence device turns off the node in a time span much longer than four to five seconds.
To disable ACPI Soft-Off, use chkconfig management and verify that the node turns off immediately when fenced. The preferred way to disable ACPI Soft-Off is with chkconfig management: however, if that method is not satisfactory for your cluster, you can disable ACPI Soft-Off with one of the following alternate methods:
  • Changing the BIOS setting to "instant-off" or an equivalent setting that turns off the node without delay

    Note

    Disabling ACPI Soft-Off with the BIOS may not be possible with some computers.
  • Appending acpi=off to the kernel boot command line of the /boot/grub/grub.conf file

    Important

    This method completely disables ACPI; some computers do not boot correctly if ACPI is completely disabled. Use this method only if the other methods are not effective for your cluster.
The following sections provide procedures for the preferred method and alternate methods of disabling ACPI Soft-Off:

2.1.1. Disabling ACPI Soft-Off with chkconfig Management

You can use chkconfig management to disable ACPI Soft-Off either by removing the ACPI daemon (acpid) from chkconfig management or by turning off acpid.

Note

This is the preferred method of disabling ACPI Soft-Off.
Disable ACPI Soft-Off with chkconfig management at each cluster node as follows:
  1. Run either of the following commands:
    • chkconfig --del acpid - This command removes acpid from chkconfig management.
      - OR -
    • chkconfig --level 2345 acpid off - This command turns off acpid.
  2. Reboot the node.
  3. When the cluster is configured and running, verify that the node turns off immediately when fenced.

    Note

    You can fence the node with the fence_node command or Conga.

2.1.2. Disabling ACPI Soft-Off with the BIOS

The preferred method of disabling ACPI Soft-Off is with chkconfig management (Section 2.1.1, "Disabling ACPI Soft-Off with chkconfig Management"). However, if the preferred method is not effective for your cluster, follow the procedure in this section.

Note

Disabling ACPI Soft-Off with the BIOS may not be possible with some computers.
You can disable ACPI Soft-Off by configuring the BIOS of each cluster node as follows:
  1. Reboot the node and start the BIOS CMOS Setup Utility program.
  2. Navigate to the Power menu (or equivalent power management menu).
  3. At the Power menu, set the Soft-Off by PWR-BTTN function (or equivalent) to Instant-Off (or the equivalent setting that turns off the node via the power button without delay). Example 2.1, "BIOS CMOS Setup Utility: Soft-Off by PWR-BTTN set to Instant-Off" shows a Power menu with ACPI Function set to Enabled and Soft-Off by PWR-BTTN set to Instant-Off.

    Note

    The equivalents to ACPI Function, Soft-Off by PWR-BTTN, and Instant-Off may vary among computers. However, the objective of this procedure is to configure the BIOS so that the computer is turned off via the power button without delay.
  4. Exit the BIOS CMOS Setup Utility program, saving the BIOS configuration.
  5. When the cluster is configured and running, verify that the node turns off immediately when fenced.

    Note

    You can fence the node with the fence_node command or Conga.

Example 2.1. BIOS CMOS Setup Utility: Soft-Off by PWR-BTTN set to Instant-Off

+------------------------------------------|-----------------+|   ACPI Function [Enabled] |  Item Help  ||   ACPI Suspend Type [S1(POS)] |-----------------|| x Run VGABIOS if S3 Resume  [Auto]   | Menu Level   *  ||   Suspend Mode  [Disabled]   | ||   HDD Power Down [Disabled]   | ||   Soft-Off by PWR-BTTN  [Instant-Off]| ||   CPU THRM-Throttling   [50.0%]  | ||   Wake-Up by PCI card   [Enabled] | ||   Power On by Ring  [Enabled] | ||   Wake Up On LAN [Enabled] | || x USB KB Wake-Up From S3 [Disabled]   | ||   Resume by Alarm   [Disabled]   | || x  Date(of Month) Alarm   0  | || x  Time(hh:mm:ss) Alarm   0 :  0 :   | ||   POWER ON Function [BUTTON ONLY]| || x KB Power ON Password   Enter   | || x Hot Key Power ON   Ctrl-F1 | |+------------------------------------------|-----------------+
This example shows ACPI Function set to Enabled, and Soft-Off by PWR-BTTN set to Instant-Off.

2.1.3. Disabling ACPI Completely in the grub.conf File

The preferred method of disabling ACPI Soft-Off is with chkconfig management (Section 2.1.1, "Disabling ACPI Soft-Off with chkconfig Management"). If the preferred method is not effective for your cluster, you can disable ACPI Soft-Off with the BIOS power management (Section 2.1.2, "Disabling ACPI Soft-Off with the BIOS"). If neither of those methods is effective for your cluster, you can disable ACPI completely by appending acpi=off to the kernel boot command line in the grub.conf file.

Important

This method completely disables ACPI; some computers do not boot correctly if ACPI is completely disabled. Use this method only if the other methods are not effective for your cluster.
You can disable ACPI completely by editing the grub.conf file of each cluster node as follows:
  1. Open /boot/grub/grub.conf with a text editor.
  2. Append acpi=off to the kernel boot command line in /boot/grub/grub.conf (refer to Example 2.2, "Kernel Boot Command Line with acpi=off Appended to It").
  3. Reboot the node.
  4. When the cluster is configured and running, verify that the node turns off immediately when fenced.

    Note

    You can fence the node with the fence_node command or Conga.

Example 2.2. Kernel Boot Command Line with acpi=off Appended to It

# grub.conf generated by anaconda## Note that you do not have to rerun grub after making changes to this file# NOTICE:  You have a /boot partition.  This means that#  all kernel and initrd paths are relative to /boot/, eg.#  root (hd0,0)#  kernel /vmlinuz-version ro root=/dev/mapper/vg_doc01-lv_root #  initrd /initrd-[generic-]version.img#boot=/dev/hdadefault=0timeout=5serial --unit=0 --speed=115200terminal --timeout=5 serial consoletitle Red Hat Enterprise Linux Server (2.6.32-193.el6.x86_64) root (hd0,0) kernel /vmlinuz-2.6.32-193.el6.x86_64 ro root=/dev/mapper/vg_doc01-lv_root console=ttyS0,115200n8 acpi=off initrd /initramrs-2.6.32-131.0.15.el6.x86_64.img
In this example, acpi=off has been appended to the kernel boot command line - the line starting with "kernel /vmlinuz-2.6.32-193.el6.x86_64.img".

Chapter 3. Configuring Fencing with the ccs Command

3.1. Configuring Fence Devices
3.2. Listing Fence Devices and Fence Device Options
3.3. Configuring Fencing for Cluster Members
3.3.1. Configuring a Single Power-Based Fence Device for a Node
3.3.2. Configuring a Single Storage-Based Fence Device for a Node
3.3.3. Configuring a Backup Fence Device
3.3.4. Configuring a Node with Redundant Power
3.3.5. Removing Fence Methods and Fence Instances
As of the Red Hat Enterprise Linux 6.1 release and later, the Red Hat High Availability Add-On provides support for the ccs cluster configuration command. The ccs command allows an administrator to create, modify and view the cluster.conf cluster configuration file. You can use the ccs command to configure a cluster configuration file on a local file system or on a remote node. Using the ccs command, an administrator can also start and stop the cluster services on one or all of the nodes in a configured cluster.
This chapter describes how to configure the Red Hat High Availability Add-On cluster configuration file using the ccs command.
This chapter consists of the following sections:

Important

Make sure that your deployment of High Availability Add-On meets your needs and can be supported. Consult with an authorized Red Hat representative to verify your configuration prior to deployment. In addition, allow time for a configuration burn-in period to test failure modes.

Important

This chapter references commonly used cluster.conf elements and attributes. For a comprehensive list and description of cluster.conf elements and attributes, refer to the cluster schema at /usr/share/cluster/cluster.rng, and the annotated schema at /usr/share/doc/cman-X.Y.ZZ/cluster_conf.html (for example /usr/share/doc/cman-3.0.12/cluster_conf.html).

3.1. Configuring Fence Devices

Configuring fence devices consists of creating, updating, and deleting fence devices for the cluster. You must create and name the fence devices in a cluster before you can configure fencing for the nodes in the cluster. For information on configuring fencing for the individual nodes in the cluster, see Section 3.3, "Configuring Fencing for Cluster Members".
Before configuring your fence devices, you may want to modify some of the fence daemon properties for your system from the default values. The values you configure for the fence daemon are general values for the cluster. The general fencing properties for the cluster you may want to modify are summarized as follows:
  • The post_fail_delay attribute is the number of seconds the fence daemon (fenced) waits before fencing a node (a member of the fence domain) after the node has failed. The post_fail_delay default value is 0. Its value may be varied to suit cluster and network performance.
  • The post-join_delay attribute is the number of seconds the fence daemon (fenced) waits before fencing a node after the node joins the fence domain. The post_join_delay default value is 6. A typical setting for post_join_delay is between 20 and 30 seconds, but can vary according to cluster and network performance.
You reset the values of the post_fail_delay and post_join_delay attributes with the --setfencedaemon option of the ccs command. Note, however, that executing the ccs --setfencedaemon command overwrites all existing fence daemon properties.
For example, to configure a value for the post_fail_delay attribute, execute the following command. This command will overwrite the values of all other exisiting fence daemon properties that you can set with this command.
ccs -h host --setfencedaemon post_fail_delay=value
To configure a value for the post_join_delay attribute, execute the following command. This command will overwrite the values of all other exisiting fence daemon properties that you can set with this command.
ccs -h host --setfencedaemon post_join_delay=value
To configure a value for both the the post_join_delay attribute and the post_fail_delay attribute, execute the following command:
ccs -h host --setfencedaemon post_fail_delay=value post_join_delay=value

Note

For more information about the post_join_delay and post_fail_delay attributes as well as the additional fence daemon properties you can modify, refer to the fenced(8) man page and refer to the cluster schema at /usr/share/cluster/cluster.rng, and the annotated schema at /usr/share/doc/cman-X.Y.ZZ/cluster_conf.html.
To configure a fence device for a cluster, execute the following command:
ccs -h host --addfencedevdevicename[fencedeviceoptions]
For example, to configure an APC fence device in the configuration file on the cluster node node1 named myfence with an IP address of apc_ip_example, a login of login_example, and a password of password_example, execute the following command:
ccs -h node1 --addfencedev myfence agent=fence_apc ipaddr=apc_ip_example login=login_example passwd=password_example
The following example shows the fencedevices section of the cluster.conf configuration file after you have added this APC fence device:
<fencedevices>  <fencedevice agent="fence_apc" ipaddr="apc_ip_example" login="login_example" name="myfence" passwd="password_example"/></fencedevices>
When configuring fence devices for a cluster, you may find it useful to see a listing of available devices for your cluster and the options available for each device. You may also find it useful to see a listing of fence devices currently configured for your cluster. For information on using the ccs command to print a list of available fence devices and options or to print a list of fence devices currently configured for your cluster, refer to Section 3.2, "Listing Fence Devices and Fence Device Options".
To remove a fence device from your cluster configuration, execute the following command:
ccs -h host --rmfencedev fence_device_name
For example, to remove a fence device that you have named myfence from the cluster configuration file on cluster node node1, execute the following command:
ccs -h node1 --rmfencedev myfence
If you need to modify the attributes of a fence device you have already configured, you must first remove that fence device then add it again with the modified attributes.
Note that when you have finished configuring all of the components of your cluster, you will need to sync the cluster configuration file to all of the nodes.

3.2. Listing Fence Devices and Fence Device Options

You can use the ccs command to print a list of available fence devices and to print a list of options for each available fence type. You can also use the ccs command to print a list of fence devices currently configured for your cluster.
To print a list of fence devices currently available for your cluster, execute the following command:
ccs -h host --lsfenceopts
For example, the following command lists the fence devices available on the cluster node node1, showing sample output.
[root@ask-03 ~]# ccs -h node1 --lsfenceoptsfence_rps10 - RPS10 Serial Switchfence_vixel - No description availablefence_egenera - No description availablefence_xcat - No description availablefence_na - Node Assassinfence_apc - Fence agent for APC over telnet/sshfence_apc_snmp - Fence agent for APC over SNMPfence_bladecenter - Fence agent for IBM BladeCenterfence_bladecenter_snmp - Fence agent for IBM BladeCenter over SNMPfence_cisco_mds - Fence agent for Cisco MDSfence_cisco_ucs - Fence agent for Cisco UCSfence_drac5 - Fence agent for Dell DRAC CMC/5fence_eps - Fence agent for ePowerSwitchfence_ibmblade - Fence agent for IBM BladeCenter over SNMPfence_ifmib - Fence agent for IF MIBfence_ilo - Fence agent for HP iLOfence_ilo_mp - Fence agent for HP iLO MPfence_intelmodular - Fence agent for Intel Modularfence_ipmilan - Fence agent for IPMI over LANfence_kdump - Fence agent for use with kdumpfence_rhevm - Fence agent for RHEV-M REST APIfence_rsa - Fence agent for IBM RSAfence_sanbox2 - Fence agent for QLogic SANBox2 FC switchesfence_scsi - fence agent for SCSI-3 persistent reservationsfence_virsh - Fence agent for virshfence_virt - Fence agent for virtual machinesfence_vmware - Fence agent for VMwarefence_vmware_soap - Fence agent for VMware over SOAP APIfence_wti - Fence agent for WTIfence_xvm - Fence agent for virtual machines
To print a list of the options you can specify for a particular fence type, execute the following command:
ccs -h host --lsfenceopts fence_type
For example, the following command lists the fence options for the fence_wti fence agent.
[root@ask-03 ~]# ccs -h node1 --lsfenceopts fence_wtifence_wti - Fence agent for WTI  Required Options:  Optional Options: option: No description available action: Fencing Action ipaddr: IP Address or Hostname login: Login Name passwd: Login password or passphrase passwd_script: Script to retrieve password cmd_prompt: Force command prompt secure: SSH connection identity_file: Identity file for ssh port: Physical plug number or name of virtual machine inet4_only: Forces agent to use IPv4 addresses only inet6_only: Forces agent to use IPv6 addresses only ipport: TCP port to use for connection with device verbose: Verbose mode debug: Write debug information to given file version: Display version information and exit help: Display help and exit separator: Separator for CSV created by operation list power_timeout: Test X seconds for status change after ON/OFF shell_timeout: Wait X seconds for cmd prompt after issuing command login_timeout: Wait X seconds for cmd prompt after login power_wait: Wait X seconds after issuing ON/OFF delay: Wait X seconds before fencing is started retry_on: Count of attempts to retry power on
To print a list of fence devices currently configured for your cluster, execute the following command:
ccs -h host --lsfencedev

3.3. Configuring Fencing for Cluster Members

Once you have completed the initial steps of creating a cluster and creating fence devices, you need to configure fencing for the cluster nodes. To configure fencing for the nodes after creating a new cluster and configuring the fencing devices for the cluster, follow the steps in this section. Note that you must configure fencing for each node in the cluster.
This section documents the following procedures:

3.3.1. Configuring a Single Power-Based Fence Device for a Node

Use the following procedure to configure a node with a single power-based fence device that uses a fence device named apc, which uses the fence_apc fencing agent.
  1. Add a fence method for the node, providing a name for the fence method.
    ccs -h host --addmethod method node
    For example, to configure a fence method named APC for the node node-01.example.com in the configuration file on the cluster node node-01.example.com, execute the following command:
    ccs -h node01.example.com --addmethod APC node01.example.com
  2. Add a fence instance for the method. You must specify the fence device to use for the node, the node this instance applies to, the name of the method, and any options for this method that are specific to this node:
    ccs -h host --addfenceinst fencedevicename node method [options]
    For example, to configure a fence instance in the configuration file on the cluster node node-01.example.com that uses the APC switch power port 1 on the fence device named apc to fence cluster node node-01.example.com using the method named APC, execute the following command:
    ccs -h node01.example.com --addfenceinst apc node01.example.com APC port=1
You will need to add a fence method for each node in the cluster. The following commands configure a fence method for each node with the method name APC. The device for the fence method specifies apc as the device name, which is a device previously configured with the --addfencedev option, as described in Section 3.1, "Configuring Fence Devices". Each node is configured with a unique APC switch power port number: The port number for node-01.example.com is 1, the port number for node-02.example.com is 2, and the port number for node-03.example.com is 3.
ccs -h node01.example.com --addmethod APC node01.example.comccs -h node01.example.com --addmethod APC node02.example.comccs -h node01.example.com --addmethod APC node03.example.comccs -h node01.example.com --addfenceinst apc node01.example.com APC port=1ccs -h node01.example.com --addfenceinst apc node02.example.com APC port=2ccs -h node01.example.com --addfenceinst apc node03.example.com APC port=3
Example 3.1, "cluster.conf After Adding Power-Based Fence Methods " shows a cluster.conf configuration file after you have added these fencing methods and instances to each node in the cluster.

Example 3.1. cluster.conf After Adding Power-Based Fence Methods 

<cluster name="mycluster" config_version="3">   <clusternodes> <clusternode name="node-01.example.com" nodeid="1"> <fence> <method name="APC">  <device name="apc" port="1"/> </method> </fence> </clusternode> <clusternode name="node-02.example.com" nodeid="2"> <fence> <method name="APC">  <device name="apc" port="2"/> </method> </fence> </clusternode> <clusternode name="node-03.example.com" nodeid="3"> <fence> <method name="APC">  <device name="apc" port="3"/> </method> </fence> </clusternode>   </clusternodes>   <fencedevices> <fencedevice agent="fence_apc" ipaddr="apc_ip_example" login="login_example" name="apc" passwd="password_example"/>   </fencedevices>   <rm>   </rm></cluster>

Note that when you have finished configuring all of the components of your cluster, you will need to sync the cluster configuration file to all of the nodes.

3.3.2. Configuring a Single Storage-Based Fence Device for a Node

When using non-power fencing methods (that is, SAN/storage fencing) to fence a node, you must configure unfencing for the fence device. This ensures that a fenced node is not re-enabled until the node has been rebooted. When you configure unfencing for a node, you specify a device that mirrors the corresponding fence device you have configured for the node with the notable addition of the explicit action of on or enable.
For more information about unfencing a node, refer to the fence_node(8) man page.
Use the following procedure to configure a node with a single storage-based fence device that uses a fence device named sanswitch1, which uses the fence_sanbox2 fencing agent.
  1. Add a fence method for the node, providing a name for the fence method.
    ccs -h host --addmethod method node
    For example, to configure a fence method named SAN for the node node-01.example.com in the configuration file on the cluster node node-01.example.com, execute the following command:
    ccs -h node01.example.com --addmethod SAN  node01.example.com
  2. Add a fence instance for the method. You must specify the fence device to use for the node, the node this instance applies to, the name of the method, and any options for this method that are specific to this node:
    ccs -h host --addfenceinst fencedevicename node method [options]
    For example, to configure a fence instance in the configuration file on the cluster node node-01.example.com that uses the SAN switch power port 11 on the fence device named sanswitch1 to fence cluster node node-01.example.com using the method named SAN, execute the following command:
    ccs -h node01.example.com --addfenceinst sanswitch1 node01.example.com SAN port=11
  3. To configure unfencing for the storage based fence device on this node, execute the following command:
    ccs -h host --addunfence fencedevicename node action=on|off
You will need to add a fence method for each node in the cluster. The following commands configure a fence method for each node with the method name SAN. The device for the fence method specifies sanswitch as the device name, which is a device previously configured with the --addfencedev option, as described in Section 3.1, "Configuring Fence Devices". Each node is configured with a unique SAN physical port number: The port number for node-01.example.com is 11, the port number for node-02.example.com is 12, and the port number for node-03.example.com is 13.
ccs -h node01.example.com --addmethod SAN node01.example.comccs -h node01.example.com --addmethod SAN node02.example.comccs -h node01.example.com --addmethod SAN node03.example.comccs -h node01.example.com --addfenceinst sanswitch1 node01.example.com SAN port=11ccs -h node01.example.com --addfenceinst sanswitch1 node02.example.com SAN port=12ccs -h node01.example.com --addfenceinst sanswitch1 node03.example.com SAN port=13ccs -h node01.example.com --addunfence sanswitch1 node01.example.com port=11 action=onccs -h node01.example.com --addunfence sanswitch1 node02.example.com port=12 action=onccs -h node01.example.com --addunfence sanswitch1 node03.example.com port=13 action=on
Example 3.2, "cluster.conf After Adding Storage-Based Fence Methods " shows a cluster.conf configuration file after you have added fencing methods, fencing instances, and unfencing to each node in the cluster.

Example 3.2. cluster.conf After Adding Storage-Based Fence Methods 

<cluster name="mycluster" config_version="3">   <clusternodes> <clusternode name="node-01.example.com" nodeid="1"> <fence> <method name="SAN">  <device name="sanswitch1" port="11"/> </method> </fence> <unfence> <device name="sanswitch1" port="11" action="on"/>  </unfence> </clusternode> <clusternode name="node-02.example.com" nodeid="2"> <fence> <method name="SAN">  <device name="sanswitch1" port="12"/> </method> </fence> <unfence> <device name="sanswitch1" port="12" action="on"/>  </unfence> </clusternode> <clusternode name="node-03.example.com" nodeid="3"> <fence> <method name="SAN">  <device name="sanswitch1" port="13"/> </method> </fence> <unfence> <device name="sanswitch1" port="13" action="on"/>  </unfence> </clusternode>   </clusternodes>   <fencedevices> <fencedevice agent="fence_sanbox2" ipaddr="san_ip_example"login="login_example" name="sanswitch1" passwd="password_example"/>   </fencedevices>   <rm>   </rm></cluster>

Note that when you have finished configuring all of the components of your cluster, you will need to sync the cluster configuration file to all of the nodes.

3.3.3. Configuring a Backup Fence Device

You can define multiple fencing methods for a node. If fencing fails using the first method, the system will attempt to fence the node using the second method, followed by any additional methods you have configured. To configure a backup fencing method for a node, you configure two methods for a node, configuring a fence instance for each method.

Note

The order in which the system will use the fencing methods you have configured follows their order in the cluster configuration file. The first method you configure with the ccs command is the primary fencing method, and the second method you configure is the backup fencing method. To change the order, you can remove the primary fencing method from the configuration file, then add that method back.
Note that at any time you can print a list of fence methods and instances currently configured for a node by executing the following command. If you do not specify a node, this command will list the fence methods and instances currently configured for all nodes.
ccs -h host --lsfenceinst [node]
Use the following procedure to configure a node with a primary fencing method that uses a fence device named apc, which uses the fence_apc fencing agent, and a backup fencing device that uses a fence device named sanswitch1, which uses the fence_sanbox2 fencing agent. Since the sanswitch1 device is a storage-based fencing agent, you will need to configure unfencing for that device as well.
  1. Add a primary fence method for the node, providing a name for the fence method.
    ccs -h host --addmethod method node
    For example, to configure a fence method named APC as the primary method for the node node-01.example.com in the configuration file on the cluster node node-01.example.com, execute the following command:
    ccs -h node01.example.com --addmethod APC node01.example.com
  2. Add a fence instance for the primary method. You must specify the fence device to use for the node, the node this instance applies to, the name of the method, and any options for this method that are specific to this node:
    ccs -h host --addfenceinst fencedevicename node method [options]
    For example, to configure a fence instance in the configuration file on the cluster node node-01.example.com that uses the APC switch power port 1 on the fence device named apc to fence cluster node node-01.example.com using the method named APC, execute the following command:
    ccs -h node01.example.com --addfenceinst apc node01.example.com APC port=1
  3. Add a backup fence method for the node, providing a name for the fence method.
    ccs -h host --addmethod method node
    For example, to configure a backup fence method named SAN for the node node-01.example.com in the configuration file on the cluster node node-01.example.com, execute the following command:
    ccs -h node01.example.com --addmethod SAN  node01.example.com
  4. Add a fence instance for the backup method. You must specify the fence device to use for the node, the node this instance applies to, the name of the method, and any options for this method that are specific to this node:
    ccs -h host --addfenceinst fencedevicename node method [options]
    For example, to configure a fence instance in the configuration file on the cluster node node-01.example.com that uses the SAN switch power port 11 on the fence device named sanswitch1 to fence cluster node node-01.example.com using the method named SAN, execute the following command:
    ccs -h node01.example.com --addfenceinst sanswitch1 node01.example.com SAN port=11
  5. Since the sanswitch1 device is a storage-based device, you must configure unfencing for this device.
    ccs -h node01.example.com --addunfence sanswitch1 node01.example.com port=11 action=on
You can continue to add fencing methods as needed.
This procedure configures a fence device and a backup fence device for one node in the cluster. You will need to configure fencing for the other nodes in the cluster as well.
Example 3.3, "cluster.conf After Adding Backup Fence Methods " shows a cluster.conf configuration file after you have added a power-based primary fencing method and a storage-based backup fencing method to each node in the cluster.

Example 3.3. cluster.conf After Adding Backup Fence Methods 

<cluster name="mycluster" config_version="3">   <clusternodes> <clusternode name="node-01.example.com" nodeid="1"> <fence> <method name="APC">  <device name="apc" port="1"/> </method> <method name="SAN">  <device name="sanswitch1" port="11"/> </method> </fence> <unfence> <device name="sanswitch1" port="11" action="on"/>  </unfence </clusternode> <clusternode name="node-02.example.com" nodeid="2"> <fence> <method name="APC">  <device name="apc" port="2"/> </method> <method name="SAN">  <device name="sanswitch1" port="12"/> </method> </fence> <unfence> <device name="sanswitch1" port="12" action="on"/>  </unfence </clusternode> <clusternode name="node-03.example.com" nodeid="3"> <fence> <method name="APC">  <device name="apc" port="3"/> </method> <method name="SAN">  <device name="sanswitch1" port="13"/> </method> </fence> <unfence> <device name="sanswitch1" port="13" action="on"/>  </unfence </clusternode>   </clusternodes>   <fencedevices> <fencedevice agent="fence_apc" ipaddr="apc_ip_example" login="login_example" name="apc" passwd="password_example"/> <fencedevice agent="fence_sanbox2" ipaddr="san_ip_example" login="login_example" name="sanswitch1" passwd="password_example"/>   </fencedevices>   <rm>   </rm></cluster>

Note that when you have finished configuring all of the components of your cluster, you will need to sync the cluster configuration file to all of the nodes.

Note

The order in which the system will use the fencing methods you have configured follows their order in the cluster configuration file. The first method you configure is the primary fencing method, and the second method you configure is the backup fencing method. To change the order, you can remove the primary fencing method from the configuration file, then add that method back.

3.3.4. Configuring a Node with Redundant Power

If your cluster is configured with redundant power supplies for your nodes, you must be sure to configure fencing so that your nodes fully shut down when they need to be fenced. If you configure each power supply as a separate fence method, each power supply will be fenced separately; the second power supply will allow the system to continue running when the first power supply is fenced and the system will not be fenced at all. To configure a system with dual power supplies, you must configure your fence devices so that both power supplies are shut off and the system is taken completely down. This requires that you configure two instances within a single fencing method, and that for each instance you configure both fence devices with an action attribute of off before configuring each of the devices with an action attribute of on.
To configure fencing for a node with dual power supplies, follow the steps in this section.
  1. Before you can configure fencing for a node with redundant power, you must configure each of the power switches as a fence device for the cluster. For information on configuring fence devices, see Section 3.1, "Configuring Fence Devices".
    To print a list of fence devices currently configured for your cluster, execute the following command:
    ccs -h host --lsfencedev
  2. Add a fence method for the node, providing a name for the fence method.
    ccs -h host --addmethod method node
    For example, to configure a fence method named APC-dual for the node node-01.example.com in the configuration file on the cluster node node-01.example.com, execute the following command:
    ccs -h node01.example.com --addmethod APC-dual node01.example.com
  3. Add a fence instance for the first power supply to the fence method. You must specify the fence device to use for the node, the node this instance applies to, the name of the method, and any options for this method that are specific to this node. At this point you configure the action attribute as off.
    ccs -h host --addfenceinst fencedevicename node method [options] action=off
    For example, to configure a fence instance in the configuration file on the cluster node node-01.example.com that uses the APC switch power port 1 on the fence device named apc1 to fence cluster node node-01.example.com using the method named APC-dual, and setting the action attribute to off, execute the following command:
    ccs -h node01.example.com --addfenceinst apc1 node01.example.com APC-dual port=1 action=off
  4. Add a fence instance for the second power supply to the fence method. You must specify the fence device to use for the node, the node this instance applies to, the name of the method, and any options for this method that are specific to this node. At this point you configure the action attribute as off for this instance as well:
    ccs -h host --addfenceinst fencedevicename node method [options] action=off
    For example, to configure a second fence instance in the configuration file on the cluster node node-01.example.com that uses the APC switch power port 1 on the fence device named apc2 to fence cluster node node-01.example.com using the same method as you specified for the first instance named APC-dual, and setting the action attribute to off, execute the following command:
    ccs -h node01.example.com --addfenceinst apc2 node01.example.com APC-dual port=1 action=off
  5. At this point, add another fence instance for the first power supply to the fence method, configuring the action attribute as on. You must specify the fence device to use for the node, the node this instance applies to, the name of the method, and any options for this method that are specific to this node, and specifying the action attribute as on:
    ccs -h host --addfenceinst fencedevicename node method [options] action=on
    For example, to configure a fence instance in the configuration file on the cluster node node-01.example.com that uses the APC switch power port 1 on the fence device named apc1 to fence cluster node node-01.example.com using the method named APC-dual, and setting the action attribute to on, execute the following command:
    ccs -h node01.example.com --addfenceinst apc1 node01.example.com APC-dual port=1 action=on
  6. Add another fence instance for second power supply to the fence method, specifying the action attribute as on for this instance. You must specify the fence device to use for the node, the node this instance applies to, the name of the method, and any options for this method that are specific to this node as well as the action attribute of on.
    ccs -h host --addfenceinst fencedevicename node method [options] action=on
    For example, to configure a second fence instance in the configuration file on the cluster node node-01.example.com that uses the APC switch power port 1 on the fence device named apc2 to fence cluster node node-01.example.com using the same method as you specified for the first instance named APC-dual and setting the action attribute to on, execute the following command:
    ccs -h node01.example.com --addfenceinst apc2 node01.example.com APC-dual port=1 action=on
Example 3.4, "cluster.conf After Adding Dual-Power Fencing " shows a cluster.conf configuration file after you have added fencing for two power supplies for each node in a cluster.

Example 3.4. cluster.conf After Adding Dual-Power Fencing 

<cluster name="mycluster" config_version="3">   <clusternodes> <clusternode name="node-01.example.com" nodeid="1"> <fence> <method name="APC-dual">  <device name="apc1" port="1"action="off"/>  <device name="apc2" port="1"action="off"/>  <device name="apc1" port="1"action="on"/>  <device name="apc2" port="1"action="on"/> </method> </fence> </clusternode> <clusternode name="node-02.example.com" nodeid="2"> <fence> <method name="APC-dual">  <device name="apc1" port="2"action="off"/>  <device name="apc2" port="2"action="off"/>  <device name="apc1" port="2"action="on"/>  <device name="apc2" port="2"action="on"/> </method> </fence> </clusternode> <clusternode name="node-03.example.com" nodeid="3"> <fence> <method name="APC-dual">  <device name="apc1" port="3"action="off"/>  <device name="apc2" port="3"action="off"/>  <device name="apc1" port="3"action="on"/>  <device name="apc2" port="3"action="on"/> </method> </fence> </clusternode>   </clusternodes>   <fencedevices>   <fencedevice agent="fence_apc" ipaddr="apc_ip_example" login="login_example" name="apc1" passwd="password_example"/>   <fencedevice agent="fence_apc" ipaddr="apc_ip_example" login="login_example" name="apc2" passwd="password_example"/>   </fencedevices>   <rm>   </rm></cluster>

Note that when you have finished configuring all of the components of your cluster, you will need to sync the cluster configuration file to all of the nodes.

3.3.5. Removing Fence Methods and Fence Instances

To remove a fence method from your cluster configuration, execute the following command:
ccs -h host --rmmethod method node
For example, to remove a fence method that you have named APC that you have configured for node01.example.com from the cluster configuration file on cluster node node01.example.com, execute the following command:
ccs -h node01.example.com  --rmmethod APC node01.example.com
To remove all fence instances of a fence device from a fence method, execute the following command:
ccs -h host --rmfenceinst fencedevicename node method
For example, to remove all instances of the fence device named apc1 from the method named APC-dual configured for node01.example.com from the cluster configuration file on cluster node node01.example.com, execute the following command:
ccs -h node01.example.com --rmfenceinst apc1 node01.example.com APC-dual
(Sebelumnya) 1 : Index - Installation Guide2 : Chapter 4. Configuring Fen ... (Berikutnya)