Example -- Setting Up a Five-node Cluster

This section describes step by step how to create a cluster of two LVS routers and three Web/FTP servers. First, collect information and set up the five systems as explained in the next section. Then, implement the example either from a shell (explained in the section called Implementing the Example from the Shell) or by starting the GUI configuration tool (explained in the section called Implementing the Example with Piranha).

Figure 3 shows the network that will exist after you've set up the LVS routers and real servers. All network addresses shown are for purposes of illustration only.

Figure 3. Layout of the Example Network

|-------|------------------------------------------|---------| Public network
        |eth0=1.2.3.2                              |eth0=1.2.3.3
        |eth0:1=1.2.3.1 (vs1)                      |
  ------|-----                              -------|-----
  |  active  |                              |   backup  |
  |  router  |                              |   router  |  
  |          |                              |           | 
  ------|-----                              -------|-----
        |eth1=192.168.1.1                          |eth1=192.168.1.2
        |eth1:1=192.168.1.254 (NAT router)         |
|-------|-|------------------|-----------------|---|----------| Private network
          |eth0=192.168.1.3  |eth0=192.168.1.4 |eth0=192.168.1.5 
          |                  |                 |
          |---------|        |---------|       |---------|
          |   rs1   |        |   rs2   |       |   rs3   |  
          |_________|        |_________|       |_________|

 

Preliminary Setup

  1. From the webmaster, obtain a virtual server IP address. In our example this will be 1.2.3.1. Requests for service at the LVS cluster will be addressed to a fully-qualified domain name associated with this address.

  2. Locate 5 servers and designate their roles: 1 primary LVS router, 1 backup LVS router, 3 real servers. The LVS routers must be Linux boxes running Red Hat 6.1 or later. The real servers may be any platform running any operating system and Web server.

    Steps 3-7 set up the LVS routers.

  3. On each LVS router, install two ethernet adapter cards, eth0 and eth1. Create a public IP interface on eth0 and a private IP interface on eth1. The public interface device (eth0) is the heartbeat device. The virtual server address is aliased to this device.

    Primary node Backup node
    eth0 1.2.3.2 1.2.3.3
    eth1 192.168.1.1 192.168.1.2

  4. Designate an IP address (192.168.1.254) for the router device (eth1) connecting the active LVS router to the private network. This floating IP address will be aliased to the router device as eth1:1, and will be the gateway to the private network and the default route used by each real server to communicate with the active router.

  5. On each LVS router:

    1. Enable packet forwarding. To do this at system boot, make sure the file /etc/sysconf/network contains the line FORWARD_IPV4=yes. To enable packet forwarding without rebooting, as root issue this command:


          echo "1" > /proc/sys/net/ipv4/ip_forward

    2. Enable packet defragmenting. To do this at system boot, make sure the file /etc/sysconf/network contains the line DEFRAG_IPV4=yes. To enable packet defragmenting without rebooting, as root issue this command:


          echo "1" > /proc/sys/net/ipv4/ip_always_defrag

    3. Masquerade the private network. Issue this command and put it in /etc/rc.d/rc.local:


          ipchains -A forward -j MASQ -s 192.168.1.0/24 -d 0.0.0.0

  6. Decide whether to use the rsh or ssh families for synchronizing LVS cluster files. Verify that your choice is installed such that the LVS routers can log in to one another as root without administrator intervention (see Table 2). In this example, we will choose rsh.

  7. On each LVS router, verify that the LVS cluster software is installed (see the section called Installing the Software.

    Steps 8-11 set up real servers.

  8. On each real server, install an ethernet network card, eth0, create an IP address on the same private subnet as in Step 3, and assign a weight to each server indicating its processing capacity relative to that of the others. In this example, rs1 has twice the capacity (two processors) of rs2 and rs3.

    rs1 rs2 rs3
    eth0 192.168.1.3 192.168.1.4 192.168.1.5
    weight 2000 1000 1000

 

  1.  
Layout Area

On each real server, verify that the address named in Step 4 (192.168.1.254) is its default route for communicating with the active LVS router.

  • Decide which program (uptime, ruptime, rup) will be used by the active router to monitor the workload on the real servers. If you choose uptime, each LVS router must be able to connect with each real server without administrator intervention, using the tool family you selected in Step 6. See Table 2 for general enablement instructions. If the selected tool cannot be enabled (for example, one of the real servers is an NT box), the scheduling algorithms that use dynamic load information will still work but the user-assigned weights will be statically applied rather than dynamically adjusted based on load.

  • Verify that each real server runs an installed and configured httpd server. Note that the real servers must listen on the same port (80 in the example) as the corresponding virtual server.

  • Verify (for example, using telnet or ping) that each real server can reach hosts on the public LAN. If a real server on the private network cannot reach a host on your LAN, this probably indicates a communication failure between the server and the active router. See Steps 8 and 9.

  • Determine the runtime parameters. For some of these, you may need to experiment over time to obtain optimal values. In this example, we will use the values listed.

    Value Parameter Description
    1050 Number of the heartbeat listening port on the primary and backup routers.
    2 Number of seconds between heartbeats.
    10 Number of seconds to wait for a non-responding router to respond before initiating failover.
    10 Number of seconds to wait for a non-responding real server to respond before removing it from the routing table.
    180 When a real server that has been removed from the routing table starts responding again, wait this number of seconds before re-adding the server to the routing table.
    wlc Use the Weighted least-connections load-balancing algorithm (assign more jobs to servers that are least busy relative to their load-adjusted weight). See Table 1 for a description of the choices.
    http Application. The alternative is ftp.
    80 Virtual server port number. The listening port selected for the virtual server is used on the real servers as well.

    Now we are ready to implement the example. You can do this from a shell as explained in the next section. Or you can use the GUI configuration tool as explained in the section called Implementing the Example with Piranha.

    • Layout Area
    		 

    Home

    2008 7/25/2008 12:09:11 PM