keepalived 实现 Nginx 高可用安装

庞小辉

关注

发布于: 2021 年 01 月 27 日

一、关闭操作系统防火墙

systemctl status firewalld
systemctl stop firewalld
systemctl disable firewalld

二、关闭 selinux

编辑文件 vi /etc/sysconfig/selinux

修改属性值：SELINUX=disabled

三、安装 gcc 及 openssl openssl-devel

联网状态安装：gcc -> yum install gccopenssl -> yum -y install openssl openssl-devel
未联网状态：寻找一台已联网，且未安装 openssl 的虚拟机 yum -y install openssl openssl-devel --downloadonly --downloaddir=/opt/install/yum/openssl（不执行安装，下载依赖包到指定目录）然后将依赖包放入未联网服务器的 yum 源，进行 yum 安装

四、安装 keepalived

进入网址：https://www.keepalived.org/download.html 下载 keepalived-2.1.5
解压 keepalived-2.1.5 到自定义的源码目录
进入源码目录
执行命令：./configure --prefix=/usr/local/keepalived
执行命令：make && make install

五、移动 keepalived 相关文件到指定目录

keepalived启动脚本变量引用文件，默认文件路径是/etc/sysconfig/，也可以不做软链接，直接修改启动脚本中文件路径即可（安装目录下）cp /usr/local/keepalived/etc/sysconfig/keepalived  /etc/sysconfig/keepalived 

将keepalived主程序加入到环境变量（安装目录下）cp /usr/local/keepalived/sbin/keepalived /usr/sbin/keepalived

将keepalived启动脚本（源码目录下，我的源码目录是在 install 下）放到/etc/init.d/目录下就可以使用service命令便捷调用cp /opt/install/keepalived/keepalived-2.1.5/keepalived/etc/init.d/keepalived /etc/init.d/keepalived

将配置文件放到默认路径下mkdir /etc/keepalivedcp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf

复制代码

六、编写监测 Nginx 存活脚本路径：（usr/local/nginx）名称：(nginx_check.sh)

#!/bin/bashcounter=$(docker ps | grep wisebot_services_nginx | wc -l)echo `date "+%Y-%m-%d %H:%M:%S"`'-执行用户:'`whoami` >> /usr/local/nginx/logs/logecho `date "+%Y-%m-%d %H:%M:%S"`'-进入执行脚本...' >> /usr/local/nginx/logs/logif [ "${counter}" = "0" ]; then    echo `date "+%Y-%m-%d %H:%M:%S"`'-Nginx服务停止，尝试重启...' >> /usr/local/nginx/logs/log    docker restart wisebot_services_nginx    sleep 2    counter=$(docker ps | grep wisebot_services_nginx | wc -l)    if [ "${counter}" = "0" ]; then        echo `date "+%Y-%m-%d %H:%M:%S"`'-Nginx服务重启失败，停止Keepalived...' >> /usr/local/nginx/logs/log        systemctl stop keepalived    fifi

复制代码

七、修改 Master 节点的 keepalived.conf 配置文件

! Configuration File for keepalived

global_defs {   notification_email {     acassen@firewall.loc     failover@firewall.loc     sysadmin@firewall.loc   }   notification_email_from Alexandre.Cassen@firewall.loc   #当前服务器IP地址   smtp_server 10.10.10.175   smtp_connect_timeout 30   router_id LVS_DEVEL   vrrp_skip_check_adv_addr   vrrp_strict   vrrp_garp_interval 0   vrrp_gna_interval 0}

#添加检测脚本vrrp_script chk_http_port {    # script "/usr/local/nginx/nginx_check.sh"    script "sh -x /usr/local/nginx/nginx_check.sh"    interval 5    weight 2}

vrrp_instance VI_1 {    #主机这里是MASTER 从机是BACKUP    state MASTER    #网卡名称，使用 ip addr 命令查看获取    interface enp0s3    #主、从机的virtual_router_id必须相同    virtual_router_id 51    #主备机取不同的优先级，主机优先级大    priority 100    #心跳检测间隔时间（秒）    advert_int 1    authentication {        auth_type PASS        auth_pass 1111    }    #虚拟对外开放虚拟IP地址，可以换行输入多个进行绑定    virtual_ipaddress {        10.10.10.166    }    #执行监测脚本配置    track_script {        chk_http_port    }}

virtual_server 192.168.200.100 443 {    delay_loop 6    lb_algo rr    lb_kind NAT    persistence_timeout 50    protocol TCP

    real_server 192.168.201.100 443 {        weight 1        SSL_GET {            url {              path /              digest ff20ad2481f97b1754ef3e12ecd3a9cc            }            url {              path /mrtg/              digest 9b3a0c85a887a256d6939da88aabd8cd            }            connect_timeout 3            retry 3            delay_before_retry 3        }    }}

virtual_server 10.10.10.2 1358 {    delay_loop 6    lb_algo rr    lb_kind NAT    persistence_timeout 50    protocol TCP

    sorry_server 192.168.200.200 1358

    real_server 192.168.200.2 1358 {        weight 1        HTTP_GET {            url {              path /testurl/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334d            }            url {              path /testurl2/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334d            }            url {              path /testurl3/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334d            }            connect_timeout 3            retry 3            delay_before_retry 3        }    }

    real_server 192.168.200.3 1358 {        weight 1        HTTP_GET {            url {              path /testurl/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334c            }            url {              path /testurl2/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334c            }            connect_timeout 3            retry 3            delay_before_retry 3        }    }}

virtual_server 10.10.10.3 1358 {    delay_loop 3    lb_algo rr    lb_kind NAT    persistence_timeout 50    protocol TCP

    real_server 192.168.200.4 1358 {        weight 1        HTTP_GET {            url {              path /testurl/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334d            }            url {              path /testurl2/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334d            }            url {              path /testurl3/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334d            }            connect_timeout 3            retry 3            delay_before_retry 3        }    }

    real_server 192.168.200.5 1358 {        weight 1        HTTP_GET {            url {              path /testurl/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334d            }            url {              path /testurl2/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334d            }            url {              path /testurl3/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334d            }            connect_timeout 3            retry 3            delay_before_retry 3        }    }}

复制代码

八、修改 Slave 节点的 keepalived.conf 配置文件

! Configuration File for keepalived



global_defs {   notification_email {     acassen@firewall.loc     failover@firewall.loc     sysadmin@firewall.loc   }   notification_email_from Alexandre.Cassen@firewall.loc   #当前服务器IP地址   smtp_server 10.10.10.175   smtp_connect_timeout 30   router_id LVS_DEVEL   vrrp_skip_check_adv_addr   vrrp_strict   vrrp_garp_interval 0   vrrp_gna_interval 0}



#添加检测脚本vrrp_script chk_http_port {    # script "/usr/local/nginx/nginx_check.sh"    script "sh -x /usr/local/nginx/nginx_check.sh"    interval 5    weight 2}



vrrp_instance VI_1 {    #主机这里是MASTER 从机是BACKUP    state BACKUP    #网卡名称，使用 ip addr 命令查看获取    interface enp0s3    #主、从机的virtual_router_id必须相同    virtual_router_id 51    #主备机取不同的优先级，主机优先级大    priority 100    #心跳检测间隔时间（秒）    advert_int 1    authentication {        auth_type PASS        auth_pass 1111    }    #虚拟对外开放虚拟IP地址，可以换行输入多个进行绑定    virtual_ipaddress {        10.10.10.166    }    #执行监测脚本配置    track_script {        chk_http_port    }}



virtual_server 192.168.200.100 443 {    delay_loop 6    lb_algo rr    lb_kind NAT    persistence_timeout 50    protocol TCP



    real_server 192.168.201.100 443 {        weight 1        SSL_GET {            url {              path /              digest ff20ad2481f97b1754ef3e12ecd3a9cc            }            url {              path /mrtg/              digest 9b3a0c85a887a256d6939da88aabd8cd            }            connect_timeout 3            retry 3            delay_before_retry 3        }    }}



virtual_server 10.10.10.2 1358 {    delay_loop 6    lb_algo rr    lb_kind NAT    persistence_timeout 50    protocol TCP



    sorry_server 192.168.200.200 1358



    real_server 192.168.200.2 1358 {        weight 1        HTTP_GET {            url {              path /testurl/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334d            }            url {              path /testurl2/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334d            }            url {              path /testurl3/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334d            }            connect_timeout 3            retry 3            delay_before_retry 3        }    }



    real_server 192.168.200.3 1358 {        weight 1        HTTP_GET {            url {              path /testurl/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334c            }            url {              path /testurl2/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334c            }            connect_timeout 3            retry 3            delay_before_retry 3        }    }}



virtual_server 10.10.10.3 1358 {    delay_loop 3    lb_algo rr    lb_kind NAT    persistence_timeout 50    protocol TCP



    real_server 192.168.200.4 1358 {        weight 1        HTTP_GET {            url {              path /testurl/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334d            }            url {              path /testurl2/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334d            }            url {              path /testurl3/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334d            }            connect_timeout 3            retry 3            delay_before_retry 3        }    }



    real_server 192.168.200.5 1358 {        weight 1        HTTP_GET {            url {              path /testurl/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334d            }            url {              path /testurl2/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334d            }            url {              path /testurl3/test.jsp              digest 640205b7b0fc66c1ea91c463fac6334d            }            connect_timeout 3            retry 3            delay_before_retry 3        }    }}

复制代码

九、启动 keepalived master 节点和 backup 节点

# 启动systemctl start keepalived
# 停止systemctl stop keepalived
# 查看状态systemctl status keepalived
# 查看keepalived 日志tail -f /var/log/messages

复制代码

十、启动之后，使用命令 ip addr 查看，虚拟 IP 已经飘到 Master 节点对应的网卡

注意事项-踩坑总结：

需要注意，keepalived 中配置文件的执行时间间隔，不能与 nginx 检测脚本中的 sleep 时间间隔相等。这样容易出现 nginx 挂了以后，不能使 keepalived 自动停止。
在离线服务器安装的时候，我这边出现了一个问题。安装 openssl-devel 的时候，服务器已经安装了一个相对较低版本的 krb5-libs。导致 yum 源中升级的时候，一直升级不成功。所以，openssl-devel 就安装不成功，进而导致 keepalived 不能安装成功。所以需要在 yum 安装额时候，加上 -v 参数，查看更信息的报错信息。然后找到具体那个包安装不成功，直接使用 yum remove 命令卸载掉，重新安装。就可以安装成功。这个是个大坑。

发布于: 2021 年 01 月 27 日阅读数: 916

原文链接:【http://xie.infoq.cn/article/57b8ed07021340e0ade22d41b】。文章转载请联系作者。