集群介绍,keepalived介绍,用keepalived配置高可用集群
温馨提示:这篇文章已超过860天没有更新,请注意相关的内容是否还可用!
集群介绍
根据功能划分为两大类:高可用和负载均衡
1、高可用集群通常为两台服务器,一台工作,另外一台作为冗余,当提供服务的机器宕机,冗余将接替继续提供服务
实现高可用的开源软件有:heartbeat、keepalived
2、负载均衡集群,需要有一台服务器作为分发器,它负责把用户的请求分发给后端的服务器处理,在这个集群里,除了分发器外,就是给用户提供服务的服务器了,这些服务器数量至少为2
实现负载均衡的开源软件有LVS、keepalived、haproxy、nginx,商业的有F5、Netscaler
各有个优势
keepalived介绍
在这里我们使用keepalived来实现高可用集群,因为heartbeat在centos6上有一些问题,影响实验效果。
keepalived通过VRRP(Virtual Router Redundancy Protocl,虚拟路由冗余协议)来实现高可用。
1、它是实现路由高可用的一种通信协议,在这个协议里会将多台功能相同的路由器组成一个小组,这个小组里会有1个master角色和N(N>=1)个backup角色。
2、master会通过组播的形式向各个backup发送VRRP协议的数据包,当backup收不到master发来的VRRP数据包时,就会认为master宕机了。此时就需要根据各个backup的优先级来决定谁成为新的mater。
3、Keepalived要有三个模块,分别是core、check和vrrp。其中core模块为keepalived的核心,负责主进程的启动、维护以及全局配置文件的加载和解析,check模块负责健康检查,vrrp模块是来实现VRRP协议的。
用Keepalived配置高可用集群
生产环境中,很多企业把Nginx作为负载均衡器来用,它的重要性很高,一旦宕机会导致整个站点不能访问,所以有必要再准备一台备用Nginx,Keepalived用在这种场景下非常合适。
1.准备工作
准备两台机器
master:172.11.1.3 安装keepaliver+nginx
backup:172.11.1.6 安装keepaliver+nginx
vip:172.11.1.10
VIP的英文名字是“Virtual IP",即“虚拟IP",也有人把它叫作“浮动IP”,因为这个IP是由Keepalived给服务器配置上的,服务器靠这个VIP对外提供服务,当master机器宕机,VIP被分配到backup上,这样用户看来是无感知的。
master和backup都安装keepalived+nginx
# yum install -y keepalived
如nginx服务没有则yum安装
# yum install -y epel-release
# yum install nginx -y
2.编辑master机器的keepalived配置文件
# > /etc/keepalived/keepalived.conf //清空配置文件 #vi /etc/keepalived/keepalived.conf #定义报警人邮箱 global_defs {notification_email { ahao@ahaoyw.com } #定义发件地址 notification_email_from root@aminglinux.com smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL} #chk_nginx为自定义名字,后面会用到 vrrp_script chk_nginx { #自定义脚本,该脚本为监控nginx服务的脚本 script "/usr/local/sbin/check_ng.sh" interval 3 } #定义master还是 backupvrrp_instance VI_1 { state MASTER } #针对哪个网卡监听 vipinterface ens33 #定义路由 idvirtual_router_id 51 #定义权重 priority 100 advert_int 1 authentication { #定义密码,可以自定义 auth_type PASS auth_pass ahaoyw>com }定义VIP公用的 virtual_ipaddress { 172.11.1.10 } #加载脚本 track_script { chk_nginx } }
3.编辑master机器的监控脚本
//脚本名字是自定义的,与keepalived配置文件要一致 #!/bin/bash #时间变量,用于记录日志 d=`date --date today +%Y%m%d_%H:%M:%S` #计算nginx进程数量 n=`ps -C nginx --no-heading|wc -l` #如果进程为0,则启动nginx,并且再次检测nginx进程数量, #如果还为0,说明nginx无法启动,此时需要关闭keepalived if [ $n -eq "0" ]; then /etc/init.d/nginx start n2=`ps -C nginx --no-heading|wc -l` if [ $n2 -eq "0" ]; then echo "$d nginx down,keepalived will stop" >> /var/log/check_ng.log systemctl stop keepalived fi fi
3.1、设置脚本权限
# chmod 755 /usr/local/sbin/check_ng.sh //需要给它权限,否则无法被keepalived调用加载
3.2、启动服务
# systemctl start keepalived //启动master上的keepalived,如果nginx服务没有启动,它会自动拉起来,并监听VIP
//启动master上的keepalived,如果nginx服务没有启动,它会自动拉起来,并监听VIP
[root@ahao-01 ~]# ps aux |grep keep
root 24268 0.0 0.1 118588 1384 ? Ss 23:31 0:00 /usr/sbin/keepalived -D
root 24269 0.0 0.3 127452 3288 ? S 23:31 0:00 /usr/sbin/keepalived -D
root 24270 0.0 0.2 127392 2832 ? S 23:31 0:00 /usr/sbin/keepalived -D
root 24290 0.0 0.0 112680 976 pts/0 R+ 23:31 0:00 grep --color=auto keep
日志存放地址:/var/log/messages
4.两台机器都查看防火墙及SElinux是否开启,开启需要关闭
[root@ahao01 ~]# setenforce 0 //临时关闭SELinux//开机关闭SELinux编辑/etc/selinux/config文件,将SELINUX的值设置为disabled
[root@ahao01 ~]# getenforce //查看SElinux是否关闭Disabled
[root@ahao01 ~]# systemctl stop firewalld.service //关闭防火墙
[root@ahao01 ~]# iptables -nvL //查看防火墙
Chain INPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination
5.编辑backup的Keepalived配置文件
[root@ahao02 ~]# > /etc/keepalived/keepalived.conf [root@ahao02 ~]# vim /etc/keepalived/keepalived.conf 增加如下配置内容: global_defs { notification_email { ahao@ahaoyw.com } notification_email_from root@ahaoyw.com smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_nginx { script "/usr/local/sbin/check_ng.sh" interval 3 } vrrp_instance VI_1 { state BACKUP interface ens33 virtual_router_id 51 priority 90 advert_int 1 authentication { auth_type PASS auth_pass ahaoyw>com } virtual_ipaddress { 192.168.5.100 } track_script { chk_nginx } } 注意:其中state和priority有变化
6.编辑backup机器的监控脚本
编辑增加如下配置内容: #时间变量,用于记录日志 d=`date --date today +%Y%m%d_%H:%M:%S` #计算nginx进程数量 n=`ps -C nginx --no-heading|wc -l` #如果进程为0,则启动nginx,并且再次检测nginx进程数量, #如果还为0,说明nginx无法启动,此时需要关闭keepalived if [ $n -eq "0" ]; then systemctl start nginx n2=`ps -C nginx --no-heading|wc -l` if [ $n2 -eq "0" ]; then echo "$d nginx down,keepalived will stop" >> /var/log/check_ng.log systemctl stop keepalived fi fi
[root@ahao02 ~]# chmod 755 /usr/local/sbin/check_ng.sh //更改权限 [root@ahao02 ~]# systemctl start keepalived //启动服务 [root@ahao02 ~]# ps aux |grep keep root 16039 0.0 0.1 120720 1400 ? Rs 19:51 0:00 /usr/sbin/keepalived -D root 16040 0.0 0.3 122792 3104 ? S 19:51 0:00 /usr/sbin/keepalived -D root 16041 0.1 0.2 127116 2656 ? S 19:51 0:00 /usr/sbin/keepalived -D root 16059 0.0 0.0 112676 976 pts/0 S+ 19:51 0:00 grep --color=auto keep [root@ahao02 ~]# ps aux |grep nginx root 15771 0.0 0.0 46308 948 ? Ss 16:56 0:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf nginx 15772 0.0 0.2 46692 2148 ? S 16:56 0:00 nginx: worker process root 16077 0.0 0.0 112676 972 pts/0 S+ 19:51 0:00 grep --color=auto nginx
7.测试高可用
确定好两台机器上nginx差异,通过curl -I 来查看nginx版本
##master机器 [root@ahao01 ~]# curl 172.11.1.3 -I HTTP/1.1 200 OK Server: nginx/1.12.1 Date: Tue, 22 May 2018 02:00:05 GMT Content-Type: text/html Content-Length: 612 Last-Modified: Tue, 22 May 2018 01:49:47 GMT Connection: keep-alive ETag: "5b03773b-264" Accept-Ranges: bytes ##backup机器 [root@ahao02 ~]# curl 172.11.1.6 -I HTTP/1.1 200 OK Server: nginx/1.12.2 Date: Tue, 22 May 2018 02:01:15 GMT Content-Type: text/html Content-Length: 3700 Last-Modified: Tue, 06 Mar 2018 09:26:21 GMT Connection: keep-alive ETag: "5a9e5ebd-e74" Accept-Ranges: bytes 这个时候vip在1.3上 [root@ahao01 ~]# curl -I 172.11.1.10 HTTP/1.1 200 OK Server: nginx/1.12.1 Date: Tue, 22 May 2018 02:01:43 GMT Content-Type: text/html Content-Length: 612 Last-Modified: Tue, 22 May 2018 01:49:47 GMT Connection: keep-alive ETag: "5b03773b-264" Accept-Ranges: bytes
测试1:关闭master上的nginx服务
1、停止master的nginx服务 [root@ahao01 ~]# /etc/init.d/nginxd stop 稍等个片刻后,在来检测端口发现端口有被启动了 [root@ahao01 ~]# ps aux |grep nginx root 2607 0.0 0.0 112724 964 pts/0 R+ 14:34 0:00 grep --color=auto nginx [root@ahao01 ~]# ps aux |grep nginx root 2628 0.0 0.0 20540 612 ? Ss 14:34 0:00 nginx: master process /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf nobody 2632 0.0 0.1 20980 1064 ? S 14:34 0:00 nginx: worker process root 2634 0.0 0.0 112724 964 pts/0 S+ 14:34 0:00 grep --color=auto nginx
测试2:在master上增加iptabls规则限制vrrp发包
[root@ahao01 ~]# iptables -I OUTPUT -p vrrp -j DROP //把主上VRRP协议出去的包封掉 [root@ahao01 ~]# iptables -nvL //查看防火墙 Chain INPUT (policy ACCEPT 22 packets, 1608 bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 15 packets, 1428 bytes) pkts bytes target prot opt in out source destination 19 760 DROP 112 -- * * 0.0.0.0/0 0.0.0.0/0
在backup上查看
浏览器地址查看
[root@ahao01 ~]# iptables -F //在master恢复防火墙
[root@ahao01 ~]# tail /var/log/messagesMay 22 14:45:18 ahao01 Keepalived[2244]: Stopping
May 22 14:45:18 ahao01 systemd: Stopping LVS and VRRP High Availability Monitor...
May 22 14:45:18 ahao01 Keepalived_vrrp[2246]: VRRP_Instance(VI_1) sent 0 priority
May 22 14:45:18 ahao01 Keepalived_vrrp[2246]: VRRP_Instance(VI_1) removing protocol VIPs.
May 22 14:45:18 ahao01 Keepalived_healthcheckers[2245]: Stopped
May 22 14:45:19 ahao01 Keepalived_vrrp[2246]: Stopped
May 22 14:45:19 ahao01 Keepalived[2244]: Stopped Keepalived v1.3.5 (03/19,2017), git commit v1.3.5-6-g6fa32f2
May 22 14:45:19 ahao01 systemd: Stopped LVS and VRRP High Availability Monitor.
May 22 15:01:01 ahao01 systemd: Started Session 3 of user root.
May 22 15:01:01 ahao01 systemd: Starting Session 3 of user root.
浏览器地址查看: