Keepalived+HAProxy实现MySQL高可用负载均衡的配置

文章目录

[隐藏]

  • Keepalived
  • HAProxy
  • 总结
Keepalived

由于在生产环境使用了mysqlcluster,需要实现高可用负载均衡,这里提供了keepalived+haproxy来实现.

keepalived主要功能是实现真实机器的故障隔离及负载均衡器间的失败切换.可在第3,4,5层交换.它通过VRRPv2(Virtual Router Redundancy Protocol) stack实现的.

Layer3:Keepalived会定期向服务器群中的服务器.发送一个ICMP的数据包(既我们平时用的Ping程序),如果发现某台服务的IP地址没有激活,Keepalived便报告这台服务器失效,并将它从服务器群中剔除,这种情况的典型例子是某台服务器被非法关机。Layer3的方式是以服务器的IP地址是否有效作为服务器工作正常与否的标准。

Layer4:主要以TCP端口的状态来决定服务器工作正常与否。如web server的服务端口一般是80,如果Keepalived检测到80端口没有启动,则Keepalived将把这台服务器从服务器群中剔除。

Layer5:在网络上占用的带宽也要大一些。Keepalived将根据用户的设定检查服务器程序的运行是否正常,如果与用户的设定不相符,则Keepalived将把服务器从服务器群中剔除。

Software Design

keepalived启动后会有单个进程

8352 ?    Ss   0:00 /usr/sbin/keepalived  8353 ?    S   0:00 _ /usr/sbin/keepalived  8356 ?    S   0:01 _ /usr/sbin/keepalived  

父进程:内存管理,子进程管理等等

子进程:VRRP子进程

子进程:Healthchecking 子进程

实例

2台mysqlcluster 10.1.6.203 master  10.1.6.205 backup    vip 10.1.6.173  

目的访问10.1.6.173 3366端口 分别轮询通过haproxy转发到10.1.6.203 3306 和10.1.6.205 3306

mysqlcluster搭建参照之前博客,这里在2台机上安装keepalived

[email protected]:~# apt-get install keepalived  [email protected]:~# cat /etc/keepalived/keepalived.conf  vrrp_script chk_haproxy {      script "killall -0 haproxy"  # verify the pid existance      interval 2          # check every 2 seconds      weight -2          # add 2 points of prio if OK  }    vrrp_instance VI_1 {      interface eth4        # interface to monitor      state MASTER      virtual_router_id 51     # Assign one ID for this route      priority 101         # 101 on master,100 on backup      nopreempt      debug        virtual_ipaddress {          10.1.6.173      }        track_script {    #注意大括号空格          chk_haproxy      }        notify_master /etc/keepalived/scripts/start_haproxy.sh #表示当切换到master状态时,要执行的脚本      notify_fault /etc/keepalived/scripts/stop_keepalived.sh #故障时执行的脚本      notify_stop  /etc/keepalived/scripts/stop_haproxy.sh #keepalived停止运行前运行notify_stop指定的脚本 }  

VRRPD配置包括三个类:

  • VRRP同步组(synchroization group)
  • VRRP实例(VRRP Instance)
  • VRRP脚本

这里使用了 VRRP实例,VRRP脚本

注意配置选项:

stat:指定instance(Initial)的初始状态,就是说在配置好后,这台服务器的初始状态就是这里指定的,但这里指定的不算,还是得要通过竞选通过优先级来确定,里如果这里设置为master,但如若他的优先级不及另外一台,那么这台在发送通告时,会发送自己的优先级,另外一台发现优先级不如自己的高,那么他会就回抢占为master

  • interface:实例绑定的网卡,因为在配置虚拟IP的时候必须是在已有的网卡上添加的
  • priority 101:设置本节点的优先级,优先级高的为master
  • debug:debug级别
  • nopreempt:设置为不抢占
vrrp_script chk_haproxy {      script "killall -0 haproxy"  # verify the pid existance      interval 2          # check every 2 seconds 脚本执行间隔      weight -2          # add 2 points of prio if OK 脚本结果导致的优先级变更:2表示优先级+2;-2则表示优先级-2  }  

然后在实例(vrrp_instance)里面引用,有点类似脚本里面的函数引用一样:先定义,后引用函数名

    track_script {          chk_haproxy      }  

注意:VRRP脚本(vrrp_script)和VRRP实例(vrrp_instance)属于同一个级别

[email protected]:scripts# cat start_haproxy.sh  #!/bin/bash    sleep 5  get=`ip addr |grep 10.1.6.173 |wc -l`  echo $get >> /etc/keepalived/scripts/start_ha.log    if [ $get -eq 1 ]  then      echo "`date +%c` success to get vip" >> /etc/keepalived/scripts/start_ha.log      /usr/local/sbin/haproxy -f /etc/haproxy/haproxy.cfg  else      echo "`date +%c` can not get vip" >> /etc/keepalived/scripts/start_ha.log  fi  [email protected]:scripts# cat stop_keepalived.sh  #!/bin/bash    p" ]  then    echo "`date +%c` no keepalived process id" >> /etc/keepalived/scripts/stop_keep.log  else    echo "`date +%c` will stop keepalived " >> /etc/keepalived/scripts/stop_keep.log    /etc/init.d/keepalived stop  fi    /etc/init.d/keepalived stop    [email protected]:scripts# cat stop_haproxy.sh  #!/bin/bash    p`date +%c` stop haproxy" >> /etc/keepalived/scripts/stop_ha.log  kill -9 $pid  

同理配置10.1.6.205

[email protected]:~# cat /etc/keepalived/keepalived.conf  vrrp_script chk_haproxy {    script "killall -0 haproxy"  # verify the pid existance    interval 2          # check every 2 seconds    weight 2           # add 2 points of prio if OK  }    vrrp_instance VI_1 {    interface eth4        # interface to monitor    state BACKUP    virtual_router_id 51     # Assign one ID for this route    priority 100         # 101 on master,100 on backup    virtual_ipaddress {      10.1.6.173    }      track_script {      chk_haproxy    }    notify_master /etc/keepalived/scripts/start_haproxy.sh  notify_fault /etc/keepalived/scripts/stop_keepalived.sh  notify_stop /etc/keepalived/scripts/stop_haproxy.sh    }  
HAProxy

下面再介绍下haproxy

HAProxy是一款基于TCP(第四层)和HTTP(第七层)应用的代理软件,它也可作为负载均衡器.可以支持数以万计的并发连接.同时可以保护服务器不暴露到网络上,通过端口映射.它还自带监控服务器状态的页面.

安装haproxy

wget -O/tmp/haproxy-1.4.22.tar.gz http://haproxy.1wt.eu/download/1.4/src/haproxy-1.4.22.tar.gz  tar xvfz /tmp/haproxy-1.4.22.tar.gz -C /tmp/  cd /tmp/haproxy-1.4.22  make TARGET=linux26  make install  

haproxy需要对每一个mysqlcluster服务器进行健康检查

1.在2台主机分别配置haproxy.cfg

[email protected]:scripts# cat /etc/haproxy/haproxy.cfg  global      maxconn 51200 #默认最大连接数      #uid 99      #gid 99      daemon    #以后台形式运行haproxy      #quiet      nbproc 1   #进程数量(可以设置多个进程提高性能)      pidfile /etc/haproxy/haproxy.pid #haproxy的pid存放路径,启动进程的用户必须有权限访问此文件    defaults      mode tcp      #所处理的类别 (#7层 http;4层tcp )      option redispatch  #serverId对应的服务器挂掉后,强制定向到其他健康的服务器      option abortonclose #当服务器负载很高的时候,自动结束掉当前队列处理比较久的连接      timeout connect 5000s  #连接超时      timeout client 50000s #客户端超时      timeout server 50000s  #服务器超时      log 127.0.0.1 local0  #错误日志记录      balance roundrobin  #默认的负载均衡的方式,轮询方式    listen proxy      bind 10.1.6.173:3366  #监听端口      mode tcp        #http的7层模式      option httpchk    #心跳检测的文件      server db1 10.1.6.203:3306 weight 1 check port 9222 inter 12000 rise 3 fall 3   #服务器定义,check inter 12000是检测心跳频率 rise 3是3次正确认为服务器可用, fall 3是3次失败认为服务器不可用,weight代表权重      server db2 10.1.6.205:3306 weight 1 check port 9222 inter 12000 rise 3 fall 3    listen haproxy_stats      mode http      bind 10.1.6.173:8888      option httplog      stats refresh 5s      stats uri /status #网站健康检测URL,用来检测HAProxy管理的网站是否可以用,正常返回200,不正常返回503      stats realm Haproxy Manager      stats auth admin:p@a1SZs24 #账号密码    [email protected]:~$ cat /etc/haproxy/haproxy.cfg  global      maxconn 51200      #uid 99      #gid 99      daemon      #quiet      nbproc 1      pidfile /etc/haproxy/haproxy.pid    defaults      mode tcp      option redispatch      option abortonclose      timeout connect 5000s      timeout client 50000s      timeout server 50000s      log 127.0.0.1 local0      balance roundrobin    listen proxy      bind 10.1.6.173:3366      mode tcp      option httpchk      server db1 10.1.6.203:3306 weight 1 check port 9222 inter 12000 rise 3 fall 3      server db2 10.1.6.205:3306 weight 1 check port 9222 inter 12000 rise 3 fall 3    listen haproxy_stats      mode http      bind 10.1.6.173:8888      option httplog      stats refresh 5s      stats uri /status      stats realm Haproxy Manager      stats auth admin:p@a1SZs24  

2.安装xinetd

[email protected]:~# apt-get install xinetd  

3.在每个节点添加xinetd服务脚本和mysqlchk端口号

[email protected]:~# vim /etc/xinetd.d/mysqlchk  # default: on  # description: mysqlchk  service mysqlchk         #需要在servive定义  {      flags      = REUSE      socket_type   = stream      port      = 9222      wait      = no      user      = nobody      server     = /opt/mysqlchk      log_on_failure += USERID      disable     = no      per_source   = UNLIMITED      bind      = 10.1.6.173  }    [email protected]:~# vim /etc/services  mysqlchk    9222/tcp            # mysqlchk  

4.编写mysqlchk监控服务脚本

[email protected]:~# ls -l /opt/mysqlchk  -rwxr--r-- 1 nobody root 1994 2013-09-17 11:27 /opt/mysqlchk  [email protected]:~# cat /opt/mysqlchk  #!/bin/bash  #  # This script checks if a mysql server is healthy running on localhost. It will  # return:  # "HTTP/1.x 200 OKr" (if mysql is running smoothly)  # - OR -  # "HTTP/1.x 500 Internal Server Errorr" (else)  #  # The purpose of this script is make haproxy capable of monitoring mysql properly  #    MYSQL_HOST="localhost"  MYSQL_SOCKET="/var/run/mysqld/mysqld.sock"  MYSQL_USERNAME="mysqlchkusr"   #该账户密码需要在mysql里添加  MYSQL_PASSWORD="secret"  MYSQL_OPTS="-N -q -A"  TMP_FILE="/dev/shm/mysqlchk.$$.out"  ERR_FILE="/dev/shm/mysqlchk.$$.err"  FORCE_FAIL="/dev/shm/proxyoff"  MYSQL_BIN="/opt/mysqlcluster/mysql-cluster-gpl-7.2.6-linux2.6-x86_64/bin/mysql"  CHECK_QUERY="select 1"    preflight_check()  {    for I in "$TMP_FILE" "$ERR_FILE"; do      if [ -f "$I" ]; then        if [ ! -w $I ]; then          echo -e "HTTP/1.1 503 Service Unavailablern"          echo -e "Content-Type: Content-Type: text/plainrn"          echo -e "rn"          echo -e "Cannot write to $Irn"          echo -e "rn"          exit 1        fi      fi    done  }    return_ok()  {    echo -e "HTTP/1.1 200 OKrn"    echo -e "Content-Type: text/htmlrn"    echo -e "Content-Length: 43rn"    echo -e "rn"    echo -e "<html><body>MySQL is running.</body></html>rn"    echo -e "rn"    rm $ERR_FILE $TMP_FILE    exit 0  }  return_fail()  {    echo -e "HTTP/1.1 503 Service Unavailablern"    echo -e "Content-Type: text/htmlrn"    echo -e "Content-Length: 42rn"    echo -e "rn"    echo -e "<html><body>MySQL is *down*.</body></html>rn"    sed -e 's/n$/rn/' $ERR_FILE    echo -e "rn"    rm $ERR_FILE $TMP_FILE    exit 1  }  preflight_check  if [ -f "$FORCE_FAIL" ]; then      echo "$FORCE_FAIL found" > $ERR_FILE      return_fail;  fi  $MYSQL_BIN $MYSQL_OPTS --host=$MYSQL_HOST --socket=$MYSQL_SOCKET --user=$MYSQL_USERNAME --password=$MYSQL_PASSWORD -e "$CHECK_QUERY" > $TMP_FILE 2> $ERR_FILE  if [ $? -ne 0 ]; then      return_fail;  fi  return_ok;  

测试

2个节点开启keepalived(主节点会获得vip,自动拉起haproxy),xinetd

[email protected]:~# ip add  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo  2: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000    link/ether 00:26:b9:36:0f:81 brd ff:ff:ff:ff:ff:ff    inet 211.151.105.186/26 brd 211.151.105.191 scope global eth0  3: eth4: <BROADCAST,MULTICAST,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000    link/ether 00:26:b9:36:0f:83 brd ff:ff:ff:ff:ff:ff    inet 10.1.6.203/24 brd 10.1.6.255 scope global eth4    inet 10.1.6.173/32 scope global eth4  4: eth5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000    link/ether 00:26:b9:36:0f:85 brd ff:ff:ff:ff:ff:ff  5: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000    link/ether 00:26:b9:36:0f:87 brd ff:ff:ff:ff:ff:ff  [email protected]:~# netstat -tunlp | grep ha  tcp    0   0 10.1.6.173:3366   0.0.0.0:*        LISTEN   1042/haproxy  tcp    0   0 10.1.6.203:8888   0.0.0.0:*        LISTEN   1042/haproxy  udp    0   0 0.0.0.0:56562      0.0.0.0:*              1042/haproxy  [email protected]:~# netstat -tunlp | grep xine  tcp    0   0 10.1.6.203:9222   0.0.0.0:*        LISTEN   30897/xinetd  [email protected]:~# ps -ef | grep haproxy  root   1042   1 0 Sep17 ?    00:00:00 /usr/local/sbin/haproxy -f /etc/haproxy/haproxy.cfg  

测试

通过vip10.1.6.173 3366访问cluster数据库(注意账户dave权限需要加3个ip10.1.6.203,10.1.6.205,10.1.6.173)

[email protected]:mgm# mysql -udave -p -h 10.1.6.173 -P 3366  Enter password:  Welcome to the MySQL monitor. Commands end with ; or g.  Your MySQL connection id is 1344316  Server version: 5.5.22-ndb-7.2.6-gpl-log MySQL Cluster Community Server (GPL)    Type 'help;' or 'h' for help. Type 'c' to clear the buffer.    mysql> show databases;  +--------------------+  | Database      |  +--------------------+  | information_schema |  | dave       |  | test        |  +--------------------+  3 rows in set (0.01 sec)    mysql>  

手动分别使keepalive,haproxy,数据库挂掉.vip10.1.6.173会自动漂到10.1.6.205从上,并不影响vip的访问

通过vip,haproxy查看各节点状态

http://10.1.6.173:8888/status  

总结

以上是脚本之家为你收集整理的Keepalived+HAProxy实现MySQL高可用负载均衡的配置全部内容,希望文章能够帮你解决Keepalived+HAProxy实现MySQL高可用负载均衡的配置所遇到的程序开发问题。

原文出处:jb51 -> https://m.jb51.cc/mssql/502260.html

本站所发布的一切资源仅限用于学习和研究目的;不得将上述内容用于商业或者非法用途,否则,一切后果请用户自负。本站信息来自网络,版权争议与本站无关。您必须在下载后的24个小时之内,从您的电脑中彻底删除上述内容。如果您喜欢该程序,请支持正版软件,购买注册,得到更好的正版服务。如果侵犯你的利益,请发送邮箱到 [email protected],我们会很快的为您处理。
超哥软件库 » Keepalived+HAProxy实现MySQL高可用负载均衡的配置