写点什么

记一次 dr_auto_sync 计划内的切换测试过程

  • 2024-07-05
    北京
  • 本文字数:2773 字

    阅读完需:约 9 分钟

作者: paulli 原文来源:https://tidb.net/blog/78c9a0ff

一、背景

TiDB 作为一款开源的分布式关系型数据库,提供了强大的数据同步和容灾能力,其中dr_auto_sync功能允许数据库在预定的时间或条件下自动从主数据中心切换到从数据中心,保证服务的连续性和数据的一致性。本文将记录一次 TiDB dr_auto_sync计划内的切换测试过程。


二、测试环境

  • TiDB 集群版本:7.1.5(请根据实际版本替换)

  • 主数据中心:DC1,包含 2 个 TiDB 节点、2 个 TiKV 节点 (voter) 和 2 个 PD 节点

  • 从数据中心:DC2,包含 2 个 TiDB 节点、2 个 TiKV 节点 (follower/learner) 和 1 个 PD 节点

集群节点拓扑如下:

集群副本配置如下:

副本 leader 全部运行在 dc1 中心


三、切换前准备

检查集群运行状态

granfna -> dr_test_overview -> service port status


检查集群主备同步状态

granfna -> dr_test_pd-> dr_autosync


准备切换文件

rules_reverse.json


[{  "group_id": "pd",  "group_index": 0,  "group_override": false,  "rules": [    {      "group_id": "pd",      "id": "voters",      "start_key": "",      "end_key": "",      "role": "voter",      "count": 2,      "location_labels": ["dc", "logic", "rack", "host"],      "label_constraints": [{"key": "dc", "op": "in", "values": ["dc2"]}]    },    {      "group_id": "pd",      "id": "followers",      "start_key": "",      "end_key": "",      "role": "follower",      "count": 1,      "location_labels": ["dc", "logic", "rack", "host"],      "label_constraints": [{"key": "logic", "op": "in", "values": ["logic2"]}]    },    {      "group_id": "pd",      "id": "learners",      "start_key": "",      "end_key": "",      "role": "learner",      "count": 1,      "location_labels": ["dc", "logic", "rack", "host"],      "label_constraints": [{"key": "logic", "op": "in", "values": ["logic1"]}]    }  ]}]
复制代码

四、切换过程

PD Leader 切换

[tidb@tidb1 dr_auto_sync]$ tiup ctl:v7.1.5 pd -u http://10.211.55.20:2379 member leader transfer pd-10.211.55.23-2379Starting component ctl: /home/tidb/.tiup/components/ctl/v7.1.5/ctl pd -u http://10.211.55.20:2379 member leader transfer pd-10.211.55.23-2379Success![tidb@tidb1 dr_auto_sync]$ tiup ctl:v7.1.5 pd -u http://10.211.55.20:2379 member leader showStarting component ctl: /home/tidb/.tiup/components/ctl/v7.1.5/ctl pd -u http://10.211.55.20:2379 member leader show{"name": "pd-10.211.55.23-2379","member_id": 11296243912059560318,"peer_urls": [  "http://10.211.55.23:2380"],"client_urls": [  "http://10.211.55.23:2379"]}
复制代码

PD 优先级

调高 dc2 中 pd 的优先级级别,调小 dc1 的 pd 的优先级别


[tidb@tidb1 dr_auto_sync]$ tiup ctl:v7.1.5 pd -u http://10.211.55.20:2379 member leader_priority pd-10.211.55.20-2379 1Starting component ctl: /home/tidb/.tiup/components/ctl/v7.1.5/ctl pd -u http://10.211.55.20:2379 member leader_priority pd-10.211.55.20-2379 1Success![tidb@tidb1 dr_auto_sync]$ [tidb@tidb1 dr_auto_sync]$ tiup ctl:v7.1.5 pd -u http://10.211.55.20:2379 member leader_priority pd-10.211.55.23-2379 5Starting component ctl: /home/tidb/.tiup/components/ctl/v7.1.5/ctl pd -u http://10.211.55.20:2379 member leader_priority pd-10.211.55.23-2379 5Success!
复制代码

修改 Placement-rules

[tidb@tidb1 dr_auto_sync]$ tiup ctl:v7.1.5 pd -u http://10.211.55.20:2379 config placement-rules rule-bundle save --in="/home/tidb/dr_auto_sync/rules_reverse.json" Starting component ctl: /home/tidb/.tiup/components/ctl/v7.1.5/ctl pd -u http://10.211.55.20:2379 config placement-rules rule-bundle save --in=/home/tidb/dr_auto_sync/rules_reverse.json"Update rules and groups successfully."[tidb@tidb1 dr_auto_sync]$ tiup ctl:v7.1.5 pd -u http://10.211.55.20:2379 config placement-rules show Starting component ctl: /home/tidb/.tiup/components/ctl/v7.1.5/ctl pd -u http://10.211.55.20:2379 config placement-rules show[{  "group_id": "pd",  "id": "followers",  "start_key": "",  "end_key": "",  "role": "follower",  "is_witness": false,  "count": 1,  "label_constraints": [    {      "key": "logic",      "op": "in",      "values": [        "logic2"      ]    }  ],  "location_labels": [    "dc",    "logic",    "rack",    "host"  ],  "create_timestamp": 1719635156},{  "group_id": "pd",  "id": "learners",  "start_key": "",  "end_key": "",  "role": "learner",  "is_witness": false,  "count": 1,  "label_constraints": [    {      "key": "logic",      "op": "in",      "values": [        "logic1"      ]    }  ],  "location_labels": [    "dc",    "logic",    "rack",    "host"  ],  "version": 1},{  "group_id": "pd",  "id": "voters",  "start_key": "",  "end_key": "",  "role": "voter",  "is_witness": false,  "count": 2,  "label_constraints": [    {      "key": "dc",      "op": "in",      "values": [        "dc2"      ]    }  ],  "location_labels": [    "dc",    "logic",    "rack",    "host"  ],  "version": 1}]
复制代码

五、切换后校验

检查主备同步状态

符合预期主备状态同步正常


检查副本分布状态

副本的 leader 角色全部切换至 dc2 中心


检查数据一致性

[tidb@tidb1 dr_auto_sync]$ mysql -h10.211.55.20 -P4000 -uroot -prootWelcome to the MariaDB monitor. Commands end with ; or \g.Your MySQL connection id is 405Server version: 5.7.25-TiDB-v7.1.5 TiDB Server (Apache License 2.0) Enterprise Edition, MySQL 5.7 compatibleCopyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.Reading table information for completion of table and column namesYou can turn off this feature to get a quicker startup with -ADatabase changedMySQL [test]> select count(*) from test_part;+----------+| count(*) |+----------+| 1190420 |+----------+1 row in set (0.78 sec)
复制代码


发布于: 刚刚阅读数: 3
用户头像

TiDB 社区官网:https://tidb.net/ 2021-12-15 加入

TiDB 社区干货传送门是由 TiDB 社区中布道师组委会自发组织的 TiDB 社区优质内容对外宣布的栏目,旨在加深 TiDBer 之间的交流和学习。一起构建有爱、互助、共创共建的 TiDB 社区 https://tidb.net/

评论

发布
暂无评论
记一次dr_auto_sync计划内的切换测试过程_7.x 实践_TiDB 社区干货传送门_InfoQ写作社区