写点什么

pd-ctl 选项 --jq 格式化语法使用案例详解

  • 2022 年 8 月 26 日
    北京
  • 本文字数:14667 字

    阅读完需:约 48 分钟

作者: OnTheRoad 原文来源:https://tidb.net/blog/d71e3609

1. json 格式化工具 jq

在阅读关于 PD Control 工具使用的文档时,发现文档未对 –jq 格式化选项的语法作太多详细介绍。于是,诞生了本文档,希望对如我这种新手小白有一定的帮助

1.1. jq 简介

在介绍 pd-ctl 的命令行选项 –jq 之前,首先了解一下 Linux 中的命令行工具 jq。jq 是一个命令行的 JSON 文本格式化工具。jq 将给定的 Filter(过滤器)应用于其输入的 JSON 文本中,并将执行结果以 JSON 格式返回到标准输出中。


官方文档链接:https://stedolan.github.io/jq/manual/

1.2. jq 安装

# Install For CentOS7/RHEL7~]$ sudo yum install -y jq
# Install For Ubuntu/Debian~]$ sudo apt-get install jq
复制代码

1.3. jq 使用

1.3.1. 语法格式

jq [选项] <过滤器> [待处理的文件]
复制代码


~]$ jq --help
jq - commandline JSON processor [version 1.6]
Usage: jq [options] <jq filter> [file...] jq [options] --args <jq filter> [strings...] jq [options] --jsonargs <jq filter> [JSON_TEXTS...]
jq is a tool for processing JSON inputs, applying the given filter to its JSON text inputs and producing the filter's results as JSON on standard output.
The simplest filter is ., which copies jq's input to its output unmodified (except for formatting, but note that IEEE754 is used for number representation internally, with all that that implies).
For more advanced filters see the jq(1) manpage ("man jq") and/or https://stedolan.github.io/jq
Example:
$ echo '{"foo": 0}' | jq . { "foo": 0 }
Some of the options include: -c compact instead of pretty-printed output; -n use `null` as the single input value; -e set the exit status code based on the output; -s read (slurp) all inputs into an array; apply filter to it; -r output raw strings, not JSON texts; -R read raw strings, not JSON texts; -C colorize JSON; -M monochrome (don't colorize JSON); -S sort keys of objects on output; --tab use tabs for indentation; --arg a v set variable $a to value <v>; --argjson a v set variable $a to JSON value <v>; --slurpfile a f set variable $a to an array of JSON texts read from <f>; --rawfile a f set variable $a to a string consisting of the contents of <f>; --args remaining arguments are string arguments, not files; --jsonargs remaining arguments are JSON arguments, not files; -- terminates argument processing;
Named arguments are also available as $ARGS.named[], whilepositional arguments are available as $ARGS.positional[].
See the manpage for more options.
复制代码

1.3.2. jq 常用示例

1.3.2.1. 准备测试文本

首先,准备 2 份测试文件。文件 1(jsonstr.txt)内容为一段未格式化的 JSON 字符串;文件 2(jsonfmt.txt)内容为一段格式化后的 JSON 文本。


~]$ cat > jsonstr.txt << EOF{"Skills": ["Oracle", "Python", "MySQL", "Latex"], "Age": 38, "Birthday": "19th Jan", "Name": "Jack", "Email": "Jack@outlook.com", "Education":{"University":"LNTU","College":"Electronics & Information Engineering","Professonal":"Computer Science & Technology","Year":"2007"}}EOF
复制代码


{  "count": 3,  "stores": [    {      "store": {        "id": 1,        "address": "192.168.3.225:20160",        "version": "6.1.0",        "status_address": "192.168.3.225:20180",        "git_hash": "080d086832ae5ce2495352dccaf8df5d40f30687",        "start_timestamp": 1659078152,        "deploy_path": "/tidb-deploy/tikv-20160/bin",        "last_heartbeat": 1660965837130921869,        "state_name": "Up"      },      "status": {        "capacity": "19.56GiB",        "available": "15.7GiB",        "used_size": "1.01GiB",        "leader_count": 6,        "leader_weight": 1,        "leader_score": 6,        "leader_size": 417,        "region_count": 22,        "region_weight": 1,        "region_score": 6859819744.928351,        "region_size": 1811,        "slow_score": 1,        "start_ts": "2022-07-29T15:02:32+08:00",        "last_heartbeat_ts": "2022-08-20T11:23:57.130921869+08:00",        "uptime": "524h21m25.130921869s"      }    },    {      "store": {        "id": 4,        "address": "192.168.3.224:20160",        "version": "6.1.0",        "status_address": "192.168.3.224:20180",        "git_hash": "080d086832ae5ce2495352dccaf8df5d40f30687",        "start_timestamp": 1659078147,        "deploy_path": "/tidb-deploy/tikv-20160/bin",        "last_heartbeat": 1660965841338055613,        "state_name": "Up"      },      "status": {        "capacity": "19.56GiB",        "available": "15.69GiB",        "used_size": "1.011GiB",        "leader_count": 11,        "leader_weight": 1,        "leader_score": 11,        "leader_size": 932,        "region_count": 22,        "region_weight": 1,        "region_score": 6861359887.412361,        "region_size": 1811,        "slow_score": 1,        "start_ts": "2022-07-29T15:02:27+08:00",        "last_heartbeat_ts": "2022-08-20T11:24:01.338055613+08:00",        "uptime": "524h21m34.338055613s"      }    },    {      "store": {        "id": 5,        "address": "192.168.3.226:20160",        "version": "6.1.0",        "status_address": "192.168.3.226:20180",        "git_hash": "080d086832ae5ce2495352dccaf8df5d40f30687",        "start_timestamp": 1659078162,        "deploy_path": "/tidb-deploy/tikv-20160/bin",        "last_heartbeat": 1660965837041716427,        "state_name": "Up"      },      "status": {        "capacity": "19.56GiB",        "available": "15.69GiB",        "used_size": "1.031GiB",        "leader_count": 5,        "leader_weight": 1,        "leader_score": 5,        "leader_size": 462,        "region_count": 22,        "region_weight": 1,        "region_score": 6862644983.297757,        "region_size": 1811,        "slow_score": 1,        "start_ts": "2022-07-29T15:02:42+08:00",        "last_heartbeat_ts": "2022-08-20T11:23:57.041716427+08:00",        "uptime": "524h21m15.041716427s"      }    }  ]}
复制代码


注意】该测试文本为一个 TiDB 集群数据库中的 store 信息。可通过如下方式获取类似格式的 JSON 文本。

> ~]$ tiup ctl:v6.1.0 pd -u http://192.168.3.221:2379 store > jsonfmt.txt
> ```
>
> 其中 `http://192.168.3.221:2379` 为 PD 实例的地址,获取的json信息通过`>`重定向输入到`jsonfmt.txt`。

#### 1.3.2.2. JSON 字符串格式化

1. 直接通过 jq 命令,格式化文本


~]$ jq –tab -S ‘.’ jsonstr.txt{ “Skills”: [ “1.Oracle”, “2.Python”, “3.MySQL”, “4.Latex” ], “Age”: 38, “Birthday”: “19th Jan”, “Name”: “Jack”, “Email”: “Jack@outlook.com”, “Education”: { “University”: “LNTU”, “College”: “Electronics & Information Engineering”, “Professonal”: “Computer Science & Technology”, “Year”: “2007” }}


这里的过滤器`'.'`,表示只对输入的内容做 JSON 格式化,而不修改其内容,效果与下面的过滤器 'values' 一致。


~]$ cat jsonstr.txt | jq ‘values’{ “Skills”: [ “1.Oracle”, “2.Python”, “3.MySQL”, “4.Latex” ], “Age”: 38, “Birthday”: “19th Jan”, “Name”: “Jack”, “Email”: “Jack@outlook.com”, “Education”: { “University”: “LNTU”, “College”: “Electronics & Information Engineering”, “Professnal”: “Computer Science & Technology”, “Year”: “2007” }}


2. 通过管道符调用 jq,格式化文本


~]$ cat jsonstr.txt | jq –tab -S ‘.’{ “Age”: 38, “Birthday”: “19th Jan”, “Education”: { “College”: “Electronics & Information Engineering”, “Professonal”: “Computer Science & Technology”, “University”: “LNTU”, “Year”: “2007” }, “Email”: “Jack@outlook.com”, “Name”: “Jack”, “Skills”: [ “1.Oracle”, “2.Python”, “3.MySQL”, “4.Latex” ]}


默认的缩进为 2 个空格,`--tab` 表示将缩进替换为 tab 制表符;`-S` 表示对格式化后的 JSON 文本按 `KEY` 排序。

> **【注意】**为简化描述,避免文档过于冗长,后续示例均以管道方式来描述 jq 的使用。

#### 1.3.2.3. 以列表形式获取 Key(只能获取 1 级的 Key 信息)


~]$ cat jsonstr.txt | jq ‘keys’[ “Age”, “Birthday”, “Education”, “Email”, “Name”, “Skills”]


这里的 `keys` 为关键字

#### 1.3.2.4. 提取指定 Key 的 Value(Key Filter)


jq ‘.Key1 名称, .Key2 名称, …’


1. 获取某个 Key 的 Value

当 JSON 文本过长,只想提取其中某个 Key 的 Value。可通过如下语法完成。


~]$ jq ‘.Education’ jsonstr.txt{ “University”: “LNTU”, “College”: “Electronics & Information Engineering”, “Professonal”: “Computer Science & Technology”, “Year”: “2007”}


~]$ cat jsonstr.txt |jq ‘.Education’{ “University”: “LNTU”, “College”: “Electronics & Information Engineering”, “Professonal”: “Computer Science & Technology”, “Year”: “2007”}



2. 获取某几个 Key 的 Value
逗号分隔多个 Key,可获取指定的多个 Key 的 Value。

复制代码


~]$ cat jsonstr.txt | jq ‘.Education, .Skills’{ “University”: “LNTU”, “College”: “Electronics & Information Engineering”, “Professnal”: “Computer Science & Technology”, “Year”: “2007”}[ “1.Oracle”, “2.Python”, “3.MySQL”, “4.Latex”]



#### 1.3.2.5. 提取指定 Key 的 KV(Key-Value Filter)

复制代码


jq ‘{Key1 名称}, {Key2 名称}, …’



1. 获取某个 Key 的 KV

复制代码


~]$ cat jsonstr.txt | jq ‘{Education}’{ “Education”: { “University”: “LNTU”, “College”: “Electronics & Information Engineering”, “Professnal”: “Computer Science & Technology”, “Year”: “2007” }}



2. 获取多个 Key 的 KV

复制代码


~]$ cat jsonstr.txt | jq ‘{Education}, {Skills}’{ “Education”: { “University”: “LNTU”, “College”: “Electronics & Information Engineering”, “Professnal”: “Computer Science & Technology”, “Year”: “2007” }}{ “Skills”: [ “1.Oracle”, “2.Python”, “3.MySQL”, “4.Latex” ]}



#### 1.3.2.6. 通过 ‘.\<Key名称>\[]’ 递归指定 Key 下的 Value 信息
1. 递归获取 1 级的所有 Value
不指定 Key 名称时,则获取 1 级的 Value 信息

复制代码


~]$ cat jsonstr.txt | jq ‘.[]’[ “1.Oracle”, “2.Python”, “3.MySQL”, “4.Latex”]38“19th Jan”“Jack”“Jack@outlook.com”{ “University”: “LNTU”, “College”: “Electronics & Information Engineering”, “Professnal”: “Computer Science & Technology”, “Year”: “2007”}



2. 获取 `Education`下的所有 Value 信息
指定 Key 名称时,则获取指定 Key 下的所有 Value 信息

复制代码


~]$ cat jsonstr.txt | jq ‘.Education[]’“LNTU”“Electronics & Information Engineering”“Computer Science & Technology”“2007”



> 注意过滤器 `'.Education[]'` 与 `'.Education'` 的区别。>> ```> ~]$ cat jsonstr.txt | jq '.Education'> {> "University": "LNTU",> "College": "Electronics & Information Engineering",> "Professnal": "Computer Science & Technology",> "Year": "2007"> }> ```
#### 1.3.2.7. 通过 select 检索包含指定 KV 的内容

复制代码

通过 .stores[] 递归取出 stores 中每个元素(元素中包含 store 和 status 两部分内容);

再通过 .store 过滤出每个元素中的 store 部分的内容。

~]$ cat jsonfmt.txt | jq ‘.stores[].store’{ “id”: 1, “address”: “192.168.3.225:20160”, “version”: “6.1.0”, “status_address”: “192.168.3.225:20180”, “git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”, “start_timestamp”: 1659078152, “deploy_path”: “/tidb-deploy/tikv-20160/bin”, “last_heartbeat”: 1660966497198077000, “state_name”: “Up”}{ “id”: 4, “address”: “192.168.3.224:20160”, “version”: “6.1.0”, “status_address”: “192.168.3.224:20180”, “git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”, “start_timestamp”: 1659078147, “deploy_path”: “/tidb-deploy/tikv-20160/bin”, “last_heartbeat”: 1660966491404376800, “state_name”: “Up”}{ “id”: 5, “address”: “192.168.3.226:20160”, “version”: “6.1.0”, “status_address”: “192.168.3.226:20180”, “git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”, “start_timestamp”: 1659078162, “deploy_path”: “/tidb-deploy/tikv-20160/bin”, “last_heartbeat”: 1660966497108902000, “state_name”: “Up”}

通过 .stores[].store 过滤出多个 store

对多个 store 应用 select 条件(id ==5)查询,筛选出包含 id == 5 的 store。

~]$ cat jsonfmt.txt | jq ‘.stores[].store | select(.id == 5)’{ “id”: 5, “address”: “192.168.3.226:20160”, “version”: “6.1.0”, “status_address”: “192.168.3.226:20180”, “git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”, “start_timestamp”: 1659078162, “deploy_path”: “/tidb-deploy/tikv-20160/bin”, “last_heartbeat”: 1660966497108902000, “state_name”: “Up”}



#### 1.3.2.8. 列表中的元素提取或切片(Index Filter)
当某个 Key 对应的 Value 是一个列表时,jq 可通过如下语法对 Value 进行元素提取或切片。
1. 列表中单个元素的提取

复制代码


jq ‘.Key 名称[元素下标, 元素下标,,]’




复制代码


~]$ cat jsonstr.txt | jq ‘.Skills[]’“1.Oracle”“2.Python”“3.MySQL”“4.Latex”


~] cat jsonstr.txt | jq ‘.Skills[1]’“2.Python”~] cat jsonstr.txt | jq ‘.Skills[1,3]’“2.Python”“4.Latex”



2. 列表中连续多个元素的切片

复制代码


jq ‘.Key 名称[起始下标: 结束下标(不含)]’



`[切片起始下标:切片结束下标(不含)]` 切片区间是一个左闭右开的区间,即`[起始下标, 结束下标)`,并且 `结束下标` 需大于 `起始下标`。如 `[1,3]` 表示提取下标为 `1 至 3,但不包含 3` 的元素。
当下标为负数时,表示从末尾反向计算下标。倒数第1个元素下标为`-1`,倒数第2个为 `-2` 依次类推。
![image.png](https://tidb-blog.oss-cn-beijing.aliyuncs.com/media/image-1660988702328.png)

复制代码


~] cat jsonstr.txt | jq ‘.Skills[0:1]’[ “1.Oracle”]


~]$ cat jsonstr.txt | jq ‘.Skills[0:2]’[ “1.Oracle”, “2.Python”]


~]$ cat jsonstr.txt | jq ‘.Skills[1:10]’[ “2.Python”, “3.MySQL”, “4.Latex”]


~]$ cat jsonstr.txt | jq ‘.Skills[-2,3]’“3.MySQL”“4.Latex”


~]$ cat jsonstr.txt | jq ‘.Skills[-2:1]’[]



注意与 Python 列表切片的区别
#### 1.3.2.9. 管道嵌套
1. 利用管道符,计算每个 Key 的 Value 长度

复制代码


~]$ cat jsonstr.txt | jq ‘.Education’{ “University”: “LNTU”, “College”: “Electronics & Information Engineering”, “Professnal”: “Computer Science & Technology”, “Year”: “2007”}


~]$ cat jsonstr.txt | jq ‘.Education | length’4



length 过滤器,计算元素长度。对于对象,表示对象里的元素个数;对于字符串,表示字符串字符数;对于列表,表示列表元素个数。
2. 利用管道符,逐级过滤

复制代码


~]$ cat jsonstr.txt | jq ‘.Education | .College’“Electronics & Information Engineering”



#### 1.3.2.10. 递归展开所有层级的 KV

复制代码


~]$ cat jsonstr.txt | jq ‘..’{ “Skills”: [ “1.Oracle”, “2.Python”, “3.MySQL”, “4.Latex” ], “Age”: 38, “Birthday”: “19th Jan”, “Name”: “Jack”, “Email”: “Jack@outlook.com”, “Education”: { “University”: “LNTU”, “College”: “Electronics & Information Engineering”, “Professnal”: “Computer Science & Technology”, “Year”: “2007” }}[ “1.Oracle”, “2.Python”, “3.MySQL”, “4.Latex”]“1.Oracle”“2.Python”“3.MySQL”“4.Latex”38“19th Jan”“Jack”“Jack@outlook.com”{ “University”: “LNTU”, “College”: “Electronics & Information Engineering”, “Professnal”: “Computer Science & Technology”, “Year”: “2007”}“LNTU”“Electronics & Information Engineering”“Computer Science & Technology”“2007”



#### 1.3.2.11. map 或 map\_value 列表遍历
1. 通过 map 遍历每个 KEY,并应用 length 计算每个 Key 对应的 Value 长度
针对对象,计算其元素个数;针对字符串计算其字符串长度。

复制代码


~]$ cat jsonstr.txt | jq ‘map(. | length)’[ 4, 38, 8, 4, 16, 4]



#### 1.3.2.12. 重构 JSON(\[]与{})
用各种 Filter 格式化后的 JSON 内容,可通过 `[]` 和 `{}` 来重新组织,生成新的 JSON 文本。

复制代码


~]$ cat jsonstr.txt | jq ‘.Education, .Skills’{ “University”: “LNTU”, “College”: “Electronics & Information Engineering”, “Professnal”: “Computer Science & Technology”, “Year”: “2007”}[ “1.Oracle”, “2.Python”, “3.MySQL”, “4.Latex”]



1. 利用 `[]`,将输出重新组织成列表

复制代码


~]$ cat jsonstr.txt | jq ‘[.Education, .Skills]’[ { “University”: “LNTU”, “College”: “Electronics & Information Engineering”, “Professnal”: “Computer Science & Technology”, “Year”: “2007” }, [ “1.Oracle”, “2.Python”, “3.MySQL”, “4.Latex” ]]



2. 利用 `{}`,将输出重新组织成 JSON 对象

复制代码


~]$ cat jsonstr.txt | jq ‘{Education, Skills}’{ “Education”: { “University”: “LNTU”, “College”: “Electronics & Information Engineering”, “Professnal”: “Computer Science & Technology”, “Year”: “2007” }, “Skills”: [ “1.Oracle”, “2.Python”, “3.MySQL”, “4.Latex” ]}



3. 利用 `{}`,重构 JSON对象,并指定新的 Key 名称。

复制代码


~]$ cat jsonstr.txt | jq ‘{EducationDetail: .Education, SkillInfo: .Skills}’{ “EducationDetail”: { “University”: “LNTU”, “College”: “Electronics & Information Engineering”, “Professnal”: “Computer Science & Technology”, “Year”: “2007” }, “SkillInfo”: [ “1.Oracle”, “2.Python”, “3.MySQL”, “4.Latex” ]}




复制代码


~]$ cat jsonstr.txt | jq ‘{EducationDetail: {Education}, SkillInfo: {Skills}}’{ “EducationDetail”: { “Education”: { “University”: “LNTU”, “College”: “Electronics & Information Engineering”, “Professnal”: “Computer Science & Technology”, “Year”: “2007” } }, “SkillInfo”: { “Skills”: [ “1.Oracle”, “2.Python”, “3.MySQL”, “4.Latex” ] }}



# 2. PD Control 中通过 jq 格式化输出
## 2.1. pd-ctl 命令的 --jq 选项使用说明
了解 jq 的常见使用方式后,便可通过如下三种方式,格式化输出 PD 的配置信息:
1. 在使用 pd-ctl 工具时指定 `--jq <过滤器>` 来格式化 PD 配置信息的输出。`--jq` 本质上也是调用系统中的 jq 工具,来执行格式化输出。因此,其该参数的语法与 `jq` 完全一致。2. 可利用管道符`|`,将 PD 配置信息的内容重定向到 `jq` 命令,作格式化输出。3. 混合使用 pd-ctl 的 `--jq` 选项与 `jq` 命令。
> **【注意】**因 pd-ctl 的 `--jq`选项本质上就是调用系统中的 `jq` 命令,因此系统中需要安装 `jq` 工具包。
## 2.2. 常用 jq 格式化 PD 配置信息示例
### 2.2.1. 格式化显示 PD 实例信息
1. 格式化显示完整的 PD 调度信息

复制代码


~]$ export PD_ADDR=http://192.168.3.222:2379~]$ tiup ctl:v6.1.0 pd config showStarting component ctl: /home/tidb/.tiup/components/ctl/v6.1.0/ctl pd config show{ “replication”: { “enable-placement-rules”: “true”, “enable-placement-rules-cache”: “false”, “isolation-level”: “”, “location-labels”: “”, “max-replicas”: 3, “strictly-match-label”: “false” }, “schedule”: { “enable-cross-table-merge”: “true”, “enable-joint-consensus”: “true”, “high-space-ratio”: 0.7, “hot-region-cache-hits-threshold”: 3, “hot-region-schedule-limit”: 4, “hot-regions-reserved-days”: 0, “hot-regions-write-interval”: “10m0s”, “leader-schedule-limit”: 4, “leader-schedule-policy”: “count”, “low-space-ratio”: 0.8, “max-merge-region-keys”: 200000, “max-merge-region-size”: 20, “max-pending-peer-count”: 64, “max-snapshot-count”: 64, “max-store-down-time”: “30m0s”, “max-store-preparing-time”: “48h0m0s”, “merge-schedule-limit”: 8, “patrol-region-interval”: “10ms”, “region-schedule-limit”: 2048, “region-score-formula-version”: “v2”, “replica-schedule-limit”: 64, “split-merge-interval”: “1h0m0s”, “tolerant-size-ratio”: 0 }}



2. 格式化显示完整的 Store 信息

复制代码


~]$ tiup ctl:v6.1.0 pd storeStarting component ctl: /home/tidb/.tiup/components/ctl/v6.1.0/ctl pd store{ “count”: 3, “stores”: [ { “store”: { “id”: 1, “address”: “192.168.3.225:20160”, “version”: “6.1.0”, “status_address”: “192.168.3.225:20180”, “git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”, “start_timestamp”: 1659078152, “deploy_path”: “/tidb-deploy/tikv-20160/bin”, “last_heartbeat”: 1660978688453748544, “state_name”: “Up” }, “status”: { “capacity”: “19.56GiB”, “available”: “15.7GiB”, “used_size”: “1.011GiB”, “leader_count”: 6, “leader_weight”: 1, “leader_score”: 6, “leader_size”: 417, “region_count”: 22, “region_weight”: 1, “region_score”: 6860013051.732055, “region_size”: 1811, “slow_score”: 1, “start_ts”: “2022-07-29T15:02:32+08:00”, “last_heartbeat_ts”: “2022-08-20T14:58:08.453748544+08:00”, “uptime”: “527h55m36.453748544s” } }, { “store”: { “id”: 4, “address”: “192.168.3.224:20160”, “version”: “6.1.0”, “status_address”: “192.168.3.224:20180”, “git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”, “start_timestamp”: 1659078147, “deploy_path”: “/tidb-deploy/tikv-20160/bin”, “last_heartbeat”: 1660978692660915975, “state_name”: “Up” }, “status”: { “capacity”: “19.56GiB”, “available”: “15.69GiB”, “used_size”: “1.011GiB”, “leader_count”: 11, “leader_weight”: 1, “leader_score”: 11, “leader_size”: 932, “region_count”: 22, “region_weight”: 1, “region_score”: 6861560216.947071, “region_size”: 1811, “slow_score”: 1, “start_ts”: “2022-07-29T15:02:27+08:00”, “last_heartbeat_ts”: “2022-08-20T14:58:12.660915975+08:00”, “uptime”: “527h55m45.660915975s” } }, { “store”: { “id”: 5, “address”: “192.168.3.226:20160”, “version”: “6.1.0”, “status_address”: “192.168.3.226:20180”, “git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”, “start_timestamp”: 1659078162, “deploy_path”: “/tidb-deploy/tikv-20160/bin”, “last_heartbeat”: 1660978688364585582, “state_name”: “Up” }, “status”: { “capacity”: “19.56GiB”, “available”: “15.69GiB”, “used_size”: “1.031GiB”, “leader_count”: 5, “leader_weight”: 1, “leader_score”: 5, “leader_size”: 462, “region_count”: 22, “region_weight”: 1, “region_score”: 6862846131.837539, “region_size”: 1811, “slow_score”: 1, “start_ts”: “2022-07-29T15:02:42+08:00”, “last_heartbeat_ts”: “2022-08-20T14:58:08.364585582+08:00”, “uptime”: “527h55m26.364585582s” } } ]}



3. 格式化显示完整的 Region 信息

复制代码


~]$ tiup ctl:v6.1.0 pd region | jq ‘.’Starting component ctl: /home/tidb/.tiup/components/ctl/v6.1.0/ctl pd region{ “count”: 22, “regions”: [ { “id”: 172, “start_key”: “7480000000000000FF5C00000000000000F8”, “end_key”: “7480000000000000FF5E00000000000000F8”, “epoch”: { “conf_ver”: 47, “version”: 68 }, “peers”: [ { “id”: 173, “store_id”: 1, “role_name”: “Voter” }, { “id”: 4467, “store_id”: 4, “role_name”: “Voter” }, { “id”: 4497, “store_id”: 5, “role_name”: “Voter” } ], “leader”: { “id”: 4467, “store_id”: 4, “role_name”: “Voter” }, “written_bytes”: 0, “read_bytes”: 0, “written_keys”: 0, “read_keys”: 0, “approximate_size”: 85, “approximate_keys”: 1079434 }, ……


]}



### 2.2.2. 按需查询 Store 信息
#### 2.2.2.1. 简化查询 Store 信息
1. 通过 `KV Filter` 过滤出 `store` 与 `status` 完整信息

复制代码


~]$ tiup ctl:v6.1.0 pd store –jq=“.stores[]” |jq ‘{store}, {status}’Starting component ctl: /home/tidb/.tiup/components/ctl/v6.1.0/ctl pd store –jq=.stores[]{ “store”: { “id”: 4, “address”: “192.168.3.224:20160”, “version”: “6.1.0”, “status_address”: “192.168.3.224:20180”, “git_hash”: “080d086832ae5ce2495352dccaf8df5d40f30687”, “start_timestamp”: 1659078147, “deploy_path”: “/tidb-deploy/tikv-20160/bin”, “last_heartbeat”: 1660980592856686300, “state_name”: “Up” }}{ “status”: { “capacity”: “19.56GiB”, “available”: “15.69GiB”, “used_size”: “1.012GiB”, “leader_count”: 11, “leader_weight”: 1, “leader_score”: 11, “leader_size”: 932, “region_count”: 22, “region_weight”: 1, “region_score”: 6861587385.861746, “region_size”: 1811, “slow_score”: 1, “start_ts”: “2022-07-29T15:02:27+08:00”, “last_heartbeat_ts”: “2022-08-20T15:29:52.856686305+08:00”, “uptime”: “528h27m25.856686305s” }}……



2. 筛选字段,重构 JSON 文本
首先,通过 `KV Filter` 筛选出 `store.id`、`store.address`、`store.state_name`、`status.available` 字段;再通过 `{}` 重构 JSON 内容,重构时指定新的 KEY 字段名称。

复制代码


~]$ tiup ctl:v6.1.0 pd store –jq=“.stores[]” |jq ‘{store_id: .store.id, addr: .store.address, state: .store.state_name, freespace: .status.available}’Starting component ctl: /home/tidb/.tiup/components/ctl/v6.1.0/ctl pd store –jq=.stores[]{ “store_id”: 1, “addr”: “192.168.3.225:20160”, “state”: “Up”, “freespace”: “15.7GiB”}{ “store_id”: 4, “addr”: “192.168.3.224:20160”, “state”: “Up”, “freespace”: “15.69GiB”}{ “store_id”: 5, “addr”: “192.168.3.226:20160”, “state”: “Up”, “freespace”: “15.69GiB”}



#### 2.2.2.2. 按序查询状态为 Down 的 Store 信息

复制代码


~]$ tiup ctl:v6.1.0 pd store –jq=“.stores[]” |jq ‘{store_id: .store.id, addr: .store.address, state: .store.state_name, freespace: .status.available}’ | jq ‘. | select(.state=“Up”)’



#### 2.2.2.3. 查询 TiKV 的信息
判断 Store 为 TiKV 的依据为 “如果 Store 的部署目录为 `/tidb-deploy/tikv-20160/bin`,则可断定其为 TiKV”。依据的前提是,各组件均规范化部署。

复制代码


~]$ tiup ctl:v6.1.0 pd store –jq=“.stores[]” |jq ‘{store_id: .store.id, addr: .store.address, state: .store.state_name, path: .store.deploy_path, freespace: .status.available}’ |jq ‘. |select(.path == “/tidb-deploy/tikv-20160/bin”)’


Starting component ctl: /home/tidb/.tiup/components/ctl/v6.1.0/ctl pd store –jq=.stores[]{ “store_id”: 1, “addr”: “192.168.3.225:20160”, “state”: “Up”, “path”: “/tidb-deploy/tikv-20160/bin”, “freespace”: “15.7GiB”}{ “store_id”: 4, “addr”: “192.168.3.224:20160”, “state”: “Up”, “path”: “/tidb-deploy/tikv-20160/bin”, “freespace”: “15.69GiB”}{ “store_id”: 5, “addr”: “192.168.3.226:20160”, “state”: “Up”, “path”: “/tidb-deploy/tikv-20160/bin”, “freespace”: “15.69GiB”}



### 2.2.3. 按序查询 Region 信息
#### 2.2.3.1. 查看 Region 总体分布情况
1. 查看 Region 分布情况,包括 Region ID、副本分布、Leader分布

复制代码


~]$ tiup ctl:v6.1.0 pd region | jq ‘.regions[] | {region_id: .id, peer_stores: [.peers[].store_id], leader_store: .leader.store_id}’Starting component ctl: /home/tidb/.tiup/components/ctl/v6.1.0/ctl pd region{ “region_id”: 192, “peer_stores”: [ 1, 5, 4 ], “leader_store”: 4}{ “region_id”: 244, “peer_stores”: [ 5, 1, 4 ], “leader_store”: 5}……



2. 查看 Region 分布情况,包括 Region ID、起始Key值、副本分布、Leader分布

复制代码


~]$ tiup ctl:v6.1.0 pd region | jq ‘.regions[] | {region_id: .id, key: [.start_key,.end_key], peer_stores: [.peers[].store_id], leader_store: .leader.store_id}’


Starting component ctl: /home/tidb/.tiup/components/ctl/v6.1.0/ctl pd region{ “region_id”: 196, “key”: [ “7480000000000000FF645F728000000000FF04F8C70000000000FA”, “7480000000000000FF645F728000000000FF0630A00000000000FA” ], “peer_stores”: [ 1, 4, 5 ], “leader_store”: 4}{ “region_id”: 204, “key”: [ “7480000000000000FF645F728000000000FF09CDB50000000000FA”, “7480000000000000FF645F728000000000FF0D6E840000000000FA” ], “peer_stores”: [ 5, 4, 1 ], “leader_store”: 4}……



#### 2.2.3.2. 根据 Region 副本数,过滤 Region
查找副本数不等于3的Region

复制代码


~]$ tiup ctl:v6.1.0 pd region –jq=“.regions[]” | jq ‘{region_id: .id, peer_stores: [.peers[].store_id] | select(length != 3)}’


Starting component ctl: /home/tidb/.tiup/components/ctl/v6.1.0/ctl pd region –jq=.regions[]{ “region_id”: 252, “peer_stores”: [ 4, 5 ]}{ “region_id”: 216, “peer_stores”: [ 1, 4 ]}



#### 2.2.3.3. 查询指定 Store 上的 Region
查看 Store ID 为 4 上的 Region。

复制代码


~]$ tiup ctl:v6.1.0 pd region |jq ‘.regions[] | {region_id: .id, peer_stores: [.peers[].store_id] | select(any(.==4))}’


Starting component ctl: /home/tidb/.tiup/components/ctl/v6.1.0/ctl pd region{ “region_id”: 252, “peer_stores”: [ 1, 4, 5 ]}{ “region_id”: 216, “peer_stores”: [ 5, 1, 4 ]}……




复制代码


~]$ tiup ctl:v6.1.0 pd region |jq ‘.regions[] | {region_id: .id, peer_stores: [.peers[].store_id] | select(any(.==(1,4)))}’



#### 2.2.3.4. 查询指定 Store 上的 Leader
查询指定 Store 上的 Leade,以及该 Leader 的其他副本分布情况。

复制代码


~]$ tiup ctl:v6.1.0 pd region | jq ‘.regions[] | select(.leader.store_id == 1) | {region_id: .id, peer_stores: [.peers[].store_id]}’Starting component ctl: /home/tidb/.tiup/components/ctl/v6.1.0/ctl pd region{ “region_id”: 200, “peer_stores”: [ 5, 4, 1 ]}{ “region_id”: 180, “peer_stores”: [ 5, 4, 1 ]}```


发布于: 刚刚阅读数: 4
用户头像

TiDB 社区官网:https://tidb.net/ 2021.12.15 加入

TiDB 社区干货传送门是由 TiDB 社区中布道师组委会自发组织的 TiDB 社区优质内容对外宣布的栏目,旨在加深 TiDBer 之间的交流和学习。一起构建有爱、互助、共创共建的 TiDB 社区 https://tidb.net/

评论

发布
暂无评论
pd-ctl 选项 --jq 格式化语法使用案例详解_性能调优_TiDB 社区干货传送门_InfoQ写作社区