Easysearch 性能测试方法概要
作者：极限实验室
- 2024-08-26 北京
本文字数：5246 字
阅读完需：约 17 分钟
INFINI Easysearch
INFINI Easysearch 是一个分布式的近实时搜索与分析引擎，核心引擎基于开源的 Apache Lucene。Easysearch 衍生自基于开源协议 Apache 2.0 的 Elasticsearch 7.10 版本，完善和支持更多的企业级功能，优化搜索业务场景，以保证更佳的数据探索与分析体验。
Easysearch 的主要特点：
兼容 Elasticsearch，业务代码无需改动；可平滑迁移
企业级安全；更稳定可靠
企业级管理后台（多集群统一管理，实现运营标准化、自动化）
信创适配（全面适配国产 CPU/OS/服务器，支持国密算法）
Easysearch 快速体验，请参阅：
Loadgen
Loadgen 是 Elasticsearch 专属压测工具，用来对 Easysearch 或者网关进行压力测试。
Loadgen 的特点：
性能强劲
轻量级无依赖
支持模板化参数随机
支持高并发
支持压测端均衡流量控制
支持服务端返回值校验
当前最新版本为 1.26.1-598，下载链接如下：
https://release.infinilabs.com/loadgen/stable/
下载并解压安装包后，得到二进制文件。
查看版本信息。
$ ./loadgen-linux-amd64 -v __ ___ _ ___ ___ __ __ / / /___\/_\ / \/ _ \ /__\/\ \ \ / / // ///_\\ / /\ / /_\//_\ / \/ // /__/ \_// _ \/ /_// /_\\//__/ /\ /\____|___/\_/ \_/___,'\____/\__/\_\ \/
[LOADGEN] A http load generator and testing suite.[LOADGEN] 1.26.1#598, 2024-08-14 04:50:03, 2024-12-31 10:10:10, 00f15fd86834d7ea539f6d66ee608e3915eef0e3loadgen 1.26.1 598 2024-08-14 04:50:03 +0000 UTC 2024-12-31 10:10:10 +0000 UTC 00f15fd86834d7ea539f6d66ee608e3915eef0e3
复制代码
Loadgen 测试
Loadgen 使用非常简单，下面演示如何进行查询压测。
建立索引，根据节点数调节分片数。
curl -X PUT -H "Content-Type: application/json" \-ku admin:xxx "https://localhost:9200/loadtest" -d'{ "settings":{ "number_of_shards":"3", "number_of_replicas":1, "translog":{ "durability":"async" } }}'{"acknowledged":true,"shards_acknowledged":true,"index":"loadtest"}
复制代码
创建
loadgen.dsl.shawnyan配置文件，定义查询
GET loadtest/_search{"query": {"term": {"id.keyword": {"value": $[[id]]}}}}
复制代码
执行压测，
-d表示秒数，-c表示并发数。
$ ES_ENDPOINT=https://localhost:9200 ES_USERNAME=admin ES_PASSWORD=xxx ./loadgen-linux-amd64 -run loadgen.dsl.shawnyan -d 10 -c 2 __ ___ _ ___ ___ __ __ / / /___\/_\ / \/ _ \ /__\/\ \ \ / / // ///_\\ / /\ / /_\//_\ / \/ // /__/ \_// _ \/ /_// /_\\//__/ /\ /\____|___/\_/ \_/___,'\____/\__/\_\ \/
[LOADGEN] A http load generator and testing suite.[LOADGEN] 1.26.1#598, 2024-08-14 04:50:03, 2024-12-31 10:10:10, 00f15fd86834d7ea539f6d66ee608e3915eef0e3[08-19 16:43:27] [INF] [env.go:179] configuration auto reload enabled[08-19 16:43:27] [INF] [app.go:285] initializing loadgen, pid: 30792[08-19 16:43:27] [INF] [app.go:286] using config: /data/es/loadgen.yml[08-19 16:43:27] [INF] [module.go:155] started plugin: statsd[08-19 16:43:27] [INF] [module.go:161] all modules are started[08-19 16:43:27] [INF] [instance.go:78] workspace: /data/es/data/loadgen/nodes/cr1gabqr90cng685o2s0[08-19 16:43:27] [INF] [app.go:511] loadgen is up and running now.[08-19 16:43:27] [INF] [main.go:403] loading config: /data/es/loadgen.dsl.shawnyan
21108 requests finished in 9.79061677s, 1.08MB sent, 3.22MB received
[Loadgen Client Metrics]Requests/sec: 2110.63Request Traffic/sec: 110.22KBTotal Transfer/sec: 440.00KBFastest Request: 680.198µsSlowest Request: 12.409574msStatus 200: 21108
[Latency Metrics]10000 samples of 21108 eventsCumulative: 8.732205871sHMean: 851.869µsAvg.: 873.22µsp50: 839.498µsp75: 914.298µsp95: 1.059197msp99: 1.327098msp999: 4.83579msLong 5%: 1.394411msShort 5%: 724.226µsMax: 11.618475msMin: 680.198µsRange: 10.938277msStdDev: 289.216µsRate/sec.: 2110.63
[Latency Distribution] 680µs - 1.774ms ------------------------------ 1.774ms - 2.867ms - 2.867ms - 3.961ms - 3.961ms - 5.055ms - 5.055ms - 6.149ms - 6.149ms - 7.243ms - 7.243ms - 8.336ms - 8.336ms - 9.43ms - 9.43ms - 10.524ms - 10.524ms - 11.618ms -
[Estimated Server Metrics]Requests/sec: 2155.94Avg Req Time: 927.668µsTransfer/sec: 449.45KB
复制代码
更多内容请参阅文档：
https://infinilabs.cn/docs/latest/gateway/getting-started/benchmark/
Rally
如果对 Elasticsearch 熟悉的同学，也可尝试使用 Rally 对 Easysearch 进行测试。
Rally 是 Elasticsearch 的基准测试框架。它可以帮助您完成以下任务：
安装 Elasticsearch 集群以进行基准测试
跨 Elasticsearch 版本管理基准数据和规范
运行基准测试并记录结果
通过连接遥测设备来查找性能问题
比较性能结果
安装 Rally。
[root@easysearch /]# pip3 install esrallyRequirement already satisfied: esrally in /usr/local/lib/python3.9/site-packages (2.11.0)Requirement already satisfied: py-cpuinfo==7.0.0 in /usr/local/lib/python3.9/site-packages (from esrally) (7.0.0)Requirement already satisfied: elastic-transport==8.4.1 in /usr/local/lib/python3.9/site-packages (from esrally) (8.4.1)Requirement already satisfied: google-resumable-media[requests]==1.1.0 in /usr/local/lib/python3.9/site-packages (from esrally) (1.1.0)Requirement already satisfied: urllib3==1.26.18 in /usr/local/lib/python3.9/site-packages (from esrally) (1.26.18)Requirement already satisfied: google-auth==1.22.1 in /usr/local/lib/python3.9/site-packages (from esrally) (1.22.1)Requirement already satisfied: thespian==3.10.1 in /usr/local/lib/python3.9/site-packages (from esrally) (3.10.1)Requirement already satisfied: jsonschema==3.1.1 in /usr/local/lib/python3.9/site-packages (from esrally) (3.1.1)Requirement already satisfied: markupsafe==2.0.1 in /usr/local/lib64/python3.9/site-packages (from esrally) (2.0.1)Requirement already satisfied: yappi==1.5.1 in /usr/local/lib64/python3.9/site-packages (from esrally) (1.5.1)Requirement already satisfied: psutil==5.9.4 in /usr/local/lib64/python3.9/site-packages (from esrally) (5.9.4)Requirement already satisfied: certifi in /usr/local/lib/python3.9/site-packages (from esrally) (2024.7.4)Requirement already satisfied: elasticsearch[async]==8.6.1 in /usr/local/lib/python3.9/site-packages (from esrally) (8.6.1)Requirement already satisfied: ijson==2.6.1 in /usr/local/lib/python3.9/site-packages (from esrally) (2.6.1)Requirement already satisfied: jinja2==3.1.4 in /usr/local/lib/python3.9/site-packages (from esrally) (3.1.4)Requirement already satisfied: requests<2.32.0 in /usr/local/lib/python3.9/site-packages (from esrally) (2.31.0)Requirement already satisfied: tabulate==0.8.9 in /usr/local/lib/python3.9/site-packages (from esrally) (0.8.9)Requirement already satisfied: zstandard==0.21.0 in /usr/local/lib64/python3.9/site-packages (from esrally) (0.21.0)Requirement already satisfied: docker==6.0.0 in /usr/local/lib/python3.9/site-packages (from esrally) (6.0.0)Requirement already satisfied: websocket-client>=0.32.0 in /usr/local/lib/python3.9/site-packages (from docker==6.0.0->esrally) (1.8.0)Requirement already satisfied: packaging>=14.0 in /usr/lib/python3.9/site-packages (from docker==6.0.0->esrally) (20.9)Requirement already satisfied: aiohttp<4,>=3 in /usr/local/lib64/python3.9/site-packages (from elasticsearch[async]==8.6.1->esrally) (3.10.4)Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.9/site-packages (from google-auth==1.22.1->esrally) (0.4.0)Requirement already satisfied: six>=1.9.0 in /usr/lib/python3.9/site-packages (from google-auth==1.22.1->esrally) (1.15.0)Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.9/site-packages (from google-auth==1.22.1->esrally) (4.9)Requirement already satisfied: setuptools>=40.3.0 in /usr/lib/python3.9/site-packages (from google-auth==1.22.1->esrally) (53.0.0)Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.9/site-packages (from google-auth==1.22.1->esrally) (4.2.4)Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in /usr/local/lib64/python3.9/site-packages (from google-resumable-media[requests]==1.1.0->esrally) (1.5.0)Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.9/site-packages (from jsonschema==3.1.1->esrally) (24.2.0)Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.9/site-packages (from jsonschema==3.1.1->esrally) (8.2.0)Requirement already satisfied: pyrsistent>=0.14.0 in /usr/local/lib64/python3.9/site-packages (from jsonschema==3.1.1->esrally) (0.20.0)Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib64/python3.9/site-packages (from aiohttp<4,>=3->elasticsearch[async]==8.6.1->esrally) (1.9.4)Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib64/python3.9/site-packages (from aiohttp<4,>=3->elasticsearch[async]==8.6.1->esrally) (1.4.1)Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.9/site-packages (from aiohttp<4,>=3->elasticsearch[async]==8.6.1->esrally) (1.3.1)Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.9/site-packages (from aiohttp<4,>=3->elasticsearch[async]==8.6.1->esrally) (4.0.3)Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /usr/local/lib/python3.9/site-packages (from aiohttp<4,>=3->elasticsearch[async]==8.6.1->esrally) (2.3.7)Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib64/python3.9/site-packages (from aiohttp<4,>=3->elasticsearch[async]==8.6.1->esrally) (6.0.5)Requirement already satisfied: pyparsing>=2.0.2 in /usr/lib/python3.9/site-packages (from packaging>=14.0->docker==6.0.0->esrally) (2.4.7)Requirement already satisfied: pyasn1<0.7.0,>=0.4.6 in /usr/local/lib/python3.9/site-packages (from pyasn1-modules>=0.2.1->google-auth==1.22.1->esrally) (0.6.0)Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/site-packages (from requests<2.32.0->esrally) (3.7)Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib64/python3.9/site-packages (from requests<2.32.0->esrally) (3.3.2)Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.9/site-packages (from importlib-metadata->jsonschema==3.1.1->esrally) (3.20.0)WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv[root@easysearch /]# esrally --versionesrally 2.11.0[root@easysearch /]#
复制代码
关于 Rally 的更多内容，请参阅官方文档：
https://esrally.readthedocs.io/en/stable/
作者：少安事务所
原文：https://mp.weixin.qq.com/s/9eEH38kgsw4i150CJqyxvQ
划线
评论
复制
发布于: 刚刚阅读数: 5
版权声明: 本文为 InfoQ 作者【极限实验室】的原创文章。
原文链接:【http://xie.infoq.cn/article/b82a3ad09b8edf9ff239c36d1】。
本文遵守【CC-BY 4.0】协议，转载请保留原文出处及本版权声明。
极限实验室
关注
简单、易用、极致、创新 2021-11-22 加入
极限实验室（INFINI Labs）致力于打造极致易用的数据探索与分析体验。
评论