写点什么

Elasticsearch VS Easysearch 性能测试

作者:极限实验室
  • 2025-01-02
    北京
  • 本文字数:2517 字

    阅读完需:约 8 分钟

Elasticsearch VS Easysearch 性能测试

压测环境

虚拟机配置

使用阿里云上规格:ecs.u1-c1m4.4xlarge,PL2: 单盘 IOPS 性能上限 10 万 (适用的云盘容量范围:461GiB - 64TiB)


Easysearch 配置

7 节点集群,版本:1.9.0


Elasticsearch 配置

7 节点集群,版本:7.10.2


监控集群配置

单节点 Easysearch 集群,版本:1.9.0


压测 loadgen 配置

loadgen 版本:1.25.0


4 台压 Easysearch,4 台压 Elasticsearch。


压测索引 Mapping

PUT nginx{  "mappings": {    "properties": {      "method": {        "type": "keyword"      },      "bandwidth": {        "type": "integer"      },      "service_name": {        "type": "keyword"      },      "ip": {        "type": "ip"      },      "memory_usage": {        "type": "integer"      },      "upstream_time": {        "type": "float"      },      "url": {        "type": "keyword"      },      "response_size": {        "type": "integer"      },      "request_time": {        "type": "float"      },      "request_body_size": {        "type": "integer"      },      "error_code": {        "type": "keyword"      },      "metrics": {        "properties": {          "queue_size": {            "type": "integer"          },          "memory_usage": {            "type": "integer"          },          "thread_count": {            "type": "integer"          },          "cpu_usage": {            "type": "integer"          },          "active_connections": {            "type": "integer"          }        }      },      "cpu_usage": {        "type": "integer"      },      "user_agent": {        "type": "keyword"      },      "connections": {        "type": "integer"      },      "timestamp": {        "type": "date",        "format": "yyyy-MM-dd'T'HH:mm:ss.SSS"      },      "status": {        "type": "integer"      }    }  },  "settings": {    "number_of_shards": 7,    "number_of_replicas": 0,    "refresh_interval": "30s"  }}
复制代码

压测方法

每 4 个 loadgen 使用批量写入接口 bulk 轮询压测同一集群的 7 个节点,每个请求写入 10000 个文档。


具体请求如下:


requests:  - request: #prepare some docs      method: POST      runtime_variables:#        batch_no: uuid      runtime_body_line_variables:#        routing_no: uuid#      url: $[[env.ES_ENDPOINT]]/_bulk      url: $[[ip]]/_bulk      body_repeat_times: 10000      basic_auth:       username: "$[[env.ES_USERNAME]]"       password: "$[[env.ES_PASSWORD]]"      body: |        {"index": {"_index": "nginx", "_type": "_doc", "_id": "$[[uuid]]"}}        $[[message]]
复制代码

压测数据样本

{"method":"DELETE","bandwidth":1955,"service_name":"cart-service","ip":"120.204.26.240","memory_usage":1463,"upstream_time":"1.418","url":"/health","response_size":421,"request_time":"0.503","request_body_size":1737,"error_code":"SYSTEM_ERROR","metrics":{"queue_size":769,"memory_usage":1183,"thread_count":65,"cpu_usage":68,"active_connections":837},"cpu_usage":70,"user_agent":"Mozilla/5.0 (iPad; CPU OS 14_6 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.1","connections":54,"timestamp":"2024-11-16T14:25:21.423","status":500}{"method":"OPTIONS","bandwidth":10761,"service_name":"product-service","ip":"223.99.83.60","memory_usage":567,"upstream_time":"0.907","url":"/static/js/app.js","response_size":679,"request_time":"1.287","request_body_size":1233,"error_code":"NOT_FOUND","metrics":{"queue_size":565,"memory_usage":1440,"thread_count":148,"cpu_usage":39,"active_connections":1591},"cpu_usage":87,"user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.1","connections":354,"timestamp":"2024-11-16T05:37:28.423","status":502}{"method":"HEAD","bandwidth":10257,"service_name":"recommendation-service","ip":"183.60.242.143","memory_usage":1244,"upstream_time":"0.194","url":"/api/v1/recommendations","response_size":427,"request_time":"1.449","request_body_size":1536,"error_code":"UNAUTHORIZED","metrics":{"queue_size":848,"memory_usage":866,"thread_count":86,"cpu_usage":29,"active_connections":3846},"cpu_usage":71,"user_agent":"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)","connections":500,"timestamp":"2024-11-16T15:14:30.424","status":403}
复制代码

压测索引 1 主分片 0 副本

Elastic 吞吐


Elastic 线程及队列

资源消耗



Easysearch 吞吐


Easysearch 线程及队列

资源消耗



对比

压测索引 1 主分片 1 副本

Elastic 吞吐


Elastic 线程及队列

资源消耗

Easysearch 吞吐


Easysearch 线程及队列

资源消耗

对比

压测索引 7 主分片

Elastic 吞吐


Elastic 线程及队列

资源消耗



网络


单节点平均接收 26MB/s,对应带宽:1456 Mb/s



5 千万文档,总存储 105 GB,单节点 15 GB


Easysearch 吞吐


Easysearch 线程及队列

资源消耗




对比

总结

通过对不同场景的压测结果进行对比分析,得出以下结论:


  • Easysearch 相比 Elasticsearch 的索引性能显著提升

  • Easysearch 集群的吞吐性能提升了 40% - 70%,且随着分片数量的增加,性能提升效果更为显著。

  • Easysearch 相比 Elasticsearch 的磁盘压缩效率大幅提高

  • Easysearch 集群的磁盘压缩效率提升了 2.5 - 3 倍,并且随着数据量的增加,压缩效果愈发明显。


此测试结果表明,Easysearch 在日志处理场景中具有更高的性能与存储效率优势,尤其适用于大规模分片与海量数据的使用场景。


如有任何问题,请随时联系我,期待与您交流!



发布于: 刚刚阅读数: 3
用户头像

简单、易用、极致、创新 2021-11-22 加入

极限实验室(INFINI Labs)致力于打造极致易用的数据探索与分析体验。

评论

发布
暂无评论
Elasticsearch VS Easysearch 性能测试_elasticsearch_极限实验室_InfoQ写作社区