elasticsearch-restful-api 笔记
发布于: 2020 年 07 月 22 日
原文 http://weikeqin.com/2020/01/19/elasticsearch-restful-api/
elasticsearch 的 resuful api 笔记
(1) 查看 ES 及索引信息
(1.1) 查看 ES 基本信息
[wkq@VM_77_25_centos ~]$ curl 'http://localhost:9200?pretty'{ "name" : "elasticsearch_001_data", "cluster_name" : "elasticsearch_test", "cluster_uuid" : "NsxYKhI1Qw63MzaPKl34dA", "version" : { "number" : "6.6.2", "build_flavor" : "default", "build_type" : "tar", "build_hash" : "3bd3e59", "build_date" : "2019-03-06T15:16:26.864148Z", "build_snapshot" : false, "lucene_version" : "7.6.0", "minimum_wire_compatibility_version" : "5.6.0", "minimum_index_compatibility_version" : "5.0.0" }, "tagline" : "You Know, for Search"}复制代码
(1.2) 计算集群中文档的数量
$ curl 'http://localhost:9200/_count?pretty' { "count" : 3, "_shards" : { "total" : 5, "successful" : 5, "skipped" : 0, "failed" : 0 }}复制代码
可以看到返回结果是 3
(1.3) 查看 ES 里所有索引
[wkq@VM_77_25_centos ~]$ curl 'localhost:9200/_cat/indices?v'health status index uuid pri rep docs.count docs.deleted store.size pri.store.sizeyellow open megacorp EUu3nzyRQN2Mp7tRw2u_nQ 5 1 3 0 17.5kb 17.5kbyellow open gb kAXZAJZTRg2X0Wm72-n5qQ 5 1 1 0 5.7kb 5.7kbyellow open us Q6ubyCClQvyMB5iX8f9BZA 5 1 1 0 5.7kb 5.7kbyellow open tweet hNRmZ9RETWWPoUMIK3BivA 5 1 12 0 26kb 26kbyellow open website 1Ptx5N-iTR2nDNtVgVMEpw 5 1 5 0 21.2kb 21.2kbyellow open blogs bW_JTJkfS2GVN8FE_gX-Hg 3 2 0 0 783b 783byellow open user rOprq90rQsuyP0mad7I6iQ 5 1 2 0 10.2kb 10.2kb[wkq@VM_77_25_centos ~]$复制代码
(1.3) 插入数据
$ curl -X PUT "localhost:9200/megacorp/employee/1" -H 'Content-Type: application/json' -d'> {> "first_name" : "John",> "last_name" : "Smith",> "age" : 25,> "about" : "I love to go rock climbing",> "interests": [ "sports", "music" ]> }> '{"_index":"megacorp","_type":"employee","_id":"1","_version":2,"result":"updated","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":1,"_primary_term":1}
复制代码
<!--more-->
(1.4) 根据 id 查询数据
$ curl -X GET "localhost:9200/megacorp/employee/1?pretty"{ "_index" : "megacorp", "_type" : "employee", "_id" : "1", "_version" : 2, "_seq_no" : 1, "_primary_term" : 1, "found" : true, "_source" : { "first_name" : "John", "last_name" : "Smith", "age" : 25, "about" : "I love to go rock climbing", "interests" : [ "sports", "music" ] }}复制代码
(1.5) 查询某个索引里的所有数据
$ curl -X GET "localhost:9200/megacorp/employee/_search?pretty"{ "took" : 17, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 3, "max_score" : 1.0, "hits" : [ { "_index" : "megacorp", "_type" : "employee", "_id" : "2", "_score" : 1.0, "_source" : { "first_name" : "Jane", "last_name" : "Smith", "age" : 32, "about" : "I like to collect rock albums", "interests" : [ "music" ] } }, { "_index" : "megacorp", "_type" : "employee", "_id" : "1", "_score" : 1.0, "_source" : { "first_name" : "John", "last_name" : "Smith", "age" : 25, "about" : "I love to go rock climbing", "interests" : [ "sports", "music" ] } }, { "_index" : "megacorp", "_type" : "employee", "_id" : "3", "_score" : 1.0, "_source" : { "first_name" : "Douglas", "last_name" : "Fir", "age" : 35, "about" : "I like to build cabinets", "interests" : [ "forestry" ] } } ] }}复制代码
(1.5) 根据某个字段搜索
$ curl -X GET "localhost:9200/megacorp/employee/_search?q=last_name:Smith&pretty"{ "took" : 112, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 2, "max_score" : 0.2876821, "hits" : [ { "_index" : "megacorp", "_type" : "employee", "_id" : "2", "_score" : 0.2876821, "_source" : { "first_name" : "Jane", "last_name" : "Smith", "age" : 32, "about" : "I like to collect rock albums", "interests" : [ "music" ] } }, { "_index" : "megacorp", "_type" : "employee", "_id" : "1", "_score" : 0.2876821, "_source" : { "first_name" : "John", "last_name" : "Smith", "age" : 25, "about" : "I love to go rock climbing", "interests" : [ "sports", "music" ] } } ] }}复制代码
(1.6) 查看索引 mapping
$ curl 'http://localhost:9200/megacorp/_mapping?pretty=true'{ "megacorp" : { "mappings" : { "employee" : { "properties" : { "about" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "age" : { "type" : "long" }, "first_name" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "interests" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } }, "fielddata" : true }, "last_name" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } } } } } }}复制代码
(1.x)
GET /_search{ "_source": { "includes": [ "obj1.*", "obj2.*" ], "excludes": [ "*.description" ] }, "query" : { "term" : { "user" : "kimchy" } }}复制代码
（2) 领域特定语言(DSL)搜索
领域特定语言(DSL)，使用 JSON 构造了一个请求。
(2.1) 计算集群中文档的数量
$ curl -XGET 'http://localhost:9200/_count?pretty' -H "Content-Type: application/json" -d '> {> "query": {> "match_all": {}> }> }> '{ "count" : 3, "_shards" : { "total" : 5, "successful" : 5, "skipped" : 0, "failed" : 0 }}复制代码
可以看到结果是 3
(2.2) 使用查询表达式搜索
$ curl -X GET "localhost:9200/megacorp/employee/_search?pretty" -H 'Content-Type: application/json' -d'> {> "query" : {> "match" : {> "last_name" : "Smith"> }> }> }> '{ "took" : 7, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 2, "max_score" : 0.2876821, "hits" : [ { "_index" : "megacorp", "_type" : "employee", "_id" : "2", "_score" : 0.2876821, "_source" : { "first_name" : "Jane", "last_name" : "Smith", "age" : 32, "about" : "I like to collect rock albums", "interests" : [ "music" ] } }, { "_index" : "megacorp", "_type" : "employee", "_id" : "1", "_score" : 0.2876821, "_source" : { "first_name" : "John", "last_name" : "Smith", "age" : 25, "about" : "I love to go rock climbing", "interests" : [ "sports", "music" ] } } ] }}$复制代码
(2.3) 多条件查询
搜索姓氏为 Smith 的员工，但这次我们只需要年龄大于 30 的
$ curl -X GET "localhost:9200/megacorp/employee/_search?pretty" -H 'Content-Type: application/json' -d'> {> "query" : {> "bool": {> "must": {> "match" : {> "last_name" : "smith"> }> },> "filter": {> "range" : {> "age" : { "gt" : 30 }> }> }> }> }> }> '{ "took" : 32, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 1, "max_score" : 0.2876821, "hits" : [ { "_index" : "megacorp", "_type" : "employee", "_id" : "2", "_score" : 0.2876821, "_source" : { "first_name" : "Jane", "last_name" : "Smith", "age" : 32, "about" : "I like to collect rock albums", "interests" : [ "music" ] } } ] }}$复制代码
range 过滤器，它能找到年龄大于 30 的文档，其中 gt 表示大于(great than)
(2.4) 全文搜索
搜索下所有喜欢攀岩（rock climbing）的员工
$ curl -X GET "localhost:9200/megacorp/employee/_search?pretty" -H 'Content-Type: application/json' -d'> {> "query" : {> "match" : {> "about" : "rock climbing"> }> }> }> '{ "took" : 13, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 2, "max_score" : 0.5753642, "hits" : [ { "_index" : "megacorp", "_type" : "employee", "_id" : "1", "_score" : 0.5753642, "_source" : { "first_name" : "John", "last_name" : "Smith", "age" : 25, "about" : "I love to go rock climbing", "interests" : [ "sports", "music" ] } }, { "_index" : "megacorp", "_type" : "employee", "_id" : "2", "_score" : 0.2876821, "_source" : { "first_name" : "Jane", "last_name" : "Smith", "age" : 32, "about" : "I like to collect rock albums", "interests" : [ "music" ] } } ] }}复制代码
(2.5) 短语搜索
精确匹配一系列单词或者短语
$ curl -X GET "localhost:9200/megacorp/employee/_search?pretty" -H 'Content-Type: application/json' -d'> {> "query" : {> "match_phrase" : {> "about" : "rock climbing"> }> }> }> '{ "took" : 16, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 1, "max_score" : 0.5753642, "hits" : [ { "_index" : "megacorp", "_type" : "employee", "_id" : "1", "_score" : 0.5753642, "_source" : { "first_name" : "John", "last_name" : "Smith", "age" : 25, "about" : "I love to go rock climbing", "interests" : [ "sports", "music" ] } } ] }}复制代码
(2.6) 高亮搜索
$ curl -X GET "localhost:9200/megacorp/employee/_search?pretty" -H 'Content-Type: application/json' -d'> {> "query" : {> "match_phrase" : {> "about" : "rock climbing"> }> },> "highlight": {> "fields" : {> "about" : {}> }> }> }> '{ "took" : 305, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 1, "max_score" : 0.5753642, "hits" : [ { "_index" : "megacorp", "_type" : "employee", "_id" : "1", "_score" : 0.5753642, "_source" : { "first_name" : "John", "last_name" : "Smith", "age" : 25, "about" : "I love to go rock climbing", "interests" : [ "sports", "music" ] }, "highlight" : { "about" : [ "I love to go <em>rock</em> <em>climbing</em>" ] } } ] }}复制代码
(2.7) 聚合
聚合与 SQL 中的 GROUP BY 类似但更强大。
挖掘出员工中最受欢迎的兴趣爱好
$ curl -X GET "localhost:9200/megacorp/employee/_search?pretty" -H 'Content-Type: application/json' -d'> {> "aggs": {> "all_interests": {> "terms": { "field": "interests" }> }> }> }> '{ "took" : 139, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 3, "max_score" : 1.0, "hits" : [ { "_index" : "megacorp", "_type" : "employee", "_id" : "2", "_score" : 1.0, "_source" : { "first_name" : "Jane", "last_name" : "Smith", "age" : 32, "about" : "I like to collect rock albums", "interests" : [ "music" ] } }, { "_index" : "megacorp", "_type" : "employee", "_id" : "1", "_score" : 1.0, "_source" : { "first_name" : "John", "last_name" : "Smith", "age" : 25, "about" : "I love to go rock climbing", "interests" : [ "sports", "music" ] } }, { "_index" : "megacorp", "_type" : "employee", "_id" : "3", "_score" : 1.0, "_source" : { "first_name" : "Douglas", "last_name" : "Fir", "age" : 35, "about" : "I like to build cabinets", "interests" : [ "forestry" ] } } ] }, "aggregations" : { "all_interests" : { "doc_count_error_upper_bound" : 0, "sum_other_doc_count" : 0, "buckets" : [ { "key" : "music", "doc_count" : 2 }, { "key" : "forestry", "doc_count" : 1 }, { "key" : "sports", "doc_count" : 1 } ] } }}复制代码
可以看到，两位员工对音乐感兴趣，一位对林业感兴趣，一位对运动感兴趣。这些聚合的结果数据并非预先统计，而是根据匹配当前查询的文档即时生成的。
(2.8) 条件聚合
想知道叫 Smith 的员工中最受欢迎的兴趣爱好
$ curl -X GET "localhost:9200/megacorp/employee/_search?pretty" -H 'Content-Type: application/json' -d'> {> "query": {> "match": {> "last_name": "smith"> }> },> "aggs": {> "all_interests": {> "terms": {> "field": "interests"> }> }> }> }> '{ "took" : 11, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 2, "max_score" : 0.2876821, "hits" : [ { "_index" : "megacorp", "_type" : "employee", "_id" : "2", "_score" : 0.2876821, "_source" : { "first_name" : "Jane", "last_name" : "Smith", "age" : 32, "about" : "I like to collect rock albums", "interests" : [ "music" ] } }, { "_index" : "megacorp", "_type" : "employee", "_id" : "1", "_score" : 0.2876821, "_source" : { "first_name" : "John", "last_name" : "Smith", "age" : 25, "about" : "I love to go rock climbing", "interests" : [ "sports", "music" ] } } ] }, "aggregations" : { "all_interests" : { "doc_count_error_upper_bound" : 0, "sum_other_doc_count" : 0, "buckets" : [ { "key" : "music", "doc_count" : 2 }, { "key" : "sports", "doc_count" : 1 } ] } }}复制代码
(2.9) 聚合条件汇总
查询特定兴趣爱好员工的平均年龄
$ curl -X GET "localhost:9200/megacorp/employee/_search?pretty" -H 'Content-Type: application/json' -d'> {> "aggs" : {> "all_interests" : {> "terms" : { "field" : "interests" },> "aggs" : {> "avg_age" : {> "avg" : { "field" : "age" }> }> }> }> }> }> '{ "took" : 36, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 3, "max_score" : 1.0, "hits" : [ { "_index" : "megacorp", "_type" : "employee", "_id" : "2", "_score" : 1.0, "_source" : { "first_name" : "Jane", "last_name" : "Smith", "age" : 32, "about" : "I like to collect rock albums", "interests" : [ "music" ] } }, { "_index" : "megacorp", "_type" : "employee", "_id" : "1", "_score" : 1.0, "_source" : { "first_name" : "John", "last_name" : "Smith", "age" : 25, "about" : "I love to go rock climbing", "interests" : [ "sports", "music" ] } }, { "_index" : "megacorp", "_type" : "employee", "_id" : "3", "_score" : 1.0, "_source" : { "first_name" : "Douglas", "last_name" : "Fir", "age" : 35, "about" : "I like to build cabinets", "interests" : [ "forestry" ] } } ] }, "aggregations" : { "all_interests" : { "doc_count_error_upper_bound" : 0, "sum_other_doc_count" : 0, "buckets" : [ { "key" : "music", "doc_count" : 2, "avg_age" : { "value" : 28.5 } }, { "key" : "forestry", "doc_count" : 1, "avg_age" : { "value" : 35.0 } }, { "key" : "sports", "doc_count" : 1, "avg_age" : { "value" : 25.0 } } ] } }}复制代码
(3) ES 修改配置语句
(3.1) es 5.x 开启全文检索语句
5.x 后对排序，聚合这些操作用单独的数据结构(fielddata)缓存到内存里了，需要单独开启
$ curl -X PUT "localhost:9200/megacorp/_mapping/employee/" -H 'Content-Type: application/json' -d'> {> "properties": {> "interests": {> "type": "text",> "fielddata": true> }> }> }> '{"acknowledged":true}复制代码
推荐使用 keyword 聚合
(3.2) 创建索引并设置分片数和副本数
$ curl -X PUT "localhost:9200/blogs?pretty" -H 'Content-Type: application/json' -d'> {> "settings" : {> "number_of_shards" : 3,> "number_of_replicas" : 1> }> }> '{ "acknowledged" : true, "shards_acknowledged" : true, "index" : "blogs"}复制代码
(3.3) 设置副本数
$ curl -X PUT "localhost:9200/blogs/_settings?pretty" -H 'Content-Type: application/json' -d'> {> "number_of_replicas" : 2> }> '{ "acknowledged" : true}复制代码
(3.4) get set up
$ curl -X GET "localhost:9200/_nodes/transport?error_trace=true&pretty=true"{ "_nodes" : { "total" : 1, "successful" : 1, "failed" : 0 }, "cluster_name" : "elasticsearch_test", "nodes" : { "urmXtplyRmyt_LKCTC6_3w" : { "name" : "elasticsearch_001_data", "transport_address" : "127.0.0.1:9300", "host" : "127.0.0.1", "ip" : "127.0.0.1", "version" : "6.6.2", "build_flavor" : "default", "build_type" : "tar", "build_hash" : "3bd3e59", "roles" : [ "master", "data", "ingest" ], "attributes" : { "ml.machine_memory" : "1927528448", "xpack.installed" : "true", "ml.max_open_jobs" : "20", "ml.enabled" : "true" }, "transport" : { "bound_address" : [ "127.0.0.1:9300" ], "publish_address" : "127.0.0.1:9300", "profiles" : { } } } }}复制代码
References
[1] 索引员工文档
[2] 检索文档
[3] 轻量搜索
[4] 使用查询表达式搜索
[5] 更复杂的搜索
[6] 全文搜索
[7] 短语搜索
[8] 高亮搜索
[9] 分析
划线
评论
复制
发布于: 2020 年 07 月 22 日阅读数: 84
版权声明: 本文为 InfoQ 作者【wkq2786130】的原创文章。
原文链接:【http://xie.infoq.cn/article/7fe4f47d9d589af080ee16dbd】。文章转载请联系作者。
wkq2786130
关注
hello 2018.09.28 加入
http://weikeqin.com/
评论