写点什么

快速上手 Elasticsearch bboss restclient

作者:大河
  • 2022 年 8 月 29 日
    湖南
  • 本文字数:8068 字

    阅读完需:约 26 分钟

快速上手Elasticsearch bboss restclient

Quick Start


Elasticsearch Bboss--高性能 Elasticsearch Java RestClient


  1. A highlevel rest client.

  2. A high performence o/r mapping rest client.

  3. A dsl and sql rest client.

  4. Support Elasticsearch 1.x,2.x,5.x,6.x,7.x,8.x,+

  5. Support Spring boot 1.x,2.x

1.Bboss 集成

导入 BBoss maven 坐标:


    <dependency>        <groupId>com.bbossgroups.plugins</groupId>        <artifactId>bboss-elasticsearch-rest-jdbc</artifactId>        <version>6.7.2</version>    </dependency>
复制代码


如果是 spring boot 项目,还需导入以下 maven 坐标:


    <dependency>        <groupId>com.bbossgroups.plugins</groupId>        <artifactId>bboss-elasticsearch-spring-boot-starter</artifactId>        <version>6.7.2</version>    </dependency>
复制代码

2.Bboss 配置和使用

在项目 resources 目录下修改 application.properties 文件(如果不存在则新建 application.properties 文件),根据项目类型做添加相应配置:

2.1 普通 maven 项目配置

#Cluster addresses are separated by commas
#elasticsearch.rest.hostNames=10.180.211.27:9200,10.180.211.28:9200,10.180.211.29:9200
elasticsearch.rest.hostNames=10.21.20.168:9200
复制代码


如需启用 HTTPS 协议, 则在 elasticsearch 地址前面添加 https 协议头:


elasticsearch.rest.hostNames=https://10.180.211.27:9280,https://10.180.211.27:9281,https://10.180.211.27:9282
复制代码


如果启用了 x-pack or searchguard 安全认证机制, 则还需在 application.properties 文件配置认证账号和口令:


#x-pack or searchguard security authentication and password configurationelasticUser=elasticelasticPassword=changeme
复制代码


接下来就可以编写 java 代码验证集成是否成功:


import org.frameworkset.elasticsearch.ElasticSearchHelper;import org.frameworkset.elasticsearch.client.ClientInterface;import org.frameworkset.elasticsearch.entity.ESDatas;import org.frameworkset.elasticsearch.scroll.HandlerInfo;import org.frameworkset.elasticsearch.scroll.ScrollHandler;import org.junit.Test;
import java.util.List;import java.util.Map;
public class TestInit { @Test public void init(){
ClientInterface clientUtil = ElasticSearchHelper.getRestClientUtil(); //get elasticsearch cluster state String result = clientUtil.executeHttp("_cluster/state?pretty",ClientInterface.HTTP_GET); //check indice twitter and index type tweet exist or not. //适用于elasticsearch 6及以下版本有类型,7以上版本无类型 boolean exist1 = clientUtil.existIndiceType("twitter","tweet"); System.out.println("twitter tweet type exist:"+exist1); //适用于 Elasticsearch7以上的版本,check indice twitter exist or not, exist1 = clientUtil.existIndice("twitter"); System.out.println("twitter exist:"+exist1); //count documents in indice twitter long count = clientUtil.countAll("twitter"); System.out.println(count); //Get All documents of indice twitter,DEFAULT_FETCHSIZE is 5000 //返回对象类型为Map,也可以指定为特定的PO对象,适用于记录数量不大的表 ESDatas<Map> esDatas = clientUtil.searchAll("twitter", Map.class); //从esDatas中获取检索到的记录集合 List<Map> datas = esDatas.getDatas(); //从esDatas中获取检索到的记录总数 long totalSize = esDatas.getTotalSize(); //Get All documents of indice twitter,Set fetchsize to 10000, Using ScrollHandler to process each batch of datas. //指定批处理器分批次处理数据,适用于记录数量比较大的全表表数据查询 clientUtil.searchAll("twitter",10000,new ScrollHandler<Map>() { public void handle(ESDatas<Map> esDatas, HandlerInfo handlerInfo) throws Exception { List<Map> dataList = esDatas.getDatas(); System.out.println("TotalSize:"+esDatas.getTotalSize()); if(dataList != null) { System.out.println("dataList.size:" + dataList.size()); } else { System.out.println("dataList.size:0"); } //do something other such as do a db query. //SQLExecutor.queryList(Map.class,"select * from td_sm_user"); } },Map.class); //Use slice parallel scoll query all documents of indice twitter by 2 thread tasks. DEFAULT_FETCHSIZE is 5000 //You can also use ScrollHandler to process each batch of datas on your own. esDatas = clientUtil.searchAllParallel("twitter", Map.class,2); //指定批处理器分批次处理数据(适用于数据量比较大的表),并行检索和处理表数据源,线程数量为2, clientUtil.searchAllParallel("twitter",10000,new ScrollHandler<Map>() { public void handle(ESDatas<Map> esDatas, HandlerInfo handlerInfo) throws Exception { List<Map> dataList = esDatas.getDatas(); System.out.println("TotalSize:"+esDatas.getTotalSize()); if(dataList != null) { System.out.println("dataList.size:" + dataList.size()); } else { System.out.println("dataList.size:0"); } //do something other such as do a db query. //SQLExecutor.queryList(Map.class,"select * from td_sm_user"); } },Map.class,2); }}
复制代码


maven project 案例下载地址:


https://gitee.com/bboss/eshelloword-booter


参考文档


集成和配置 Elasticsearch bboss:


https://esdoc.bbossgroups.com/#/common-project-with-bboss

2.2 spring boot maven 项目配置

spring.elasticsearch.bboss.elasticsearch.rest.hostNames=127.0.0.1:9200
复制代码


如需启用 HTTPS 协议, 则在 elasticsearch 地址前面添加 https 协议头:


spring.elasticsearch.bboss.elasticsearch.rest.hostNames=https://10.180.211.27:9280,https://10.180.211.27:9281,https://10.180.211.27:9282
复制代码


如果启用了 x-pack or searchguard 安全认证机制, 则还需在 application.properties 文件配置认证账号和口令:


##support x-pack and searchguardspring.elasticsearch.bboss.elasticUser=elasticspring.elasticsearch.bboss.elasticPassword=changeme
复制代码


Spring boot 需要使用注解 @Autowired 注入一个 BBossESStarter 对象来获取 elasticsearch rest client api 实例对象(每次获取到的实例是单实例多线程安全的):


@Autowiredprivate BBossESStarter bbossESStarter;//Create a client tool to load configuration files, single instance multithreaded securityClientInterface configClientUtil = bbossESStarter.getConfigRestClient(mappath);    //Build a create/modify/get/delete document client object, single instance multi-thread securityClientInterface clientUtil = bbossESStarter.getRestClient();    
复制代码


Spring boot 示例代码:


The dsl xml file this example used esmapper/demo.xml


package com.example.esbboss.service;
import com.example.esbboss.entity.Demo;import com.example.esbboss.entity.DemoSearchResult;import org.frameworkset.elasticsearch.ElasticSearchException;import org.frameworkset.elasticsearch.boot.BBossESStarter;import org.frameworkset.elasticsearch.client.ClientInterface;import org.frameworkset.elasticsearch.entity.ESDatas;import org.slf4j.Logger;import org.slf4j.LoggerFactory;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Service;
import java.text.DateFormat;import java.text.ParseException;import java.text.SimpleDateFormat;import java.util.Date;import java.util.HashMap;import java.util.List;import java.util.Map;

@Servicepublic class DocumentCRUD7 { private Logger logger = LoggerFactory.getLogger(DocumentCRUD7.class); @Autowired private BBossESStarter bbossESStarter; //DSL config file path private String mappath = "esmapper/demo7.xml";

public void dropAndCreateAndGetIndice(){ //Create a client tool to load configuration files, single instance multithreaded security ClientInterface clientUtil = bbossESStarter.getConfigRestClient(mappath); try { //To determine whether the indice demo exists, it returns true if it exists and false if it does not boolean exist = clientUtil.existIndice("demo");
//Delete mapping if the indice demo already exists if(exist) { String r = clientUtil.dropIndice("demo"); logger.debug("clientUtil.dropIndice(\"demo\") response:"+r); } //Create index demo clientUtil.createIndiceMapping("demo",//The indice name "createDemoIndice");//Index mapping DSL script name, defined createDemoIndice in esmapper/demo.xml String demoIndice = clientUtil.getIndice("demo");//Gets the newly created indice structure logger.info("after createIndiceMapping clientUtil.getIndice(\"demo\") response:"+demoIndice); } catch (ElasticSearchException e) { // TODO Auto-generated catch block e.printStackTrace(); }
}


public void addAndUpdateDocument() { //Build a create/modify/get/delete document client object, single instance multi-thread security ClientInterface clientUtil = bbossESStarter.getRestClient(); //Build an object as index document Demo demo = new Demo(); demo.setDemoId(2l);//Specify the document id, the unique identity, and mark with the @ESId annotation. If the demoId already exists, modify the document; otherwise, add the document demo.setAgentStarttime(new Date()); demo.setAgentStarttimezh(new Date()); demo.setApplicationName("blackcatdemo2"); demo.setContentbody("this is content body2"); demo.setName("liudehua"); demo.setOrderId("NFZF15045871807281445364228"); demo.setContrastStatus(2);

//Add the document and force refresh String response = clientUtil.addDocument("demo",//indice name demo,"refresh=true");


logger.debug("Print the result:addDocument-------------------------"); logger.debug(response); demo = new Demo(); demo.setDemoId(3l);//Specify the document id, the unique identity, and mark with the @ESId annotation. If the demoId already exists, modify the document; otherwise, add the document demo.setAgentStarttime(new Date()); demo.setApplicationName("blackcatdemo3"); demo.setContentbody("this is content body3"); demo.setName("zhangxueyou"); demo.setOrderId("NFZF15045871807281445364228"); demo.setContrastStatus(3); demo.setAgentStarttime(new Date()); demo.setAgentStarttimezh(new Date()); //Add the document and force refresh response = clientUtil.addDocument("demo",//indice name demo,"refresh=true"); //Get the document object according to the document id, and return the Demo object demo = clientUtil.getDocument("demo",//indice name "2",//document id Demo.class); //update document demo = new Demo(); demo.setDemoId(2l);//Specify the document id, the unique identity, and mark with the @ESId annotation. If the demoId already exists, modify the document; otherwise, add the document demo.setAgentStarttime(new Date()); demo.setApplicationName("blackcatdemo2"); demo.setContentbody("this is modify content body2"); demo.setName("刘德华modify\t"); demo.setOrderId("NFZF15045871807281445364228"); demo.setContrastStatus(2); demo.setAgentStarttimezh(new Date()); //Execute update and force refresh response = clientUtil.addDocument("demo",//index name demo,"refresh=true");

//Get the modified document object according to the document id and return the json message string response = clientUtil.getDocument("demo",//indice name "2");//document id logger.debug("Print the modified result:getDocument-------------------------"); logger.debug(response);



logger.debug("Print the modified result:getDocument-------------------------"); logger.debug(response);

}
public void deleteDocuments(){ //Build a create/modify/get/delete document client object, single instance multi-thread security ClientInterface clientUtil = bbossESStarter.getRestClient(); //Batch delete documents clientUtil.deleteDocuments("demo",//indice name new String[]{"2","3"});//Batch delete document ids }
/** * Use slice parallel scoll query all documents of indice demo by 2 thread tasks. DEFAULT_FETCHSIZE is 5000 */ public void searchAllPararrel(){ ClientInterface clientUtil = bbossESStarter.getRestClient(); ESDatas<Demo> esDatas = clientUtil.searchAllParallel("demo", Demo.class,2); }


/** * Search the documents */ public DemoSearchResult search() { //Create a load DSL file client instance to retrieve documents, single instance multithread security ClientInterface clientUtil = bbossESStarter.getConfigRestClient(mappath); //Set query conditions, pass variable parameter values via map,key for variable names in DSL //There are four variables in the DSL: // applicationName1 // applicationName2 // startTime // endTime Map<String,Object> params = new HashMap<String,Object>(); //Set the values of applicationName1 and applicationName2 variables params.put("applicationName1","blackcatdemo2"); params.put("applicationName2","blackcatdemo3"); DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"); //Set the time range, and accept the long value as the time parameter try { params.put("startTime",dateFormat.parse("2017-09-02 00:00:00").getTime()); } catch (ParseException e) { e.printStackTrace(); } params.put("endTime",new Date().getTime());

//Execute the query ESDatas<Demo> esDatas = //ESDatas contains a collection of currently retrieved records, up to 1000 records, specified by the size attribute in the DSL clientUtil.searchList("demo/_search",//demo as the indice, _search as the search action "searchDatas",//DSL statement name defined in esmapper/demo.xml params,//Query parameters Demo.class);//Data object type Demo returned //Gets a list of result objects and returns max up to 1000 records (specified in DSL) List<Demo> demos = esDatas.getDatas();
// String json = clientUtil.executeRequest("demo/_search",//demo as the index table, _search as the search action// "searchDatas",//DSL statement name defined in esmapper/demo.xml// params);//Query parameters// String json = com.frameworkset.util.SimpleStringUtil.object2json(demos); //Gets the total number of records long totalSize = esDatas.getTotalSize(); DemoSearchResult demoSearchResult = new DemoSearchResult(); demoSearchResult.setDemos(demos); demoSearchResult.setTotalSize(totalSize); return demoSearchResult; }

}
复制代码


A spring boot Web demo github url:包含本文案例及 dsl xml 配置文件 demo7.xml


https://gitee.com/bboss/springboot-elasticsearch


参考文档


Spring boot 集成和配置 Elasticsearch:


https://esdoc.bbossgroups.com/#/spring-booter-with-bboss


Springboot 集成 bboss Elasticsearch 和 Apollo:


https://esdoc.bbossgroups.com/#/springboot-bbosses-apollo

3.获取 Elasticsearch 客户端组件实例方法

一般项目通过 ElasticSearchHelper 获取 elasticsearch rest client api 实例:


//创建加载配置文件的客户端实例,单实例多线程安全ClientInterface clientUtil = ElasticSearchHelper.getConfigRestClientUtil("esmapper/demo.xml");//创建直接操作dsl的客户端实例,单实例多线程安全ClientInterface clientUtil = ElasticSearchHelper.getRestClientUtil() ;
复制代码


Spring boot 项目使用 BBossESStarter 获取 elasticsearch rest client api 实例:


@Autowiredprivate BBossESStarter bbossESStarter;//Create a client tool to load configuration files, single instance multithreaded securityClientInterface clientUtil = bbossESStarter.getConfigRestClient("esmapper/demo.xml");    //Build a create/modify/get/delete document client object, single instance multi-thread security    ClientInterface clientUtil = bbossESStarter.getRestClient();    
复制代码


Elasticsearch bboss 开发指南:


https://esdoc.bbossgroups.com/#/development

4.从源码构建 Elasticsearch BBoss

首先下载下面的两个源码工程


https://gitee.com/bboss/bboss-elastic


https://gitee.com/bboss/bboss-elastic-tran


然后通过 gradle 依次按顺序构建 bboss-elasticsearch 和 bboss-elastic-tran:


gradle clean publishToMavenLocalGradle 环境搭建和配置教程


https://esdoc.bbossgroups.com/#/bboss-build

5.开发交流

bboss elasticsearch 交流 QQ 群:21220580,166471282


bboss elasticsearch 微信公众号:



用户头像

大河

关注

还未添加个人签名 2022.08.15 加入

还未添加个人简介

评论

发布
暂无评论
快速上手Elasticsearch bboss restclient_elasticsearch_大河_InfoQ写作社区