所谓"工欲善其事,必先利其器"，同样要想成为 es 的顶级玩家,研究其源码肯定是必不可少的。研究源码的第一步就是搭建源码运行环境。
本文主要演示搭建 elasticsearch 的 debug 环境的具体步骤。目标是使读者能根据本文在本地搭建自己的 es 开发环境。
笔者使用的环境如下:
elasticsearch 7.6
Mac OS X 10.16 (x86_64)
jdk 12、13、11、8、9
gradle 6.3
idea 2020.3
笔者之前搭建开发环境也是在互联网找各种大牛的教程,但是经过几番尝试之后发现最后都失败了,究其原因不是因为大牛的方法不对,可能与不同的开发环境等都有关系。所以笔者就想既然是顶级开源项目肯定教我们如何做开源的相关文档,所以万事回归本源,最好的资料就是源码以及官方文档.
打开 elasticsearch 的 github 主页 https://github.com/elastic/elasticsearch 在 readme.md 中期待找有关如何编译项目的说明,很遗憾没有说明.
所以笔者想在源码中的某一些文档中肯定有说明,所以果断下载源码到本地,这里推荐先将源码 fork 一份都自己的 github 主页中，这样方便后期看源码时候可以添加一些注释,方便提交日后复习.
下载 es 源码,切换到 7.6 分支，因为笔者工作中使用 es 版本是 7.6 所以笔者切换到 7.6 分支，读者可以根据自己的需要切换到任意分支,方法都是一样的。
可以在 elasticsearch 的 github 主页选择自己的版本.
git clone https://github.com/elastic/elasticsearch.gitgit checkout v7.6.0
复制代码
打开 elastisearch 的源码,这一步可以用任意的工具打开,我这里用 vscode 打开,这一步主要是查找文档的说明。
到这一步我们就需要在源码中查找有关的说明文档,一般来说就是在源码的根目录中,所以经过一番艰难的英语阅读之后,笔者终于在 CONTRIBUTING.md 中找到答案了.真是源码面前无任何秘密可言.打开 CONTRIBUTING.md 中有这么一段说明.
Contributing to the Elasticsearch codebase
------------------------------------------
**Repository:** [https://github.com/elastic/elasticsearch](https://github.com/elastic/elasticsearch)
JDK 13 is required to build Elasticsearch. You must have a JDK 13 installation
with the environment variable `JAVA_HOME` referencing the path to Java home for
your JDK 13 installation. By default, tests use the same runtime as `JAVA_HOME`.
However, since Elasticsearch supports JDK 8, the build supports compiling with
JDK 13 and testing on a JDK 8 runtime; to do this, set `RUNTIME_JAVA_HOME`
pointing to the Java home of a JDK 8 installation. Note that this mechanism can
be used to test against other JDKs as well, this is not only limited to JDK 8.
> Note: It is also required to have `JAVA8_HOME`, `JAVA9_HOME`, `JAVA10_HOME`
and `JAVA11_HOME`, and `JAVA12_HOME` available so that the tests can pass.
Elasticsearch uses the Gradle wrapper for its build. You can execute Gradle
using the wrapper via the `gradlew` script on Unix systems or `gradlew.bat`
script on Windows in the root of the repository. The examples below show the
usage on Unix.
We support development in IntelliJ versions IntelliJ 2019.2 and
onwards. We would like to support Eclipse, but few of us use it and has fallen
into [disrepair][eclipse].
英语不好的可以借助有道词典或者 google 翻译,这句话的大致意思是如何给 es 贡献代码. 我们需要将 JAVA_HOME 设置为 jdk13，并且需要设置 RUNTIME_JAVA_HOME 环境变量指向 JDK8，而且由于 es 中的很多测试套件使用的是 JDK8/JDK9/JDK10 所以我们还需要设置 JAVA8_HOME、JAVA10_HOME、JAVA11_HOME、JAVA12_HOME 等环境变量。所以根据指示,笔者在 mac 电脑的本地环境变量文件.bash_profile 中做了如下设置
这里我把 JDK8 开始到 JDK14 全部设置了一遍,那是因为 es 的不同版本要求的 JDK 不一样，所以干脆所有版本的 JDK 都下载下来,需要哪一个版本随时切换都可以了.
下载 JDK 不同版本地址信息 https://www.oracle.com/java/technologies/oracle-java-archive-downloads.html
设置玩环境变量之后,根据 CONTRIBUTING.md 的说明,我们就可以将 es 源码导入到 idea 中
注意笔者这里可以正常访问国外网站,而且网速通常也比较快,所以就没有单独设置 es 依赖的仓库信息,gradle 版本信息也没有单独下载,因为在导入 es 时候 idea 会帮我们自动下载指定的 gralde 版本.
将源码导入到 ES 中之后，我们要将 idea 的 JAVA 版本正确设置。主要有两个地方
设置完这些之后.因为我导入的是 idea 中,idea 会帮我们做一些自动配置. 紧接着就是一段漫长的等待过程,这种 idea 在中间会下载一些列的依赖等信息。
一切下载完成之后,如果一切顺利的话,那么我们就可以正常的打开代码了.
运行 elasticsearch
打开 es 的启动类 Elasticsearch.java 执行 main 方法运行。发现报错如下
ERROR: the system property [es.path.conf] must be set
复制代码
这是告诉我们需要指定一个包含配置文件的目录，我们可以直接下载一个发行版的 ES，然后在 idea 的VM Options指定这个这个发行版 ES 的 config 目录，然后同时需要指定es.path.home，配置如下：
-Des.path.conf=/Users/xxx/software/elasticsearch-7.6.0/config-Des.path.home=/Users/xxx/software/elasticsearch-7.6.0/config
// 注意这里要下载对应的es发行版本例如我这里编译的是es 7.6那么我们就需要下载对应的release 7.6版本
复制代码
然后运行，有报错：
Exception in thread "main" java.lang.NullPointerException at org.elasticsearch.node.InternalSettingsPreparer.checkSettingsForTerminalDeprecation(InternalSettingsPreparer.java:119) at org.elasticsearch.node.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:91) at org.elasticsearch.bootstrap.Bootstrap.createEnvironment(Bootstrap.java:267) at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:306) at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) at org.elasticsearch.cli.Command.main(Command.java:90) at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92)
复制代码
debug 跟到源码里，发现是node.name这值是空导致的，那简单，在配置文件里加上就行了。修改elasticsearch.yml，加上下面这个
然后继续运行，再次报错：
2020-12-20 10:20:53,679 main ERROR Could not register mbeans java.security.AccessControlException: access denied ("javax.management.MBeanTrustPermission" "register") at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:444) at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanTrustPermission(DefaultMBeanServerInterceptor.java:1805) at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:318) at java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at org.apache.logging.log4j.core.jmx.Server.register(Server.java:393) at org.apache.logging.log4j.core.jmx.Server.reregisterMBeansAfterReconfigure(Server.java:168) at org.apache.logging.log4j.core.jmx.Server.reregisterMBeansAfterReconfigure(Server.java:141) at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:558) at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:263)
复制代码
这是 log4j JMX 报错了。JMX 是一个管理应用程序的接口，log4j 提供了对 JMX 的支持，实现了远程动态修改配置等功能。
这个其实不是 ES 的核心功能，既然报错了，干脆就把它关闭吧
-Dlog4j2.disable.jmx=true
复制代码
继续运行，还是报错：
[2020-12-20T10:22:36,426][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [node-1] fatal error in thread [main], exitingjava.lang.NoClassDefFoundError: org/elasticsearch/plugins/ExtendedPluginsClassLoader at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:545) ~[main/:?] at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:471) ~[main/:?] at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:163) ~[main/:?] at org.elasticsearch.node.Node.<init>(Node.java:313) ~[main/:?] at org.elasticsearch.node.Node.<init>(Node.java:257) ~[main/:?] at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:221) ~[main/:?] at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) ~[main/:?] at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[main/:?] at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[main/:?] at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) ~[main/:?] at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[main/:?] at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) ~[main/:?] at org.elasticsearch.cli.Command.main(Command.java:90) ~[main/:?] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) ~[main/:?] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[main/:?]
复制代码
这个错误是跟 idea 的环境有关
idea > preferences > Build, Execution, Deployment > Build Tools > Gradle
把Build and run using gradle 改成 Build and run using IntelliJ IDEA。
另一处修改是 Edit Configuration，找到Include dependencies with Provided scope，选中。
注意 idea 2020.3 这里的Include dependencies with Provided scope 非常难找，我也是参考一篇文章找到的。具体可以参考
https://youtrack.jetbrains.com/issue/IDEA-257364
运行，接着报错：
java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "createClassLoader") at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) at java.base/java.security.AccessController.checkPermission(AccessController.java:1036) at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:408) at java.base/java.lang.SecurityManager.checkCreateClassLoader(SecurityManager.java:470) at java.base/java.lang.ClassLoader.checkCreateClassLoader(ClassLoader.java:369) at java.base/java.lang.ClassLoader.checkCreateClassLoader(ClassLoader.java:359) at java.base/java.lang.ClassLoader.<init>(ClassLoader.java:456) at org.elasticsearch.plugins.ExtendedPluginsClassLoader.<init>(ExtendedPluginsClassLoader.java:36) at org.elasticsearch.plugins.ExtendedPluginsClassLoader.lambda$create$0(ExtendedPluginsClassLoader.java:57) at java.base/java.security.AccessController.doPrivileged(AccessController.java:312) at org.elasticsearch.plugins.ExtendedPluginsClassLoader.create(ExtendedPluginsClassLoader.java:56) at org.elasticsearch.plugins.PluginLoaderIndirection.createLoader(PluginLoaderIndirection.java:31) at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:545) at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:471) at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:163) at org.elasticsearch.node.Node.<init>(Node.java:313) at org.elasticsearch.node.Node.<init>(Node.java:257) at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:221) at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) at org.elasticsearch.cli.Command.main(Command.java:90) at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92)
复制代码
这个问题查了一些资料，解决方案如下：
在发行版的 config 目录下新建 java.policy 文件，填入下面内容:
grant { permission java.lang.RuntimePermission "createClassLoader";};
复制代码
然后在 VM options 加入 java.security.policy 的设置，指向该文件即可-Djava.security.policy=/Users/xxx/software/elasticsearch-7.6.0/config/java.policy
运行,继续报错
ElasticsearchException[Failure running machine learning native code. This could be due to running on an unsupported OS or distribution, missing OS libraries, or a problem with the temp directory. To bypass this problem by running Elasticsearch without machine learning functionality set [xpack.ml.enabled: false].] at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:587) at org.elasticsearch.node.Node.lambda$new$9(Node.java:456) at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1621) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) at org.elasticsearch.node.Node.<init>(Node.java:459) at org.elasticsearch.node.Node.<init>(Node.java:257) at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:221) at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) at org.elasticsearch.cli.Command.main(Command.java:90) at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92)
复制代码
根据上面的说明提示在 elasticsearch.yml 中添加如下设置xpack.ml.enabled: false
运行之后，success，终于成功了.
浏览器输入 localhost:9200
{ "name" : "node-1", "cluster_name" : "elasticsearch", "cluster_uuid" : "tGwLMOdeRvye5ZaIF_68FQ", "version" : { "number" : "7.6.0", "build_flavor" : "unknown", "build_type" : "unknown", "build_hash" : "unknown", "build_date" : "unknown", "build_snapshot" : true, "lucene_version" : "8.4.0", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search"}
复制代码
为了更加方便的连接 es,我们可以下载对应的 kibana 来管理 es,在这里我下载 kibana-7-6-0
https://www.elastic.co/cn/downloads/past-releases/kibana-7-6-0
启动 kibana，如果有以下报错
{"type":"log","@timestamp":"2020-12-21T08:19:45Z","tags":["fatal","root"],"pid":4166,"message":"{ Error: getaddrinfo ENOTFOUND localhost\n at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:56:26)\n errno: 'ENOTFOUND',\n code: 'ENOTFOUND',\n syscall: 'getaddrinfo',\n hostname: 'localhost' }"}
FATAL Error: getaddrinfo ENOTFOUND localhost
复制代码
解决办法是打开 kibana 中的配置文件 kibana.yml，修改一下两处
server.host: "localhost" ------>server.host: "127.0.0.1"
elasticsearch.hosts: ["http://localhost:9200"] ---->elasticsearch.hosts: ["http://127.0.0.1:9200"]然后重新启动就 OK 了.
如图所示:
评论