一、概述
Elasticsearch 是一个基于 Lucene 的搜索引擎。它提供了具有 HTTP Web 界面和无架构 JSON 文档的分布式，多租户能力的全文搜索引擎。Elasticsearch 是用 Java 开发的，根据 Apache 许可条款作为开源发布。
官方文档：https://www.elastic.co/guide/en/elasticsearch/reference/master/getting-started.htmlGitHub: https://github.com/elastic/elasticsearch
也可以参考我这篇文章：分布式实时搜索和分析引擎——Elasticsearch
二、Elasticsearch 编排部署
地址：https://artifacthub.io/packages/helm/elastic/elasticsearch
1）添加源并下载编排部署包
helm repo add elastic https://helm.elastic.cohelm pull elastic/elasticsearch --version 7.17.3tar -xf elasticsearch-7.17.3.tgz
复制代码
2）构建镜像
Elasticsearch 各版本下载地址：https://www.elastic.co/cn/downloads/past-releases#elasticsearch这里就不重新构建镜像了，有不知道怎么构建镜像的小伙伴可以给我留言或私信，这里是将远程的镜像推送到我们本地 harbor，加速拉取镜像。
docker pull docker.elastic.co/elasticsearch/elasticsearch:7.17.3docker tag docker.elastic.co/elasticsearch/elasticsearch:7.17.3 myharbor.com/bigdata/elasticsearch:7.17.3# 上传镜像docker push myharbor.com/bigdata/elasticsearch:7.17.3# 删除镜像docker rmi myharbor.com/bigdata/elasticsearch:7.17.3crictl rmi myharbor.com/bigdata/elasticsearch:7.17.3
复制代码
3）修改 yaml 编排
image: "myharbor.com/bigdata/elasticsearch"......### 去掉这几行volumeClaimTemplate: accessModes: ["ReadWriteOnce"] resources: requests: storage: 30Gi....persistence: enabled: true labels: # Add default labels for the volumeClaimTemplate of the StatefulSet enabled: false annotations: {} accessModes: - ReadWriteOnce size: 1Gi storageClass: "elasticsearch-local-storage" local: - name: elasticsearch-0 host: "local-168-182-110" path: "/opt/bigdata/servers/elasticsearch/data/data1" - name: elasticsearch-1 host: "local-168-182-111" path: "/opt/bigdata/servers/elasticsearch/data/data1" - name: elasticsearch-2 host: "local-168-182-112" path: "/opt/bigdata/servers/elasticsearch/data/data1"...protocol: httphttpPort: 9200transportPort: 9300service: enabled: true type: NodePort nodePort: 30920 httpPortName: http
复制代码
kind: StorageClassapiVersion: storage.k8s.io/v1metadata: name: {{ .Values.persistence.storageClass }}provisioner: kubernetes.io/no-provisioner
复制代码
{{- range .Values.persistence.local }}---apiVersion: v1kind: PersistentVolumemetadata: name: {{ .name }} labels: name: {{ .name }}spec: storageClassName: {{ $.Values.persistence.storageClass }} capacity: storage: {{ $.Values.persistence.size }} accessModes: {{- range $.Values.persistence.accessModes }} - {{ . | quote }} {{- end }} local: path: {{ .path }} nodeAffinity: required: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/hostname operator: In values: - {{ .host }}---{{- end }}
复制代码
spec: volumeClaimTemplates: spec:# 去掉这行{{ toYaml .Values.volumeClaimTemplate | indent 6 }}# 新增以下内容： accessModes: {{- range .Values.persistence.accessModes }} - {{ . | quote }} {{- end }} resources: requests: storage: {{ .Values.persistence.size | quote }} {{- if .Values.persistence.storageClass }} {{- if (eq "-" .Values.persistence.storageClass) }} storageClassName: "" {{- else }} storageClassName: "{{ .Values.persistence.storageClass }}" {{- end }} {{- end }}
复制代码
4）开始部署
# 先创建本地存储目录mkdir -p /opt/bigdata/servers/elasticsearch/data/data1chmod -R 777 /opt/bigdata/servers/elasticsearch/data/data1helm install my-elasticsearch ./elasticsearch -n elasticsearch --create-namespace# 查看helm get notes my-elasticsearch -n elasticsearchkubectl get pods,svc -n elasticsearch -owide
复制代码
NOTES
NAME: my-elasticsearchLAST DEPLOYED: Wed Oct 12 23:47:17 2022NAMESPACE: elasticsearchSTATUS: deployedREVISION: 1NOTES:1. Watch all cluster members come up. $ kubectl get pods --namespace=elasticsearch -l app=elasticsearch-master -w2. Test cluster health using Helm test. $ helm --namespace=elasticsearch test my-elasticsearch
复制代码
5）测试验证
curl http://192.168.182.110:30920/curl http://192.168.182.110:30920/_cat/nodescurl http://192.168.182.110:30920/_cat/health?pretty
复制代码
6）elasticsearch-head
elasticsearch-head GitHub 下载地址：https://github.com/mobz/elasticsearch-headGoogle 浏览器 elasticsearch-head 插件：
链接：https://pan.baidu.com/s/1kYcTjBDPmSWVzsku2hEW7w?pwd=67v4 提取码：67v4
7）卸载
helm uninstall my-elasticsearch -n elasticsearchkubectl delete ns elasticsearch --force
rm -fr /opt/bigdata/servers/elasticsearch/data/data1/*ssh local-168-182-111 "rm -fr /opt/bigdata/servers/elasticsearch/data/data1/*"ssh local-168-182-112 "rm -fr /opt/bigdata/servers/elasticsearch/data/data1/*"docker rmi myharbor.com/bigdata/elasticsearch:7.17.3crictl rmi myharbor.com/bigdata/elasticsearch:7.17.3ssh local-168-182-111 "crictl rmi myharbor.com/bigdata/elasticsearch:7.17.3"ssh local-168-182-112 "crictl rmi myharbor.com/bigdata/elasticsearch:7.17.3"
复制代码
三、Kibana 编排部署
地址：https://artifacthub.io/packages/helm/bitnami/kibana?modal=install
1）添加源并下载编排部署包
helm repo add bitnami https://charts.bitnami.com/bitnamihelm pull bitnami/kibana --version 10.2.6tar -xf kibana-10.2.6.tgz
复制代码
2）构建镜像
这里也不重新构建镜像了，只是将镜像推送到本地 harbor 加速，对构建镜像不清楚的可以留言或私信。【注意】版本号需要跟 ES 相对应，版本不一样，可能会存储不兼容的情况。
docker pull docker.io/bitnami/kibana:7.17.3docker tag docker.io/bitnami/kibana:7.17.3 myharbor.com/bigdata/kibana:7.17.3# 上传镜像docker push myharbor.com/bigdata/kibana:7.17.3# 删除镜像docker rmi myharbor.com/bigdata/kibana:7.17.3crictl rmi myharbor.com/bigdata/kibana:7.17.3
复制代码
3）修改 yaml 编排
image: registry: myharbor.com repository: bigdata/kibana tag: 8.4.3-debian-11-r1...replicaCount: 1...persistence: enabled: true accessModes: - ReadWriteOnce size: 1Gi storageClass: "kibana-local-storage" local: - name: kibana-0 host: "local-168-182-111" path: "/opt/bigdata/servers/kibana/data/data1"...service: ports: http: 5601 type: NodePort nodePorts: http: "30601"...elasticsearch: hosts: - elasticsearch-master.elasticsearch port: "9200"
复制代码
{{- range .Values.persistence.local }}---apiVersion: v1kind: PersistentVolumemetadata: name: {{ .name }} labels: name: {{ .name }}spec: storageClassName: {{ $.Values.persistence.storageClass }} capacity: storage: {{ $.Values.persistence.size }} accessModes: {{- range $.Values.persistence.accessModes }} - {{ . | quote }} {{- end }} local: path: {{ .path }} nodeAffinity: required: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/hostname operator: In values: - {{ .host }}---{{- end }}
复制代码
kind: StorageClassapiVersion: storage.k8s.io/v1metadata: name: {{ .Values.persistence.storageClass }}provisioner: kubernetes.io/no-provisioner
复制代码
{{- range .Values.persistence.local }}---apiVersion: v1kind: PersistentVolumemetadata: name: {{ .name }} labels: name: {{ .name }}spec: storageClassName: {{ $.Values.persistence.storageClass }} capacity: storage: {{ $.Values.persistence.size }} accessModes: {{- range $.Values.persistence.accessModes }} - {{ . | quote }} {{- end }} local: path: {{ .path }} nodeAffinity: required: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/hostname operator: In values: - {{ .host }}---{{- end }}
复制代码
4）开始部署
# 先创建本地存储目录mkdir -p /opt/bigdata/servers/kibana/data/data1chmod -R 777 /opt/bigdata/servers/kibana/data/data1helm install my-kibana ./kibana -n kibana --create-namespace# 查看helm get notes my-kibana -n kibana kubectl get pods,svc -n kibana -owide
复制代码
NOTES
NAME: my-kibanaLAST DEPLOYED: Thu Oct 13 22:43:30 2022NAMESPACE: kibanaSTATUS: deployedREVISION: 1TEST SUITE: NoneNOTES:CHART NAME: kibanaCHART VERSION: 10.2.6APP VERSION: 8.4.3** Please be patient while the chart is being deployed **######################################################################################################### ERROR: You did not provide the Elasticsearch external host or port in your 'helm install' call #########################################################################################################Complete your Kibana deployment by running:
helm upgrade --namespace kibana my-kibana my-repo/kibana \ --set elasticsearch.hosts[0]=YOUR_ES_HOST,elasticsearch.port=YOUR_ES_PORT
Replacing "YOUR_ES_HOST" and "YOUR_ES_PORT" placeholders by the proper values of your Elasticsearch deployment.
复制代码
5）测试验证
web：http://192.168.182.111:30601
6）卸载
helm uninstall my-kibana -n kibanakubectl delete ns kibana --forcessh local-168-182-111 rm -fr /opt/bigdata/servers/kibana/data/data1/*docker rmi myharbor.com/bigdata/bigdata:8.4.3-debian-11-r1crictl rmi myharbor.com/bigdata/bigdata:8.4.3-debian-11-r1
复制代码
elasticsearch-on-k8s 下载地址：https://gitee.com/hadoop-bigdata/elasticsearch-on-k8skibana-on-k8s 下载地址：https://gitee.com/hadoop-bigdata/kibana-on-k8s
Elasticsearch + kibana on k8s 编排部署讲解与实战操作就先到这里了，这里只是简单的演示了查询，更多的操作，可以参考官方文档，有疑问的小伙欢迎给我留言，后续会持续更新一、概述
Elasticsearch 是一个基于 Lucene 的搜索引擎。它提供了具有 HTTP Web 界面和无架构 JSON 文档的分布式，多租户能力的全文搜索引擎。Elasticsearch 是用 Java 开发的，根据 Apache 许可条款作为开源发布。
官方文档：https://www.elastic.co/guide/en/elasticsearch/reference/master/getting-started.htmlGitHub: https://github.com/elastic/elasticsearch
也可以参考我这篇文章：分布式实时搜索和分析引擎——Elasticsearch
二、Elasticsearch 编排部署
地址：https://artifacthub.io/packages/helm/elastic/elasticsearch
1）添加源并下载编排部署包
helm repo add elastic https://helm.elastic.cohelm pull elastic/elasticsearch --version 7.17.3tar -xf elasticsearch-7.17.3.tgz
复制代码
2）构建镜像
Elasticsearch 各版本下载地址：https://www.elastic.co/cn/downloads/past-releases#elasticsearch这里就不重新构建镜像了，有不知道怎么构建镜像的小伙伴可以给我留言或私信，这里是将远程的镜像推送到我们本地 harbor，加速拉取镜像。
docker pull docker.elastic.co/elasticsearch/elasticsearch:7.17.3docker tag docker.elastic.co/elasticsearch/elasticsearch:7.17.3 myharbor.com/bigdata/elasticsearch:7.17.3# 上传镜像docker push myharbor.com/bigdata/elasticsearch:7.17.3# 删除镜像docker rmi myharbor.com/bigdata/elasticsearch:7.17.3crictl rmi myharbor.com/bigdata/elasticsearch:7.17.3
复制代码
3）修改 yaml 编排
image: "myharbor.com/bigdata/elasticsearch"......### 去掉这几行volumeClaimTemplate: accessModes: ["ReadWriteOnce"] resources: requests: storage: 30Gi....persistence: enabled: true labels: # Add default labels for the volumeClaimTemplate of the StatefulSet enabled: false annotations: {} accessModes: - ReadWriteOnce size: 1Gi storageClass: "elasticsearch-local-storage" local: - name: elasticsearch-0 host: "local-168-182-110" path: "/opt/bigdata/servers/elasticsearch/data/data1" - name: elasticsearch-1 host: "local-168-182-111" path: "/opt/bigdata/servers/elasticsearch/data/data1" - name: elasticsearch-2 host: "local-168-182-112" path: "/opt/bigdata/servers/elasticsearch/data/data1"...protocol: httphttpPort: 9200transportPort: 9300service: enabled: true type: NodePort nodePort: 30920 httpPortName: http
复制代码
kind: StorageClassapiVersion: storage.k8s.io/v1metadata: name: {{ .Values.persistence.storageClass }}provisioner: kubernetes.io/no-provisioner
复制代码
{{- range .Values.persistence.local }}---apiVersion: v1kind: PersistentVolumemetadata: name: {{ .name }} labels: name: {{ .name }}spec: storageClassName: {{ $.Values.persistence.storageClass }} capacity: storage: {{ $.Values.persistence.size }} accessModes: {{- range $.Values.persistence.accessModes }} - {{ . | quote }} {{- end }} local: path: {{ .path }} nodeAffinity: required: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/hostname operator: In values: - {{ .host }}---{{- end }}
复制代码
spec: volumeClaimTemplates: spec:# 去掉这行{{ toYaml .Values.volumeClaimTemplate | indent 6 }}# 新增以下内容： accessModes: {{- range .Values.persistence.accessModes }} - {{ . | quote }} {{- end }} resources: requests: storage: {{ .Values.persistence.size | quote }} {{- if .Values.persistence.storageClass }} {{- if (eq "-" .Values.persistence.storageClass) }} storageClassName: "" {{- else }} storageClassName: "{{ .Values.persistence.storageClass }}" {{- end }} {{- end }}
复制代码
4）开始部署
# 先创建本地存储目录mkdir -p /opt/bigdata/servers/elasticsearch/data/data1chmod -R 777 /opt/bigdata/servers/elasticsearch/data/data1helm install my-elasticsearch ./elasticsearch -n elasticsearch --create-namespace# 查看helm get notes my-elasticsearch -n elasticsearchkubectl get pods,svc -n elasticsearch -owide
复制代码
NOTES
NAME: my-elasticsearchLAST DEPLOYED: Wed Oct 12 23:47:17 2022NAMESPACE: elasticsearchSTATUS: deployedREVISION: 1NOTES:1. Watch all cluster members come up. $ kubectl get pods --namespace=elasticsearch -l app=elasticsearch-master -w2. Test cluster health using Helm test. $ helm --namespace=elasticsearch test my-elasticsearch
复制代码
5）测试验证
curl http://192.168.182.110:30920/curl http://192.168.182.110:30920/_cat/nodescurl http://192.168.182.110:30920/_cat/health?pretty
复制代码
6）elasticsearch-head
elasticsearch-head GitHub 下载地址：https://github.com/mobz/elasticsearch-headGoogle 浏览器 elasticsearch-head 插件：
链接：https://pan.baidu.com/s/1kYcTjBDPmSWVzsku2hEW7w?pwd=67v4 提取码：67v4
7）卸载
helm uninstall my-elasticsearch -n elasticsearchkubectl delete ns elasticsearch --force
rm -fr /opt/bigdata/servers/elasticsearch/data/data1/*ssh local-168-182-111 "rm -fr /opt/bigdata/servers/elasticsearch/data/data1/*"ssh local-168-182-112 "rm -fr /opt/bigdata/servers/elasticsearch/data/data1/*"docker rmi myharbor.com/bigdata/elasticsearch:7.17.3crictl rmi myharbor.com/bigdata/elasticsearch:7.17.3ssh local-168-182-111 "crictl rmi myharbor.com/bigdata/elasticsearch:7.17.3"ssh local-168-182-112 "crictl rmi myharbor.com/bigdata/elasticsearch:7.17.3"
复制代码
三、Kibana 编排部署
地址：https://artifacthub.io/packages/helm/bitnami/kibana?modal=install
1）添加源并下载编排部署包
helm repo add bitnami https://charts.bitnami.com/bitnamihelm pull bitnami/kibana --version 10.2.6tar -xf kibana-10.2.6.tgz
复制代码
2）构建镜像
这里也不重新构建镜像了，只是将镜像推送到本地 harbor 加速，对构建镜像不清楚的可以留言或私信。【注意】版本号需要跟 ES 相对应，版本不一样，可能会存储不兼容的情况。
docker pull docker.io/bitnami/kibana:7.17.3docker tag docker.io/bitnami/kibana:7.17.3 myharbor.com/bigdata/kibana:7.17.3# 上传镜像docker push myharbor.com/bigdata/kibana:7.17.3# 删除镜像docker rmi myharbor.com/bigdata/kibana:7.17.3crictl rmi myharbor.com/bigdata/kibana:7.17.3
复制代码
3）修改 yaml 编排
image: registry: myharbor.com repository: bigdata/kibana tag: 8.4.3-debian-11-r1...replicaCount: 1...persistence: enabled: true accessModes: - ReadWriteOnce size: 1Gi storageClass: "kibana-local-storage" local: - name: kibana-0 host: "local-168-182-111" path: "/opt/bigdata/servers/kibana/data/data1"...service: ports: http: 5601 type: NodePort nodePorts: http: "30601"...elasticsearch: hosts: - elasticsearch-master.elasticsearch port: "9200"
复制代码
{{- range .Values.persistence.local }}---apiVersion: v1kind: PersistentVolumemetadata: name: {{ .name }} labels: name: {{ .name }}spec: storageClassName: {{ $.Values.persistence.storageClass }} capacity: storage: {{ $.Values.persistence.size }} accessModes: {{- range $.Values.persistence.accessModes }} - {{ . | quote }} {{- end }} local: path: {{ .path }} nodeAffinity: required: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/hostname operator: In values: - {{ .host }}---{{- end }}
复制代码
kind: StorageClassapiVersion: storage.k8s.io/v1metadata: name: {{ .Values.persistence.storageClass }}provisioner: kubernetes.io/no-provisioner
复制代码
{{- range .Values.persistence.local }}---apiVersion: v1kind: PersistentVolumemetadata: name: {{ .name }} labels: name: {{ .name }}spec: storageClassName: {{ $.Values.persistence.storageClass }} capacity: storage: {{ $.Values.persistence.size }} accessModes: {{- range $.Values.persistence.accessModes }} - {{ . | quote }} {{- end }} local: path: {{ .path }} nodeAffinity: required: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/hostname operator: In values: - {{ .host }}---{{- end }}
复制代码
4）开始部署
# 先创建本地存储目录mkdir -p /opt/bigdata/servers/kibana/data/data1chmod -R 777 /opt/bigdata/servers/kibana/data/data1helm install my-kibana ./kibana -n kibana --create-namespace# 查看helm get notes my-kibana -n kibana kubectl get pods,svc -n kibana -owide
复制代码
NOTES
NAME: my-kibanaLAST DEPLOYED: Thu Oct 13 22:43:30 2022NAMESPACE: kibanaSTATUS: deployedREVISION: 1TEST SUITE: NoneNOTES:CHART NAME: kibanaCHART VERSION: 10.2.6APP VERSION: 8.4.3** Please be patient while the chart is being deployed **######################################################################################################### ERROR: You did not provide the Elasticsearch external host or port in your 'helm install' call #########################################################################################################Complete your Kibana deployment by running:
helm upgrade --namespace kibana my-kibana my-repo/kibana \ --set elasticsearch.hosts[0]=YOUR_ES_HOST,elasticsearch.port=YOUR_ES_PORT
Replacing "YOUR_ES_HOST" and "YOUR_ES_PORT" placeholders by the proper values of your Elasticsearch deployment.
复制代码
5）测试验证
web：http://192.168.182.111:30601
6）卸载
helm uninstall my-kibana -n kibanakubectl delete ns kibana --forcessh local-168-182-111 rm -fr /opt/bigdata/servers/kibana/data/data1/*docker rmi myharbor.com/bigdata/bigdata:8.4.3-debian-11-r1crictl rmi myharbor.com/bigdata/bigdata:8.4.3-debian-11-r1
复制代码
elasticsearch-on-k8s 下载地址：https://gitee.com/hadoop-bigdata/elasticsearch-on-k8skibana-on-k8s 下载地址：https://gitee.com/hadoop-bigdata/kibana-on-k8s
Elasticsearch + kibana on k8s 编排部署讲解与实战操作就先到这里了，这里只是简单的演示了查询，更多的操作，可以参考官方文档，有疑问的小伙欢迎给我留言，后续会持续更新
评论