写点什么

极客时间运维进阶训练营第五周作业

作者:9527
  • 2022-11-25
    美国
  • 本文字数:8664 字

    阅读完需:约 28 分钟

20221120 作业:

1.完全基于 pipline 实现完整的代码部署流水线

2.熟悉 ELK 各组件的功能、elasticsearch 的节点角色类型

3.熟悉索引、doc、分片与副本的概念

4.掌握不同环境的 ELK 部署规划,基于 deb 或二进制部署 elasticsearch 集群

5.了解 elasticsearch API 的简单使用,安装 head 插件管理 ES 的数据

6.安装 logstash 收集不同类型的系统日志并写入到 ES 的不同 index

7.安装 kibana、查看 ES 集群的数据

扩展:

1.了解 heartbeat 和 metricbeat 的使用

Mandatory Assignment

Jenkins Pipeline Full Integration

  • Jenkinsfile

#!/usr/bin/env groovy
pipeline { agent { label 'slave1' } options { parallelsAlwaysFailFast() } stages { stage('init') { steps { println "Build started" } } stage('Linux CICD') { parallel { stage('linux') { agent { label "master" } stages { // Active the Python virtual env stage('setup') { steps { sh label: 'Active python venv', script: ''' echo "==========Create virtual env==========" python3.8 -m venv venv source venv/bin/activate pip install --upgrade pip pip install -r requirements.txt echo "==========End virtual env==========" ''' } } stage('build') { steps { sh label: 'build', script: ''' echo "==========Testing and building==========" export PYTHONPATH="./app" python -m unittest echo "==========Testing and building==========" source venv/bin/activate python setup.py bdist_wheel ls ''' } } stage('deploy') { when { branch 'main' } steps { script { sh label: 'build', script: ''' echo "Deploy to s3 bucket..." VERSION=`cat app/VERSION` aws s3 cp dist/infinity-$VERSION* s3://bucket_name/ ''' } } } stage('run') { when { branch 'main' } steps { script { sh label: 'build', script: ''' echo "Download package from s3 bucket..." VERSION=`cat app/VERSION` aws s3 cp s3://bucket_name/infinity-$VERSION-py3-none-any.whl /tmp source venv/bin/activate which pip pip install /tmp/infinity-$VERSION-py3-none-any.whl pip list export PYTHONPATH="./app" cd app pwd nohup uvicorn main:app --host 0.0.0.0 --port 8000 & ''' } } } } post { // Slack notifications success { sh label: 'success', script: 'echo "Success"' slackSend channel: 'jenkins-noti', color: 'good', message: "<${env.BUILD_URL}|${env.JOB_NAME} #${env.BUILD_NUMBER}> - Build completed successfully" } failure { sh label: 'failure', script: 'echo "Failure"' slackSend channel: 'jenkins-noti', color: 'danger', message: "<${env.BUILD_URL}|${env.JOB_NAME} #${env.BUILD_NUMBER}> - Build failed" } aborted { sh label: 'aborted', script: 'echo "Aborted"' slackSend channel: 'jenkins-noti', color: 'danger', message: "<${env.BUILD_URL}|${env.JOB_NAME} #${env.BUILD_NUMBER}> - Build aborted" } always { sh label: 'Linux build done', script: 'echo "Done Linux build"' //cleanWs()// } } } } } }}
复制代码

ELK Intro

  • What is ELK



  • ELK features

  • ELK shards

Install ES

  •  Install the Elasticsearch RPM package on each EC2 instance as instructed below.

$ sudo rpm -i https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/rpm/elasticsearch/2.3.3/elasticsearch-2.3.3.rpmwarning: /var/tmp/rpm-tmp.R0MYGs: Header V4 RSA/SHA1 Signature, key ID d88e42b4: NOKEYCreating elasticsearch group... OKCreating elasticsearch user... OK
复制代码
  • Register as system service

$ systemctl daemon-reload$ systemctl enable elasticsearch.service
复制代码
  • Install plugins

$ cd /usr/share/elasticsearch/$  bin/plugin install cloud-aws-> Installing cloud-aws...Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/cloud-aws/2.3.3/cloud-aws-2.3.3.zip ...Downloading ...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................DONEVerifying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/cloud-aws/2.3.3/cloud-aws-2.3.3.zip checksums if available ...Downloading .DONE@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@     WARNING: plugin requires additional permissions     @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@* java.lang.RuntimePermission getClassLoaderSee http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.htmlfor descriptions of what these permissions allow and the associated risks.
Continue with installation? [y/N]yInstalled cloud-aws into /usr/share/elasticsearch/plugins/cloud-aws
复制代码
  • Update configs, Open "/etc/sysconfig/elasticsearch" on each EC2 instance with your favorite editor and set the "ES_HEAP_SIZE" and "MAX_LOCKED_MEMORY" parameters.

ES_HEAP_SIZE=4g MAX_LOCKED_MEMORY=unlimited
复制代码
  • Open "/etc/elasticsearch/elasticsearch.yml" 

cluster.name: testesbootstrap.mlockall: true discovery.zen.ping.unicast.hosts: [_ip_address_,…] network.host: [_ip_address_]
复制代码
  • If use Java11, you need to comment out the following from, otherwise you will run into error "Unrecognized VM option 'UseParNewGC'"

$ vim /usr/share/elasticsearch/bin/elasticsearch.in.sh
#ES_GC_OPTS="$ES_GC_OPTS -XX:+UseParNewGC"
复制代码
  • Start ES service

$ service elasticsearch startStarting elasticsearch (via systemctl):                    [  OK  ]
systemctl status elasticsearch● elasticsearch.service - Elasticsearch Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled) Active: active (running) since Thu 2022-11-24 15:16:32 UTC; 2s ago Docs: http://www.elastic.co Process: 6409 ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec (code=exited, status=0/SUCCESS) Main PID: 6412 (java) CGroup: /system.slice/elasticsearch.service └─6412 /bin/java -Xms4g -Xmx4g -Djava.awt.headless=true -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapD...
Nov 24 15:16:32 ip-172-31-68-222.ec2.internal systemd[1]: Starting Elasticsearch...Nov 24 15:16:32 ip-172-31-68-222.ec2.internal systemd[1]: Started Elasticsearch.Nov 24 15:16:32 ip-172-31-68-222.ec2.internal elasticsearch[6412]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will ...release.Nov 24 15:16:34 ip-172-31-68-222.ec2.internal elasticsearch[6412]: [2022-11-24 15:16:34,757][INFO ][node ] [Blood Spider] version[2.3.3], pid[6412...:40:04Z]Nov 24 15:16:34 ip-172-31-68-222.ec2.internal elasticsearch[6412]: [2022-11-24 15:16:34,758][INFO ][node ] [Blood Spider] initializing ...
复制代码
  • Check cluster health

$ curl localhost:9200/_cluster/health?pretty{  "cluster_name" : "testes",  "status" : "green",  "timed_out" : false,  "number_of_nodes" : 1,  "number_of_data_nodes" : 1,  "active_primary_shards" : 0,  "active_shards" : 0,  "relocating_shards" : 0,  "initializing_shards" : 0,  "unassigned_shards" : 0,  "delayed_unassigned_shards" : 0,  "number_of_pending_tasks" : 0,  "number_of_in_flight_fetch" : 0,  "task_max_waiting_in_queue_millis" : 0,  "active_shards_percent_as_number" : 100.0}
复制代码

Install ES with RPM

  • Download rpm

$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.5.2-x86_64.rpm$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.5.2-x86_64.rpm.sha512$ shasum -a 512 -c elasticsearch-8.5.2-x86_64.rpm.sha512 $ rpm --install elasticsearch-8.5.2-x86_64.rpmwarning: elasticsearch-8.5.2-x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID d88e42b4: NOKEYCreating elasticsearch group... OKCreating elasticsearch user... OK--------------------------- Security autoconfiguration information ------------------------------
Authentication and authorization are enabled.TLS for the transport and HTTP layers is enabled and configured.
The generated password for the elastic built-in superuser is : SfKIw1jzMz=1lshM=ZpX
If this node should join an existing cluster, you can reconfigure this with'/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token <token-here>'after creating an enrollment token on your existing cluster.
You can complete the following actions at any time:
Reset the password of the elastic built-in superuser with'/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic'.
Generate an enrollment token for Kibana instances with '/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana'.
Generate an enrollment token for Elasticsearch nodes with'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node'.
-------------------------------------------------------------------------------------------------### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd sudo systemctl daemon-reload sudo systemctl enable elasticsearch.service### You can start elasticsearch service by executing sudo systemctl start elasticsearch.service
复制代码
  • Check ES is running:

$ curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic https://localhost:9200Enter host password for user 'elastic':{  "name" : "ip-172-31-69-91.ec2.internal",  "cluster_name" : "elasticsearch",  "cluster_uuid" : "YN-BAx-3SR-LZpokzkEKOA",  "version" : {    "number" : "8.5.2",    "build_flavor" : "default",    "build_type" : "rpm",    "build_hash" : "a846182fa16b4ebfcc89aa3c11a11fd5adf3de04",    "build_date" : "2022-11-17T18:56:17.538630285Z",    "build_snapshot" : false,    "lucene_version" : "9.4.1",    "minimum_wire_compatibility_version" : "7.17.0",    "minimum_index_compatibility_version" : "7.0.0"  },  "tagline" : "You Know, for Search"}
复制代码

Install Kibana

  • Download and install yum config file

$ wget https://artifacts.elastic.co/downloads/kibana/kibana-8.5.2-x86_64.rpm$ shasum -a 512 kibana-8.5.2-x86_64.rpm $ rpm --install kibana-8.5.2-x86_64.rpm
复制代码
  • Generate an enrollment token for Kibana

$ /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibanaeyJ2ZXIiOiI4LjUuMiIsImFkciI6WyIxNzIuMzEuNjkuOTE6OTIwMCJdLCJmZ3IiOiI5YWExZWVjOGIyNDg0YjJjZjAwMWMwMzlkYjIyMzIzMTFiYThjOThkMzgyOTc1NmFiODk4ZWM4MThmYmE4NDMxIiwia2V5Ijoid3ZoTnFvUUIybVd0THU0NDRSZmQ6NzdnU1FHWWRScFM1RkNvV2c5QUZVZyJ9
复制代码

Install Logstash

  • The Logstash binaries are available from https://www.elastic.co/downloads. Download the Logstash installation file for your host environment—​TAR.GZ, DEB, ZIP, or RPM.

  • Install

$ rpm -i logstash-8.5.2-x86_64.rpm
复制代码
  • Run logstash

$ logstash -f logstash.conf
复制代码

Run Kibana as system service

  • Configure systemd

$ systemctl daemon-reload$ systemctl enable kibana.service
复制代码
  • Start service

$ systemctl start kibana.service
复制代码
  • Access Kibana

  • Login

  • Explore sample data


  • Explore logs


Optional Assignment


  1. 了解 heartbeat 和 metricbeat 的使用

Heartbeat

What is heartbeat

Heartbeat is a lightweight daemon that you install on a remote server to periodically check the status of your services and determine whether they are available. 

Heartbeat currently supports monitors for checking hosts via:

  • ICMP (v4 and v6) Echo Requests. Use the icmp monitor when you simply want to check whether a service is available. This monitor requires root access.

  • TCP. Use the tcp monitor to connect via TCP. You can optionally configure this monitor to verify the endpoint by sending and/or receiving a custom payload.

  • HTTP. Use the http monitor to connect via HTTP. You can optionally configure this monitor to verify that the service returns the expected response, such as a specific status code, response header, or content.

Install heartbeat

  • Unlike most Beats, which you install on edge nodes, you typically install Heartbeat as part of a monitoring service that runs on a separate machine and possibly even outside of the network where the services that you want to monitor are running.

  • Install

$ curl -L -O https://artifacts.elastic.co/downloads/beats/heartbeat/heartbeat-8.5.2-x86_64.rpm$ rpm -vi heartbeat-8.5.2-x86_64.rpm
复制代码
  • Connect to ES

  • Set host

output.elasticsearch:  hosts: ["https://myEShost:9200"]  username: "heartbeat_internal"  password: "YOUR_PASSWORD"   ssl:    enabled: true    ca_trusted_fingerprint: "b9a10bbe64ee9826abeda6546fc988c8bf798b41957c33d05db736716513dc9c"
复制代码
  • Configure the Kibana endpoint

setup.kibana:    host: "mykibanahost:5601"     username: "my_kibana_user"      password: "{pwd}"
复制代码
  • Configure heartbeat monitor

heartbeat.monitors:- type: icmp  schedule: '*/5 * * * * * *'   hosts: ["myhost"]  id: my-icmp-service  name: My ICMP Service- type: tcp  schedule: '@every 5s'   hosts: ["myhost:12345"]  mode: any   id: my-tcp-service- type: http  schedule: '@every 5s'  urls: ["http://example.net"]  service.name: apm-service-name   id: my-http-service  name: My HTTP Service
复制代码

Metricbeat

What is metricbeat

Metricbeat is a lightweight shipper that you can install on your servers to periodically collect metrics from the operating system and from services running on the server. Metricbeat takes the metrics and statistics that it collects and ships them to the output that you specify, such as Elasticsearch or Logstash.


Metricbeat helps you monitor your servers and the services they host by collecting metrics from the operating system and services.

This guide describes how to get started quickly with metrics collection. You’ll learn how to:

  • install Metricbeat on each system you want to monitor

  • specify the metrics you want to collect

  • send the metrics to Elasticsearch

  • visualize the metrics data in Kibana


Install metricbeat

  • Install

$ curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-8.5.2-x86_64.rpm$ rpm -vi metricbeat-8.5.2-x86_64.rpm
复制代码
  • Connect to ES

output.elasticsearch:  hosts: ["https://myEShost:9200"]  username: "metricbeat_internal"  password: "YOUR_PASSWORD"   ssl:    enabled: true    ca_trusted_fingerprint: "b9a10bbe64ee9826abeda6546fc988c8bf798b41957c33d05db736716513dc9c" 
复制代码
  • Connect to Kibana

setup.kibana:    host: "mykibanahost:5601"     username: "my_kibana_user"      password: "{pwd}"
复制代码


发布于: 刚刚阅读数: 3
用户头像

9527

关注

还未添加个人签名 2020-04-22 加入

还未添加个人简介

评论

发布
暂无评论
极客时间运维进阶训练营第五周作业_9527_InfoQ写作社区