Graylog 部署文档

用户头像
蚍蜉
关注
发布于: 2020 年 05 月 06 日
Graylog部署文档

前言

Graylog的架构主要有两种,一种是最小化Graylog,用于较小的非关键或测试环境,各组成组件之间不冗余,设置简单快捷;另一种是针对更大生产环境下的Graylog,各组成组件之间组成集群架构,在Graylog节点前部署负载均衡,适用于生产环境。

最小化Graylog所有的组件都部署在一台服务器之上,其架构如下:

生产环境下Graylog架构如下:

一、准备环境

此次部署既要满足测试功能,又要能够对生产环境下的架构有清楚认识,决定先部署一台单节点的最小化graylog,然后再扩展加入集群。

现场环境准备3台虚拟机进行安装部署,各组件版本选型分别是:elasticsearch-6.6.2、mongodb-4.0、graylog-3.0.0、openjdk1.8.0_181等。

3台虚拟机配置如下:

二、单节点部署

首先部署单节点最小化graylog,部署步骤如下(部署在192.168.80.111上):

  1. 配置可通外网网络或搭建内部最新yum源(此处不做详细介绍)

  2. 关闭selinux、iptable、firewalld等防火墙

$ setenforce 0
# 将SELINUX=enable修改成SELINUX=disabled,保存退出
$ vim /etc/selinux/config
$ systemctl stop firewalld
$ systemctl disable firewalld
  1. 系统优化

$ vim /etc/security/limits.conf
# 加入以下内容
* soft nofile 655360
* hard nofile 655360
* soft nproc 655360
* hard nproc 655360
* soft memlock unlimited
* hard memlock unlimited
$ vim /etc/sysctl.conf
# 加入以下内容
vm.max_map_count = 655360
$ source /etc/sysctl.conf
  1. 安装jdk以及pwgen(用来生成密码校验码)

$ yum install java-1.8.0-openjdk-headless.x86_64
$ yum install epel-release
$ yum install pwgen
  1. 安装mongodb

# 新建mongodb的repo文件
$ vim /etc/yum.repos.d/mongodb-org-4.0.repo
# 加入以下内容:
[mongodb-org-4.0]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/4.0/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-4.0.asc
$ yum install mongodb-org
$ systemctl daemon-reload
$ systemctl enable mongod.service
$ systemctl start mongod.service
  1. 安装Elasticsearch

$ rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
# 新建Elasticsearch的repo文件
$ vim /etc/yum.repos.d/elasticsearch.repo
# 加入以下内容:
[elasticsearch-6.x]
name=Elasticsearch repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/oss-6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
$ yum install elasticsearch-oss
$ vim /etc/elasticsearch/elasticsearch.yml
# 修改elasticsearch配置文件,加入以下内容
cluster.name: graylog
action.auto_create_index: false
$ chkconfig --add elasticsearch
$ systemctl daemon-reload
$ systemctl enable elasticsearch.service
$ systemctl start elasticsearch.service
  1. 安装graylog

$ rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-3.0-repository_latest.rpm
$ yum install graylog-server
$ vim /etc/graylog/server/server.conf
# 修改graylog配置文件,将password_secret和root_password_sha2的值填上,password_secret的值通过<pwgen -N 1 -s 96>命令生成,root_password_sha2的值通过< echo -n "Enter Password: " && head -1 </dev/stdin | tr -d '\n' | sha256sum | cut -d" " -f1>命令生成,填上后,保存退出。如:
password_secret = ESVfAErGXhpih8FLTncNP3tDAoz4V6Oz25PWdca8r4aqLJJNt8Yl5MRv5EGyJnjBDvUFSBgCaL8Ymm8D3s3Oub0j8Un5Zuvy
root_password_sha2 = fee074307074d18c93ebeb597aed4dfcc87d856864afb2d4216183ccc65687c2
# 修改http_bind_address的值,以方便访问
http_bind_address = 0.0.0.0:9000
$ chkconfig --add graylog-server
$ systemctl daemon-reload
$ systemctl enable graylog-server.service
$ systemctl start graylog-server.service
  1. 测试graylog

打开浏览器访问graylog服务器地址,如:http://192.168.80.111:9000

添加input测试收集是否正常,system→inputs→select input→Raw/Plaintext TCP→Launch new input,选择Node,添加Title,点击Save即可。



然后到节点上输入命令:

$ echo "Hello, Just Test!" | nc 192.168.80.111 5555



点击Show received messages,即可查看到日志信息。

三、集群部署

接下来,介绍如何部署生产环境下graylog集群,部署步骤如下(部署在192.168.80.112和192.168.80.113上):

  1. 配置可通外网网络或搭建内部最新yum源(此处不做详细介绍)

  2. 关闭selinux、iptable、firewalld等防火墙

$ setenforce 0
# 将SELINUX=enable修改成SELINUX=disabled,保存退出
$ vim /etc/selinux/config
$ systemctl stop firewalld
$ systemctl disable firewalld
  1. 系统优化

$ vim /etc/security/limits.conf
# 加入以下内容
soft nofile 655360
hard nofile 655360
soft nproc 655360
hard nproc 655360
soft memlock unlimited
hard memlock unlimited
$ vim /etc/sysctl.conf
# 加入以下内容
vm.maxmapcount = 655360
$ source /etc/sysctl.conf
  1. 安装jdk以及pwgen(用来生成密码校验码)

$ yum install java-1.8.0-openjdk-headless.x86_64
$ yum install epel-release
$ yum install pwgen
  1. 安装mongodb

# 新建mongodb的repo文件
$ vim /etc/yum.repos.d/mongodb-org-4.0.repo
# 加入以下内容:
[mongodb-org-4.0]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/4.0/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-4.0.asc
$ yum install mongodb-org
$ vim /etc/mongodb.conf
# 修改其中内容如下,保存退出:
net:
port: 27017
bindIp: 0.0.0.0
replication:
replSetName: "rs0"
$ systemctl daemon-reload
$ systemctl enable mongod.service
$ systemctl start mongod.service
  1. 配置mongodb副本集

# 在主节点(192.168.80.111)上执行:
$ mongo
> rs.initiate( {
_id : "rs0",
members: [
{ _id: 0, host: "192.168.80.111:27017" },
{ _id: 1, host: "192.168.80.112:27017" },
{ _id: 2, host: "192.168.80.113:27017" }
]
})
> rs.conf() #查看配置是否生效
> rs.status() #查看节点状态,如果可以看到其他节点,表示副本集配置成功
  1. 安装Elasticsearch

$ rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
# 新建Elasticsearch的repo文件
$ vim /etc/yum.repos.d/elasticsearch.repo
# 加入以下内容:
[elasticsearch-6.x]
name=Elasticsearch repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/oss-6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
$ yum install elasticsearch-oss
$ chkconfig --add elasticsearch
$ systemctl daemon-reload
$ systemctl enable elasticsearch.service
  1. 配置Elasticsearch集群

# 修改主节点(192.168.80.111)上elasticsearch的配置文件
$ vim /etc/elasticsearch/elasticsearch.yml
# 修改elasticsearch配置文件,加入以下内容
cluster.name: graylog
node.name: es-node01
network.host: 192.168.80.111
http.port: 9200
discovery.zen.ping.unicast.hosts: ["192.168.80.112", "192.168.80.113"]
action.autocreateindex: false
# 修改从节点(192.168.80.112)上elasticsearch的配置文件
$ vim /etc/elasticsearch/elasticsearch.yml
# 修改elasticsearch配置文件,加入以下内容
cluster.name: graylog
node.name: es-node02
network.host: 192.168.80.112
http.port: 9200
discovery.zen.ping.unicast.hosts: ["192.168.80.111", "192.168.80.113"]
action.autocreateindex: false
# 修改从节点(192.168.80.113)上elasticsearch的配置文件
$ vim /etc/elasticsearch/elasticsearch.yml
# 修改elasticsearch配置文件,加入以下内容
cluster.name: graylog
node.name: es-node03
network.host: 192.168.80.113
http.port: 9200
discovery.zen.ping.unicast.hosts: ["192.168.80.111", "192.168.80.112"]
action.autocreateindex: false
$ systemctl restart elasticsearch.service
$ curl -XGET 'http://192.168.80.111:9200/_cat/nodes?pretty' #查看elasticsearch节点状态,*为主节点
  1. 安装graylog

$ rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-3.0-repository_latest.rpm
$ yum install graylog-server
$ chkconfig --add graylog-server
$ systemctl daemon-reload
$ systemctl enable graylog-server.service
  1. 配置graylog

# 修改主节点(192.168.80.111)上graylog配置文件
$ vim /etc/graylog/server/server.conf
# 修改graylog配置文件,修改以下选项值,内容如下:
is_master = true
password_secret = ESVfAErGXhpih8FLTncNP3tDAoz4V6Oz25PWdca8r4aqLJJNt8Yl5MRv5EGyJnjBDvUFSBgCaL8Ymm8D3s3Oub0j8Un5ZUvy
rootpasswordsha2 = fee074307074d18c93ebeb597aed4dfcc87d856864afb2d4216183ccc65687c2
root_timezone = Asia/Shanghai
httpbindaddress = 0.0.0.0:9000
elasticsearch_hosts = http://192.168.80.111:9200,http://192.168.80.112:9200,http://192.168.80.113:9200
mongodb_uri = mongodb://192.168.80.111:27017,192.168.80.112:27017,192.168.80.113:27017/graylog?replicaSet=rs0
# 修改从节点(192.168.80.112)上graylog配置文件
$ vim /etc/graylog/server/server.conf
is_master = false
password_secret = ESVfAErGXhpih8FLTncNP3tDAoz4V6Oz25PWdca8r4aqLJJNt8Yl5MRv5EGyJnjBDvUFSBgCaL8Ymm8D3s3Oub0j8Un5ZUvy
rootpasswordsha2 = fee074307074d18c93ebeb597aed4dfcc87d856864afb2d4216183ccc65687c2
root_timezone = Asia/Shanghai
httpbindaddress = 0.0.0.0:9000
elasticsearch_hosts = http://192.168.80.111:9200,http://192.168.80.112:9200,http://192.168.80.113:9200
mongodb_uri = mongodb://192.168.80.111:27017,192.168.80.112:27017,192.168.80.113:27017/graylog?replicaSet=rs0
# 修改从节点(192.168.80.113)上graylog配置文件
$ vim /etc/graylog/server/server.conf
is_master = false
password_secret = ESVfAErGXhpih8FLTncNP3tDAoz4V6Oz25PWdca8r4aqLJJNt8Yl5MRv5EGyJnjBDvUFSBgCaL8Ymm8D3s3Oub0j8Un5ZUvy
rootpasswordsha2 = fee074307074d18c93ebeb597aed4dfcc87d856864afb2d4216183ccc65687c2
root_timezone = Asia/Shanghai
httpbindaddress = 0.0.0.0:9000
elasticsearch_hosts = http://192.168.80.111:9200,http://192.168.80.112:9200,http://192.168.80.113:9200
mongodb_uri = mongodb://192.168.80.111:27017,192.168.80.112:27017,192.168.80.113:27017/graylog?replicaSet=rs0
$ systemctl restart graylog-server.service
  1. 测试graylog

打开浏览器访问graylog服务器地址,如:http://192.168.80.111:9000

点击System→Nodes,即可查看到graylog节点状态

至此,graylog集群部署完成。

发布于: 2020 年 05 月 06 日 阅读数: 94
用户头像

蚍蜉

关注

蜉蝣朝生暮死,蚍蜉亦撼大树! 2019.04.11 加入

如果存在更高级的多维空间,我还是我嘛?

评论

发布
暂无评论
Graylog部署文档