写点什么

K8s 事件监控之 kube-eventer

用户头像
雪雷
关注
发布于: 2020 年 08 月 21 日
K8s事件监控之kube-eventer

一 背景



目前k8s监控可以分为:资源监控,性能监控,安全健康等,但是在K8s中,如何表示一个资源对象的状态及一些列的资源状态转换,需要事件监控来表示,目前阿里有开源的K8s事件监控项目kube-eventer, 其将事件分为两种,一种是Warning事件,表示产生这个事件的状态转换是在非预期的状态之间产生的;另外一种是Normal事件,表示期望到达的状态,和目前达到的状态是一致的。



可以收集pod/node/kubelet等资源对象的event,还可以收集自定义资源对象的event,汇聚处理发送到配置好好的接受端,架构图如下所示。





二 安装部署



使用阿里的开源k8s事件监控项目,可以收集k8s的事件日志,来自定义等级告警



2.1 钉钉机器人配置



目前新加的自定义机器人需要安全配置,可以配置标签,然后在sink的label中定义







记录保存webhook



https://oapi.dingtalk.com/robot/send?access_token=e1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx8fe



2.2 部署资源



apiVersion: apps/v1beta2
kind: Deployment
metadata:
labels:
name: kube-eventer
name: kube-eventer
namespace: kube-system
spec:
replicas: 1
selector:
matchLabels:
app: kube-eventer
template:
metadata:
labels:
app: kube-eventer
spec:
dnsPolicy: ClusterFirstWithHostNet
serviceAccount: kube-eventer
containers:
- image: registry.aliyuncs.com/acs/kube-eventer-amd64:v1.1.0-63e7f98-aliyun
name: kube-eventer
command:
- "/kube-eventer"
- "--source=kubernetes:https://kubernetes.default"
## .e.g,dingtalk sink demo
- --sink=dingtalk:https://oapi.dingtalk.com/robot/send?access_token=exxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx28fe&label=anchnet-kubesphere&level=Warning&msg_type=markdown
env:
# If TZ is assigned, set the TZ value as the time zone
- name: TZ
value: America/New_York
volumeMounts:
- name: localtime
mountPath: /etc/localtime
readOnly: true
- name: zoneinfo
mountPath: /usr/share/zoneinfo
readOnly: true
resources:
requests:
cpu: 100m
memory: 100Mi
limits:
cpu: 500m
memory: 250Mi
volumes:
- name: localtime
hostPath:
path: /etc/localtime
- name: zoneinfo
hostPath:
path: /usr/share/zoneinfo
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kube-eventer
rules:
- apiGroups:
- ""
resources:
- events
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
annotations:
name: kube-eventer
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: kube-eventer
subjects:
- kind: ServiceAccount
name: kube-eventer
namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: kube-eventer
namespace: kube-system



三 接入告警测试



3.1 钉钉接入测试



--sink=dingtalk:https://oapi.dingtalk.com/robot/send?access_token=exxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx28fe&label=anchnet-kubesphere&level=Warning&msg_type=markdown





3.2 企业微信接入测试



--sink=wechat:?corp_id=wwxxxxxxxxxx1a&corp_secret=gxxxxxxxxxxxxxxxxxxxxxxxxxxx4U&agent_id=10xxxxxx7&to_user=&level=Warning&label=K8S-Alert-Prod&msg_type=markdown





参考链接



  • https://github.com/AliyunContainerService/kube-eventer/blob/master/docs/en/dingtalk-sink.md

  • https://github.com/AliyunContainerService/kube-eventer



发布于: 2020 年 08 月 21 日阅读数: 361
用户头像

雪雷

关注

stay hungry stay foolish 2019.08.16 加入

Devops,python,shell,云原生,云架构,kubernetes https://github.com/redhatxl

评论

发布
暂无评论
K8s事件监控之kube-eventer