Spring Cloud Kubernetes 之实战一配置管理
作者：Xiao8
- 2022 年 7 月 15 日
本文字数：11857 字
阅读完需：约 39 分钟
一直以来，玩 springcloud 的，基本都是在玩 Springboot1.x，Springcloud(Dalston 版)的众多相关组件来做配置中心、服务注册与发现，网关用的是 Netflix 公司对 springboot 做的 LB，等等，但是这些东西太过沉重，复杂了。在一个以 k8s 为基础的 iaas 服务系统，如果不用 k8s 的特性来做这些事，那是说不过去的。理由这就不重复述说了。一句话：减少系统服务的复杂性。
本文主要介绍 springcloud 结合 k8s，做配置管理，避免更多服务组件的冗余，完美填坑版！
环境：
ubuntu16.04
docker18.04
k8s1.13.x +
maven3.5.3
java1.8 +
springboot 2.1.1
spring-cloud-kubernetes：1.0.1.RELEAS
复制代码
前提
Ubuntu下安装docker18.04 or 其它较高版本，k8s1.13.x及以上，jvm环境等。创建项目 基础依赖：
<parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>2.1.8.RELEASE</version> <relativePath/> </parent> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> <java.version>1.8</java.version> <swagger.version>2.6.1</swagger.version> <xstream.version>1.4.7</xstream.version> <pageHelper.version>4.1.6</pageHelper.version> <fastjson.version>1.2.51</fastjson.version> <shiro.version>1.3.0</shiro.version> <!-- <kubernetes-client-version>6.0.1</kubernetes-client-version> --> <kubernetes-client-version>5.0.0</kubernetes-client-version> <fabric8-kubernetes-client.version>4.6.1</fabric8-kubernetes-client.version><!-- 对应k8s v1.15.3 --> <springcloud.version>Greenwich.SR4</springcloud.version> <springcloud.kubernetes.version>1.1.1.RELEASE</springcloud.kubernetes.version> <mysql.version>5.1.46</mysql.version> </properties>
<dependencyManagement> <dependencies> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-dependencies</artifactId> <version>${springcloud.version}</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement>
复制代码
核心依赖：
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> <exclusions> <exclusion> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-tomcat</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-undertow</artifactId> </dependency>
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-actuator</artifactId> </dependency>
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-actuator-autoconfigure</artifactId> </dependency>
<dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-kubernetes-config</artifactId> </dependency>
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency>
<!-- springcloud-k8s-discovery -->
<dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-commons</artifactId> </dependency>
<dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-kubernetes-core</artifactId> </dependency>
<dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-kubernetes-discovery</artifactId> </dependency>
<dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-kubernetes-ribbon</artifactId> </dependency>
<dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-ribbon</artifactId> </dependency>
<dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-hystrix</artifactId> </dependency>
复制代码
**本次依赖引入配置管理、服务的发现(即消费者)。**
复制代码
如果有操作 redis 和 db 的话，引入相应的依赖：
<!-- mybatis --> <dependency> <groupId>org.mybatis.spring.boot</groupId> <artifactId>mybatis-spring-boot-starter</artifactId> <version>1.1.1</version> </dependency> <dependency> <groupId>mysql</groupId> <artifactId>mysql-connector-java</artifactId> <version>${mysql.version}</version> </dependency>
<!--分页插件--> <dependency> <groupId>com.github.pagehelper</groupId> <artifactId>pagehelper</artifactId> <version>${pageHelper.version}</version> </dependency>
<!-- datasource pool--> <dependency> <groupId>com.alibaba</groupId> <artifactId>druid</artifactId> <version>1.1.3</version> </dependency>
<dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-lang3</artifactId> </dependency>
<dependency> <groupId>commons-collections</groupId> <artifactId>commons-collections</artifactId> <version>3.2.2</version> </dependency>
<!-- 对redis支持,引入的话项目缓存就支持redis了,所以必须加上redis的相关配置,否则操作相关缓存会报异常 --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-redis</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-redis</artifactId> <version>1.4.7.RELEASE</version> </dependency>
<dependency> <groupId>com.google.guava</groupId> <artifactId>guava</artifactId> <version>19.0</version> </dependency>
复制代码
剩下的就是构建镜像时的插件：
<build> <finalName>${project.artifactId}</finalName> <plugins> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> <configuration> <jvmArguments>-Dfile.encoding=UTF-8</jvmArguments> <fork>true</fork> </configuration> </plugin> <plugin> <groupId>org.jacoco</groupId> <artifactId>jacoco-maven-plugin</artifactId> <version>0.7.8</version> <executions> <execution> <goals> <goal>prepare-agent</goal> <goal>report</goal> </goals> </execution> </executions> </plugin> </plugins> </build>
复制代码
接下来，我们创建主类：
@SpringBootApplication(scanBasePackages = { "com.leinao" })@EnableConfigurationProperties(EnvConfig.class)@EnableDiscoveryClient@EnableHystrix@EnableSchedulingpublic class AdminApp { public static void main(String[] args) { SpringApplication.run(AdminApp.class, args); }}
复制代码
注意这里创建启动类时，对 springboot 的项目进行了优化，避免启动时加载很多，启动繁重，具体深度优化，可参考：https://mp.weixin.qq.com/s?__biz=MzU2NjIzNDk5NQ==&mid=2247487954&idx=1&sn=2426451f3bd83161cfe1237f82d6b448&key=f8fb043b3d2681a794e51a46e142af77355722dff712776af12b1f3c831218df6dfc329df63c8e5e550b3d88d58f0f178c4c3c16b141733e0e3344fa595e2bc25241d864d45132753fd99279b832de85&ascene=1&uin=MzQzMzI2NjAxMQ%3D%3D&devicetype=Windows+10&version=62070158&lang=zh_CN&pass_ticket=pnSSI9jAq0M11V5hYMmkoVm5qO%2FWk9l3UUUJMglbdtdDOzLHa7iHsDmwSzs486sD。
然后我们在进行配置，注意：据官方说，项目的 src\main\resources 路径下不要创建 application.yml 文件，只创建名为 bootstrap.yml 的文件：
management: endpoint: restart: enabled: true health: enabled: true info: enabled: true
spring: application: name: edge-admin cloud: kubernetes: config: sources: - name: ${spring.application.name} namespace: default discovery: all-namespaces: true reload: #自动更新配置的开关设置为打开 enabled: true #更新配置信息的模式：polling是主动拉取，event是事件通知 mode: polling #主动拉取的间隔时间是500毫秒 period: 500 http: encoding: charset: UTF-8 enabled: true force: true mvc: throw-exception-if-no-handler-found: true main: allow-bean-definition-overriding: true # 当遇到同样名称时，是否允许覆盖注册
复制代码
这里，我创建了 bootstrap 文件，同时也加了 application 文件，启动时会先加载 bootstrap，验证有效。
在 application.yaml 中，我们加入如下内容：
server: port: 9999 undertow: accesslog: enabled: false pattern: combined servlet: session: timeout: PT120M
logging: path: /data/${spring.application.name}/logs
management: endpoint: restart: enabled: true health: enabled: true info: enabled: trueclient: http: request: connectTimeout: 8000 readTimeout: 30000
mybatis: mapperLocations: classpath:mapper/*.xml typeAliasesPackage: com.demo.*.model
backend: ribbon: eureka: enabled: false client: enabled: true ServerListRefreshInterval: 5000hystrix.command.BackendCall.execution.isolation.thread.timeoutInMilliseconds: 5000hystrix.threadpool.BackendCallThread.coreSize: 5
复制代码
注意：这里的 server 设置 session 的超时时间，对于 springboot2.0 与 1.0 版本完全不一样了，具体看内容。
其他的 application-test.yaml 等配置文件，配置的是日志的级别：
logging: level: com: leinao: INFO org: springframework: web: INFO
复制代码
接下来配置环境配置：
EnvConfig.java 类作为环境变量配置，注解 ConfigurationProperties 的 prefix="spring_cloud"，
表示该类用到的配置项都是名为"spring_cloud"的配置项的子内容 ：
package com.demo.config;
import org.springframework.boot.context.properties.ConfigurationProperties;import org.springframework.context.annotation.Configuration;
/** * 配置信息 * @author Damon * @date 2019年10月25日 下午1:54:01 * */
@Configuration@ConfigurationProperties(prefix = "greeting")public class EnvConfig {
private String message = "This is a dummy message";
private String container_command; private String model_dir_path; private String so_path; private String config_path; private String task_role_name; private String container_name; private String container_workdir; private String init_containers_image; private String service_account_name; private String spring_mq_host; private String spring_mq_port; private String spring_mq_user; private String spring_mq_pwd; private String jdbc_driverClassName; private String jdbc_url; private String jdbc_username; private String jdbc_password; private String spring_redis_host; private String spring_redis_port; private String spring_redis_pwd; private String kube_apiserver_address; private String image_path; private String volume_image_path; private String inference_job_namespace; private String api_version; private String remote_deployment_url; private String remote_pods_url; private String remote_deployment_pod_log_url; private String base_path; private String chunk_size; private String cas_url; private String create_job_url; private String abnormal_data_dir;
private Long expire_time= 600000L;
public String getMessage() { return this.message; }
public void setMessage(String message) { this.message = message; }
public String getContainer_command() { return container_command; }
public void setContainer_command(String container_command) { this.container_command = container_command; }
public String getModel_dir_path() { return model_dir_path; }
public void setModel_dir_path(String model_dir_path) { this.model_dir_path = model_dir_path; }
public String getSo_path() { return so_path; }
public void setSo_path(String so_path) { this.so_path = so_path; }
public String getConfig_path() { return config_path; }
public void setConfig_path(String config_path) { this.config_path = config_path; }
public String getTask_role_name() { return task_role_name; }
public void setTask_role_name(String task_role_name) { this.task_role_name = task_role_name; }
public String getContainer_name() { return container_name; }
public void setContainer_name(String container_name) { this.container_name = container_name; }
public String getContainer_workdir() { return container_workdir; }
public void setContainer_workdir(String container_workdir) { this.container_workdir = container_workdir; }
public String getInit_containers_image() { return init_containers_image; }
public void setInit_containers_image(String init_containers_image) { this.init_containers_image = init_containers_image; }
public String getService_account_name() { return service_account_name; }
public void setService_account_name(String service_account_name) { this.service_account_name = service_account_name; }
public String getSpring_mq_host() { return spring_mq_host; }
public void setSpring_mq_host(String spring_mq_host) { this.spring_mq_host = spring_mq_host; }
public String getSpring_mq_port() { return spring_mq_port; }
public void setSpring_mq_port(String spring_mq_port) { this.spring_mq_port = spring_mq_port; }
public String getSpring_mq_user() { return spring_mq_user; }
public void setSpring_mq_user(String spring_mq_user) { this.spring_mq_user = spring_mq_user; }
public String getSpring_mq_pwd() { return spring_mq_pwd; }
public void setSpring_mq_pwd(String spring_mq_pwd) { this.spring_mq_pwd = spring_mq_pwd; }
public String getJdbc_driverClassName() { return jdbc_driverClassName; }
public void setJdbc_driverClassName(String jdbc_driverClassName) { this.jdbc_driverClassName = jdbc_driverClassName; }
public String getJdbc_url() { return jdbc_url; }
public void setJdbc_url(String jdbc_url) { this.jdbc_url = jdbc_url; }
public String getJdbc_username() { return jdbc_username; }
public void setJdbc_username(String jdbc_username) { this.jdbc_username = jdbc_username; }
public String getJdbc_password() { return jdbc_password; }
public void setJdbc_password(String jdbc_password) { this.jdbc_password = jdbc_password; }
public String getSpring_redis_host() { return spring_redis_host; }
public void setSpring_redis_host(String spring_redis_host) { this.spring_redis_host = spring_redis_host; }
public String getSpring_redis_port() { return spring_redis_port; }
public void setSpring_redis_port(String spring_redis_port) { this.spring_redis_port = spring_redis_port; }
public String getSpring_redis_pwd() { return spring_redis_pwd; }
public void setSpring_redis_pwd(String spring_redis_pwd) { this.spring_redis_pwd = spring_redis_pwd; }
public String getKube_apiserver_address() { return kube_apiserver_address; }
public void setKube_apiserver_address(String kube_apiserver_address) { this.kube_apiserver_address = kube_apiserver_address; }
public String getImage_path() { return image_path; }
public void setImage_path(String image_path) { this.image_path = image_path; }
public String getVolume_image_path() { return volume_image_path; }
public void setVolume_image_path(String volume_image_path) { this.volume_image_path = volume_image_path; }
public String getInference_job_namespace() { return inference_job_namespace; }
public void setInference_job_namespace(String inference_job_namespace) { this.inference_job_namespace = inference_job_namespace; }
public String getApi_version() { return api_version; }
public void setApi_version(String api_version) { this.api_version = api_version; }
public String getRemote_deployment_url() { return remote_deployment_url; }
public void setRemote_deployment_url(String remote_deployment_url) { this.remote_deployment_url = remote_deployment_url; }
public String getRemote_pods_url() { return remote_pods_url; }
public void setRemote_pods_url(String remote_pods_url) { this.remote_pods_url = remote_pods_url; }
public String getRemote_deployment_pod_log_url() { return remote_deployment_pod_log_url; }
public void setRemote_deployment_pod_log_url(String remote_deployment_pod_log_url) { this.remote_deployment_pod_log_url = remote_deployment_pod_log_url; }
public String getBase_path() { return base_path; }
public void setBase_path(String base_path) { this.base_path = base_path; }
public String getChunk_size() { return chunk_size; }
public void setChunk_size(String chunk_size) { this.chunk_size = chunk_size; }
public Long getExpire_time() { return expire_time; }
public void setExpire_time(Long expire_time) { this.expire_time = expire_time; }
public String getCas_url() { return cas_url; }
public void setCas_url(String cas_url) { this.cas_url = cas_url; }
public String getCreate_job_url() { return create_job_url; }
public void setCreate_job_url(String create_job_url) { this.create_job_url = create_job_url; }
public String getAbnormal_data_dir() { return abnormal_data_dir; }
public void setAbnormal_data_dir(String abnormal_data_dir) { this.abnormal_data_dir = abnormal_data_dir; }
}
测试demo类：
/** * @author Damon * @date 2019年12月27日 上午9:16:41 * */
@RestControllerpublic class DemoController {
@Autowired private EnvConfig envConfig;
/** * * @author Damon * @date 2019年12月26日 * */
@GetMapping(value = "/getTest") public String getTest() { return envConfig.getBase_path(); }}
复制代码
重点：默认的 svc 是没有权限访问 k8s 的 API Server 的资源的，执行如下脚本，可以提升权限，允许其访问 configmap 的可读权限：
#使用这个代表集群最高权限，deployment中无需引入serviceAccount: config-readerapiVersion: rbac.authorization.k8s.io/v1kind: ClusterRoleBindingmetadata: name: fabric8-rbacsubjects: - kind: ServiceAccount # Reference to upper's `metadata.name` name: default # Reference to upper's `metadata.namespace` namespace: defaultroleRef: kind: ClusterRole name: cluster-admin apiGroup: rbac.authorization.k8s.io
复制代码
配置 configmap：
kind: ConfigMapapiVersion: v1metadata: name: edge-admindata: application.yaml: |- greeting: message: Say Hello to the World --- spring: profiles: dev greeting: message: Say Hello to the Developers --- spring: profiles: test greeting: message: Say Hello to the Test --- spring: profiles: prod greeting: message: Say Hello to the Prod
复制代码
接下来就是执行 deployment 启动项目了：
apiVersion: apps/v1kind: Deploymentmetadata: name: edge-admin-deployment labels: app: edge-adminspec: replicas: 1 selector: matchLabels: app: edge-admin template: metadata: labels: app: edge-admin spec: nodeSelector: edge-admin: "true" containers: - name: edge-admin image: 10.11.2.20:8000/harbor/edge-admin imagePullPolicy: IfNotPresent ports: - name: admin01 containerPort: 1002 volumeMounts: - mountPath: /home/edge-admin name: edge-admin-path - mountPath: /data/edge-admin name: edge-admin-log-path - mountPath: /etc/kubernetes name: kube-config-path - mountPath: /abnormal_data_dir name: abnormal-data-dir args: ["sh", "-c", "nohup java $JAVA_OPTS -jar -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=128m -Xms1024m -Xmx1024m -Xmn256m -Xss256k -XX:SurvivorRatio=8 -XX:+UseConcMarkSweepGC edge-admin.jar --spring.profiles.active=dev", "&"] hostAliases: - ip: "10.10.1.5" hostnames: - "k8s.api.server" - "foo.remote" - ip: "127.0.0.1" hostnames: - "foo.localhost" - ip: "0.0.0.0" hostnames: - "foo.all" #利用admin-rbac.yaml来获取权限 #serviceAccount: config-reader #serviceAccountName: config-reader volumes: - name: edge-admin-path hostPath: path: /var/pai/edge-admin - name: edge-admin-log-path hostPath: path: /data/edge-admin - name: kube-config-path hostPath: path: /etc/kubernetes - name: abnormal-data-dir hostPath: path: /data/images/detect_result/defect
复制代码
其中，前面说的，项目启动参数对其性能优化，是对 jvm 的参数设置。分别执行 kubectl apply -f deployment.yaml 和 configmap.yaml，创建 demo 时所用的 configmap 的资源以及利用 k8s 部署启动项目。
最后打开浏览器：执行 ip:port/hello，即可看到 configmap 中对应的属性值，这里就不展示了，有兴趣的可以试试。
以上即是对 springcloud 和 k8s 首次结合后利用其 configmap 特性，来做配置管理，摒弃 springcloud-config、spring-boot-starter-actuator 的组件，减少系统的复杂性，毕竟 k8s 是肯定会被用到的，所以可以直接用其特性来做系统服务的环境配置管理。
划线
评论
复制
发布于: 刚刚阅读数: 3
版权声明: 本文为 InfoQ 作者【Xiao8】的原创文章。
原文链接:【http://xie.infoq.cn/article/a4e60a563ed9c614827e693fa】。文章转载请联系作者。
Xiao8
关注
God bless the fighters. 2020.03.11 加入
欢迎关注公众号：程序猿Damon，长期从事Java开发，研究Springcloud的微服务架构设计。目前主要从事基于K8s云原生架构研发的工作，Golang开发，长期研究边缘计算框架KubeEdge、调度框架Volcano、容器云KubeSphere研究
评论