重写 Nacos 服务发现：多个服务器如何跨命名空间，访问公共服务？
作者：LigaAI
- 2022-12-16 广东
本文字数：12735 字
阅读完需：约 42 分钟
一、问题背景
在开发某个公共应用时，笔者发现该公共应用的数据是所有测试环境（假设存在 dev/dev2/dev3）通用的。
这就意味着只需部署一个应用，就能满足所有测试环境的需求；也意味着所有测试环境都需要调用该公共应用，而不同测试环境的应用注册在不同的 Nacos 命名空间。
二、两种解决方案
如果所有测试环境都需要调用该公共应用，有两种可行的方案。第一种，将该公共服务同时注册到不同的测试环境所对应的命名空间中。
第二种，将公共应用注册到单独的命名空间，不同的测试环境能够跨命名空间访问该应用。
三、详细的问题解决过程
先行交代笔者的版本号配置。Nacos 客户端版本号为 NACOS 1.4.1；Java 项目的 Nacos 版本号如下。
最初想法是将该公共应用同时注册到多个命名空间下。在查找资料的过程中，团队成员在 GitHub 上发现了一篇类似问题的博客分享：Registration Center: Can services in different namespaces be called from each other? #1176。
01 注册多个命名空间
从该博客中，我们看到其他程序员朋友也遇到了类似的公共服务的需求。在本篇文章中，笔者将进一步分享实现思路以及示例代码。
说明：以下代码内容来自用户 chuntaojun 的分享。
shareNamespace={namespaceId[:group]},{namespaceId[:group]}
复制代码
@RunWith(SpringRunner.class)@SpringBootTest(classes = NamingApp.class, properties = {"server.servlet.context-path=/nacos"}, webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)public class SelectServiceInShareNamespace_ITCase {
private NamingService naming1; private NamingService naming2; @LocalServerPort private int port; @Before public void init() throws Exception{ NamingBase.prepareServer(port); if (naming1 == null) { Properties properties = new Properties(); properties.setProperty(PropertyKeyConst.SERVER_ADDR, "127.0.0.1"+":"+port); properties.setProperty(PropertyKeyConst.SHARE_NAMESPACE, "57425802-3058-4507-9a73-3229b9f00a36"); naming1 = NamingFactory.createNamingService(properties);
Properties properties2 = new Properties(); properties2.setProperty(PropertyKeyConst.SERVER_ADDR, "127.0.0.1"+":"+port); properties2.setProperty(PropertyKeyConst.NAMESPACE, "57425802-3058-4507-9a73-3229b9f00a36"); naming2 = NamingFactory.createNamingService(properties2); } while (true) { if (!"UP".equals(naming1.getServerStatus())) { Thread.sleep(1000L); continue; } break; } }
@Test public void testSelectInstanceInShareNamespaceNoGroup() throws NacosException, InterruptedException { String service1 = randomDomainName(); String service2 = randomDomainName(); naming1.registerInstance(service1, "127.0.0.1", 90); naming2.registerInstance(service2, "127.0.0.2", 90);
Thread.sleep(1000);
List<Instance> instances = naming1.getAllInstances(service2); Assert.assertEquals(1, instances.size()); Assert.assertEquals(service2, NamingUtils.getServiceName(instances.get(0).getServiceName())); }
@Test public void testSelectInstanceInShareNamespaceWithGroup() throws NacosException, InterruptedException { String service1 = randomDomainName(); String service2 = randomDomainName(); naming2.registerInstance(service1, groupName, "127.0.0.1", 90); naming3.registerInstance(service2, "127.0.0.2", 90);
Thread.sleep(1000);
List<Instance> instances = naming3.getAllInstances(service1); Assert.assertEquals(1, instances.size()); Assert.assertEquals(service1, NamingUtils.getServiceName(instances.get(0).getServiceName())); Assert.assertEquals(groupName, NamingUtils.getServiceName(NamingUtils.getGroupName(instances.get(0).getServiceName()))); }
}
复制代码
进一步考虑后发现该解决方案可能不太契合当前遇到的问题。公司目前的开发测试环境有很多个，并且不确定以后会不会继续增加。
如果每增加一个环境，都需要修改一次公共服务的配置，并且重启一次公共服务，着实太麻烦了。倒不如反其道而行，让其他的服务器实现跨命名空间访问公共服务。
02 跨命名空间访问
针对实际问题查找资料时，我们找到了类似的参考分享《重写 Nacos 服务发现逻辑动态修改远程服务IP地址》。
跟着博客思路看代码，笔者了解到服务发现的主要相关类是 NacosNamingService, NacosDiscoveryProperties, NacosDiscoveryAutoConfiguration。
然后，笔者将博客的示例代码复制过来，试着进行如下调试：
@Slf4j@Configuration@ConditionalOnNacosDiscoveryEnabled@ConditionalOnProperty( name = {"spring.profiles.active"}, havingValue = "dev")@AutoConfigureBefore({NacosDiscoveryClientAutoConfiguration.class})public class DevEnvironmentNacosDiscoveryClient {
@Bean @ConditionalOnMissingBean public NacosDiscoveryProperties nacosProperties() { return new DevEnvironmentNacosDiscoveryProperties(); }
static class DevEnvironmentNacosDiscoveryProperties extends NacosDiscoveryProperties {
private NamingService namingService;
@Override public NamingService namingServiceInstance() { if (null != this.namingService) { return this.namingService; } else { Properties properties = new Properties(); properties.put("serverAddr", super.getServerAddr()); properties.put("namespace", super.getNamespace()); properties.put("com.alibaba.nacos.naming.log.filename", super.getLogName()); if (super.getEndpoint().contains(":")) { int index = super.getEndpoint().indexOf(":"); properties.put("endpoint", super.getEndpoint().substring(0, index)); properties.put("endpointPort", super.getEndpoint().substring(index + 1)); } else { properties.put("endpoint", super.getEndpoint()); }
properties.put("accessKey", super.getAccessKey()); properties.put("secretKey", super.getSecretKey()); properties.put("clusterName", super.getClusterName()); properties.put("namingLoadCacheAtStart", super.getNamingLoadCacheAtStart());
try { this.namingService = new DevEnvironmentNacosNamingService(properties); } catch (Exception var3) { log.error("create naming service error!properties={},e=,", this, var3); return null; }
return this.namingService; } }
}
static class DevEnvironmentNacosNamingService extends NacosNamingService {
public DevEnvironmentNacosNamingService(Properties properties) { super(properties); }
@Override public List<Instance> selectInstances(String serviceName, List<String> clusters, boolean healthy) throws NacosException { List<Instance> instances = super.selectInstances(serviceName, clusters, healthy); instances.stream().forEach(instance -> instance.setIp("10.101.232.24")); return instances; } }
}
复制代码
调试后发现博客提供的代码并不能满足笔者的需求，还得进一步深入探索。
但幸运的是，调试过程发现 Nacos 服务发现的关键类是 com.alibaba.cloud.nacos.discovery.NacosServiceDiscovery，其中的关键方法是 getInstances() 和 getServices()，即「返回指定服务 ID 的所有服务实例」和「获取所有服务的名称」。
也就是说，对 getInstances() 方法进行重写肯定能实现本次目标——跨命名空间访问公共服务。
/** * Return all instances for the given service. * @param serviceId id of service * @return list of instances * @throws NacosException nacosException */public List<ServiceInstance> getInstances(String serviceId) throws NacosException { String group = discoveryProperties.getGroup(); List<Instance> instances = discoveryProperties.namingServiceInstance() .selectInstances(serviceId, group, true); return hostToServiceInstanceList(instances, serviceId);}
/** * Return the names of all services. * @return list of service names * @throws NacosException nacosException */public List<String> getServices() throws NacosException { String group = discoveryProperties.getGroup(); ListView<String> services = discoveryProperties.namingServiceInstance() .getServicesOfServer(1, Integer.MAX_VALUE, group); return services.getData();}
复制代码
03 最终解决思路及代码示例
具体的解决方案思路大致如下：
生成一个共享配置类
NacosShareProperties，用来配置共享公共服务的namespace和group；
重写配置类
NacosDiscoveryProperties（新：NacosDiscoveryPropertiesV2），将新增的共享配置类作为属性放进该配置类，后续会用到；
重写服务发现类
NacosServiceDiscovery（新：NacosServiceDiscoveryV2），这是最关键的逻辑；
重写自动配置类
NacosDiscoveryAutoConfiguration，将自定义相关类比 Nacos 原生类更早的注入容器。
最终代码中用到了一些工具类，可以自行补充完整。
/** * <pre> * @description: 共享nacos属性 * @author: rookie0peng * @date: 2022/8/29 15:22 * </pre> */@Configuration@ConfigurationProperties(prefix = "nacos.share")public class NacosShareProperties {
private final Map<String, Set<String>> NAMESPACE_TO_GROUP_NAME_MAP = new ConcurrentHashMap<>();
/** * 共享nacos实体列表 */ private List<NacosShareEntity> entities;
public List<NacosShareEntity> getEntities() { return entities; }
public void setEntities(List<NacosShareEntity> entities) { this.entities = entities; }
public Map<String, Set<String>> getNamespaceGroupMap() { safeStream(entities).filter(entity -> nonNull(entity) && nonNull(entity.getNamespace())) .forEach(entity -> { Set<String> groupNames = NAMESPACE_TO_GROUP_NAME_MAP.computeIfAbsent(entity.getNamespace(), k -> new HashSet<>()); if (nonNull(entity.getGroupNames())) groupNames.addAll(entity.getGroupNames()); }); return new HashMap<>(NAMESPACE_TO_GROUP_NAME_MAP); }
@Override public String toString() { return "NacosShareProperties{" + "entities=" + entities + '}'; }
/** * 共享nacos实体 */ public static final class NacosShareEntity {
/** * 命名空间 */ private String namespace;
/** * 分组 */ private List<String> groupNames;
public String getNamespace() { return namespace; }
public void setNamespace(String namespace) { this.namespace = namespace; }
public List<String> getGroupNames() { return groupNames; }
public void setGroupNames(List<String> groupNames) { this.groupNames = groupNames; }
@Override public String toString() { return "NacosShareEntity{" + "namespace='" + namespace + '\'' + ", groupNames=" + groupNames + '}'; } }}复制代码
/** * @description: naocs服务发现属性重写 * @author: rookie0peng * @date: 2022/8/30 1:19 */public class NacosDiscoveryPropertiesV2 extends NacosDiscoveryProperties {
private static final Logger log = LoggerFactory.getLogger(NacosDiscoveryPropertiesV2.class);
private final NacosShareProperties nacosShareProperties;
private static final Map<String, NamingService> NAMESPACE_TO_NAMING_SERVICE_MAP = new ConcurrentHashMap<>();
public NacosDiscoveryPropertiesV2(NacosShareProperties nacosShareProperties) { super(); this.nacosShareProperties = nacosShareProperties; }
public Map<String, NamingService> shareNamingServiceInstances() { if (!NAMESPACE_TO_NAMING_SERVICE_MAP.isEmpty()) { return new HashMap<>(NAMESPACE_TO_NAMING_SERVICE_MAP); } List<NacosShareProperties.NacosShareEntity> entities = Optional.ofNullable(nacosShareProperties) .map(NacosShareProperties::getEntities).orElse(Collections.emptyList()); entities.stream().filter(entity -> nonNull(entity) && nonNull(entity.getNamespace())) .filter(PredicateUtil.distinctByKey(NacosShareProperties.NacosShareEntity::getNamespace)) .forEach(entity -> { try { NamingService namingService = NacosFactory.createNamingService(getNacosProperties(entity.getNamespace())); if (namingService != null) { NAMESPACE_TO_NAMING_SERVICE_MAP.put(entity.getNamespace(), namingService); } } catch (Exception e) { log.error("create naming service error! properties={}, e=", this, e); } }); return new HashMap<>(NAMESPACE_TO_NAMING_SERVICE_MAP); }
private Properties getNacosProperties(String namespace) { Properties properties = new Properties(); properties.put(SERVER_ADDR, getServerAddr()); properties.put(USERNAME, Objects.toString(getUsername(), "")); properties.put(PASSWORD, Objects.toString(getPassword(), "")); properties.put(NAMESPACE, namespace); properties.put(UtilAndComs.NACOS_NAMING_LOG_NAME, getLogName()); String endpoint = getEndpoint(); if (endpoint.contains(":")) { int index = endpoint.indexOf(":"); properties.put(ENDPOINT, endpoint.substring(0, index)); properties.put(ENDPOINT_PORT, endpoint.substring(index + 1)); } else { properties.put(ENDPOINT, endpoint); }
properties.put(ACCESS_KEY, getAccessKey()); properties.put(SECRET_KEY, getSecretKey()); properties.put(CLUSTER_NAME, getClusterName()); properties.put(NAMING_LOAD_CACHE_AT_START, getNamingLoadCacheAtStart());
// enrichNacosDiscoveryProperties(properties); return properties; }}
复制代码
/** * @description: naocs服务发现重写 * @author: rookie0peng * @date: 2022/8/30 1:10 */public class NacosServiceDiscoveryV2 extends NacosServiceDiscovery {
private final NacosDiscoveryPropertiesV2 discoveryProperties;
private final NacosShareProperties nacosShareProperties;
private final NacosServiceManager nacosServiceManager;
public NacosServiceDiscoveryV2(NacosDiscoveryPropertiesV2 discoveryProperties, NacosShareProperties nacosShareProperties, NacosServiceManager nacosServiceManager) { super(discoveryProperties, nacosServiceManager); this.discoveryProperties = discoveryProperties; this.nacosShareProperties = nacosShareProperties; this.nacosServiceManager = nacosServiceManager; }
/** * Return all instances for the given service. * @param serviceId id of service * @return list of instances * @throws NacosException nacosException */ public List<ServiceInstance> getInstances(String serviceId) throws NacosException { String group = discoveryProperties.getGroup(); List<Instance> instances = discoveryProperties.namingServiceInstance() .selectInstances(serviceId, group, true); if (isEmpty(instances)) { Map<String, Set<String>> namespaceGroupMap = nacosShareProperties.getNamespaceGroupMap(); Map<String, NamingService> namespace2NamingServiceMap = discoveryProperties.shareNamingServiceInstances(); for (Map.Entry<String, NamingService> entry : namespace2NamingServiceMap.entrySet()) { String namespace; NamingService namingService; if (isNull(namespace = entry.getKey()) || isNull(namingService = entry.getValue())) continue; Set<String> groupNames = namespaceGroupMap.get(namespace); List<Instance> shareInstances; if (isEmpty(groupNames)) { shareInstances = namingService.selectInstances(serviceId, group, true); if (nonEmpty(shareInstances)) break; } else { shareInstances = new ArrayList<>(); for (String groupName : groupNames) { List<Instance> subShareInstances = namingService.selectInstances(serviceId, groupName, true); if (nonEmpty(subShareInstances)) { shareInstances.addAll(subShareInstances); } } } if (nonEmpty(shareInstances)) { instances = shareInstances; break; } } } return hostToServiceInstanceList(instances, serviceId); }
/** * Return the names of all services. * @return list of service names * @throws NacosException nacosException */ public List<String> getServices() throws NacosException { String group = discoveryProperties.getGroup(); ListView<String> services = discoveryProperties.namingServiceInstance() .getServicesOfServer(1, Integer.MAX_VALUE, group); return services.getData(); }
public static List<ServiceInstance> hostToServiceInstanceList( List<Instance> instances, String serviceId) { List<ServiceInstance> result = new ArrayList<>(instances.size()); for (Instance instance : instances) { ServiceInstance serviceInstance = hostToServiceInstance(instance, serviceId); if (serviceInstance != null) { result.add(serviceInstance); } } return result; }
public static ServiceInstance hostToServiceInstance(Instance instance, String serviceId) { if (instance == null || !instance.isEnabled() || !instance.isHealthy()) { return null; } NacosServiceInstance nacosServiceInstance = new NacosServiceInstance(); nacosServiceInstance.setHost(instance.getIp()); nacosServiceInstance.setPort(instance.getPort()); nacosServiceInstance.setServiceId(serviceId);
Map<String, String> metadata = new HashMap<>(); metadata.put("nacos.instanceId", instance.getInstanceId()); metadata.put("nacos.weight", instance.getWeight() + ""); metadata.put("nacos.healthy", instance.isHealthy() + ""); metadata.put("nacos.cluster", instance.getClusterName() + ""); metadata.putAll(instance.getMetadata()); nacosServiceInstance.setMetadata(metadata);
if (metadata.containsKey("secure")) { boolean secure = Boolean.parseBoolean(metadata.get("secure")); nacosServiceInstance.setSecure(secure); } return nacosServiceInstance; }
private NamingService namingService() { return nacosServiceManager .getNamingService(discoveryProperties.getNacosProperties()); }}复制代码
/** * @description: 重写nacos服务发现的自动配置 * @author: rookie0peng * @date: 2022/8/30 1:08 */@Configuration(proxyBeanMethods = false)@ConditionalOnDiscoveryEnabled@ConditionalOnNacosDiscoveryEnabled@AutoConfigureBefore({NacosDiscoveryAutoConfiguration.class})public class NacosDiscoveryAutoConfigurationV2 {
@Bean @ConditionalOnMissingBean public NacosDiscoveryPropertiesV2 nacosProperties(NacosShareProperties nacosShareProperties) { return new NacosDiscoveryPropertiesV2(nacosShareProperties); }
@Bean @ConditionalOnMissingBean public NacosServiceDiscovery nacosServiceDiscovery( NacosDiscoveryPropertiesV2 discoveryPropertiesV2, NacosShareProperties nacosShareProperties, NacosServiceManager nacosServiceManager ) { return new NacosServiceDiscoveryV2(discoveryPropertiesV2, nacosShareProperties, nacosServiceManager); }}复制代码
本以为问题到这就结束了，但最后自测时发现程序根本不走 Nacos 的服务发现逻辑，而是执行 Ribbon 的负载均衡逻辑com.netflix.loadbalancer.AbstractLoadBalancerRule。
不过实现类是 com.alibaba.cloud.nacos.ribbon.NacosRule，继续基于 NacosRule 重写负载均衡。
/** * @description: 共享nacos命名空间规则 * @author: rookie0peng * @date: 2022/8/31 2:04 */public class ShareNacosNamespaceRule extends AbstractLoadBalancerRule {
private static final Logger LOGGER = LoggerFactory.getLogger(ShareNacosNamespaceRule.class);
@Autowired private NacosDiscoveryPropertiesV2 nacosDiscoveryPropertiesV2; @Autowired private NacosShareProperties nacosShareProperties;
/** * 重写choose方法 * * @param key * @return */ @SneakyThrows @Override public Server choose(Object key) { try { String clusterName = this.nacosDiscoveryPropertiesV2.getClusterName(); DynamicServerListLoadBalancer loadBalancer = (DynamicServerListLoadBalancer) getLoadBalancer(); String name = loadBalancer.getName();
NamingService namingService = nacosDiscoveryPropertiesV2 .namingServiceInstance(); List<Instance> instances = namingService.selectInstances(name, true); if (CollectionUtils.isEmpty(instances)) { LOGGER.warn("no instance in service {}, then to get share service's instance", name); List<Instance> shareNamingService = this.getShareNamingService(name); if (nonEmpty(shareNamingService)) instances = shareNamingService; else return null; } List<Instance> instancesToChoose = instances; if (org.apache.commons.lang3.StringUtils.isNotBlank(clusterName)) { List<Instance> sameClusterInstances = instances.stream() .filter(instance -> Objects.equals(clusterName, instance.getClusterName())) .collect(Collectors.toList()); if (!CollectionUtils.isEmpty(sameClusterInstances)) { instancesToChoose = sameClusterInstances; } else { LOGGER.warn( "A cross-cluster call occurs，name = {}, clusterName = {}, instance = {}", name, clusterName, instances); } }
Instance instance = ExtendBalancer.getHostByRandomWeight2(instancesToChoose);
return new NacosServer(instance); } catch (Exception e) { LOGGER.warn("NacosRule error", e); return null; } }
@Override public void initWithNiwsConfig(IClientConfig iClientConfig) {
}
private List<Instance> getShareNamingService(String serviceId) throws NacosException { List<Instance> instances = Collections.emptyList(); Map<String, Set<String>> namespaceGroupMap = nacosShareProperties.getNamespaceGroupMap(); Map<String, NamingService> namespace2NamingServiceMap = nacosDiscoveryPropertiesV2.shareNamingServiceInstances(); for (Map.Entry<String, NamingService> entry : namespace2NamingServiceMap.entrySet()) { String namespace; NamingService namingService; if (isNull(namespace = entry.getKey()) || isNull(namingService = entry.getValue())) continue; Set<String> groupNames = namespaceGroupMap.get(namespace); List<Instance> shareInstances; if (isEmpty(groupNames)) { shareInstances = namingService.selectInstances(serviceId, true); if (nonEmpty(shareInstances)) break; } else { shareInstances = new ArrayList<>(); for (String groupName : groupNames) { List<Instance> subShareInstances = namingService.selectInstances(serviceId, groupName, true); if (nonEmpty(subShareInstances)) { shareInstances.addAll(subShareInstances); } } } if (nonEmpty(shareInstances)) { instances = shareInstances; break; } } return instances; }}复制代码
至此问题得以解决。在 Nacos 上配置好共享 namespace 和 group 后，就能够进行跨命名空间访问了。
# nacos共享命名空间配置 示例nacos.share.entities[0].namespace=e6ed2017-3ed6-4d9b-824a-db626424fc7bnacos.share.entities[0].groupNames[0]=DEFAULT_GROUP# 指定服务使用共享的负载均衡规则，service-id是注册到nacos上的服务id，ShareNacosNamespaceRule需要写全限定名service-id.ribbon.NFLoadBalancerRuleClassName=***.***.***.ShareNacosNamespaceRule
复制代码
注意：如果 Java 项目的 nacos discovery 版本用的是 2021.1，则不需要重写 Ribbon 的负载均衡类，因为该版本的 Nacos 不依赖 Ribbon。
2.2.1.RELEASE 版本的 nacos discovery 依赖 Ribbon.
2021.1 版本的 nacos discovery 不依赖 Ribbon。
四、总结
为了达到共享命名空间的预期，构思、查找资料、实现逻辑、调试，前后一共花费 4 天时间。成就感满满的同时，笔者也发现该功能仍存在共享服务缓存等可优化空间，留待后续实现。
五、参考文献
[1] Registration Center: Can services in different namespaces be called from each other? [EB/OL]. https://github.com/alibaba/nacos/issues/1176, 2019-05-07/2022-11-29.
[2] 重写 Nacos 服务发现逻辑动态修改远程服务 IP 地址 [EB/OL]. https://www.cnblogs.com/changxy-codest/p/14632574.html, 2021-04-08/2022-11-29.
了解更多敏捷开发、项目管理、行业动态等消息，可关注LigaAI获取更多咨讯，LigaAI-新一代智能研发管理平台期待与你一路同行，助力开发者扬帆远航！
划线
评论
复制
发布于: 刚刚阅读数: 6
版权声明: 本文为 InfoQ 作者【LigaAI】的原创文章。
原文链接:【http://xie.infoq.cn/article/f73a38310387b9824d07e8f08】。未经作者许可，禁止转载。
LigaAI
关注
新一代智能研发协作平台 2021-02-23 加入
AI赋能工作场景，想要做最懂开发者的智能研发管理平台~
评论