写点什么

summary.typeQuotaInfos.typeQuotaInfo[3].type

作者:刘大猫
  • 2025-09-26
    黑龙江
  • 本文字数:3826 字

    阅读完需:约 13 分钟

@[TOC]

摘要

在使用工具方法==fs.getContentSummary(path)==方法获取 hbase 库表信息时报错,报错如下:


java.io.IOException: com.google.protobuf.ServiceException: com.google.protobuf.UninitializedMessageException: Message missing required fields: summary.typeQuotaInfos.typeQuotaInfo[3].type  at org.apache.hadoop.ipc.ProtobufHelper.getRemoteException(ProtobufHelper.java:47) ~[api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getContentSummary(ClientNamenodeProtocolTranslatorPB.java:809) ~[api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at sun.reflect.GeneratedMethodAccessor163.invoke(Unknown Source) ~[na:na]  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_231]  at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_231]  at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) ~[api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) ~[api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at com.sun.proxy.$Proxy213.getContentSummary(Unknown Source) ~[na:na]  at org.apache.hadoop.hdfs.DFSClient.getContentSummary(DFSClient.java:3040) ~[api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at org.apache.hadoop.hdfs.DistributedFileSystem$15.doCall(DistributedFileSystem.java:725) ~[api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at org.apache.hadoop.hdfs.DistributedFileSystem$15.doCall(DistributedFileSystem.java:721) ~[api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) ~[api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at org.apache.hadoop.hdfs.DistributedFileSystem.getContentSummary(DistributedFileSystem.java:721) ~[api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at com.geespace.microservices.directory.assets.service.impl.DataAssetsScreenServiceImpl.updateAssetsSize(DataAssetsScreenServiceImpl.java:153) ~[api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at com.geespace.microservices.directory.assets.service.impl.DataAssetsScreenServiceImpl$$FastClassBySpringCGLIB$$cc83a803.invoke(<generated>) [api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) [api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749) [api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) [api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115) ~[api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_231]  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_231]  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_231]  at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_231]Caused by: com.google.protobuf.ServiceException: com.google.protobuf.UninitializedMessageException: Message missing required fields: summary.typeQuotaInfos.typeQuotaInfo[3].type  at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:271) ~[api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at com.sun.proxy.$Proxy212.getContentSummary(Unknown Source) ~[na:na]  at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getContentSummary(ClientNamenodeProtocolTranslatorPB.java:806) ~[api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  ... 24 common frames omittedCaused by: com.google.protobuf.UninitializedMessageException: Message missing required fields: summary.typeQuotaInfos.typeQuotaInfo[3].type  at com.google.protobuf.AbstractMessage$Builder.newUninitializedMessageException(AbstractMessage.java:770) ~[api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$GetContentSummaryResponseProto$Builder.build(ClientNamenodeProtocolProtos.java) ~[api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$GetContentSummaryResponseProto$Builder.build(ClientNamenodeProtocolProtos.java) ~[api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:261) ~[api-gateway-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]  ... 26 common frames omitted
复制代码

原因一:

服务器 hadoop 版本和 springboot 项目依赖的 hadoop 版本不一致,以服务器 hadoop 版本为主

解决方案:

修改 pom 相关的 hadoop 的 version,确保一致即可



原因二:

springboot 项目里引入多个依赖,甚至有隐藏依赖,导致出现多个版本,最终由于存在多版本导致服务器调用了不一致版本而报错,截图问题如下:


解决办法一:

依赖 hadoop-common 设置了 3.1.1 即可


<dependency>    <groupId>org.apache.hadoop</groupId>    <artifactId>hadoop-common</artifactId>    <version>3.1.1</version></dependency>
复制代码

解决办法二:

还有一种情况是 hadoop 相关依赖已经设置 3.1.1 正确版本,但是其他 hbase 依赖内部包含了 2.7.7 的版本,所以需要找到它并排除掉,最终只保留一个 3.1.1 的版本即可,不然 report 后可能会还是出现两个版本,解决方案如下添加<exclusions>


==说明:我偷懒了所以都拷贝排除了,正常只排除掉你需要排除的即可==


<dependency>    <groupId>org.apache.hbase</groupId>    <artifactId>hbase-client</artifactId>    <version>2.0.0</version>    <exclusions>        <exclusion>            <artifactId>hadoop-hdfs</artifactId>            <groupId>org.apache.hadoop</groupId>        </exclusion>        <exclusion>            <artifactId>hadoop-mapreduce-client-core</artifactId>            <groupId>org.apache.hadoop</groupId>        </exclusion>        <exclusion>            <artifactId>hadoop-annotations</artifactId>            <groupId>org.apache.hadoop</groupId>        </exclusion>        <exclusion>            <artifactId>hadoop-yarn-api</artifactId>            <groupId>org.apache.hadoop</groupId>        </exclusion>        <exclusion>            <artifactId>hadoop-yarn-common</artifactId>            <groupId>org.apache.hadoop</groupId>        </exclusion>        <exclusion>            <artifactId>hbase-hadoop-compat</artifactId>            <groupId>org.apache.hbase</groupId>        </exclusion>        <exclusion>            <artifactId>hbase-hadoop2-compat</artifactId>            <groupId>org.apache.hbase</groupId>        </exclusion>        <exclusion>            <artifactId>hadoop-common</artifactId>            <groupId>org.apache.hadoop</groupId>        </exclusion>        <exclusion>            <artifactId>hadoop-client</artifactId>            <groupId>org.apache.hadoop</groupId>        </exclusion>    </exclusions></dependency>
复制代码

其他文章答案讨论

1.比如这个 http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-issues/202012.mbox/%3CJIRA.13337627.1603879426000.278677.1607068440721@Atlassian.JIRA%3E他说是 nn 和 dn 升级后版本不一致,这个针对能否单独对 nn 和 dn 升级我不清楚,所以需要你们去评论,当然如果有升级过的可以回复我,共同学习下

用户头像

刘大猫

关注

还未添加个人签名 2022-08-23 加入

还未添加个人简介

评论

发布
暂无评论
summary.typeQuotaInfos.typeQuotaInfo[3].type_人工智能_刘大猫_InfoQ写作社区