DolphinScheduler+Sqoop 入门避坑：一文搞定数据同步常见异常
作者：白鲸开源
- 2025-07-08 天津
本文字数：2680 字
阅读完需：约 9 分钟
在 DolphinScheduler 中假如遇到创建租户时，报错，错误日志显示权限不足：
可以使用下面的语句去开启权限：
hdfs dfs -chmod 777 /
复制代码
遇见下面的错误
检查 DolphinScheduler 如何整合 Sqoop 呢？
按照上边的图片里的路径和文档在最后边添加下边的代码：
export SQOOP_HOME=/opt/installs/sqoop
export PATH=$SQOOP_HOME/bin:$HADOOP_HOME/bin:$SPARK_HOME1/bin:$SPARK_HOME2/bin:$PYTHON_HOME/bin:$JAVA_HOME/bin:$HIVE_HOME/bin:$FLINK_HOME/bin:$DATAX_HOME/bin:$SEATUNNEL_HOME/bin:$CHUNJUN_HOME/bin:$PATH
复制代码
然后重启一下 DolphinScheduler：
查看ds状态：bash ./bin/dolphinscheduler-daemon.sh status standalone-server
关闭ds：bash ./bin/dolphinscheduler-daemon.sh stop standalone-server
开启ds：bash ./bin/dolphinscheduler-daemon.sh start standalone-server
复制代码
若出现下边的错误
Caused by: java.lang.ClassNotFoundException: Class QueryResult not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2571) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2665) ... 12 more
复制代码
解决方案就是：拷贝一个 QueryResult.jar 到 sqoop 的 lib 下 【临时解决方案】
[INFO] 2024-09-25 06:19:16.083 +0000 - -> 注: /tmp/sqoop-root/compile/46c0c4b3def5aba0c202ae9664234de6/QueryResult.java使用或覆盖了已过时的 API。 注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。 通过日志看到一个位置：cd /tmp/sqoop-root/compile/46c0c4b3def5aba0c202ae9664234de6
复制代码
将此jar包拷贝cp /tmp/sqoop-root/compile/46c0c4b3def5aba0c202ae9664234de6/QueryResult.jar /opt/installs/sqoop/lib/
复制代码
永久解决方案：
在 DolphinScheduler 的配置文件中添加一句话：
在文档的最下边添加下边的代码
export HADOOP_CONF_DIR=/opt/installs/hadoop/etc/hadoop
复制代码
配置完成后，重启 DolphinScheduler。
查看ds状态：bash ./bin/dolphinscheduler-daemon.sh status standalone-server
关闭ds：bash ./bin/dolphinscheduler-daemon.sh stop standalone-server
开启ds：bash ./bin/dolphinscheduler-daemon.sh start standalone-server
复制代码
报错如下：
[INFO] 2024-09-25 06:27:53.628 +0000 - -> 2024-09-25 14:27:52,757 INFO [main] sqoop.Sqoop (Sqoop.java:<init>(96)) - Running Sqoop version: 1.4.7 2024-09-25 14:27:52,824 ERROR [main] tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(333)) - Error parsing arguments for import: 2024-09-25 14:27:52,825 ERROR [main] tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(336)) - Unrecognized argument: dt 2024-09-25 14:27:52,829 ERROR [main] tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(336)) - Unrecognized argument: 2024-09-24 2024-09-25 14:27:52,829 ERROR [main] tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(336)) - Unrecognized argument: -m 2024-09-25 14:27:52,829 ERROR [main] tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(336)) - Unrecognized argument: 1 2024-09-25 14:27:52,829 ERROR [main] tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(336)) - Unrecognized argument: --connect 2024-09-25 14:27:52,829 ERROR [main] tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(336)) - Unrecognized argument: jdbc:mysql://192.168.233.131:3306/jrxd?allowLoadLocalInfile=false&autoDeserialize=false&allowLocalInfile=false&allowUrlInLocalInfile=false 2024-09-25 14:27:52,830 ERROR [main] tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(336)) - Unrecognized argument: --username 2024-09-25 14:27:52,830 ERROR [main] tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(336)) - Unrecognized argument: root 2024-09-25 14:27:52,830 ERROR [main] tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(336)) - Unrecognized argument: --password
复制代码
通过以上可以大概猜到，是 sqoop 命令写错了。
出现这个错误说明是 DolphinScheduler 工作流中的 sqoop 语句出错了。
[INFO] 2024-09-25 06:34:34.639 +0000 - -> 2024-09-25 14:34:33,636 INFO [main] sqoop.Sqoop (Sqoop.java:<init>(96)) - Running Sqoop version: 1.4.7 2024-09-25 14:34:33,674 WARN [main] tool.BaseSqoopTool (BaseSqoopTool.java:applyCredentialsOptions(1120)) - Setting your password on the command-line is insecure. Consider using -P instead. Must specify destination with --target-dir. Try --help for usage instructions.[INFO] 2024-09-25 06:34:34.640 +0000 - FINALIZE_SESSION
复制代码
再出现错误，则按照下边的格式进行修改 ：
如果出现以下语法错误：
2024-09-25 14:36:53,044 ERROR [main] manager.SqlManager (LoggingUtils.java:logAll(43)) - Error executing statement: java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '%Y-%m-%d) = 2024-09-24 AND (1 = 0)' at line 1 java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '%Y-%m-%d) = 2024-09-24 AND (1 = 0)' at line 1 at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
复制代码
则应该将双引号变单引号。
如果出现以下错误：
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: ParseException line 45:21 cannot recognize input near ';' '<EOF>' '<EOF>' in expression specification at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:335) at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:199)
复制代码
解决方式：使用 sql 模块，不要添加分号；。
原文链接：https://blog.csdn.net/m0_51350088/article/details/148723463
划线
评论
复制
发布于: 2025-07-08阅读数: 2
白鲸开源
关注
一家开源原生的DataOps商业公司。 2022-03-18 加入
致力于打造下一代开源原生的DataOps 平台，助力企业在大数据和云时代，智能化地完成多数据源、多云及信创环境的数据集成、调度开发和治理，以提高企业解决数据问题的效率，提升企业分析洞察能力和决策能力。
评论