那些年，我们在 Apache SeaTunnel 2.1.0 部署中踩过的坑【含源码分析】

`env {    # seatunnel defined streaming batch duration in seconds    [spark.app.name](http://spark.app.name) = "SeaTunnel"    spark.executor.instances = 1    spark.executor.cores = 1    spark.executor.memory = "1g"  }
source {    jdbc {      driver = "com.mysql.jdbc.Driver"      url = "jdbc:mysql://0.0.0.0:3306/database?useUnicode=true&characterEncoding=utf8&useSSL=false"      table = "table_name"      result\_table\_name = "result\_table\_name"      user = "root"      password = "password"   }
}
transform {    # split data by specific delimiter
# you can also use other filter plugins, such as sql    # sql {    #   sql = "select * from accesslog where request_time > 1000"    # }
# If you would like to get more information about how to configure seatunnel and see full list of filter plugins,    # please go to [https://seatunnel.apache.org/docs/spark/configuration/transform-plugins/Sql](https://seatunnel.apache.org/docs/spark/configuration/transform-plugins/Sql)  }
sink {    # choose stdout output plugin to output data to console    # Console {}    jdbc {     # 这里配置driver参数，否则数据交换不成功     driver = "com.mysql.jdbc.Driver",      saveMode = "update",      url = "jdbc:mysql://ip:3306/database?useUnicode=true&characterEncoding=utf8&useSSL=false",      user = "userName",      password = "***********",      dbTable = "tableName",      customUpdateStmt = "INSERT INTO table (column1, column2, created, modified, yn) values(?, ?, now(), now(), 1) ON DUPLICATE KEY UPDATE column1 = IFNULL(VALUES (column1), column1), column2 = IFNULL(VALUES (column2), column2)"      }  }`

`./bin/start-seatunnel-spark.sh --master 'yarn' --deploy-mode client --config ./config/spark.batch.jdbc.to.jdbc.conf`

`ERROR Seatunnel:121 - Plugin[org.apache.seatunnel.spark.sink.Jdbc] contains invalid config, error: please specify [driver] as non-empty`

`env {    # seatunnel defined streaming batch duration in seconds    [spark.app.name](http://spark.app.name) = "SeaTunnel"    spark.executor.instances = 1    spark.executor.cores = 1    spark.executor.memory = "1g"    # 因为sink用到hive源，所以必须进行以下配置    spark.sql.catalogImplementation = "hive"  }
source {    jdbc {      driver = "com.mysql.jdbc.Driver"      url = "jdbc:mysql://0.0.0.0:3306/database?useUnicode=true&characterEncoding=utf8&useSSL=false"      table = "table_name"      result\_table\_name = "result\_table\_name"      user = "root"      password = "password"   }
}
transform {    # split data by specific delimiter
# you can also use other filter plugins, such as sql    # sql {    #   sql = "select * from accesslog where request_time > 1000"    # }
# If you would like to get more information about how to configure seatunnel and see full list of filter plugins,    # please go to [https://seatunnel.apache.org/docs/spark/configuration/transform-plugins/Sql](https://seatunnel.apache.org/docs/spark/configuration/transform-plugins/Sql)  }
sink {    Hive {      sql = "insert overwrite table seatunnel.test1 partition(province) select name,age,province from result\_table\_name"    }  }`

-   `./bin/start-seatunnel-spark.sh --master 'yarn' --deploy-mode client --config ./config/spark.batch.jdbc.to.jdbc.conf`

    `...... ERROR Seatunnel:191 - Exception StackTrace:org.apache.spark.sql.AnalysisException: Table or view not found:  ...... Caused by: org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database 'seatunnel' not found; ......`