OpenMLDB Weekly Update(2021.9.19-2021.9.26)

OpenMLDB

Summary

本周合并 Pull requests 8 个，新增 Pull requests 5 个，关闭 Issues 11 个，新增 Issues 20 个。总计 84 个文件修改，新增 6677 行代码，删除 511 行代码。 

Merged Pull Requests

• feat: support spark.master config to run job in yarn or local#466 

• feat: read openmldb git properties and set in return string#464

• feat: enable `TestWindowUnion` test#467

• feat: bump snakeyaml from 1.17 to 1.26 in /test/batch-test/openmldb-batch-test#416 

• feat: add openmldb jmh#367 

• test: modify cicd#465 

• feat: add src test cicd#452 

• fix: fix a bug when turn on window parallelization and skew optimization#438 

Open Pull Requests

• Fix import error of OpenMLDB Python lib when installed as zip format#439 

• fix: remove dup apply pass on the same physical op.#453 

• feat: ddlparser extracts indexes from sql#457 

• build: add deployment script#469 

• style: update hybridse header guard style#470 

Close Issues

• Support Spark local jobs in TaskManager for local OpenMLDB setup#456 

• Return the openmldb version string for Batch#463 

• Enable the unit test of TestWindowUnion#339 

• 创建索引，ttl_type=test，创建成功，期望返回错误信息#162 

• Deploy all-in-one java package#84 

• move demo to main repo#120 

• 创建absorlat类型的索引，desc展示不出来#163 

• scripts: package java sdk set cmake type to release#371 

• feat: support insert multiple rows into a table using a single SQL insert statement.#398

• Fix enable WindowSkewOpt and WindowComputeParallelOpt at the same time#444 

• add more status badges to README#313 

Open Issues

• Support passing multiple databases for parsing each SQL statement#468 

• Set index if there is no index info in create statement#462 

• Update catalog in stand alone mode#461 

• Add `SHOW DEPLOYMENT` and `DROP DEPLOYMENT` command#460 

• Recover metadata when nameserver restart in stand alone mode#459 

• Store metadata in system table#458 

• Support `SELECT INTO OUTFILE` syntax#455 

• ci: pre-build macos pkg needs to select sdk#454 

• Support querying data from Trino in oenmldb CLI#451 

• Add SHOW JOBS command for CLI and TaskManager#450 

• Support generating and storing job id for TaskManager#449 

• Integrate the ability of SQL analysis by trino in CLI#448 

• Add `DEPLOY` command#447 

• Write the result of select sql into file#446 

• Support `SET` syntax#445 

• Add `import` command#443 

• Support `engine` option in `create` statement#442 

• Use nameserver ip and port to connect openmldb#441 

• Support stand-alone mode#440 

• Fail query window+lastjoin when turn on `enable_batch_window_parallelization`#437 

Contributors

• aceforeverd (teapot@aceforeverd.com)

• Chen22 (jingchen2222@gmail.com)

• Kanekanekane (1290561498@qq.com)

• Stark (hujinlei1999@qq.com)

• tobe (tobeg3oogle@gmail.com)

• wuyou10206 (zw1020688@163.com) 

Highlights

本周在特性开发、缺陷修复、性能测试、CICD 集成方案都有进一步完善。功能上在 TaskManager 上支持了单机批处理模式，Bug 方案修复 Git 版本信息展示、批模式同时运行窗口倾斜优化以及窗口并行优化的问题，性能测试方面集成了 openmldb-jmh 模块加强对 Java 模块的性能回归测试，CICD 集成方面加入了基于源码分支的 SRC 测试和基于特定版本的 PKG 测试，包含数千个 case 的大规模集成测试也可以在 Github 中手动出发运行。本周新增 Issue 较多，侧重于高性能单机版的重新设计和实现，对高性能数据库优化感兴趣的也可以关注一下。

欢迎更多开发者关注和参与OpenMLDB开源项目。