OpenMLDB Weekly Update(2021.9.5-2021.9.12)
Summary
本周合并 Pull requests 11 个,新增 Pull requests 9 个,关闭 Issues 14 个,新增 Issues 12 个。总计 57 个文件修改,新增 3002 行代码,删除 539 行代码。
Merged Pull Requests
feat: remove junit dependencies and covert java cases as scala#396
feat: bump hadoop-common to 2.8.5 to avoid vulnerability#388
feat: support column query with the same name in window skew optimization#349
Open Pull Requests
feat: bump junit from 4.11 to 4.13.1 in /java/openmldb-batchjob#382
feat: add batch, batchjob and taskmanager as java submodules#386
fix: fix some code implementation in window skew optimization#392
feat: support insert multiple rows into a table using a single SQL insert statement#399
Close Issues
Remove junit since of the EPL license for OpenMLDB Batch#390
Avoid vulnerability by upgrading hadoop-common dependency#387
rtidb磁盘占用#389
feat: engine plan optimization for where and group with the same partition#317
是否支持批量插入#177
字段含义#378
表使用大小#373
Fix the issue to enable window skew optimization for window union case#374
Enable WindowSkewOpt by default and resolve the running issues#335
Add module for OpenMLDB Batch to run custom SQL and submit by TaskManager#351
Multiple columns with the same name can't execute when last join and over window#356
Open Issues
feat: support insert multiple rows into a table using a single SQL insert statement.#398
Bug: SQL INSERT Statement with multi rows does not work as expected#391
Make openmldb-batchjob and openmldb-taskmanager as submodules of openmldb-parent#385
Support submit and manage Kubernetes jobs for TaskManager#375
Contributors
dl239 (dl239@126.com)
tobe (tobeg3oogle@gmail.com)
Wang ZeKai (1290561498@qq.com)
wuyou10206 (zw1020688@163.com)
Highlights
本周新的三个 Java 模块,分别是 openmldb-common、openmldb-batchjob 和 openmldb-taskmanager。通过可重用的 Java 模块抽象,可以快速实现新的子模块。BatchJob 模块和 TaskManager 实现了第一个最小功能集版本,提供了基础的批处理任务管理功能。
本周通过拓展 SQL 语法解析器和物理计划优化器,实现了在线执行引擎的全表分组聚合和过滤功能,此功能难点在于需要在 SQL 语法解析阶段就区分 SQL 中的 UDF 和 UDAF/UDTF 函数,并最终生成对应的逻辑计划和优化后的物理计划。
欢迎更多开发者关注和参与OpenMLDB开源项目。
版权声明: 本文为 InfoQ 作者【第四范式开发者社区】的原创文章。
原文链接:【http://xie.infoq.cn/article/1728dfa7c495c271042b3e299】。未经作者许可,禁止转载。
评论