tool
/
starrocks
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
22,203 Commits 506 Branches 0 Tags 934 MiB
Commit Graph

47 Commits

Author SHA1 Message Date
yan zhang 38820e9c8f
[BugFix] fix paimon jni scanner (#60271) 
Signed-off-by: yan zhang <dirtysalt1987@gmail.com>
 2025-06-25 13:34:12 +08:00
yan zhang ba48ea9526
[BugFix] Fix Child Class Loader of JNI Readers (#60163) 
Signed-off-by: yan zhang <dirtysalt1987@gmail.com>
 2025-06-24 18:44:17 +08:00
miomiocat 5110ba490b
[Feature] Support paimon time type and fix paimon MV (#58292) 
Signed-off-by: miomiocat <284487410@qq.com>
 2025-04-24 11:28:55 +08:00
miomiocat 9e3fe1c4f1
[BugFix] Support paimon schema change (#56796) 
Signed-off-by: miomiocat <284487410@qq.com>
 2025-03-19 14:27:30 +08:00
Jiao Mingye b8d5c4584f
[Enhancement] Upgrade paimon version to 1.0.1 (#55760) 
Signed-off-by: Jiao Mingye <mxdzs0612@gmail.com>
 2025-02-11 10:36:02 +08:00
Jiao Mingye a8aeca0b67
[Enhancement] Upgrade paimon sdk to 1.0 version (#54796) 
Signed-off-by: Jiao Mingye <mxdzs0612@gmail.com>
 2025-02-02 09:16:28 +08:00
Jiao Mingye 0172fa1e7e
[BugFix] Fix an exception when reading a Paimon table with array<row> column (#52743) 
Signed-off-by: Jiao Mingye <mxdzs0612@gmail.com>
 2024-11-13 17:23:27 +08:00
Smith Cruise 5ef6ad22ec
[BugFix] Fix paimon HiveConf class not found error (#49865) 
Signed-off-by: Smith Cruise <chendingchao1@126.com>
 2024-08-16 14:37:57 +08:00
Smith Cruise 6629665b2f
[Enhancement] Introduce trivy to detect cve problems (#49757) 
Signed-off-by: Smith Cruise <chendingchao1@126.com>
 2024-08-13 19:03:38 +08:00
Smith Cruise b47bdebcfc
[Enhancement] Fix cve problems in java-extensions module (#49425) 
Signed-off-by: Smith Cruise <chendingchao1@126.com>
 2024-08-08 17:05:40 +08:00
Smith Cruise 6b01a165c4
[Enhancement] Fix some hudi cve problems (#49157) 
Signed-off-by: Smith Cruise <chendingchao1@126.com>
 2024-08-02 10:32:08 +08:00
Xueyan Li 9cb887c707
[Enhancement] Security Fixes (#49217) 
Signed-off-by: Astralidea <astralidea@163.com>
 2024-07-31 16:21:23 +08:00
stephen 0066149594
[Feature] support to query iceberg refs table (#48972) 
Signed-off-by: stephen <stephen5217@163.com>
 2024-07-29 15:26:41 +08:00
Smith Cruise fa92d764d4
[Enhancement] Bump paimon to 0.8.1 to resolve cve problems (#47058) 2024-06-15 19:43:51 +08:00
Smith Cruise c1ff2d2655
[Enhancement] Remove CVE jars (#46327) 
Signed-off-by: Smith Cruise <chendingchao1@126.com>
 2024-05-30 10:41:58 +08:00
Smith Cruise 0fcf3d7eba
[Enhancement] Bump FE/BE's hadoop to 3.4.0 (#45312) 
Why I'm doing:
For the CVE problem, we need to upgrade Hadoop SDK from 3.3.6 -> 3.4.0
It will introduce aws java SDK v2, so we can delete SDK v1.

Signed-off-by: Smith Cruise <chendingchao1@126.com>
 2024-05-16 14:40:26 +08:00
RyanZ 1569f589dc
[BugFix] fix empty required fields in jni scanner (#45568) 
Signed-off-by: yanz <dirtysalt1987@gmail.com>
 2024-05-14 13:59:48 +08:00
leoyy0316 3bf4701f9a
[Enhancement]Upgrade paimon version to 0.7 (#42280) 
Signed-off-by: leoyy0316 <571684903@qq.com>
 2024-03-07 22:40:48 +08:00
leoyy0316 53a35df1e5
[BugFix]Fix paimon timestamp bug (#39381) 2024-01-18 16:11:02 +00:00
leoyy0316 cdf22824ef
[Enhancement]Enhance JNI reader for date and timestamp type (#38537) 
Signed-off-by: leoyy0316 <571684903@qq.com>
 2024-01-09 21:43:08 +08:00
leoyy0316 2684e57699
[BugFix]Fix query wrong result for paimon table binay type (#38011) 
Signed-off-by: leoyy0316 <571684903@qq.com>
 2023-12-28 18:27:21 +08:00
leoyy0316 0382e47e78
[Enhancement]Support binary type for paimon table (#37904) 
Signed-off-by: leoyy0316 <571684903@qq.com>
 2023-12-27 22:11:05 +08:00
miomiocat bb6946d18d
[Feature] Support native reader for paimon table (#36824) 
Signed-off-by: miomiocat <284487410@qq.com>
 2023-12-13 11:09:15 +08:00
miomiocat 1a78ba74ac
[BugFix] Fix the issue of filesystem class conflicts due to the introduction of paimon dependency (#36627) 
Signed-off-by: miomiocat <284487410@qq.com>
 2023-12-11 11:15:09 +08:00
miomiocat a4ac4c6605
[BugFix] Fix the NPE and enable oss fileIO interface while reading paimon (#36264) 
Signed-off-by: miomiocat <284487410@qq.com>
 2023-12-05 15:50:01 +08:00
xiangakun 398f581143
[BugFix] Fix paimon jni reader may explore the hive metastore (#35777) 
Signed-off-by: xiangakun <xiangakun@126.com>
 2023-11-30 12:44:49 +08:00
xiangakun e6e875ecdc
[BugFix] fix paimon jni reader error when query all the columns (#35774) 
Signed-off-by: xiangakun <xiangakun@126.com>
 2023-11-29 14:11:06 +08:00
miomiocat e95749dc9d
[BugFix] fix the issue of reading paimon table on OSS (#34996) 
Signed-off-by: miomiocat <284487410@qq.com>
 2023-11-15 14:42:07 +08:00
leoyy0316 3fce3cfb8f
[BugFix]Fix paimon array NPE exception and deciaml type issue (#33789) 
Signed-off-by: leoyy0316 <571684903@qq.com>
 2023-11-01 10:57:38 +08:00
dirtysalt 5848af94d0
[Feature] pass `hadoop-ext` related fields to jni scanner (#33660) 
Signed-off-by: yanz <dirtysalt1987@gmail.com>
 2023-10-27 11:43:14 +08:00
leoyy0316 11cd428f4b
[Enhancement]Enhance paimon reader for decimal type (#31570) 
Signed-off-by: leoyy0316 <571684903@qq.com>
 2023-10-12 15:37:23 +08:00
leoyy0316 2460db5685
[BugFix]Fix paimon struct type error (#31069) 
Signed-off-by: leoyy0316 <571684903@qq.com>
 2023-09-14 16:30:14 +08:00
leoyy0316 23221f3577
[BugFix]Fix paimon table struct type (#30817) 
Signed-off-by: leoyy0316 <571684903@qq.com>
 2023-09-12 11:41:26 +00:00
wgcn 3787888343
[Enhancement]support paimon version 0.5 (#30532) 
Signed-off-by: wgcn <1026688210@qq.com>
 2023-09-11 18:50:08 +08:00
leoyy0316 b4a742e094
[Feature]Support struct type for paimon table (#30162) 
Signed-off-by: leoyy0316 <571684903@qq.com>
 2023-09-05 10:20:48 +08:00
miomiocat 17dd5f5217
[BugFix] Fix the exception while reading paimon table with uppercase field names and refactor the paimon reader (#29959) 
Signed-off-by: miomiocat <284487410@qq.com>
 2023-08-28 10:06:40 +08:00
leoyy0316 449aa81fef
[Feature]Support array and map type for paimon table (#29854) 
Fixes #29853

Signed-off-by: leoyy0316 <571684903@qq.com>
 2023-08-25 03:20:10 +00:00
leoyy0316 db5dc5ce69
[Feature] Support tinyint type for jni connector (#29687) 
Signed-off-by: leoyy0316 <571684903@qq.com>
 2023-08-22 18:20:45 +08:00
miomiocat 60ca057e2a
[BugFix] Fix the issue of reading paimon table on S3 (#28516) 
Fixes #28403 

Signed-off-by: miomiocat <284487410@qq.com>
 2023-08-04 10:36:03 +08:00
miomiocat 834abd5bd7
[Enhancement] Support zstd compression for paimon connector (#28551) 
Signed-off-by: miomiocat <284487410@qq.com>
 2023-08-03 07:20:47 +00:00
miomiocat 865b915962
[Enhancement] improve error message handling of JNI connector and support char type of paimon connector (#28044) 
Fixes #27992

- fail fast for unsupported types of paimon table
- output error massage of JNI to client terminal
- support char type of paimon connector
- some minor refactor

Signed-off-by: miomiocat <284487410@qq.com>
 2023-07-28 14:50:17 +08:00
miomiocat e6af8ecdf7
[BugFix] Fix the issue of calling the read interface multiple times of paimon (#27381) 
Signed-off-by: miomiocat <284487410@qq.com>
 2023-07-18 12:47:18 +08:00
miomiocat 4daa6bc420
[BugFix] Fix the exception while reading date and datetime type of paimon table (#27062) 
Signed-off-by: miomiocat <284487410@qq.com>
 2023-07-13 14:08:10 +08:00
miomiocat 9659f18df0
[Enhancement] Upgrade the hadoop version of paimon dependency to 3.3.4 (#26211) 
Fixes #issue

Signed-off-by: miomiocat <284487410@qq.com>
 2023-06-29 10:07:03 +00:00
miomiocat b06c0b63fd
[Feature] Support expression push down while reading paimon table and hive metastore type of paimon catalog (#25246) 2023-06-27 23:13:19 -07:00
miomiocat d139bde248
[Enhancement] Separate JNI logs from be.out (#25826) 
- separate JNI logs from be.out
- rolling out config 
- provide log4j-1 compatibility using [log4j 1.2
bridge](https://logging.apache.org/log4j/2.x/log4j-1.2-api.html)

Signed-off-by: miomiocat <284487410@qq.com>
 2023-06-25 17:33:00 +08:00
miomiocat 0909a313af
[Feature] Support reading paimon table (#24626) 
Fixes #23481

Basic implementation of paimon table format reading using JNI connector #9716

example:

```
CREATE EXTERNAL CATALOG paimon_catalog
properties
( 
    "type" = "paimon",
    "paimon.catalog.type" = "hive",
    "paimon.catalog.warehouse" = "hdfs://192.168.7.239:9000/user/hive/warehouse",
    "hive.metastore.uris" = "thrift://192.168.7.239:9083"
);


CREATE EXTERNAL CATALOG paimon_catalog
properties
( 
    "type" = "paimon",
    "paimon.catalog.type" = "filesystem",
    "paimon.catalog.warehouse" = "hdfs://192.168.7.239:9000/user/hive/warehouse"
);
```
 2023-06-13 09:29:32 -07:00