tool
/
starrocks
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
22,203 Commits 506 Branches 0 Tags 934 MiB
Commit Graph

97 Commits

Author SHA1 Message Date
wyb d6088ff298
[Enhancement] Bump librdkafka to 2.11.0 for kafka 4.0 (#61698) 
Signed-off-by: wyb <wybb86@gmail.com>
 2025-08-08 06:31:54 +00:00
Kevin Cai 5caf273918
[Enhancement] ugprade zstd to v1.5.7 (#56302) 
Signed-off-by: Kevin Xiaohua Cai <caixiaohua@starrocks.com>
 2025-07-31 21:25:15 +08:00
Murphy d9b23cffb4
[Enhancement] function hour_from_unixtime (#60331) 
Signed-off-by: Murphy <mofei@starrocks.com>
 2025-07-10 16:16:51 +08:00
Rohit Satardekar d96b52103a
[Enhancement] bump up breakpad to 2024.02.16 to fix crash in dump_syms (#60005) 
we are using and older version of dump_syms that doesn’t understand the .relr.dyn section

Fixes #60004

update breakpad to version breakpad-2024.02.16

$ bash thirdparty/minidump/gen_minidump_symbols.sh
starrocks_be'size (2153mb) reduced to (556mb)
symbol file is at /home/ubuntu/starrocks/output/be/symbols/starrocks_be/AC7FAB7F8B5BF82100000000000000000

Signed-off-by: Rohit Satardekar <rohitrs1983@gmail.com>
 2025-06-22 07:48:33 +08:00
wyb f2752d861e
[Enhancement] Add azure cpp sdk (#59016) 
Signed-off-by: wyb <wybb86@gmail.com>
 2025-05-16 11:15:41 +00:00
wyb 97464b08fe
[Enhancement] Bump avro to 1.12.0 and add avro cpp library (#57984) 
Signed-off-by: wyb <wybb86@gmail.com>
 2025-04-15 13:39:26 +08:00
yan zhang 38a20f8c4d
[Enhancement] add xsmid thirdparty header files (#57964) 
Signed-off-by: yan zhang <dirtysalt1987@gmail.com>
 2025-04-14 19:32:53 +08:00
eyes_on_me 4c027fe694
[Feature] upper/lower support utf8 (#56192) 
Signed-off-by: silverbullet233 <3675229+silverbullet233@users.noreply.github.com>
 2025-03-24 09:55:55 +08:00
Youngwb 296c172bba
[Enhancement] upgrade CRoaring lib (#53620) 
Signed-off-by: Youngwb <yangwenbo_mailbox@163.com>
 2024-12-06 11:30:48 +08:00
zombee0 43a09d1c56
[Tool] introduce 3rd-party lib poco (#50023) 
Signed-off-by: zombee0 <ewang2027@gmail.com>
 2024-08-21 10:06:13 +08:00
Kevin Cai 47e85eeef9
[Enhancement] Staros v3.3 rc3 and brpc glog update (#48949) 
Signed-off-by: stdpain <drfeng08@gmail.com>
Signed-off-by: Kevin Xiaohua Cai <caixiaohua@starrocks.com>
Co-authored-by: stdpain <drfeng08@gmail.com>
 2024-07-28 22:58:52 -07:00
leorishdu 9ca404197d
[Feature] Introduce TenAnn as the vector index implementation library (#48384) 
Signed-off-by: leorishdu <18771113323@163.com>
 2024-07-22 03:36:20 +00:00
Murphy ef16a69357
[Tool] upgrade lz4 to 1.9.4 (#48257) 
Signed-off-by: Murphy <mofei@starrocks.com>
 2024-07-12 16:54:06 +08:00
wyb 3ca24dc29d
[BugFix] Bump simdjson to 3.9.4 and Fix struct field columns inconsistent when loading from bad json (#47775) 
Signed-off-by: wyb <wybb86@gmail.com>
 2024-07-05 04:47:51 +00:00
Kevin Cai 43d99dbfe9
[Enhancement] adopt simdutf implementation (#47174) 
adopt simdutf library to replace the utf8 related homebrew impl.
Fixes Consider replacing src/util/simdutf8check.h #16058

Signed-off-by: Kevin Xiaohua Cai <caixiaohua@starrocks.com>
 2024-06-21 23:43:27 +08:00
Letian Jiang 3beb50d9d7
[Enhancement] Bump arrow to 16.1.0 (#46966) 
Signed-off-by: Letian Jiang <letian.jiang@outlook.com>
 2024-06-20 10:31:04 +08:00
Smith Cruise 283b91c3ed
[Enhancement] Bump BE thrift to 0.20.0 (#46521) 2024-06-04 10:01:43 +08:00
srlch af725a6307
[BugFix] Fix arm compile for gin lib (#46571) 
Signed-off-by: srlch <linzichao@starrocks.com>
 2024-06-03 15:51:51 +00:00
srlch f2aebb1324
[Enhancement] Upgrade gin for speeding up index write (#46549) 
Signed-off-by: srlch <linzichao@starrocks.com>
 2024-06-03 16:19:37 +08:00
stdpain 37cb2cf0ed
[Feature] introduce thirdparty grpc (#46247) 
Signed-off-by: stdpain <drfeng08@gmail.com>
 2024-05-28 21:04:31 +08:00
Kevin Cai faddb6c835
[Enhancement] Upgrade staros v3.3 rc0 (#43887) 
Signed-off-by: Kevin Xiaohua Cai <caixiaohua@starrocks.com>
Co-authored-by: silverbullet233 <3675229+silverbullet233@users.noreply.github.com>
 2024-04-12 11:33:59 +08:00
Gavin 30e7e97cae
[Enhancement] Remove cachelib library and dependencies to reduce the thirdparty space. (#42660) 2024-03-15 17:49:29 +08:00
凌涛 18de260a7d
[BugFix] Add valid source of boost (#38650) 
Signed-off-by: 凌涛 <lingtaolf@gmail.com>
 2024-01-09 18:39:00 +08:00
leorishdu 64c4b8ca94
[BugFix] Update clucene deps to fix memory leak bug (#36028) 
Signed-off-by: leorishdu <18771113323@163.com>
 2023-11-29 00:10:18 -08:00
leorishdu f622cba6a6
[Refactor] Upgrade clucene lib (#34822) 2023-11-10 15:36:18 -08:00
leorishdu e66a35e082
[Feature] Support clucene deps (#34134) 2023-11-08 07:58:59 -08:00
Kevin Cai 025921a93d
[Enhancement] sasl enable kerberos support (#32483) 
Signed-off-by: Kevin Xiaohua Cai <caixiaohua@starrocks.com>
Co-authored-by: Your Name <you@example.com>
 2023-10-18 14:56:53 +08:00
Felix Li 5b60d3d9f6
[Enhancement] Upgrade libcurl due to CVE-2023-38545/CVE-2023-38546 (#32562) 
Signed-off-by: Astralidea <astralidea@163.com>
 2023-10-11 22:43:44 -07:00
Letian Jiang 6a792a3fcb
[Feature] Support Tencent cos filesystem (#31758) 
Signed-off-by: Letian Jiang <letian.jiang@outlook.com>
 2023-10-09 19:33:02 +08:00
chen9t 88d4eaaf2a
[Enhancement] Upgrade pulsar client to 3.3.0 (#30684) 
Fixes #29540
 2023-09-26 23:28:54 -07:00
Gavin 32263e1eaf
[Refactor] Seperate starcache from staros to avoid build staros modules if unnecessary. (#30687) 
Signed-off-by: Gavin <yangguansuo@starrocks.com>
 2023-09-14 15:46:43 +08:00
Moonm3n bd44cf0c56 [Tool] Fix vars.sh 
Signed-off-by: Moonm3n <saxonzhan@gmail.com>
 2023-08-25 11:09:22 +08:00
Moonm3n 1d03dd74dc [Tool] Incorporate the LLVM third-party library 
Signed-off-by: Moonm3n <saxonzhan@gmail.com>
 2023-08-25 11:09:22 +08:00
Smith Cruise 8fe8a65747
[Enhancement] Replace libdeflate including way (#28695) 
Using third-party to manage libdeflate


Signed-off-by: Smith Cruise <chendingchao1@126.com>
 2023-08-07 11:09:14 +08:00
eyes_on_me 5777c3735b
[Tool] introduce fiu for fault injection (#27694) 
Signed-off-by: silverbullet233 <3675229+silverbullet233@users.noreply.github.com>
 2023-07-21 13:40:37 +08:00
Jinho Kim 78a8c87cd6
[Tool] Upgrade ftm from 7.0.3 to 8.1.1. (#26057) 
Upgrade fmt from 7.0.3 to 8.1.1.
In my fork, I need to use a library that requires fmt-8.1.1.

I have fixed a potential compatibility issue in
https://github.com/StarRocks/starrocks/pull/25806.

---------

Signed-off-by: jkim650 <117620956+jkim650@users.noreply.github.com>
 2023-07-03 13:44:19 +08:00
trueeyu 33b134228b
[BugFix] upgrade jemalloc from 5.2.1 to 5.3.0 (#26326) 
Fixes #issue

```
Thread 912 (Thread 0x7f50197ff700 (LWP 3944)):
#0  0x00007f501ad7854d in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007f501ad73e9b in _L_lock_883 () from /lib64/libpthread.so.0
#2  0x00007f501ad73d68 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x0000000006059a27 in pthread_mutex_lock_impl (mutex=<optimized out>) at src/bthread/mutex.cpp:564
#4  pthread_mutex_lock (__mutex=0x7f5019a032a8) at src/bthread/mutex.cpp:818
#5  pthread_mutex_lock (__mutex=0x7f5019a032a8) at src/bthread/mutex.cpp:817
#6  0x000000000697819f in malloc_mutex_lock_final (mutex=0x7f5019a03268) at include/jemalloc/internal/mutex.h:155
#7  je_malloc_mutex_lock_slow (mutex=mutex@entry=0x7f5019a03268) at src/mutex.c:85
#8  0x000000000696f8c8 in malloc_mutex_lock (mutex=0x7f5019a03268, tsdn=0x7f50197f9040) at include/jemalloc/internal/mutex.h:221
#9  je_extents_evict (tsdn=tsdn@entry=0x7f50197f9040, arena=arena@entry=0x7f5019a00980, r_extent_hooks=r_extent_hooks@entry=0x7f50197f84d8, extents=extents@entry=0x7f5019a03268, npages_min=npages_min@entry=18533) at src/extent.c:572
#10 0x0000000006936004 in arena_stash_decayed (decay_extents=<synthetic pointer>, npages_decay_max=492, npages_limit=<optimized out>, extents=0x7f5019a03268, r_extent_hooks=0x7f50197f84d8, arena=0x7f5019a00980, tsdn=0x7f50197f9040) at src/arena.c:833
#11 arena_decay_to_limit (tsdn=tsdn@entry=0x7f50197f9040, arena=arena@entry=0x7f5019a00980, decay=decay@entry=0x7f5019a06ca0, extents=extents@entry=0x7f5019a03268, all=all@entry=false, npages_limit=<optimized out>, npages_decay_max=<optimized out>, is_background_thread=<optimized out>) at src/arena.c:934
#12 0x00000000069363fb in arena_decay_try_purge (is_background_thread=<optimized out>, npages_limit=<optimized out>, current_npages=<optimized out>, extents=<optimized out>, decay=<optimized out>, arena=<optimized out>, tsdn=<optimized out>) at src/arena.c:722
#13 arena_decay_try_purge (is_background_thread=<optimized out>, npages_limit=<optimized out>, current_npages=<optimized out>, extents=<optimized out>, decay=<optimized out>, arena=<optimized out>, tsdn=<optimized out>) at src/arena.c:611
#14 arena_maybe_decay (is_background_thread=<optimized out>, extents=<optimized out>, decay=<optimized out>, arena=<optimized out>, tsdn=<optimized out>) at src/arena.c:762
#15 arena_maybe_decay (tsdn=<optimized out>, arena=0x7f5019a00980, decay=0x7f5019a06ca0, extents=0x7f5019a03268, is_background_thread=<optimized out>) at src/arena.c:714
#16 0x00000000069389ab in arena_decay_impl (all=false, is_background_thread=true, extents=0x7f5019a03268, decay=0x7f5019a06ca0, arena=0x7f5019a00980, tsdn=0x7f50197f9040) at src/arena.c:964
#17 arena_decay_dirty (all=false, is_background_thread=true, arena=0x7f5019a00980, tsdn=0x7f50197f9040) at src/arena.c:985
#18 je_arena_decay (tsdn=0x7f50197f9040, arena=0x7f5019a00980, is_background_thread=true, all=<optimized out>) at src/arena.c:998
#19 0x00000000069403e9 in background_work_sleep_once (ind=0, info=<optimized out>, tsdn=<optimized out>) at src/background_thread.c:295
#20 background_thread0_work (tsd=<optimized out>) at src/background_thread.c:452
#21 background_work (ind=<optimized out>, tsd=<optimized out>) at src/background_thread.c:490
#22 background_thread_entry () at src/background_thread.c:522
#23 0x00007f501ad71ea5 in start_thread () from /lib64/libpthread.so.0
#24 0x00007f501a38cb0d in clone () from /lib64/libc.so.6
```

upgrade jemalloc from 5.2.1 to 5.3.0 to fix the deadlock:

https://github.com/jemalloc/jemalloc/issues/2019

Redis also encountered this problem

https://github.com/redis/redis/pull/12115

Signed-off-by: trueeyu <lxhhust350@qq.com>
 2023-06-30 20:07:13 +08:00
miomiocat a676657597
[BugFix] Fix no such file error for ubuntu22 and aarch64 arch (#25845) 
Fixes #issue

Signed-off-by: miomiocat <284487410@qq.com>
 2023-06-26 16:33:55 +08:00
miomiocat 7305fb2f07
[Enhancement] Upgrade jindosdk version to 4.6.8 (#24809) 
Signed-off-by: miomiocat <284487410@qq.com>
 2023-06-25 15:19:20 +08:00
Smith Cruise 8be2372adf
[Enhancement] Upgrade AWS S3 SDK to 1.10.36 (#24129) 
The reason that needs to upgrade aws-sdk-cpp is already described in
https://github.com/StarRocks/starrocks/issues/18912

Consider cloud storage like KS3, OBS, if the request path
contains '=', will face 403 problems. So we need to upgrade s3 sdk, to
enable `compliantRfc3986Encoding`.

And we are using the same version as
[ClickHouse](a9d5b68946/contrib),
it will be more stable.

Previous pr about this problem: #18912 #19000 #19229 #19228

**We need to change BE's CMakeFiles also, otherwise ci will be failed.**

## IMPORTANT
Introduce a new parameter `aws_sdk_enable_compliant_rfc3986_encoding` in
be.conf, if you are using OBS/KS3 cloud storage, you should set
`aws_sdk_enable_compliant_rfc3986_encoding=true`.

---------

Signed-off-by: Smith Cruise <chendingchao1@126.com>
 2023-05-26 11:44:46 +08:00
gengjun-git 022f9d2227
[Enhancement] Add FE debug tools: async-profiler (#24044) 
Generate memory profile
```
./bin/profiler.sh -e alloc -d 300 -f alloc-profile.html pid
```

Generate cpu profile
```
./bin/profiler.sh -d 300 -f cpu-profile.html pid
```

Signed-off-by: gengjun-git <gengjun@starrocks.com>
 2023-05-26 10:52:00 +08:00
trueeyu e6ee2f3537
[Enhancement] upgrade roaring bitmap lib to v1.1.3 (#23502) 
The related pr of RoaringBitmap:
https://github.com/RoaringBitmap/CRoaring/pull/246/files
This bug will cause queries based on BitmapIndex to return wrong data,
so upgrade roaring bitmap lib

Signed-off-by: trueeyu <lxhhust350@qq.com>
 2023-05-17 15:32:34 +08:00
zaorangyang 16978cab1d
[Tool] Enable sasl option with librdkafka (#22558) 
Signed-off-by: Zaorang Yang <zaorangy@gmail.com>
 2023-05-10 14:18:44 +08:00
chen9t 1f1bfe08f3
[Enhancement] Introduce datasketches toolkit (#20835) 
* Introduce datasketches toolkit

* Build datasketches from thirdparty
 2023-05-09 05:05:26 +08:00
sfwang218 f9ed50d6c6
[BugFix] Fix hive catalog can't read hive table with textfile lzo format bug (part 1: add thirdparty) (#20191) 
Hive catalog can't read hive table with textfile lzo format
---------

Signed-off-by: wangshifa <wangshifa@58.com>
 2023-03-24 16:07:14 +08:00
Kevin Cai 31fe2a912d
[Tool][Docker] Allow full unitilize CPU during thirdparty build (#19964) 
* Allow set `PARALLEL` outside build-thirdparty.sh to full utilize build
  machine capability

Signed-off-by: Kevin Xiaohua Cai <caixiaohua@starrocks.com>
 2023-03-22 14:13:21 +08:00
Smith Cruise 8eeb503df5
[Feature] Add GCS hadoop connector into thirdparty (#19601) 2023-03-16 16:24:15 +08:00
kangkaisen 5d3cd555d5
Revert "[BugFix] [Thirdparty lib] Fix AWS signature mismatch when querying partitioned Hive table on GCS (#19228)" (#19337) 
This reverts commit 672d29a2ac.
 2023-03-10 12:44:57 +08:00
xiwenjun 672d29a2ac
[BugFix] [Thirdparty lib] Fix AWS signature mismatch when querying partitioned Hive table on GCS (#19228) 
Signed-off-by: gddezero <gdde_0@hotmail.com>
 2023-03-10 00:55:29 +08:00
zaorangyang ee4d84e26c
[Feature] Add confluet schema registry module in thirdparty (#17709) 
* [Feature] Add confluet schema registry module in thirdparty

Signed-off-by: Zaorang Yang <zaorangy@gmail.com>
 2023-02-20 12:36:58 +08:00