Why I'm doing:
Fixes#61695
What I'm doing:
Fixes#61695 , add WEEKDAY function similar to the dayofweek_iso function, but it returns numbers from 0 to 6 (corresponding to Monday to Sunday)
Signed-off-by: wangshifa.1 <wangshifa.1@jd.com>
Why I'm doing:
StarRocks supports spilling some intermediate data to disk or object storage when writing to native table. This can avoid wring too many small files under memory pressure.
The same issue is also exist when writing external table. However, now the spill procedure heavily coupled with native table and cannot be reused by external table directly.
So, it is necessary to introduce a separate module to implement the spill function, which can easily be used by native and external table.
What I'm doing:
Introduce a load chunk spiller to handle the load and merge functions.
Refactor the spill memtable sink of native table based on the load chunk spiller.
Signed-off-by: GavinMar <yangguansuo@starrocks.com>
What I'm doing:
bits_function: the implementation is wrong
change the static DCHECK to dynamic argument validation for some functions
fix some type mapping error in the logical_type.cpp
Signed-off-by: Murphy <mofei@starrocks.com>
## Why I'm doing:
Currently, users can only configure the timeout for prepared transactions through the global FE configuration `prepared_transaction_default_timeout_second`. This approach lacks flexibility as it requires all transactions to use the same timeout value. Users need the ability to specify different timeout values for different transactions based on their specific requirements, especially in production environments where precise control over transaction lifecycle is crucial.
## What I'm doing:
This PR adds support for the `prepared_timeout` configuration in transaction stream load, allowing users to specify a timeout period for transactions from PREPARED to COMMITTED state. The implementation includes:
**Backend Changes:**
- Added `HTTP_PREPARED_TIMEOUT` constant in `be/src/http/http_common.h`
- Extended `StreamLoadContext` with `prepared_timeout_second` field
- Modified `TransactionMgr` to parse `prepared_timeout` HTTP header
- Updated `StreamLoadExecutor::prepare_txn` to pass timeout to FE
- Enhanced `TransactionState` with `preparedTimeoutMs` field and timeout detection logic
- Updated Thrift interface `TLoadTxnCommitRequest` with `prepared_timeout_second` field
**Frontend Changes:**
- Modified `TransactionLoadAction` to parse `prepared_timeout` parameter
- Updated `TransactionState` with `setPreparedTimeAndTimeout` method
- Enhanced `DatabaseTransactionMgr` and `GlobalTransactionMgr` to handle prepared timeout
- Updated transaction timeout detection logic in `TransactionState::isTimeout`
**Usage Example:**
```bash
# Begin transaction
curl --location-trusted -u root: -H "label:test_txn" -H "timeout:300" -H "db:test_db" -H "table:test_table" \
-XPOST http://fe_host:8030/api/transaction/begin
# Load data
curl --location-trusted -u root: -H "label:test_txn" -H "db:test_db" -H "table:test_table" \
-d '1' -XPUT http://fe_host:8030/api/transaction/load
# Prepare transaction with custom timeout (60 seconds)
curl --location-trusted -u root: -H "label:test_txn" -H "db:test_db" \
-H "prepared_timeout:60" -XPOST http://fe_host:8030/api/transaction/prepare
# Commit transaction
curl --location-trusted -u root: -H "label:test_txn" -H "db:test_db" \
-XPOST http://fe_host:8030/api/transaction/commit
# View transaction details including PreparedTime and PreparedTimeoutMs
SHOW TRANSACTION WHERE id = <transaction_id>;
+---------------+--------+---------------+-------------------+-------------------+---------------------+---------------------+---------------------+---------------------+---------------------+--------+--------------------+------------+-----------+-------------------+--------+
| TransactionId | Label | Coordinator | TransactionStatus | LoadJobSourceType | PrepareTime | PreparedTime | CommitTime | PublishTime | FinishTime | Reason | ErrorReplicasCount | ListenerId | TimeoutMs | PreparedTimeoutMs | ErrMsg |
+---------------+--------+---------------+-------------------+-------------------+---------------------+---------------------+---------------------+---------------------+---------------------+--------+--------------------+------------+-----------+-------------------+--------+
| 1633 | test_txn | BE: 127.0.0.1 | VISIBLE | BACKEND_STREAMING | 2025-08-03 11:02:54 | 2025-08-03 11:03:10 | 2025-08-03 11:03:14 | 2025-08-03 11:03:14 | 2025-08-03 11:03:14 | | 0 | [12237] | 300000 | 60000 | |
+---------------+--------+---------------+-------------------+-------------------+---------------------+---------------------+---------------------+---------------------+---------------------+--------+--------------------+------------+-----------+-------------------+--------+
```
**Documentation:**
- Updated `Stream_Load_transaction_interface.md` with `prepared_timeout` usage instructions
- Modified `SHOW_TRANSACTION.md` to document new `PreparedTime` and `PreparedTimeoutMs` fields
- Added version information indicating support from 4.0.0 onwards
The feature provides backward compatibility by using the FE configuration `prepared_transaction_default_timeout_second` as the default value when `prepared_timeout` is not specified.
Signed-off-by: PengFei Li <lpengfei2016@gmail.com>
Signed-off-by: 絵空事スピリット <wanglichen@starrocks.com>
Co-authored-by: 絵空事スピリット <wanglichen@starrocks.com>
Fix sse_memcmp UT compilation error on aarch64.
## Why I'm doing:
```
[ 96%] Building CXX object test/CMakeFiles/starrocks_test_objs.dir/util/monotime_test.cpp.o
[ 96%] Building CXX object test/CMakeFiles/starrocks_test_objs.dir/util/mysql_row_buffer_test.cpp.o
/root/starrocks/be/test/util/memcmp_test.cpp: In member function 'virtual void starrocks::sse_memcmp_Test_Test::TestBody()':
/root/starrocks/be/test/util/memcmp_test.cpp:38:20: error: 'sse_memcmp2' was not declared in this scope
38 | int res2 = sse_memcmp2(c1, c2, 3);
| ^~~~~~~~~~~
/root/starrocks/be/test/util/memcmp_test.cpp:46:20: error: 'sse_memcmp2' was not declared in this scope
46 | int res2 = sse_memcmp2(c1, c2, 3);
| ^~~~~~~~~~~
/root/starrocks/be/test/util/memcmp_test.cpp:54:20: error: 'sse_memcmp2' was not declared in this scope
54 | int res2 = sse_memcmp2(c1, c2, 3);
| ^~~~~~~~~~~
/root/starrocks/be/test/util/memcmp_test.cpp:62:20: error: 'sse_memcmp2' was not declared in this scope
62 | int res2 = sse_memcmp2(c1, c2, 3);
| ^~~~~~~~~~~
/root/starrocks/be/test/util/memcmp_test.cpp:71:20: error: 'sse_memcmp2' was not declared in this scope
71 | int res2 = sse_memcmp2(c1, c2, 3);
| ^~~~~~~~~~~
/root/starrocks/be/test/util/memcmp_test.cpp:80:20: error: 'sse_memcmp2' was not declared in this scope
80 | int res2 = sse_memcmp2(c1, c2, 3);
| ^~~~~~~~~~~
/root/starrocks/be/test/util/memcmp_test.cpp:89:20: error: 'sse_memcmp2' was not declared in this scope
89 | int res2 = sse_memcmp2(c1, c2, 3);
| ^~~~~~~~~~~
/root/starrocks/be/test/util/memcmp_test.cpp:98:20: error: 'sse_memcmp2' was not declared in this scope
98 | int res2 = sse_memcmp2(c1, c2, 3);
| ^~~~~~~~~~~
make[2]: *** [test/CMakeFiles/starrocks_test_objs.dir/util/memcmp_test.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [test/CMakeFiles/starrocks_test_objs.dir/all] Error 2
make: *** [all] Error 2
```
Signed-off-by: qingzhongli <qingzhongli2018@gmail.com>
the cpu instruction is off either because of not wanted the target instruction set or the build machine doesn't have the instruction supported. Be respectful to the instruction switch.
Signed-off-by: Kevin Cai <kevin.cai@celerdata.com>
handle zstd decompress failure, throw runtime_error exception
fix orc_scanner tpch_10k.orc.zstd, it's corrupted. Replace it with correct test file and update the related test cases.
Signed-off-by: Kevin Cai <kevin.cai@celerdata.com>
Why I'm doing:
When sending a request to the /api/transaction/{begin,load,commit,...} endpoints, the content type is wrongly set to text/html instead of application/json.
What I'm doing:
Fixes#61130
Signed-off-by: Fatih Çatalkaya <fatih.catalkaya@yahoo.de>
Signed-off-by: Kevin Cai <kevin.cai@celerdata.com>
Co-authored-by: Kevin Cai <kevin.cai@celerdata.com>
should not create a separate evhttp_request in test body
shall leverage the input_buffer created in the evhttp_request initialized by evhttp_request_new()
Signed-off-by: Kevin Cai <kevin.cai@celerdata.com>
Why I'm doing:
When clone and drop table run concurrency, the new_tablet during clone maybe dropped and throw null exception.
Signed-off-by: sevev <qiangzh95@gmail.com>
lyw
eyes_on_me
shuming.li
zihe.liu
starrocks-xupeng
stdpain
Murphy
Yixin Luo
Simon Griebel
zhanghe
xiangguangyxg
yan zhang
Seaven
Gavin
trueeyu
zombee0
wyb
wangshifa.1
Hongkun Xu
zhangqiang
srlch
meegoo
PengFei Li
before-Sunrise
liubotao
Xie Bofan
Mathieu Baurin
qingzhongli
Kevin Cai
SevenJ
Fatih Çatalkaya
Hechem Selmi
Jun-Seok Heo
duanyyyyyyy
Evgeniy Zuikin
satanson
alexzorin