Commit Graph

1699 Commits

Author SHA1 Message Date
lyw b05e7ded50
[Feature](function) support sec_to_time (#62797) 2025-10-11 17:28:48 +08:00
satanson 78945fde78
[Enhancement] lead/lag/first_value/last_value support array type (#63547)
Signed-off-by: satanson <ranpanf@gmail.com>
2025-10-11 10:05:20 +08:00
shuming.li 6f69933e11
[UT] Add more cases to cover nested mv create (#62996)
Signed-off-by: shuming.li <ming.moriarty@gmail.com>
2025-10-10 18:27:18 +08:00
shuming.li 1785077e7b
[UT] Fix print_hit_materialized_views to ensure the result deterministic (#63862)
Signed-off-by: shuming.li <ming.moriarty@gmail.com>
2025-10-10 18:25:44 +08:00
shuming.li 7ed48a790a
[BugFix] Fix mv rewrite bugs with IllegalStateException (#63655)
Signed-off-by: shuming.li <ming.moriarty@gmail.com>
2025-10-09 19:22:29 +08:00
shuming.li fb05bb99a4
[UT] Fix mv related unstable tests (#63852)
Signed-off-by: shuming.li <ming.moriarty@gmail.com>
2025-10-09 18:21:42 +08:00
shuming.li 0dfb447b23
[BugFix] Fix mv rewrite bugs with char/varchar implicit cast (#63659)
Signed-off-by: shuming.li <ming.moriarty@gmail.com>
2025-10-09 14:30:20 +08:00
yan zhang fc35890c29
[UT] increase CBO timeout to avoid iceberg UT fail (#63766)
Signed-off-by: yan zhang <dirtysalt1987@gmail.com>
2025-10-07 09:28:55 +08:00
Youngwb 943c6f8a57
[Enhancement] Support add files from hive table for iceberg procedure(part 2) (#63656) 2025-09-29 15:05:30 +08:00
zihe.liu 4c248a560b
[BugFix] Fix cast from LARGEINT to DECIMAL128 (#63559)
Signed-off-by: zihe.liu <ziheliu1024@gmail.com>
2025-09-29 10:36:35 +08:00
Youngwb 364ff2e036
[BugFix] Fix profile lost sql statement and planner trace when use prepare statement (#63519) 2025-09-28 10:58:01 +08:00
shuming.li ee28a4b9ac
[UT] Avoid partition ttl scheduler affecting tests (#63615)
Signed-off-by: shuming.li <ming.moriarty@gmail.com>
2025-09-26 17:11:07 +08:00
before-Sunrise ca0b85e643
[BugFix] fix prepare stmt's sql is "omit" (#62988)
Signed-off-by: before-Sunrise <unclejyj@gmail.com>
2025-09-26 13:58:28 +08:00
shuming.li 874d1dbff8
[Feature] (IVM Part7) Support IVM refresh for aggreagte (#62699)
Signed-off-by: shuming.li <ming.moriarty@gmail.com>
2025-09-26 10:24:06 +08:00
stephen a736842c67
[BugFix] Fix date_trunc partition pruning with combined predicates causing EMPTYSET (#63464)
Signed-off-by: stephen <stephen5217@163.com>
2025-09-25 19:01:31 +08:00
Harbor Liu a73849377d
[UT] Fix unstability security test case (#63520) 2025-09-25 09:58:42 +08:00
stephen 7d81781448
[Feature] support asof join (BE Part) (#63236)
Signed-off-by: stephen <stephen5217@163.com>
2025-09-24 17:15:36 +08:00
yan zhang 3b0bee9731
[UT] fix unstable distance function cases because of precision (#63502)
Signed-off-by: yan zhang <dirtysalt1987@gmail.com>
2025-09-24 16:24:04 +08:00
predator4ann 7f5710459a
[BugFix] Fix dcg data not read correctly when switching from column mode to row mode in partial update (#61529)
Signed-off-by: predator4ann <yunlong.sun@hotmail.com>
2025-09-24 15:09:11 +08:00
stephen b9176a4696
[UT] fix unstable sql test (#63503)
Signed-off-by: stephen <stephen5217@163.com>
2025-09-24 15:04:56 +08:00
satanson 0c45329a51
[Enhancement] Low cardinality optimization on analytic operator obove table functions (#63378)
Signed-off-by: satanson <ranpanf@gmail.com>
2025-09-24 14:43:35 +08:00
Martin Bogusz 4f33574e11
[Enhancement] Replace assert_regexp_matches with assert_regex for python 3.12 in test/lib (#63456)
## Why I'm doing:
nose.tools maps assert_regexp_matches to unittest `AssertRegexpMatches`.
`AssertRegexpMatches` got renamed to `AssertRegex` and  is deprecated since python 3.2. The alias got removed in 3.12 ([changelog](49e83e31bd/Doc/whatsnew/3.12.rst (unittest-1))). 
When running with `python >= 3.12` this will result in `AttributeError: module 'nose.tools' has no attribute 'assert_regexp_matches'`.

## What I'm doing:
Replacing occurences of assert_regexp_matches with assert_regex.

Signed-off-by: m.bogusz <m.bogusz@celonis.com>
2025-09-24 11:19:19 +08:00
before-Sunrise 86fdf35bcf
[BugFix] Fix query detail lost audit items (#63237)
Signed-off-by: before-Sunrise <unclejyj@gmail.com>
2025-09-23 09:00:42 -07:00
srlch 5bbd2ed15a
[BugFix] Fix dcg meta inconsistency when partial update with auto increment column in column upsert mode. (#63370)
Signed-off-by: srlch <linzichao@starrocks.com>
2025-09-23 14:12:42 +08:00
stephen 30df6bb044
[Enhancement] Implement SQL standard JOIN USING with MySQL compatibility (#63312)
Signed-off-by: stephen <stephen5217@163.com>
2025-09-23 10:05:50 +08:00
Harbor Liu 0bbbae8cef
[Enhancement] Implement grant and revoke role functionality for external groups (#63258)
What I'm doing:
This pull request adds support for granting and revoking roles to user groups in the authorization system. It introduces new methods and data structures to manage the mapping between groups and roles, ensures persistence and replay of these operations, and updates the privilege loading logic to include group-based roles. The changes also update relevant SQL analyzers and statement visitors to handle group-related role grants and revokes.

Group-to-Role Grant/Revoke Functionality:

Added a new groupToRoleList map in AuthorizationMgr to track which roles are assigned to each group, along with methods to grant and revoke roles for groups (grantRoleToGroup, revokeRoleFromGroup, and their replay counterparts). These actions are now handled in the main grantRole and revokeRole methods. [1] [2] [3] [4] [5]
Introduced the UpdateGroupToRoleLog class to persist group-to-role assignments and removals, and added new operation types (OP_GRANT_ROLE_TO_GROUP, OP_REVOKE_ROLE_FROM_GROUP) for edit log handling. [1] [2]
Updated the edit log and deserialization logic to support the new group-to-role operations, ensuring they are correctly replayed during recovery. [1] [2] [3]
Privilege Resolution and Query Enhancements:

Modified privilege collection loading to include roles assigned via group membership, so users gain the union of their direct and group-based roles. [1] [2]
Added methods to query roles assigned to a group, both by role ID and by role name, for use in privilege checks and metadata queries. [1] [2]
SQL Analyzer and Visitor Updates:

Updated the SQL analyzer and statement visitor logic to support group-based role grants and revokes, ensuring correct validation and privilege checks for group operations. [1] [2] [3] [4]
Persistence and Initialization:

Ensured that the groupToRoleList map is persisted and restored during authorization manager state load, maintaining group-role relationships across restarts.
These changes collectively enable group-based role management, improving flexibility and scalability of privilege assignment in the system.
2025-09-22 14:56:37 +08:00
kyle-goodale-klaviyo ce6b0525c6
[BugFix] Fix bug where reserved words in iceberg partitions break toThrift (#63243) 2025-09-22 14:11:28 +08:00
Kevin Cai bfb83e7e9c
[BugFix] fix duplicate key table delete issue in shared-data mode (#63296)
Signed-off-by: Kevin Cai <kevin.cai@celerdata.com>
2025-09-22 09:29:03 +08:00
meegoo af7052ff34
[Enhancement] Make BE reject multi-statement transaction stream load (#63242)
Signed-off-by: meegoo <meegoo.sr@gmail.com>
2025-09-19 10:33:40 -07:00
Harbor Liu 348cd8f151
[Enhancement] Support for `IF NOT EXISTS` and `IF EXISTS` clauses to the GROUP PROVIDER (#63248)
What I'm doing:
This pull request adds support for IF NOT EXISTS and IF EXISTS clauses to the CREATE GROUP PROVIDER and DROP GROUP PROVIDER SQL statements, ensuring these operations are idempotent and error-tolerant. It also introduces comprehensive unit tests to verify the correct behavior of these clauses, and updates the parser and semantic analyzer accordingly.

Enhancements to Group Provider DDL Statements:

Added support for IF NOT EXISTS to CREATE GROUP PROVIDER and IF EXISTS to DROP GROUP PROVIDER, allowing these statements to succeed silently if the provider already exists or does not exist, respectively. The implementation includes proper error handling and DDL exception messages for cases where these clauses are not specified. [1] [2]

Updated the SQL parser (AstBuilder) to recognize and propagate the IF NOT EXISTS and IF EXISTS flags in CreateGroupProviderStmt and DropGroupProviderStmt objects.

Enhanced the semantic analyzer to respect the new flags, throwing exceptions only when appropriate (i.e., when the flags are not set and the provider's existence does not match the operation). [1] [2]

Testing Improvements:

Added a new test class GroupProviderStatementTest with comprehensive unit tests covering all scenarios for the new IF NOT EXISTS and IF EXISTS functionality, including creation, dropping, legacy constructors, and sequential operations.
Error Handling:

Wrapped the execution of dropGroupProviderStatement in the DDL statement executor with error reporting to ensure runtime exceptions are properly managed.
2025-09-19 14:01:30 +08:00
zihe.liu 82e2559533
[UT] Fix timeout SQL Test (#63246)
Signed-off-by: zihe.liu <ziheliu1024@gmail.com>
2025-09-18 15:06:34 +08:00
meegoo 2c759218fe
[Enhancement] Update default bucket size of random distribution (#63168)
Signed-off-by: meegoo <meegoo.sr@gmail.com>
2025-09-18 10:32:03 +08:00
shuming.li 7c8a29660a
[Enhancement] Optimize parsing predicates with large number of CompoundPredicates (#63139)
Signed-off-by: shuming.li <ming.moriarty@gmail.com>
2025-09-17 16:37:29 +08:00
stephen 9e641d9eaf
[Feature] Introduce asof join (FE Part) (#63070)
Signed-off-by: stephen <stephen5217@163.com>
2025-09-16 19:57:33 +08:00
shuming.li d96e17baa8
[UT] [BugFix] Fix FineGrainedRangePredicateRule rule bug (#63148)
Signed-off-by: shuming.li <ming.moriarty@gmail.com>
2025-09-16 10:32:51 +08:00
satanson b956f11544
[Enhancement] Full sort use german string for comparison (#62929)
Signed-off-by: satanson <ranpanf@gmail.com>
2025-09-15 20:23:40 +08:00
PengFei Li d60c76aa52
[Tool] Add CLEANUP block to SQL-tester for reliable per-case teardown (#63068)
## Why I’m doing
- Tests often enable temporary configs, create transient objects, or invoke external side-effects. When a test fails mid-way, ad-hoc cleanup embedded in the test body may be skipped, leaving residue that causes flakiness and cross-test interference.
- This change provides a first-class, always-run teardown mechanism that executes after each case, regardless of pass/fail, improving test reliability and isolation, and simplifying test authoring.

## What I’m doing
- Introduce a CLEANUP block to SQL-tester that defines statements to run in tearDown for each case.
- Semantics:
  - CLEANUP runs unconditionally in tearDown after each case (pass/fail/timeout).
  - Results in CLEANUP are not validated; failures are logged but do not prevent later cleanup steps.
  - Existing built-in cleanup (e.g., dropping parsed DATABASE/RESOURCE) still runs; CLEANUP is additive.
  - In record mode, the CLEANUP block is preserved in-place in the generated R file.
- Syntax and example:
```sql
-- name: my_case
...
CLEANUP {
  -- SQL must end with ';'
  DROP TABLE IF EXISTS t1;
  DROP DATABASE IF EXISTS db_${uuid0};
  -- shell and function are allowed
  shell: echo "cleanup ${uuid0}"
  function: restore_env("arg1")
} END CLEANUP
```

**Code changes**
- test/README.md
  - Document the CLEANUP grammar, usage, semantics, and example.
- test/lib/__init__.py
  - Add CLEANUP/END CLEANUP flags.
- test/lib/choose_cases.py
  - Parse CLEANUP blocks, collect commands, validate format, include in case model, support variable/uuid replacement, and record block in-place during -r mode.
- test/test_sql_cases.py
  - Execute collected CLEANUP commands in tearDown unconditionally; keep record-mode output aligned; ensure built-in database/resource cleanup still runs.

Signed-off-by: PengFei Li <lpengfei2016@gmail.com>
2025-09-15 14:27:43 +08:00
shuming.li fd76111263
[UT] Fix unstable case test_mv_rewrite_with_time_series_multi_mvs (#63077)
Signed-off-by: shuming.li <ming.moriarty@gmail.com>
2025-09-12 16:13:47 +08:00
shuming.li 54317498bc
[BugFix] Fix mv rewriter binder bugs (#62919)
Signed-off-by: shuming.li <ming.moriarty@gmail.com>
2025-09-12 14:12:35 +08:00
SevenJ c9a278a380
[BugFix] fix iceberg read null partition bug (#62934)
Signed-off-by: SevenJ <wenjun7j@gmail.com>
2025-09-12 10:10:38 +08:00
eyes_on_me efd17800bf
[BugFix] mutate input columns in functions' returning value (#62826)
Signed-off-by: silverbullet233 <3675229+silverbullet233@users.noreply.github.com>
2025-09-10 09:28:55 +00:00
shuming.li cccd31c903
[Enhancement] Ensure mv force refresh will refresh target partitions (#62627)
Signed-off-by: shuming.li <ming.moriarty@gmail.com>
2025-09-10 10:31:43 +08:00
PengFei Li 678cb8fff4
[BugFix] Fix collecting stream load profile failed (#62802)
Signed-off-by: PengFei Li <lpengfei2016@gmail.com>
2025-09-10 10:25:16 +08:00
Youngwb e9969458f4
[Enhancement] Support add files iceberg procedure (part 1) (#62886) 2025-09-10 10:05:30 +08:00
stdpain ed2dc59ef7
[Enhancement] support struct input for min/max by (#62829)
Signed-off-by: stdpain <drfeng08@gmail.com>
2025-09-09 17:01:08 +08:00
stephen ad5500f35e
[BugFix] Fix SQL syntax error in histogram statistics when MCV contains single quotes (#62853)
Signed-off-by: stephen <stephen5217@163.com>
2025-09-09 14:28:27 +08:00
srlch 7cbfdee587
[Enhancement] provide alter table xxx set auto_increment (#62767)
Signed-off-by: srlch <linzichao@starrocks.com>
Co-authored-by: Evgeniy Shishkin <eshishki@gmail.com>
2025-09-09 14:08:40 +08:00
yan zhang 594207d01a
[UT] fix sql test for low card on lake (#62771)
Signed-off-by: yan zhang <dirtysalt1987@gmail.com>
2025-09-06 21:22:03 +08:00
Rohit Satardekar 6b3d00852b
[Enhancement] add command to show the dropped meta information that can be recovered (#51007)
Signed-off-by: Rohit Satardekar <rohitrs1983@gmail.com>
2025-09-05 10:23:19 +08:00
zihe.liu 0de2419259
[UT] Fix timeout for SQL test case `test_partition_hash_join` (#62722)
Signed-off-by: zihe.liu <ziheliu1024@gmail.com>
2025-09-04 16:05:36 +08:00