[Doc] Fix filename spaces (#32525)
Signed-off-by: DanRoscigno <dan@roscigno.com>
This commit is contained in:
parent
f96c3279a9
commit
ad1d16540e
|
|
@ -69,6 +69,6 @@ Before contributing, please read this article carefully to quickly understand th
|
|||
|
||||
## Documentation template
|
||||
|
||||
- [Functions](https://github.com/StarRocks/docs/blob/main/sql-reference/sql-functions/How%20to%20Write%20Functions%20Documentation.md)
|
||||
- [SQL command template](https://github.com/StarRocks/docs/blob/main/sql-reference/sql-statements/SQL%20command%20template.md)
|
||||
- [Functions](https://github.com/StarRocks/docs/blob/main/sql-reference/sql-functions/How_to_Write_Functions_Documentation.md)
|
||||
- [SQL command template](https://github.com/StarRocks/docs/blob/main/sql-reference/sql-statements/SQL_command_template.md)
|
||||
- [Loading data template](https://github.com/StarRocks/starrocks/blob/main/docs/loading/Loading_data_template.md)
|
||||
|
|
|
|||
270
docs/TOC.md
270
docs/TOC.md
|
|
@ -100,10 +100,10 @@
|
|||
+ [Query rewrite](./using_starrocks/query_rewrite_with_materialized_views.md)
|
||||
+ [Data lake query acceleration](./using_starrocks/data_lake_query_acceleration_with_materialized_views.md)
|
||||
+ Usage
|
||||
+ [CREATE MATERIALIZED VIEW](./sql-reference/sql-statements/data-definition/CREATE%20MATERIALIZED%20VIEW.md)
|
||||
+ [ALTER MATERIALIZED VIEW](./sql-reference/sql-statements/data-definition/ALTER%20MATERIALIZED%20VIEW.md)
|
||||
+ [DROP MATERIALIZED VIEW](./sql-reference/sql-statements/data-definition/DROP%20MATERIALIZED%20VIEW.md)
|
||||
+ [SHOW MATERIALIZED VIEWS](./sql-reference/sql-statements/data-manipulation/SHOW%20MATERIALIZED%20VIEW.md)
|
||||
+ [CREATE MATERIALIZED VIEW](./sql-reference/sql-statements/data-definition/CREATE_MATERIALIZED_VIEW.md)
|
||||
+ [ALTER MATERIALIZED VIEW](./sql-reference/sql-statements/data-definition/ALTER_MATERIALIZED_VIEW.md)
|
||||
+ [DROP MATERIALIZED VIEW](./sql-reference/sql-statements/data-definition/DROP_MATERIALIZED_VIEW.md)
|
||||
+ [SHOW MATERIALIZED VIEWS](./sql-reference/sql-statements/data-manipulation/SHOW_MATERIALIZED_VIEW.md)
|
||||
+ [information_schema.materialized_views](./administration/information_schema.md#materialized_views)
|
||||
+ [Troubleshooting](./using_starrocks/troubleshooting_asynchronous_materialized_views.md)
|
||||
+ [Colocate Join](./using_starrocks/Colocate_join.md)
|
||||
|
|
@ -165,159 +165,159 @@
|
|||
+ Reference
|
||||
+ SQL Reference
|
||||
+ User Account Management
|
||||
+ [ALTER USER](./sql-reference/sql-statements/account-management/ALTER%20USER.md)
|
||||
+ [CREATE ROLE](./sql-reference/sql-statements/account-management/CREATE%20ROLE.md)
|
||||
+ [CREATE USER](./sql-reference/sql-statements/account-management/CREATE%20USER.md)
|
||||
+ [DROP ROLE](./sql-reference/sql-statements/account-management/DROP%20ROLE.md)
|
||||
+ [DROP USER](./sql-reference/sql-statements/account-management/DROP%20USER.md)
|
||||
+ [EXECUTE AS](./sql-reference/sql-statements/account-management/EXECUTE%20AS.md)
|
||||
+ [ALTER USER](./sql-reference/sql-statements/account-management/ALTER_USER.md)
|
||||
+ [CREATE ROLE](./sql-reference/sql-statements/account-management/CREATE_ROLE.md)
|
||||
+ [CREATE USER](./sql-reference/sql-statements/account-management/CREATE_USER.md)
|
||||
+ [DROP ROLE](./sql-reference/sql-statements/account-management/DROP_ROLE.md)
|
||||
+ [DROP USER](./sql-reference/sql-statements/account-management/DROP_USER.md)
|
||||
+ [EXECUTE AS](./sql-reference/sql-statements/account-management/EXECUTE_AS.md)
|
||||
+ [GRANT](./sql-reference/sql-statements/account-management/GRANT.md)
|
||||
+ [REVOKE](./sql-reference/sql-statements/account-management/REVOKE.md)
|
||||
+ [SET DEFAULT ROLE](./sql-reference/sql-statements/account-management/SET_DEFAULT_ROLE.md)
|
||||
+ [SET PASSWORD](./sql-reference/sql-statements/account-management/SET%20PASSWORD.md)
|
||||
+ [SET PROPERTY](./sql-reference/sql-statements/account-management/SET%20PROPERTY.md)
|
||||
+ [SET ROLE](./sql-reference/sql-statements/account-management/SET%20ROLE.md)
|
||||
+ [SHOW AUTHENTICATION](./sql-reference/sql-statements/account-management/SHOW%20AUTHENTICATION.md)
|
||||
+ [SHOW GRANTS](./sql-reference/sql-statements/account-management/SHOW%20GRANTS.md)
|
||||
+ [SHOW ROLES](./sql-reference/sql-statements/account-management/SHOW%20ROLES.md)
|
||||
+ [SHOW USERS](./sql-reference/sql-statements/account-management/SHOW%20USERS.md)
|
||||
+ [SET PASSWORD](./sql-reference/sql-statements/account-management/SET_PASSWORD.md)
|
||||
+ [SET PROPERTY](./sql-reference/sql-statements/account-management/SET_PROPERTY.md)
|
||||
+ [SET ROLE](./sql-reference/sql-statements/account-management/SET_ROLE.md)
|
||||
+ [SHOW AUTHENTICATION](./sql-reference/sql-statements/account-management/SHOW_AUTHENTICATION.md)
|
||||
+ [SHOW GRANTS](./sql-reference/sql-statements/account-management/SHOW_GRANTS.md)
|
||||
+ [SHOW ROLES](./sql-reference/sql-statements/account-management/SHOW_ROLES.md)
|
||||
+ [SHOW USERS](./sql-reference/sql-statements/account-management/SHOW_USERS.md)
|
||||
+ Cluster Management
|
||||
+ [ADD SQLBLACKLIST](./sql-reference/sql-statements/Administration/ADD%20SQLBLACKLIST.md)
|
||||
+ [ADMIN CANCEL REPAIR TABLE](./sql-reference/sql-statements/Administration/ADMIN%20CANCEL%20REPAIR.md)
|
||||
+ [ADMIN CHECK TABLET](./sql-reference/sql-statements/Administration/ADMIN%20CHECK%20TABLET.md)
|
||||
+ [ADMIN REPAIR TABLE](./sql-reference/sql-statements/Administration/ADMIN%20REPAIR.md)
|
||||
+ [ADMIN SET CONFIG](./sql-reference/sql-statements/Administration/ADMIN%20SET%20CONFIG.md)
|
||||
+ [ADMIN SET REPLICA STATUS](./sql-reference/sql-statements/Administration/ADMIN%20SET%20REPLICA%20STATUS.md)
|
||||
+ [ADMIN SHOW CONFIG](./sql-reference/sql-statements/Administration/ADMIN%20SHOW%20CONFIG.md)
|
||||
+ [ADMIN SHOW REPLICA DISTRIBUTION](./sql-reference/sql-statements/Administration/ADMIN%20SHOW%20REPLICA%20DISTRIBUTION.md)
|
||||
+ [ADMIN SHOW REPLICA STATUS](./sql-reference/sql-statements/Administration/ADMIN%20SHOW%20REPLICA%20STATUS.md)
|
||||
+ [ALTER RESOURCE GROUP](./sql-reference/sql-statements/Administration/ALTER%20RESOURCE%20GROUP.md)
|
||||
+ [ALTER STORAGE VOLUME](./sql-reference/sql-statements/Administration/ALTER%20STORAGE%20VOLUME.md)
|
||||
+ [ALTER SYSTEM](./sql-reference/sql-statements/Administration/ALTER%20SYSTEM.md)
|
||||
+ [CANCEL DECOMMISSION](./sql-reference/sql-statements/Administration/CANCEL%20DECOMMISSION.md)
|
||||
+ [CREATE FILE](./sql-reference/sql-statements/Administration/CREATE%20FILE.md)
|
||||
+ [CREATE RESOURCE GROUP](./sql-reference/sql-statements/Administration/CREATE%20RESOURCE%20GROUP.md)
|
||||
+ [CREATE STORAGE VOLUME](./sql-reference/sql-statements/Administration/CREATE%20STORAGE%20VOLUME.md)
|
||||
+ [DELETE SQLBLACKLIST](./sql-reference/sql-statements/Administration/DELETE%20SQLBLACKLIST.md)
|
||||
+ [DESC STORAGE VOLUME](./sql-reference/sql-statements/Administration/DESC%20STORAGE%20VOLUME.md)
|
||||
+ [DROP FILE](./sql-reference/sql-statements/Administration/DROP%20FILE.md)
|
||||
+ [DROP RESOURCE GROUP](./sql-reference/sql-statements/Administration/DROP%20RESOURCE%20GROUP.md)
|
||||
+ [DROP STORAGE VOLUME](./sql-reference/sql-statements/Administration/DROP%20STORAGE%20VOLUME.md)
|
||||
+ [ADD SQLBLACKLIST](./sql-reference/sql-statements/Administration/ADD_SQLBLACKLIST.md)
|
||||
+ [ADMIN CANCEL REPAIR TABLE](./sql-reference/sql-statements/Administration/ADMIN_CANCEL_REPAIR.md)
|
||||
+ [ADMIN CHECK TABLET](./sql-reference/sql-statements/Administration/ADMIN_CHECK_TABLET.md)
|
||||
+ [ADMIN REPAIR TABLE](./sql-reference/sql-statements/Administration/ADMIN_REPAIR.md)
|
||||
+ [ADMIN SET CONFIG](./sql-reference/sql-statements/Administration/ADMIN_SET_CONFIG.md)
|
||||
+ [ADMIN SET REPLICA STATUS](./sql-reference/sql-statements/Administration/ADMIN_SET_REPLICA_STATUS.md)
|
||||
+ [ADMIN SHOW CONFIG](./sql-reference/sql-statements/Administration/ADMIN_SHOW_CONFIG.md)
|
||||
+ [ADMIN SHOW REPLICA DISTRIBUTION](./sql-reference/sql-statements/Administration/ADMIN_SHOW_REPLICA_DISTRIBUTION.md)
|
||||
+ [ADMIN SHOW REPLICA STATUS](./sql-reference/sql-statements/Administration/ADMIN_SHOW_REPLICA_STATUS.md)
|
||||
+ [ALTER RESOURCE GROUP](./sql-reference/sql-statements/Administration/ALTER_RESOURCE_GROUP.md)
|
||||
+ [ALTER STORAGE VOLUME](./sql-reference/sql-statements/Administration/ALTER_STORAGE_VOLUME.md)
|
||||
+ [ALTER SYSTEM](./sql-reference/sql-statements/Administration/ALTER_SYSTEM.md)
|
||||
+ [CANCEL DECOMMISSION](./sql-reference/sql-statements/Administration/CANCEL_DECOMMISSION.md)
|
||||
+ [CREATE FILE](./sql-reference/sql-statements/Administration/CREATE_FILE.md)
|
||||
+ [CREATE RESOURCE GROUP](./sql-reference/sql-statements/Administration/CREATE_RESOURCE_GROUP.md)
|
||||
+ [CREATE STORAGE VOLUME](./sql-reference/sql-statements/Administration/CREATE_STORAGE_VOLUME.md)
|
||||
+ [DELETE SQLBLACKLIST](./sql-reference/sql-statements/Administration/DELETE_SQLBLACKLIST.md)
|
||||
+ [DESC STORAGE VOLUME](./sql-reference/sql-statements/Administration/DESC_STORAGE_VOLUME.md)
|
||||
+ [DROP FILE](./sql-reference/sql-statements/Administration/DROP_FILE.md)
|
||||
+ [DROP RESOURCE GROUP](./sql-reference/sql-statements/Administration/DROP_RESOURCE_GROUP.md)
|
||||
+ [DROP STORAGE VOLUME](./sql-reference/sql-statements/Administration/DROP_STORAGE_VOLUME.md)
|
||||
+ [EXPLAIN](./sql-reference/sql-statements/Administration/EXPLAIN.md)
|
||||
+ [INSTALL PLUGIN](./sql-reference/sql-statements/Administration/INSTALL%20PLUGIN.md)
|
||||
+ [INSTALL PLUGIN](./sql-reference/sql-statements/Administration/INSTALL_PLUGIN.md)
|
||||
+ [KILL](./sql-reference/sql-statements/Administration/KILL.md)
|
||||
+ [SET](./sql-reference/sql-statements/Administration/SET.md)
|
||||
+ [SET DEFAULT STORAGE VOLUME](./sql-reference/sql-statements/Administration/SET%20DEFAULT%20STORAGE%20VOLUME.md)
|
||||
+ [SHOW BACKENDS](./sql-reference/sql-statements/Administration/SHOW%20BACKENDS.md)
|
||||
+ [SHOW BROKER](./sql-reference/sql-statements/Administration/SHOW%20BROKER.md)
|
||||
+ [SHOW COMPUTE NODES](./sql-reference/sql-statements/Administration/SHOW%20COMPUTE%20NODES.md)
|
||||
+ [SHOW FILE](./sql-reference/sql-statements/Administration/SHOW%20FILE.md)
|
||||
+ [SHOW FRONTENDS](./sql-reference/sql-statements/Administration/SHOW%20FRONTENDS.md)
|
||||
+ [SHOW FULL COLUMNS](./sql-reference/sql-statements/Administration/SHOW%20FULL%20COLUMNS.md)
|
||||
+ [SHOW INDEX](./sql-reference/sql-statements/Administration/SHOW%20INDEX.md)
|
||||
+ [SHOW PLUGINS](./sql-reference/sql-statements/Administration/SHOW%20PLUGINS.md)
|
||||
+ [SHOW PROC](./sql-reference/sql-statements/Administration/SHOW%20PROC.md)
|
||||
+ [SHOW PROCESSLIST](./sql-reference/sql-statements/Administration/SHOW%20PROCESSLIST.md)
|
||||
+ [SHOW RESOURCE GROUP](./sql-reference/sql-statements/Administration/SHOW%20RESOURCE%20GROUP.md)
|
||||
+ [SHOW SQLBLACKLIST](./sql-reference/sql-statements/Administration/SHOW%20SQLBLACKLIST.md)
|
||||
+ [SHOW STORAGE VOLUMES](./sql-reference/sql-statements/Administration/SHOW%20STORAGE%20VOLUMES.md)
|
||||
+ [SHOW TABLE STATUS](./sql-reference/sql-statements/Administration/SHOW%20TABLE%20STATUS.md)
|
||||
+ [SHOW VARIABLES](./sql-reference/sql-statements/Administration/SHOW%20VARIABLES.md)
|
||||
+ [UNINSTALL PLUGIN](./sql-reference/sql-statements/Administration/UNINSTALL%20PLUGIN.md)
|
||||
+ [SET DEFAULT STORAGE VOLUME](./sql-reference/sql-statements/Administration/SET_DEFAULT_STORAGE_VOLUME.md)
|
||||
+ [SHOW BACKENDS](./sql-reference/sql-statements/Administration/SHOW_BACKENDS.md)
|
||||
+ [SHOW BROKER](./sql-reference/sql-statements/Administration/SHOW_BROKER.md)
|
||||
+ [SHOW COMPUTE NODES](./sql-reference/sql-statements/Administration/SHOW_COMPUTE_NODES.md)
|
||||
+ [SHOW FILE](./sql-reference/sql-statements/Administration/SHOW_FILE.md)
|
||||
+ [SHOW FRONTENDS](./sql-reference/sql-statements/Administration/SHOW_FRONTENDS.md)
|
||||
+ [SHOW FULL COLUMNS](./sql-reference/sql-statements/Administration/SHOW_FULL_COLUMNS.md)
|
||||
+ [SHOW INDEX](./sql-reference/sql-statements/Administration/SHOW_INDEX.md)
|
||||
+ [SHOW PLUGINS](./sql-reference/sql-statements/Administration/SHOW_PLUGINS.md)
|
||||
+ [SHOW PROC](./sql-reference/sql-statements/Administration/SHOW_PROC.md)
|
||||
+ [SHOW PROCESSLIST](./sql-reference/sql-statements/Administration/SHOW_PROCESSLIST.md)
|
||||
+ [SHOW RESOURCE GROUP](./sql-reference/sql-statements/Administration/SHOW_RESOURCE_GROUP.md)
|
||||
+ [SHOW SQLBLACKLIST](./sql-reference/sql-statements/Administration/SHOW_SQLBLACKLIST.md)
|
||||
+ [SHOW STORAGE VOLUMES](./sql-reference/sql-statements/Administration/SHOW_STORAGE_VOLUMES.md)
|
||||
+ [SHOW TABLE STATUS](./sql-reference/sql-statements/Administration/SHOW_TABLE_STATUS.md)
|
||||
+ [SHOW VARIABLES](./sql-reference/sql-statements/Administration/SHOW_VARIABLES.md)
|
||||
+ [UNINSTALL PLUGIN](./sql-reference/sql-statements/Administration/UNINSTALL_PLUGIN.md)
|
||||
+ DDL
|
||||
+ [ALTER DATABASE](./sql-reference/sql-statements/data-definition/ALTER%20DATABASE.md)
|
||||
+ [ALTER MATERIALIZED VIEW](./sql-reference/sql-statements/data-definition/ALTER%20MATERIALIZED%20VIEW.md)
|
||||
+ [ALTER TABLE](./sql-reference/sql-statements/data-definition/ALTER%20TABLE.md)
|
||||
+ [ALTER VIEW](./sql-reference/sql-statements/data-definition/ALTER%20VIEW.md)
|
||||
+ [ALTER RESOURCE](./sql-reference/sql-statements/data-definition/ALTER%20RESOURCE.md)
|
||||
+ [ANALYZE TABLE](./sql-reference/sql-statements/data-definition/ANALYZE%20TABLE.md)
|
||||
+ [ALTER DATABASE](./sql-reference/sql-statements/data-definition/ALTER_DATABASE.md)
|
||||
+ [ALTER MATERIALIZED VIEW](./sql-reference/sql-statements/data-definition/ALTER_MATERIALIZED_VIEW.md)
|
||||
+ [ALTER TABLE](./sql-reference/sql-statements/data-definition/ALTER_TABLE.md)
|
||||
+ [ALTER VIEW](./sql-reference/sql-statements/data-definition/ALTER_VIEW.md)
|
||||
+ [ALTER RESOURCE](./sql-reference/sql-statements/data-definition/ALTER_RESOURCE.md)
|
||||
+ [ANALYZE TABLE](./sql-reference/sql-statements/data-definition/ANALYZE_TABLE.md)
|
||||
+ [BACKUP](./sql-reference/sql-statements/data-definition/BACKUP.md)
|
||||
+ [CANCEL ALTER TABLE](./sql-reference/sql-statements/data-definition/CANCEL%20ALTER%20TABLE.md)
|
||||
+ [CANCEL BACKUP](./sql-reference/sql-statements/data-definition/CANCEL%20BACKUP.md)
|
||||
+ [CANCEL RESTORE](./sql-reference/sql-statements/data-definition/CANCEL%20RESTORE.md)
|
||||
+ [CREATE ANALYZE](./sql-reference/sql-statements/data-definition/CREATE%20ANALYZE.md)
|
||||
+ [CREATE DATABASE](./sql-reference/sql-statements/data-definition/CREATE%20DATABASE.md)
|
||||
+ [CREATE EXTERNAL CATALOG](./sql-reference/sql-statements/data-definition/CREATE%20EXTERNAL%20CATALOG.md)
|
||||
+ [CANCEL ALTER TABLE](./sql-reference/sql-statements/data-definition/CANCEL_ALTER_TABLE.md)
|
||||
+ [CANCEL BACKUP](./sql-reference/sql-statements/data-definition/CANCEL_BACKUP.md)
|
||||
+ [CANCEL RESTORE](./sql-reference/sql-statements/data-definition/CANCEL_RESTORE.md)
|
||||
+ [CREATE ANALYZE](./sql-reference/sql-statements/data-definition/CREATE_ANALYZE.md)
|
||||
+ [CREATE DATABASE](./sql-reference/sql-statements/data-definition/CREATE_DATABASE.md)
|
||||
+ [CREATE EXTERNAL CATALOG](./sql-reference/sql-statements/data-definition/CREATE_EXTERNAL_CATALOG.md)
|
||||
+ [CREATE FUNCTION](./sql-reference/sql-statements/data-definition/create-function.md)
|
||||
+ [CREATE INDEX](./sql-reference/sql-statements/data-definition/CREATE%20INDEX.md)
|
||||
+ [CREATE MATERIALIZED VIEW](./sql-reference/sql-statements/data-definition/CREATE%20MATERIALIZED%20VIEW.md)
|
||||
+ [CREATE REPOSITORY](./sql-reference/sql-statements/data-definition/CREATE%20REPOSITORY.md)
|
||||
+ [CREATE RESOURCE](./sql-reference/sql-statements/data-definition/CREATE%20RESOURCE.md)
|
||||
+ [CREATE TABLE](./sql-reference/sql-statements/data-definition/CREATE%20TABLE.md)
|
||||
+ [CREATE TABLE AS SELECT](./sql-reference/sql-statements/data-definition/CREATE%20TABLE%20AS%20SELECT.md)
|
||||
+ [CREATE TABLE LIKE](./sql-reference/sql-statements/data-definition/CREATE%20TABLE%20LIKE.md)
|
||||
+ [CREATE VIEW](./sql-reference/sql-statements/data-definition/CREATE%20VIEW.md)
|
||||
+ [DROP ANALYZE](./sql-reference/sql-statements/data-definition/DROP%20ANALYZE.md)
|
||||
+ [DROP CATALOG](./sql-reference/sql-statements/data-definition/DROP%20CATALOG.md)
|
||||
+ [DROP DATABASE](./sql-reference/sql-statements/data-definition/DROP%20DATABASE.md)
|
||||
+ [CREATE INDEX](./sql-reference/sql-statements/data-definition/CREATE_INDEX.md)
|
||||
+ [CREATE MATERIALIZED VIEW](./sql-reference/sql-statements/data-definition/CREATE_MATERIALIZED_VIEW.md)
|
||||
+ [CREATE REPOSITORY](./sql-reference/sql-statements/data-definition/CREATE_REPOSITORY.md)
|
||||
+ [CREATE RESOURCE](./sql-reference/sql-statements/data-definition/CREATE_RESOURCE.md)
|
||||
+ [CREATE TABLE](./sql-reference/sql-statements/data-definition/CREATE_TABLE.md)
|
||||
+ [CREATE TABLE AS SELECT](./sql-reference/sql-statements/data-definition/CREATE_TABLE_AS_SELECT.md)
|
||||
+ [CREATE TABLE LIKE](./sql-reference/sql-statements/data-definition/CREATE_TABLE_LIKE.md)
|
||||
+ [CREATE VIEW](./sql-reference/sql-statements/data-definition/CREATE_VIEW.md)
|
||||
+ [DROP ANALYZE](./sql-reference/sql-statements/data-definition/DROP_ANALYZE.md)
|
||||
+ [DROP CATALOG](./sql-reference/sql-statements/data-definition/DROP_CATALOG.md)
|
||||
+ [DROP DATABASE](./sql-reference/sql-statements/data-definition/DROP_DATABASE.md)
|
||||
+ [DROP FUNCTION](./sql-reference/sql-statements/data-definition/drop-function.md)
|
||||
+ [DROP INDEX](./sql-reference/sql-statements/data-definition/DROP%20INDEX.md)
|
||||
+ [DROP MATERIALIZED VIEW](./sql-reference/sql-statements/data-definition/DROP%20MATERIALIZED%20VIEW.md)
|
||||
+ [DROP REPOSITORY](./sql-reference/sql-statements/data-definition/DROP%20REPOSITORY.md)
|
||||
+ [DROP RESOURCE](./sql-reference/sql-statements/data-definition/DROP%20RESOURCE.md)
|
||||
+ [DROP STATS](./sql-reference/sql-statements/data-definition/DROP%20STATS.md)
|
||||
+ [DROP TABLE](./sql-reference/sql-statements/data-definition/DROP%20TABLE.md)
|
||||
+ [DROP VIEW](./sql-reference/sql-statements/data-definition/DROP%20VIEW.md)
|
||||
+ [DROP INDEX](./sql-reference/sql-statements/data-definition/DROP_INDEX.md)
|
||||
+ [DROP MATERIALIZED VIEW](./sql-reference/sql-statements/data-definition/DROP_MATERIALIZED_VIEW.md)
|
||||
+ [DROP REPOSITORY](./sql-reference/sql-statements/data-definition/DROP_REPOSITORY.md)
|
||||
+ [DROP RESOURCE](./sql-reference/sql-statements/data-definition/DROP_RESOURCE.md)
|
||||
+ [DROP STATS](./sql-reference/sql-statements/data-definition/DROP_STATS.md)
|
||||
+ [DROP TABLE](./sql-reference/sql-statements/data-definition/DROP_TABLE.md)
|
||||
+ [DROP VIEW](./sql-reference/sql-statements/data-definition/DROP_VIEW.md)
|
||||
+ [HLL](./sql-reference/sql-statements/data-definition/HLL.md)
|
||||
+ [KILL ANALYZE](./sql-reference/sql-statements/data-definition/KILL%20ANALYZE.md)
|
||||
+ [KILL ANALYZE](./sql-reference/sql-statements/data-definition/KILL_ANALYZE.md)
|
||||
+ [RECOVER](./sql-reference/sql-statements/data-definition/RECOVER.md)
|
||||
+ [REFRESH EXTERNAL TABLE](./sql-reference/sql-statements/data-definition/REFRESH%20EXTERNAL%20TABLE.md)
|
||||
+ [REFRESH EXTERNAL TABLE](./sql-reference/sql-statements/data-definition/REFRESH_EXTERNAL_TABLE.md)
|
||||
+ [RESTORE](./sql-reference/sql-statements/data-definition/RESTORE.md)
|
||||
+ [SET CATALOG](./sql-reference/sql-statements/data-definition/SET%20CATALOG.md)
|
||||
+ [SHOW ANALYZE JOB](./sql-reference/sql-statements/data-definition/SHOW%20ANALYZE%20JOB.md)
|
||||
+ [SHOW ANALYZE STATUS](./sql-reference/sql-statements/data-definition/SHOW%20ANALYZE%20STATUS.md)
|
||||
+ [SET CATALOG](./sql-reference/sql-statements/data-definition/SET_CATALOG.md)
|
||||
+ [SHOW ANALYZE JOB](./sql-reference/sql-statements/data-definition/SHOW_ANALYZE_JOB.md)
|
||||
+ [SHOW ANALYZE STATUS](./sql-reference/sql-statements/data-definition/SHOW_ANALYZE_STATUS.md)
|
||||
+ [SHOW FUNCTION](./sql-reference/sql-statements/data-definition/show-functions.md)
|
||||
+ [SHOW META](./sql-reference/sql-statements/data-definition/SHOW%20META.md)
|
||||
+ [SHOW RESOURCES](./sql-reference/sql-statements/data-definition/SHOW%20RESOURCES.md)
|
||||
+ [TRUNCATE TABLE](./sql-reference/sql-statements/data-definition/TRUNCATE%20TABLE.md)
|
||||
+ [SHOW META](./sql-reference/sql-statements/data-definition/SHOW_META.md)
|
||||
+ [SHOW RESOURCES](./sql-reference/sql-statements/data-definition/SHOW_RESOURCES.md)
|
||||
+ [TRUNCATE TABLE](./sql-reference/sql-statements/data-definition/TRUNCATE_TABLE.md)
|
||||
+ [USE](./sql-reference/sql-statements/data-definition/USE.md)
|
||||
+ DML
|
||||
+ [ALTER LOAD](./sql-reference/sql-statements/data-manipulation/ALTER%20LOAD.md)
|
||||
+ [ALTER LOAD](./sql-reference/sql-statements/data-manipulation/ALTER_LOAD.md)
|
||||
+ [ALTER ROUTINE LOAD](./sql-reference/sql-statements/data-manipulation/alter-routine-load.md)
|
||||
+ [BROKER LOAD](./sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md)
|
||||
+ [CANCEL LOAD](./sql-reference/sql-statements/data-manipulation/CANCEL%20LOAD.md)
|
||||
+ [CANCEL EXPORT](./sql-reference/sql-statements/data-manipulation/CANCEL%20EXPORT.md)
|
||||
+ [CANCEL REFRESH MATERIALIZED VIEW](./sql-reference/sql-statements/data-manipulation/CANCEL%20REFRESH%20MATERIALIZED%20VIEW.md)
|
||||
+ [CREATE ROUTINE LOAD](./sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md)
|
||||
+ [BROKER LOAD](./sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md)
|
||||
+ [CANCEL LOAD](./sql-reference/sql-statements/data-manipulation/CANCEL_LOAD.md)
|
||||
+ [CANCEL EXPORT](./sql-reference/sql-statements/data-manipulation/CANCEL_EXPORT.md)
|
||||
+ [CANCEL REFRESH MATERIALIZED VIEW](./sql-reference/sql-statements/data-manipulation/CANCEL_REFRESH_MATERIALIZED_VIEW.md)
|
||||
+ [CREATE ROUTINE LOAD](./sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md)
|
||||
+ [DELETE](./sql-reference/sql-statements/data-manipulation/DELETE.md)
|
||||
+ [DROP TASK](./sql-reference/sql-statements/data-manipulation/DROP%20TASK.md)
|
||||
+ [DROP TASK](./sql-reference/sql-statements/data-manipulation/DROP_TASK.md)
|
||||
+ [EXPORT](./sql-reference/sql-statements/data-manipulation/EXPORT.md)
|
||||
+ [GROUP BY](./sql-reference/sql-statements/data-manipulation/GROUP%20BY.md)
|
||||
+ [GROUP BY](./sql-reference/sql-statements/data-manipulation/GROUP_BY.md)
|
||||
+ [INSERT](./sql-reference/sql-statements/data-manipulation/insert.md)
|
||||
+ [PAUSE ROUTINE LOAD](./sql-reference/sql-statements/data-manipulation/PAUSE%20ROUTINE%20LOAD.md)
|
||||
+ [REFRESH MATERIALIZED VIEW](./sql-reference/sql-statements/data-manipulation/REFRESH%20MATERIALIZED%20VIEW.md)
|
||||
+ [RESUME ROUTINE LOAD](./sql-reference/sql-statements/data-manipulation/RESUME%20ROUTINE%20LOAD.md)
|
||||
+ [PAUSE ROUTINE LOAD](./sql-reference/sql-statements/data-manipulation/PAUSE_ROUTINE_LOAD.md)
|
||||
+ [REFRESH MATERIALIZED VIEW](./sql-reference/sql-statements/data-manipulation/REFRESH_MATERIALIZED_VIEW.md)
|
||||
+ [RESUME ROUTINE LOAD](./sql-reference/sql-statements/data-manipulation/RESUME_ROUTINE_LOAD.md)
|
||||
+ [SELECT](./sql-reference/sql-statements/data-manipulation/SELECT.md)
|
||||
+ [SHOW ALTER TABLE](./sql-reference/sql-statements/data-manipulation/SHOW%20ALTER.md)
|
||||
+ [SHOW ALTER MATERIALIZED VIEW](./sql-reference/sql-statements/data-manipulation/SHOW%20ALTER%20MATERIALIZED%20VIEW.md)
|
||||
+ [SHOW BACKUP](./sql-reference/sql-statements/data-manipulation/SHOW%20BACKUP.md)
|
||||
+ [SHOW CATALOGS](./sql-reference/sql-statements/data-manipulation/SHOW%20CATALOGS.md)
|
||||
+ [SHOW CREATE CATALOG](./sql-reference/sql-statements/data-manipulation/SHOW%20CREATE%20CATALOG.md)
|
||||
+ [SHOW CREATE DATABASE](./sql-reference/sql-statements/data-manipulation/SHOW%20CREATE%20DATABASE.md)
|
||||
+ [SHOW CREATE MATERIALIZED VIEW](./sql-reference/sql-statements/data-manipulation/SHOW%20CREATE%20MATERIALIZED%20VIEW.md)
|
||||
+ [SHOW CREATE TABLE](./sql-reference/sql-statements/data-manipulation/SHOW%20CREATE%20TABLE.md)
|
||||
+ [SHOW CREATE VIEW](./sql-reference/sql-statements/data-manipulation/SHOW%20CREATE%20VIEW.md)
|
||||
+ [SHOW DATA](./sql-reference/sql-statements/data-manipulation/SHOW%20DATA.md)
|
||||
+ [SHOW DATABASES](./sql-reference/sql-statements/data-manipulation/SHOW%20DATABASES.md)
|
||||
+ [SHOW DELETE](./sql-reference/sql-statements/data-manipulation/SHOW%20DELETE.md)
|
||||
+ [SHOW DYNAMIC PARTITION TABLES](./sql-reference/sql-statements/data-manipulation/SHOW%20DYNAMIC%20PARTITION%20TABLES.md)
|
||||
+ [SHOW EXPORT](./sql-reference/sql-statements/data-manipulation/SHOW%20EXPORT.md)
|
||||
+ [SHOW LOAD](./sql-reference/sql-statements/data-manipulation/SHOW%20LOAD.md)
|
||||
+ [SHOW MATERIALIZED VIEWS](./sql-reference/sql-statements/data-manipulation/SHOW%20MATERIALIZED%20VIEW.md)
|
||||
+ [SHOW PARTITIONS](./sql-reference/sql-statements/data-manipulation/SHOW%20PARTITIONS.md)
|
||||
+ [SHOW PROPERTY](./sql-reference/sql-statements/data-manipulation/SHOW%20PROPERTY.md)
|
||||
+ [SHOW REPOSITORIES](./sql-reference/sql-statements/data-manipulation/SHOW%20REPOSITORIES.md)
|
||||
+ [SHOW RESTORE](./sql-reference/sql-statements/data-manipulation/SHOW%20RESTORE.md)
|
||||
+ [SHOW ROUTINE LOAD](./sql-reference/sql-statements/data-manipulation/SHOW%20ROUTINE%20LOAD.md)
|
||||
+ [SHOW ROUTINE LOAD TASK](./sql-reference/sql-statements/data-manipulation/SHOW%20ROUTINE%20LOAD%20TASK.md)
|
||||
+ [SHOW SNAPSHOT](./sql-reference/sql-statements/data-manipulation/SHOW%20SNAPSHOT.md)
|
||||
+ [SHOW TABLES](./sql-reference/sql-statements/data-manipulation/SHOW%20TABLES.md)
|
||||
+ [SHOW TABLET](./sql-reference/sql-statements/data-manipulation/SHOW%20TABLET.md)
|
||||
+ [SHOW TRANSACTION](./sql-reference/sql-statements/data-manipulation/SHOW%20TRANSACTION.md)
|
||||
+ [SPARK LOAD](./sql-reference/sql-statements/data-manipulation/SPARK%20LOAD.md)
|
||||
+ [STOP ROUTINE LOAD](./sql-reference/sql-statements/data-manipulation/STOP%20ROUTINE%20LOAD.md)
|
||||
+ [STREAM LOAD](./sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md)
|
||||
+ [SUBMIT TASK](./sql-reference/sql-statements/data-manipulation/SUBMIT%20TASK.md)
|
||||
+ [SHOW ALTER TABLE](./sql-reference/sql-statements/data-manipulation/SHOW_ALTER.md)
|
||||
+ [SHOW ALTER MATERIALIZED VIEW](./sql-reference/sql-statements/data-manipulation/SHOW_ALTER_MATERIALIZED_VIEW.md)
|
||||
+ [SHOW BACKUP](./sql-reference/sql-statements/data-manipulation/SHOW_BACKUP.md)
|
||||
+ [SHOW CATALOGS](./sql-reference/sql-statements/data-manipulation/SHOW_CATALOGS.md)
|
||||
+ [SHOW CREATE CATALOG](./sql-reference/sql-statements/data-manipulation/SHOW_CREATE_CATALOG.md)
|
||||
+ [SHOW CREATE DATABASE](./sql-reference/sql-statements/data-manipulation/SHOW_CREATE_DATABASE.md)
|
||||
+ [SHOW CREATE MATERIALIZED VIEW](./sql-reference/sql-statements/data-manipulation/SHOW_CREATE_MATERIALIZED_VIEW.md)
|
||||
+ [SHOW CREATE TABLE](./sql-reference/sql-statements/data-manipulation/SHOW_CREATE_TABLE.md)
|
||||
+ [SHOW CREATE VIEW](./sql-reference/sql-statements/data-manipulation/SHOW_CREATE_VIEW.md)
|
||||
+ [SHOW DATA](./sql-reference/sql-statements/data-manipulation/SHOW_DATA.md)
|
||||
+ [SHOW DATABASES](./sql-reference/sql-statements/data-manipulation/SHOW_DATABASES.md)
|
||||
+ [SHOW DELETE](./sql-reference/sql-statements/data-manipulation/SHOW_DELETE.md)
|
||||
+ [SHOW DYNAMIC PARTITION TABLES](./sql-reference/sql-statements/data-manipulation/SHOW_DYNAMIC_PARTITION_TABLES.md)
|
||||
+ [SHOW EXPORT](./sql-reference/sql-statements/data-manipulation/SHOW_EXPORT.md)
|
||||
+ [SHOW LOAD](./sql-reference/sql-statements/data-manipulation/SHOW_LOAD.md)
|
||||
+ [SHOW MATERIALIZED VIEWS](./sql-reference/sql-statements/data-manipulation/SHOW_MATERIALIZED_VIEW.md)
|
||||
+ [SHOW PARTITIONS](./sql-reference/sql-statements/data-manipulation/SHOW_PARTITIONS.md)
|
||||
+ [SHOW PROPERTY](./sql-reference/sql-statements/data-manipulation/SHOW_PROPERTY.md)
|
||||
+ [SHOW REPOSITORIES](./sql-reference/sql-statements/data-manipulation/SHOW_REPOSITORIES.md)
|
||||
+ [SHOW RESTORE](./sql-reference/sql-statements/data-manipulation/SHOW_RESTORE.md)
|
||||
+ [SHOW ROUTINE LOAD](./sql-reference/sql-statements/data-manipulation/SHOW_ROUTINE_LOAD.md)
|
||||
+ [SHOW ROUTINE LOAD TASK](./sql-reference/sql-statements/data-manipulation/SHOW_ROUTINE_LOAD_TASK.md)
|
||||
+ [SHOW SNAPSHOT](./sql-reference/sql-statements/data-manipulation/SHOW_SNAPSHOT.md)
|
||||
+ [SHOW TABLES](./sql-reference/sql-statements/data-manipulation/SHOW_TABLES.md)
|
||||
+ [SHOW TABLET](./sql-reference/sql-statements/data-manipulation/SHOW_TABLET.md)
|
||||
+ [SHOW TRANSACTION](./sql-reference/sql-statements/data-manipulation/SHOW_TRANSACTION.md)
|
||||
+ [SPARK LOAD](./sql-reference/sql-statements/data-manipulation/SPARK_LOAD.md)
|
||||
+ [STOP ROUTINE LOAD](./sql-reference/sql-statements/data-manipulation/STOP_ROUTINE_LOAD.md)
|
||||
+ [STREAM LOAD](./sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md)
|
||||
+ [SUBMIT TASK](./sql-reference/sql-statements/data-manipulation/SUBMIT_TASK.md)
|
||||
+ [UPDATE](./sql-reference/sql-statements/data-manipulation/UPDATE.md)
|
||||
+ Auxiliary Commands
|
||||
+ [DESC](./sql-reference/sql-statements/Utility/DESCRIBE.md)
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@ If you have stored a large amount of data in a table, we recommend that you back
|
|||
|
||||
### Create a repository
|
||||
|
||||
Before backing up data, you need to create a repository, which is used to store data snapshots in a remote storage system. You can create multiple repositories in a StarRocks cluster. For detailed instructions, see [CREATE REPOSITORY](../sql-reference/sql-statements/data-definition/CREATE%20REPOSITORY.md).
|
||||
Before backing up data, you need to create a repository, which is used to store data snapshots in a remote storage system. You can create multiple repositories in a StarRocks cluster. For detailed instructions, see [CREATE REPOSITORY](../sql-reference/sql-statements/data-definition/CREATE_REPOSITORY.md).
|
||||
|
||||
- Create a repository in HDFS
|
||||
|
||||
|
|
@ -99,7 +99,7 @@ PROPERTIES(
|
|||
>
|
||||
> StarRocks supports creating repositories in Google GCS only according to the S3A protocol. Therefore, when you create repositories in Google GCS, you must replace the prefix in the GCS URI you pass as a repository location in `ON LOCATION` with `s3a://`.
|
||||
|
||||
After the repository is created, you can check the repository via [SHOW REPOSITORIES](../sql-reference/sql-statements/data-manipulation/SHOW%20REPOSITORIES.md). After restoring data, you can delete the repository in StarRocks using [DROP REPOSITORY](../sql-reference/sql-statements/data-definition/DROP%20REPOSITORY.md). However, data snapshots backed up in the remote storage system cannot be deleted through StarRocks. You need to delete them manually in the remote storage system.
|
||||
After the repository is created, you can check the repository via [SHOW REPOSITORIES](../sql-reference/sql-statements/data-manipulation/SHOW_REPOSITORIES.md). After restoring data, you can delete the repository in StarRocks using [DROP REPOSITORY](../sql-reference/sql-statements/data-definition/DROP_REPOSITORY.md). However, data snapshots backed up in the remote storage system cannot be deleted through StarRocks. You need to delete them manually in the remote storage system.
|
||||
|
||||
### Back up a data snapshot
|
||||
|
||||
|
|
@ -113,7 +113,7 @@ TO test_repo
|
|||
ON (sr_member);
|
||||
```
|
||||
|
||||
BACKUP is an asynchronous operation. You can check the status of a BACKUP job using [SHOW BACKUP](../sql-reference/sql-statements/data-manipulation/SHOW%20BACKUP.md), or cancel a BACKUP job using [CANCEL BACKUP](../sql-reference/sql-statements/data-definition/CANCEL%20BACKUP.md).
|
||||
BACKUP is an asynchronous operation. You can check the status of a BACKUP job using [SHOW BACKUP](../sql-reference/sql-statements/data-manipulation/SHOW_BACKUP.md), or cancel a BACKUP job using [CANCEL BACKUP](../sql-reference/sql-statements/data-definition/CANCEL_BACKUP.md).
|
||||
|
||||
## Restore or migrate data
|
||||
|
||||
|
|
@ -125,7 +125,7 @@ To migrate data to another StarRocks cluster, you need to create a repository wi
|
|||
|
||||
### Check the snapshot
|
||||
|
||||
Before restoring data, you can check the snapshots in a specified repository using [SHOW SNAPSHOT](../sql-reference/sql-statements/data-manipulation/SHOW%20SNAPSHOT.md).
|
||||
Before restoring data, you can check the snapshots in a specified repository using [SHOW SNAPSHOT](../sql-reference/sql-statements/data-manipulation/SHOW_SNAPSHOT.md).
|
||||
|
||||
The following example checks the snapshot information in `test_repo`.
|
||||
|
||||
|
|
@ -155,7 +155,7 @@ PROPERTIES (
|
|||
);
|
||||
```
|
||||
|
||||
RESTORE is an asynchronous operation. You can check the status of a RESTORE job using [SHOW RESTORE](../sql-reference/sql-statements/data-manipulation/SHOW%20RESTORE.md), or cancel a RESTORE job using [CANCEL RESTORE](../sql-reference/sql-statements/data-definition/CANCEL%20RESTORE.md).
|
||||
RESTORE is an asynchronous operation. You can check the status of a RESTORE job using [SHOW RESTORE](../sql-reference/sql-statements/data-manipulation/SHOW_RESTORE.md), or cancel a RESTORE job using [CANCEL RESTORE](../sql-reference/sql-statements/data-definition/CANCEL_RESTORE.md).
|
||||
|
||||
## Configure BACKUP or RESTORE jobs
|
||||
|
||||
|
|
|
|||
|
|
@ -407,7 +407,7 @@ Q: I have recently upgraded StarRocks v2.0 to v2.1 or later versions. When I loa
|
|||
|
||||
A: StarRocks v2.0 parses all columns as strings and then performs type conversion for loading. When you load BOOLEAN type data (`true` and `false`) in JSON format into an integer column, StarRocks v2.0 converts the data into `0` and `1` for loading. StarRocks v2.1 refactored its JSON Paerser, which directly extracts the JSON fields according to the target column type, resulting in this problem.
|
||||
|
||||
You can solve this problem by adding the expression `tmp, target=if(tmp,1,0)` to the `columns` parameter of the [Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md) command. The complete command is as follows:
|
||||
You can solve this problem by adding the expression `tmp, target=if(tmp,1,0)` to the `columns` parameter of the [Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md) command. The complete command is as follows:
|
||||
|
||||
```shell
|
||||
curl --location-trusted -u <username>:<password> \
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ FE parameters are classified into dynamic parameters and static parameters.
|
|||
|
||||
- Static parameters can only be configured and adjusted in the FE configuration file **fe.conf**. **After you modify this file, you must restart your FE for the changes to take effect.**
|
||||
|
||||
Whether a parameter is a dynamic parameter is indicated by the `IsMutable` column in the output of [ADMIN SHOW CONFIG](../sql-reference/sql-statements/Administration/ADMIN%20SHOW%20CONFIG.md). `TRUE` indicates a dynamic parameter.
|
||||
Whether a parameter is a dynamic parameter is indicated by the `IsMutable` column in the output of [ADMIN SHOW CONFIG](../sql-reference/sql-statements/Administration/ADMIN_SHOW_CONFIG.md). `TRUE` indicates a dynamic parameter.
|
||||
|
||||
Note that both dynamic and static FE parameters can be configured in the **fe.conf** file.
|
||||
|
||||
|
|
@ -22,7 +22,7 @@ After your FE is started, you can run the ADMIN SHOW FRONTEND CONFIG command on
|
|||
ADMIN SHOW FRONTEND CONFIG [LIKE "pattern"];
|
||||
```
|
||||
|
||||
For detailed description of the returned fields, see [ADMIN SHOW CONFIG](../sql-reference/sql-statements/Administration/ADMIN%20SHOW%20CONFIG.md).
|
||||
For detailed description of the returned fields, see [ADMIN SHOW CONFIG](../sql-reference/sql-statements/Administration/ADMIN_SHOW_CONFIG.md).
|
||||
|
||||
> **NOTE**
|
||||
>
|
||||
|
|
@ -30,7 +30,7 @@ After your FE is started, you can run the ADMIN SHOW FRONTEND CONFIG command on
|
|||
|
||||
### Configure FE dynamic parameters
|
||||
|
||||
You can configure or modify the settings of FE dynamic parameters using [ADMIN SET FRONTEND CONFIG](../sql-reference/sql-statements/Administration/ADMIN%20SET%20CONFIG.md).
|
||||
You can configure or modify the settings of FE dynamic parameters using [ADMIN SET FRONTEND CONFIG](../sql-reference/sql-statements/Administration/ADMIN_SET_CONFIG.md).
|
||||
|
||||
```SQL
|
||||
ADMIN SET FRONTEND CONFIG ("key" = "value");
|
||||
|
|
@ -409,13 +409,13 @@ ADMIN SET FRONTEND CONFIG ("key" = "value");
|
|||
|
||||
- **Unit**: s
|
||||
- **Default**: 15
|
||||
- **Description**: The maximum time for each Routine Load task within the cluster to consume data. Since v3.1.0, Routine Load job supports a new parameter `task_consume_second` in [job_properties](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md#job_properties). This parameter applies to individual load tasks within a Routine Load job, which is more flexible.
|
||||
- **Description**: The maximum time for each Routine Load task within the cluster to consume data. Since v3.1.0, Routine Load job supports a new parameter `task_consume_second` in [job_properties](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md#job_properties). This parameter applies to individual load tasks within a Routine Load job, which is more flexible.
|
||||
|
||||
##### routine_load_task_timeout_second
|
||||
|
||||
- **Unit**: s
|
||||
- **Default**: 60
|
||||
- **Description**: The timeout duration for each Routine Load task within the cluster. Since v3.1.0, Routine Load job supports a new parameter `task_timeout_second` in [job_properties](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md#job_properties). This parameter applies to individual load tasks within a Routine Load job, which is more flexible.
|
||||
- **Description**: The timeout duration for each Routine Load task within the cluster. Since v3.1.0, Routine Load job supports a new parameter `task_timeout_second` in [job_properties](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md#job_properties). This parameter applies to individual load tasks within a Routine Load job, which is more flexible.
|
||||
|
||||
##### max_tolerable_backend_down_num
|
||||
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ Users with the system-defined role `user_admin` can view all the user and role i
|
|||
|
||||
### View privilege information
|
||||
|
||||
You can view the privileges granted to a user or a role using [SHOW GRANTS](../sql-reference/sql-statements/account-management/SHOW%20GRANTS.md).
|
||||
You can view the privileges granted to a user or a role using [SHOW GRANTS](../sql-reference/sql-statements/account-management/SHOW_GRANTS.md).
|
||||
|
||||
- View the privileges of the current user.
|
||||
|
||||
|
|
@ -42,7 +42,7 @@ You can view the privileges granted to a user or a role using [SHOW GRANTS](../s
|
|||
|
||||
### View user property
|
||||
|
||||
You can view the property of a user using [SHOW PROPERTY](../sql-reference/sql-statements/account-management/SET%20PROPERTY.md).
|
||||
You can view the property of a user using [SHOW PROPERTY](../sql-reference/sql-statements/account-management/SET_PROPERTY.md).
|
||||
|
||||
The following example shows the property of the user `jack`:
|
||||
|
||||
|
|
@ -52,7 +52,7 @@ SHOW PROPERTY FOR jack@'172.10.1.10';
|
|||
|
||||
### View roles
|
||||
|
||||
You can view all the roles within the StarRocks cluster using [SHOW ROLES](../sql-reference/sql-statements/account-management/SHOW%20ROLES.md).
|
||||
You can view all the roles within the StarRocks cluster using [SHOW ROLES](../sql-reference/sql-statements/account-management/SHOW_ROLES.md).
|
||||
|
||||
```SQL
|
||||
SHOW ROLES;
|
||||
|
|
@ -74,7 +74,7 @@ Users with the system-defined role `user_admin` can create users, alter users, a
|
|||
|
||||
You can create a user by specifying the user identity, authentication method, and default role.
|
||||
|
||||
StarRocks supports user authentication with login credentials or LDAP authentication. For more information about StarRocks' authentication, see [Authentication](../administration/Authentication.md). For more information and advanced instructions on creating a user, see [CREATE USER](../sql-reference/sql-statements/account-management/CREATE%20USER.md).
|
||||
StarRocks supports user authentication with login credentials or LDAP authentication. For more information about StarRocks' authentication, see [Authentication](../administration/Authentication.md). For more information and advanced instructions on creating a user, see [CREATE USER](../sql-reference/sql-statements/account-management/CREATE_USER.md).
|
||||
|
||||
The following example creates the user `jack`, allows it to connect only from the IP address `172.10.1.10`, sets the password to `12345` for it, and assigns the role `example_role` to it as its default role:
|
||||
|
||||
|
|
@ -95,7 +95,7 @@ The default role of a user is automatically activated when the user connects to
|
|||
|
||||
#### Alter the default role of a user
|
||||
|
||||
You can set the default role of a user using [SET DEFAULT ROLE](../sql-reference/sql-statements/account-management/SET_DEFAULT_ROLE.md) or [ALTER USER](../sql-reference/sql-statements/account-management/ALTER%20USER.md).
|
||||
You can set the default role of a user using [SET DEFAULT ROLE](../sql-reference/sql-statements/account-management/SET_DEFAULT_ROLE.md) or [ALTER USER](../sql-reference/sql-statements/account-management/ALTER_USER.md).
|
||||
|
||||
Both of the following examples set the default role of `jack` to `db1_admin`. Note that `db1_admin` must have been assigned to `jack`.
|
||||
|
||||
|
|
@ -113,7 +113,7 @@ Both of the following examples set the default role of `jack` to `db1_admin`. No
|
|||
|
||||
#### Alter the property of a user
|
||||
|
||||
You can set the property of a user using [SET PROPERTY](../sql-reference/sql-statements/account-management/SET%20PROPERTY.md).
|
||||
You can set the property of a user using [SET PROPERTY](../sql-reference/sql-statements/account-management/SET_PROPERTY.md).
|
||||
|
||||
The following example sets the maximum number of connections for user `jack` to `1000`. User identities that have the same user name share the same property.
|
||||
|
||||
|
|
@ -125,7 +125,7 @@ SET PROPERTY FOR jack 'max_user_connections' = '1000';
|
|||
|
||||
#### Reset password for a user
|
||||
|
||||
You can reset the password for a user using [SET PASSWORD](../sql-reference/sql-statements/account-management/SET%20PASSWORD.md) or [ALTER USER](../sql-reference/sql-statements/account-management/ALTER%20USER.md).
|
||||
You can reset the password for a user using [SET PASSWORD](../sql-reference/sql-statements/account-management/SET_PASSWORD.md) or [ALTER USER](../sql-reference/sql-statements/account-management/ALTER_USER.md).
|
||||
|
||||
> **NOTE**
|
||||
>
|
||||
|
|
@ -196,7 +196,7 @@ If you have lost the password of the `root` user and cannot connect to StarRocks
|
|||
|
||||
### Drop a user
|
||||
|
||||
You can drop a user using [DROP USER](../sql-reference/sql-statements/account-management/DROP%20USER.md).
|
||||
You can drop a user using [DROP USER](../sql-reference/sql-statements/account-management/DROP_USER.md).
|
||||
|
||||
The following example drops the user `jack`:
|
||||
|
||||
|
|
@ -210,7 +210,7 @@ Users with the system-defined role `user_admin` can create, grant, revoke, or dr
|
|||
|
||||
### Create a role
|
||||
|
||||
You can create a role using [CREATE ROLE](../sql-reference/sql-statements/account-management/CREATE%20ROLE.md).
|
||||
You can create a role using [CREATE ROLE](../sql-reference/sql-statements/account-management/CREATE_ROLE.md).
|
||||
|
||||
The following example creates the role `example_role`:
|
||||
|
||||
|
|
@ -264,7 +264,7 @@ You can revoke roles from a user or another role using [REVOKE](../sql-reference
|
|||
|
||||
### Drop a role
|
||||
|
||||
You can drop a role using [DROP ROLE](../sql-reference/sql-statements/account-management/DROP%20ROLE.md).
|
||||
You can drop a role using [DROP ROLE](../sql-reference/sql-statements/account-management/DROP_ROLE.md).
|
||||
|
||||
The following example drops the role `example_role`:
|
||||
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ StarRocks stores its audit logs in the local file **fe/log/fe.audit.log** rather
|
|||
|
||||
## Create a table to store audit logs
|
||||
|
||||
Create a database and a table in your StarRocks cluster to store its audit logs. See [CREATE DATABASE](../sql-reference/sql-statements/data-definition/CREATE%20DATABASE.md) and [CREATE TABLE](../sql-reference/sql-statements/data-definition/CREATE%20TABLE.md) for detailed instructions.
|
||||
Create a database and a table in your StarRocks cluster to store its audit logs. See [CREATE DATABASE](../sql-reference/sql-statements/data-definition/CREATE_DATABASE.md) and [CREATE TABLE](../sql-reference/sql-statements/data-definition/CREATE_TABLE.md) for detailed instructions.
|
||||
|
||||
Because the fields of audit logs vary among different StarRocks versions, you must choose among the following examples to create a table that is compatible with your StarRocks.
|
||||
|
||||
|
|
@ -261,11 +261,11 @@ Execute the following statement along with the path you copied to install Audit
|
|||
INSTALL PLUGIN FROM "<absolute_path_to_package>";
|
||||
```
|
||||
|
||||
See [INSTALL PLUGIN](../sql-reference/sql-statements/Administration/INSTALL%20PLUGIN.md) for detailed instructions.
|
||||
See [INSTALL PLUGIN](../sql-reference/sql-statements/Administration/INSTALL_PLUGIN.md) for detailed instructions.
|
||||
|
||||
## Verify the installation and query audit logs
|
||||
|
||||
1. You can check if the installation is successful via [SHOW PLUGINS](../sql-reference/sql-statements/Administration/SHOW%20PLUGINS.md).
|
||||
1. You can check if the installation is successful via [SHOW PLUGINS](../sql-reference/sql-statements/Administration/SHOW_PLUGINS.md).
|
||||
|
||||
In the following example, the `Status` of the plugin `AuditLoader` is `INSTALLED`, meaning installation is successful.
|
||||
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ With file manager, you can create, view, and delete files, such as the files tha
|
|||
|
||||
- Configure the following parameters for each FE.
|
||||
- `small_file_dir`: the path in which the uploaded files are stored. The default path is `small_files/`, which is in the runtime directory of the FE. You need to specify this parameter in the **fe.conf** file and then restart the FE to allow the change to take effect.
|
||||
- `max_small_file_size_bytes`: the maximum size of a single file. The default value of this parameter is 1 MB. If the size of a file exceeds the value of this parameter, the file cannot be created. You can specify this parameter by using the [ADMIN SET CONFIG](../sql-reference/sql-statements/Administration/ADMIN%20SET%20CONFIG.md) statement.
|
||||
- `max_small_file_size_bytes`: the maximum size of a single file. The default value of this parameter is 1 MB. If the size of a file exceeds the value of this parameter, the file cannot be created. You can specify this parameter by using the [ADMIN SET CONFIG](../sql-reference/sql-statements/Administration/ADMIN_SET_CONFIG.md) statement.
|
||||
- `max_small_file_number`: the maximum number of files that can be created within a cluster. The default value of this parameter is 100. If the number of files that you have created reaches the value of this parameter, you cannot continue creating files. You can specify this parameter by using the ADMIN SET CONFIG statement.
|
||||
|
||||
> Note: Increasing the values of the two parameters causes an increase in the memory usage of the FE. Therefore, we recommend that you do not increase the values of the two parameters unless necessary.
|
||||
|
|
@ -21,15 +21,15 @@ With file manager, you can create, view, and delete files, such as the files tha
|
|||
|
||||
## Create a file
|
||||
|
||||
You can execute the CREATE FILE statement to create a file. For more information, see [CREATE FILE](../sql-reference/sql-statements/Administration/CREATE%20FILE.md). After a file is created, the file is uploaded and persisted in StarRocks.
|
||||
You can execute the CREATE FILE statement to create a file. For more information, see [CREATE FILE](../sql-reference/sql-statements/Administration/CREATE_FILE.md). After a file is created, the file is uploaded and persisted in StarRocks.
|
||||
|
||||
## View a file
|
||||
|
||||
You can execute the SHOW FILE statement to view the information about a file stored in a database. For more information, see [SHOW FILE](../sql-reference/sql-statements/Administration/SHOW%20FILE.md).
|
||||
You can execute the SHOW FILE statement to view the information about a file stored in a database. For more information, see [SHOW FILE](../sql-reference/sql-statements/Administration/SHOW_FILE.md).
|
||||
|
||||
## Delete a file
|
||||
|
||||
You can execute the DROP FILE statement to delete a file. For more information, see [DROP FILE](../sql-reference/sql-statements/Administration/DROP%20FILE.md).
|
||||
You can execute the DROP FILE statement to delete a file. For more information, see [DROP FILE](../sql-reference/sql-statements/Administration/DROP_FILE.md).
|
||||
|
||||
## How an FE and a BE use a file
|
||||
|
||||
|
|
|
|||
|
|
@ -44,7 +44,7 @@ StarRocks has optimized the metadata information provided by the tables `tables`
|
|||
| [tables](#tables) | Provides general metadata information of tables. |
|
||||
| [tables_config](#tables_config) | Provides additional table metadata information that is unique to StarRocks. |
|
||||
| [load_tracking_logs](#load_tracking_logs) | Provides error information (if any) of load jobs. |
|
||||
| [loads](#loads) | Provides the results of load jobs. This table is supported from v3.1 onwards. Currently, you can only view the results of [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md) and [Insert](../sql-reference/sql-statements/data-manipulation/insert.md) jobs from this table. |
|
||||
| [loads](#loads) | Provides the results of load jobs. This table is supported from v3.1 onwards. Currently, you can only view the results of [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md) and [Insert](../sql-reference/sql-statements/data-manipulation/insert.md) jobs from this table. |
|
||||
|
||||
### loads
|
||||
|
||||
|
|
|
|||
|
|
@ -103,7 +103,7 @@ You can also decide how to deal with these queued queries by configuring the max
|
|||
SET GLOBAL query_queue_pending_timeout_second = 480;
|
||||
```
|
||||
|
||||
You can check whether a query is pending using [SHOW PROCESSLIST](../sql-reference/sql-statements/Administration/SHOW%20PROCESSLIST.md).
|
||||
You can check whether a query is pending using [SHOW PROCESSLIST](../sql-reference/sql-statements/Administration/SHOW_PROCESSLIST.md).
|
||||
|
||||
```Plain
|
||||
mysql> SHOW PROCESSLIST;
|
||||
|
|
@ -122,7 +122,7 @@ From v3.0 onwards, StarRocks supports viewing the queries that are currently pro
|
|||
|
||||
### Monitor via MySQL client
|
||||
|
||||
1. You can view the queries that are currently processed (`current_queries`) using [SHOW PROC](../sql-reference/sql-statements/Administration/SHOW%20PROC.md).
|
||||
1. You can view the queries that are currently processed (`current_queries`) using [SHOW PROC](../sql-reference/sql-statements/Administration/SHOW_PROC.md).
|
||||
|
||||
```SQL
|
||||
SHOW PROC '/current_queries';
|
||||
|
|
@ -243,7 +243,7 @@ To enable SQL Blacklist, execute the following statement:
|
|||
ADMIN SET FRONTEND CONFIG ("enable_sql_blacklist" = "true");
|
||||
```
|
||||
|
||||
Then you can add the regular expression that represents the SQL pattern to SQL Blacklist using [ADD SQLBLACKLIST](../sql-reference/sql-statements/Administration/ADD%20SQLBLACKLIST.md).
|
||||
Then you can add the regular expression that represents the SQL pattern to SQL Blacklist using [ADD SQLBLACKLIST](../sql-reference/sql-statements/Administration/ADD_SQLBLACKLIST.md).
|
||||
|
||||
The following example adds `COUNT(DISTINCT)` to SQL Blacklist:
|
||||
|
||||
|
|
|
|||
|
|
@ -2,9 +2,9 @@
|
|||
|
||||
## Why is the error message "no permission" still reported even after the required role has been assigned to a user?
|
||||
|
||||
This error may happen if the role is not activated. You can run `select current_role();` to query the roles that have been activated for the user in the current session. If the required role is not activated, run [SET ROLE](../sql-reference/sql-statements/account-management/SET%20ROLE.md) to activate this role and perform operations using this role.
|
||||
This error may happen if the role is not activated. You can run `select current_role();` to query the roles that have been activated for the user in the current session. If the required role is not activated, run [SET ROLE](../sql-reference/sql-statements/account-management/SET_ROLE.md) to activate this role and perform operations using this role.
|
||||
|
||||
If you want roles to be automatically activated upon login, the `user_admin` role can run [SET DEFAULT ROLE](../sql-reference/sql-statements/account-management/SET_DEFAULT_ROLE.md) or [ALTER USER DEFAULT ROLE](../sql-reference/sql-statements/account-management/ALTER%20USER.md) to set a default role for each user. After the default role is set, it will be automatically activated when the user logs in.
|
||||
If you want roles to be automatically activated upon login, the `user_admin` role can run [SET DEFAULT ROLE](../sql-reference/sql-statements/account-management/SET_DEFAULT_ROLE.md) or [ALTER USER DEFAULT ROLE](../sql-reference/sql-statements/account-management/ALTER_USER.md) to set a default role for each user. After the default role is set, it will be automatically activated when the user logs in.
|
||||
|
||||
If you want all the assigned roles of all users to be automatically activated upon login, you can run the following command. This operation requires the OPERATE permission at the System level.
|
||||
|
||||
|
|
|
|||
|
|
@ -97,7 +97,7 @@ Active roles allow users to apply the privileges of the role under the current s
|
|||
|
||||
#### Default roles
|
||||
|
||||
Default roles are automatically activated when the user logs in to the cluster. It can be a role owned by one or more users. The administrator can set default roles using the `DEFAULT ROLE` keyword in [CREATE USER](../sql-reference/sql-statements/account-management/CREATE%20USER.md) and can change default roles using [ALTER USER](../sql-reference/sql-statements/account-management/ALTER%20USER.md).
|
||||
Default roles are automatically activated when the user logs in to the cluster. It can be a role owned by one or more users. The administrator can set default roles using the `DEFAULT ROLE` keyword in [CREATE USER](../sql-reference/sql-statements/account-management/CREATE_USER.md) and can change default roles using [ALTER USER](../sql-reference/sql-statements/account-management/ALTER_USER.md).
|
||||
|
||||
Users can also change their default roles using [SET DEFAULT ROLE](../sql-reference/sql-statements/account-management/SET_DEFAULT_ROLE.md).
|
||||
|
||||
|
|
@ -107,7 +107,7 @@ A user who does not have a default role still has the `public` role, which is au
|
|||
|
||||
#### Manually activate roles
|
||||
|
||||
In addition to default roles, users can also manually activate one or more existing roles within a session. You can use [SHOW GRANTS](../sql-reference/sql-statements/account-management/SHOW%20GRANTS.md) to view the privileges and roles that can be activated, and use [SET ROLE](../sql-reference/sql-statements/account-management/SET%20ROLE.md) to configure active roles that are effective in the current session.
|
||||
In addition to default roles, users can also manually activate one or more existing roles within a session. You can use [SHOW GRANTS](../sql-reference/sql-statements/account-management/SHOW_GRANTS.md) to view the privileges and roles that can be activated, and use [SET ROLE](../sql-reference/sql-statements/account-management/SET_ROLE.md) to configure active roles that are effective in the current session.
|
||||
|
||||
Note that the SET ROLE command overwrites each other. For example, after a user logs in, the `default_role` is activated by default. Then the user runs `SET ROLE role_s`. At this time, the user has only the privileges of `role_s` and their own privileges. `default_role` is overwritten.
|
||||
|
||||
|
|
|
|||
|
|
@ -55,7 +55,7 @@ You can set the capacity of a query queue and the maximum timeout of queries in
|
|||
|
||||
You can view the statistics of query queues via the following ways:
|
||||
|
||||
- Check the number of running queries, and memory and CPU usages in BE nodes using [SHOW PROC](../sql-reference/sql-statements/Administration/SHOW%20PROC.md):
|
||||
- Check the number of running queries, and memory and CPU usages in BE nodes using [SHOW PROC](../sql-reference/sql-statements/Administration/SHOW_PROC.md):
|
||||
|
||||
```Plain
|
||||
mysql> SHOW PROC '/backends'\G
|
||||
|
|
@ -66,7 +66,7 @@ mysql> SHOW PROC '/backends'\G
|
|||
CpuUsedPct: 0.0 %
|
||||
```
|
||||
|
||||
- Check if a query is in a queue (when `IsPending` is `true`) using [SHOW PROCESSLIST](../sql-reference/sql-statements/Administration/SHOW%20PROCESSLIST.md):
|
||||
- Check if a query is in a queue (when `IsPending` is `true`) using [SHOW PROCESSLIST](../sql-reference/sql-statements/Administration/SHOW_PROCESSLIST.md):
|
||||
|
||||
```Plain
|
||||
mysql> SHOW PROCESSLIST;
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
|
||||
For more information on how to create a storage volume for other object storages and set the default storage volume, see [CREATE STORAGE VOLUME](../../sql-reference/sql-statements/Administration/CREATE%20STORAGE%20VOLUME.md) and [SET DEFAULT STORAGE VOLUME](../../sql-reference/sql-statements/Administration/SET%20DEFAULT%20STORAGE%20VOLUME.md).
|
||||
For more information on how to create a storage volume for other object storages and set the default storage volume, see [CREATE STORAGE VOLUME](../../sql-reference/sql-statements/Administration/CREATE_STORAGE_VOLUME.md) and [SET DEFAULT STORAGE VOLUME](../../sql-reference/sql-statements/Administration/SET_DEFAULT_STORAGE_VOLUME.md).
|
||||
|
||||
### Create a database and a cloud-native table
|
||||
|
||||
|
|
@ -74,7 +74,7 @@ Whether to allow data to be written into object storage asynchronously. Default:
|
|||
|
||||
### View table information
|
||||
|
||||
You can view the information of tables in a specific database using `SHOW PROC "/dbs/<db_id>"`. See [SHOW PROC](../../sql-reference/sql-statements/Administration/SHOW%20PROC.md) for more information.
|
||||
You can view the information of tables in a specific database using `SHOW PROC "/dbs/<db_id>"`. See [SHOW PROC](../../sql-reference/sql-statements/Administration/SHOW_PROC.md) for more information.
|
||||
|
||||
Example:
|
||||
|
||||
|
|
|
|||
|
|
@ -572,7 +572,7 @@ SHOW RESOURCES;
|
|||
DROP RESOURCE "hive0";
|
||||
~~~
|
||||
|
||||
You can modify `hive.metastore.uris` of a Hive resource in StarRocks 2.3 and later versions. For more information, see [ALTER RESOURCE](../sql-reference/sql-statements/data-definition/ALTER%20RESOURCE.md).
|
||||
You can modify `hive.metastore.uris` of a Hive resource in StarRocks 2.3 and later versions. For more information, see [ALTER RESOURCE](../sql-reference/sql-statements/data-definition/ALTER_RESOURCE.md).
|
||||
|
||||
### Create a database
|
||||
|
||||
|
|
@ -781,7 +781,7 @@ The following table describes the related parameters.
|
|||
| iceberg.catalog.type | The catalog type of the resource. Both Hive catalog and custom catalog are supported. If you specify a Hive catalog, set the value to `HIVE`. If you specify a custom catalog, set the value to `CUSTOM`. |
|
||||
| iceberg.catalog-impl | The fully qualified class name of the custom catalog. FEs search for the catalog based on this name. If the catalog contains custom configuration items, you must add them to the `PROPERTIES` parameter as key-value pairs when you create an Iceberg external table. |
|
||||
|
||||
You can modify `hive.metastore.uris` and `iceberg.catalog-impl`of a Iceberg resource in StarRocks 2.3 and later versions. For more information, see [ALTER RESOURCE](../sql-reference/sql-statements/data-definition/ALTER%20RESOURCE.md).
|
||||
You can modify `hive.metastore.uris` and `iceberg.catalog-impl`of a Iceberg resource in StarRocks 2.3 and later versions. For more information, see [ALTER RESOURCE](../sql-reference/sql-statements/data-definition/ALTER_RESOURCE.md).
|
||||
|
||||
##### View Iceberg resources
|
||||
|
||||
|
|
@ -926,7 +926,7 @@ The following table describes the parameters.
|
|||
| type | The type of the Hudi resource. Set the vaue to hudi. |
|
||||
| hive.metastore.uris | The Thrift URI of the Hive metastore to which the Hudi resource connects. After connecting the Hudi resource to a Hive metastore, you can create and manage Hudi tables by using Hive. The Thrift URI is in the `<IP address of the Hive metastore>:<Port number of the Hive metastore>` format. The default port number is 9083. |
|
||||
|
||||
From v2.3 onwards, StarRocks allows changing the `hive.metastore.uris` value of a Hudi resource. For more information, see [ALTER RESOURCE](../sql-reference/sql-statements/data-definition/ALTER%20RESOURCE.md).
|
||||
From v2.3 onwards, StarRocks allows changing the `hive.metastore.uris` value of a Hudi resource. For more information, see [ALTER RESOURCE](../sql-reference/sql-statements/data-definition/ALTER_RESOURCE.md).
|
||||
|
||||
##### View Hudi resources
|
||||
|
||||
|
|
|
|||
|
|
@ -32,7 +32,7 @@ Currently, StarRocks provides two types of catalogs: internal catalog and extern
|
|||
|
||||
## Access catalog
|
||||
|
||||
You can use the [SET CATALOG](../../sql-reference/sql-statements/data-definition/SET%20CATALOG.md) statement to switch to a specified catalog in the current session. Then, you can query data by using that catalog.
|
||||
You can use the [SET CATALOG](../../sql-reference/sql-statements/data-definition/SET_CATALOG.md) statement to switch to a specified catalog in the current session. Then, you can query data by using that catalog.
|
||||
|
||||
## Query data
|
||||
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ StarRocks 2.3 and later provide an internal catalog to manage the internal data
|
|||
- If you use the MySQL client to connect the StarRocks cluster, you go to `default_catalog` by default after connecting.
|
||||
- If you use JDBC to connect the StarRocks cluster, you can go directly to the destination database in the default catalog by specifying `default_catalog.db_name` when connecting.
|
||||
|
||||
2. (Optional) Use [SHOW DATABASES](../../sql-reference/sql-statements/data-manipulation/SHOW%20DATABASES.md) to view databases:
|
||||
2. (Optional) Use [SHOW DATABASES](../../sql-reference/sql-statements/data-manipulation/SHOW_DATABASES.md) to view databases:
|
||||
|
||||
```SQL
|
||||
SHOW DATABASES;
|
||||
|
|
@ -22,7 +22,7 @@ StarRocks 2.3 and later provide an internal catalog to manage the internal data
|
|||
SHOW DATABASES FROM <catalog_name>;
|
||||
```
|
||||
|
||||
3. (Optional) Use [SET CATALOG](../../sql-reference/sql-statements/data-definition/SET%20CATALOG.md) to switch to the destination catalog in the current session:
|
||||
3. (Optional) Use [SET CATALOG](../../sql-reference/sql-statements/data-definition/SET_CATALOG.md) to switch to the destination catalog in the current session:
|
||||
|
||||
```SQL
|
||||
SET CATALOG <catalog_name>;
|
||||
|
|
|
|||
|
|
@ -756,13 +756,13 @@ PROPERTIES
|
|||
|
||||
## View Delta Lake catalogs
|
||||
|
||||
You can use [SHOW CATALOGS](../../sql-reference/sql-statements/data-manipulation/SHOW%20CATALOGS.md) to query all catalogs in the current StarRocks cluster:
|
||||
You can use [SHOW CATALOGS](../../sql-reference/sql-statements/data-manipulation/SHOW_CATALOGS.md) to query all catalogs in the current StarRocks cluster:
|
||||
|
||||
```SQL
|
||||
SHOW CATALOGS;
|
||||
```
|
||||
|
||||
You can also use [SHOW CREATE CATALOG](../../sql-reference/sql-statements/data-manipulation/SHOW%20CREATE%20CATALOG.md) to query the creation statement of an external catalog. The following example queries the creation statement of a Delta Lake catalog named `deltalake_catalog_glue`:
|
||||
You can also use [SHOW CREATE CATALOG](../../sql-reference/sql-statements/data-manipulation/SHOW_CREATE_CATALOG.md) to query the creation statement of an external catalog. The following example queries the creation statement of a Delta Lake catalog named `deltalake_catalog_glue`:
|
||||
|
||||
```SQL
|
||||
SHOW CREATE CATALOG deltalake_catalog_glue;
|
||||
|
|
@ -772,7 +772,7 @@ SHOW CREATE CATALOG deltalake_catalog_glue;
|
|||
|
||||
You can use one of the following methods to switch to a Delta Lake catalog and a database in it:
|
||||
|
||||
- Use [SET CATALOG](../../sql-reference/sql-statements/data-definition/SET%20CATALOG.md) to specify a Delta Lake catalog in the current session, and then use [USE](../../sql-reference/sql-statements/data-definition/USE.md) to specify an active database:
|
||||
- Use [SET CATALOG](../../sql-reference/sql-statements/data-definition/SET_CATALOG.md) to specify a Delta Lake catalog in the current session, and then use [USE](../../sql-reference/sql-statements/data-definition/USE.md) to specify an active database:
|
||||
|
||||
```SQL
|
||||
-- Switch to a specified catalog in the current session:
|
||||
|
|
@ -789,7 +789,7 @@ You can use one of the following methods to switch to a Delta Lake catalog and a
|
|||
|
||||
## Drop a Delta Lake catalog
|
||||
|
||||
You can use [DROP CATALOG](../../sql-reference/sql-statements/data-definition/DROP%20CATALOG.md) to drop an external catalog.
|
||||
You can use [DROP CATALOG](../../sql-reference/sql-statements/data-definition/DROP_CATALOG.md) to drop an external catalog.
|
||||
|
||||
The following example drops a Delta Lake catalog named `deltalake_catalog_glue`:
|
||||
|
||||
|
|
@ -815,7 +815,7 @@ You can use one of the following syntaxes to view the schema of a Delta Lake tab
|
|||
|
||||
## Query a Delta Lake table
|
||||
|
||||
1. Use [SHOW DATABASES](../../sql-reference/sql-statements/data-manipulation/SHOW%20DATABASES.md) to view the databases in your Delta Lake cluster:
|
||||
1. Use [SHOW DATABASES](../../sql-reference/sql-statements/data-manipulation/SHOW_DATABASES.md) to view the databases in your Delta Lake cluster:
|
||||
|
||||
```SQL
|
||||
SHOW DATABASES FROM <catalog_name>
|
||||
|
|
@ -841,7 +841,7 @@ INSERT INTO default_catalog.olap_db.olap_tbl SELECT * FROM deltalake_table
|
|||
|
||||
### Manual update
|
||||
|
||||
By default, StarRocks caches the metadata of Delta Lake and automatically updates the metadata in asynchronous mode to deliver better performance. Additionally, after some schema changes or table updates are made on a Delta Lake table, you can also use [REFRESH EXTERNAL TABLE](../../sql-reference/sql-statements/data-definition/REFRESH%20EXTERNAL%20TABLE.md) to manually update its metadata, thereby ensuring that StarRocks can obtain up-to-date metadata at its earliest opportunity and generate appropriate execution plans:
|
||||
By default, StarRocks caches the metadata of Delta Lake and automatically updates the metadata in asynchronous mode to deliver better performance. Additionally, after some schema changes or table updates are made on a Delta Lake table, you can also use [REFRESH EXTERNAL TABLE](../../sql-reference/sql-statements/data-definition/REFRESH_EXTERNAL_TABLE.md) to manually update its metadata, thereby ensuring that StarRocks can obtain up-to-date metadata at its earliest opportunity and generate appropriate execution plans:
|
||||
|
||||
```SQL
|
||||
REFRESH EXTERNAL TABLE <table_name>
|
||||
|
|
|
|||
|
|
@ -773,13 +773,13 @@ PROPERTIES
|
|||
|
||||
## View Hive catalogs
|
||||
|
||||
You can use [SHOW CATALOGS](../../sql-reference/sql-statements/data-manipulation/SHOW%20CATALOGS.md) to query all catalogs in the current StarRocks cluster:
|
||||
You can use [SHOW CATALOGS](../../sql-reference/sql-statements/data-manipulation/SHOW_CATALOGS.md) to query all catalogs in the current StarRocks cluster:
|
||||
|
||||
```SQL
|
||||
SHOW CATALOGS;
|
||||
```
|
||||
|
||||
You can also use [SHOW CREATE CATALOG](../../sql-reference/sql-statements/data-manipulation/SHOW%20CREATE%20CATALOG.md) to query the creation statement of an external catalog. The following example queries the creation statement of a Hive catalog named `hive_catalog_glue`:
|
||||
You can also use [SHOW CREATE CATALOG](../../sql-reference/sql-statements/data-manipulation/SHOW_CREATE_CATALOG.md) to query the creation statement of an external catalog. The following example queries the creation statement of a Hive catalog named `hive_catalog_glue`:
|
||||
|
||||
```SQL
|
||||
SHOW CREATE CATALOG hive_catalog_glue;
|
||||
|
|
@ -789,7 +789,7 @@ SHOW CREATE CATALOG hive_catalog_glue;
|
|||
|
||||
You can use one of the following methods to switch to a Hive catalog and a database in it:
|
||||
|
||||
- Use [SET CATALOG](../../sql-reference/sql-statements/data-definition/SET%20CATALOG.md) to specify a Hive catalog in the current session, and then use [USE](../../sql-reference/sql-statements/data-definition/USE.md) to specify an active database:
|
||||
- Use [SET CATALOG](../../sql-reference/sql-statements/data-definition/SET_CATALOG.md) to specify a Hive catalog in the current session, and then use [USE](../../sql-reference/sql-statements/data-definition/USE.md) to specify an active database:
|
||||
|
||||
```SQL
|
||||
-- Switch to a specified catalog in the current session:
|
||||
|
|
@ -806,7 +806,7 @@ You can use one of the following methods to switch to a Hive catalog and a datab
|
|||
|
||||
## Drop a Hive catalog
|
||||
|
||||
You can use [DROP CATALOG](../../sql-reference/sql-statements/data-definition/DROP%20CATALOG.md) to drop an external catalog.
|
||||
You can use [DROP CATALOG](../../sql-reference/sql-statements/data-definition/DROP_CATALOG.md) to drop an external catalog.
|
||||
|
||||
The following example drops a Hive catalog named `hive_catalog_glue`:
|
||||
|
||||
|
|
@ -832,7 +832,7 @@ You can use one of the following syntaxes to view the schema of a Hive table:
|
|||
|
||||
## Query a Hive table
|
||||
|
||||
1. Use [SHOW DATABASES](../../sql-reference/sql-statements/data-manipulation/SHOW%20DATABASES.md) to view the databases in your Hive cluster:
|
||||
1. Use [SHOW DATABASES](../../sql-reference/sql-statements/data-manipulation/SHOW_DATABASES.md) to view the databases in your Hive cluster:
|
||||
|
||||
```SQL
|
||||
SHOW DATABASES FROM <catalog_name>
|
||||
|
|
@ -890,7 +890,7 @@ GRANT SELECT ON ALL VIEWS IN ALL DATABASES TO ROLE hive_role_table;
|
|||
|
||||
### Manual update
|
||||
|
||||
By default, StarRocks caches the metadata of Hive and automatically updates the metadata in asynchronous mode to deliver better performance. Additionally, after some schema changes or table updates are made on a Hive table, you can also use [REFRESH EXTERNAL TABLE](../../sql-reference/sql-statements/data-definition/REFRESH%20EXTERNAL%20TABLE.md) to manually update its metadata, thereby ensuring that StarRocks can obtain up-to-date metadata at its earliest opportunity and generate appropriate execution plans:
|
||||
By default, StarRocks caches the metadata of Hive and automatically updates the metadata in asynchronous mode to deliver better performance. Additionally, after some schema changes or table updates are made on a Hive table, you can also use [REFRESH EXTERNAL TABLE](../../sql-reference/sql-statements/data-definition/REFRESH_EXTERNAL_TABLE.md) to manually update its metadata, thereby ensuring that StarRocks can obtain up-to-date metadata at its earliest opportunity and generate appropriate execution plans:
|
||||
|
||||
```SQL
|
||||
REFRESH EXTERNAL TABLE <table_name>
|
||||
|
|
@ -989,9 +989,9 @@ From v2.5.5 onwards, StarRocks can periodically refresh the cached metadata of t
|
|||
|
||||
| Configuration item | Default | Description |
|
||||
| ------------------------------------------------------------ | ------------------------------------ | ------------------------------------ |
|
||||
| enable_background_refresh_connector_metadata | `true` in v3.0<br />`false` in v2.5 | Whether to enable the periodic Hive metadata cache refresh. After it is enabled, StarRocks polls the metastore (Hive Metastore or AWS Glue) of your Hive cluster, and refreshes the cached metadata of the frequently accessed Hive catalogs to perceive data changes. `true` indicates to enable the Hive metadata cache refresh, and `false` indicates to disable it. This item is an [FE dynamic parameter](../../administration/Configuration.md#configure-fe-dynamic-parameters). You can modify it using the [ADMIN SET FRONTEND CONFIG](../../sql-reference/sql-statements/Administration/ADMIN%20SET%20CONFIG.md) command. |
|
||||
| background_refresh_metadata_interval_millis | `600000` (10 minutes) | The interval between two consecutive Hive metadata cache refreshes. Unit: millisecond. This item is an [FE dynamic parameter](../../administration/Configuration.md#configure-fe-dynamic-parameters). You can modify it using the [ADMIN SET FRONTEND CONFIG](../../sql-reference/sql-statements/Administration/ADMIN%20SET%20CONFIG.md) command. |
|
||||
| background_refresh_metadata_time_secs_since_last_access_secs | `86400` (24 hours) | The expiration time of a Hive metadata cache refresh task. For the Hive catalog that has been accessed, if it has not been accessed for more than the specified time, StarRocks stops refreshing its cached metadata. For the Hive catalog that has not been accessed, StarRocks will not refresh its cached metadata. Unit: second. This item is an [FE dynamic parameter](../../administration/Configuration.md#configure-fe-dynamic-parameters). You can modify it using the [ADMIN SET FRONTEND CONFIG](../../sql-reference/sql-statements/Administration/ADMIN%20SET%20CONFIG.md) command. |
|
||||
| enable_background_refresh_connector_metadata | `true` in v3.0<br />`false` in v2.5 | Whether to enable the periodic Hive metadata cache refresh. After it is enabled, StarRocks polls the metastore (Hive Metastore or AWS Glue) of your Hive cluster, and refreshes the cached metadata of the frequently accessed Hive catalogs to perceive data changes. `true` indicates to enable the Hive metadata cache refresh, and `false` indicates to disable it. This item is an [FE dynamic parameter](../../administration/Configuration.md#configure-fe-dynamic-parameters). You can modify it using the [ADMIN SET FRONTEND CONFIG](../../sql-reference/sql-statements/Administration/ADMIN_SET_CONFIG.md) command. |
|
||||
| background_refresh_metadata_interval_millis | `600000` (10 minutes) | The interval between two consecutive Hive metadata cache refreshes. Unit: millisecond. This item is an [FE dynamic parameter](../../administration/Configuration.md#configure-fe-dynamic-parameters). You can modify it using the [ADMIN SET FRONTEND CONFIG](../../sql-reference/sql-statements/Administration/ADMIN_SET_CONFIG.md) command. |
|
||||
| background_refresh_metadata_time_secs_since_last_access_secs | `86400` (24 hours) | The expiration time of a Hive metadata cache refresh task. For the Hive catalog that has been accessed, if it has not been accessed for more than the specified time, StarRocks stops refreshing its cached metadata. For the Hive catalog that has not been accessed, StarRocks will not refresh its cached metadata. Unit: second. This item is an [FE dynamic parameter](../../administration/Configuration.md#configure-fe-dynamic-parameters). You can modify it using the [ADMIN SET FRONTEND CONFIG](../../sql-reference/sql-statements/Administration/ADMIN_SET_CONFIG.md) command. |
|
||||
|
||||
Using the periodic Hive metadata cache refresh feature and the metadata automatic asynchronous update policy together significantly accelerates data access, reduces the read load from external data sources, and improves query performance.
|
||||
|
||||
|
|
|
|||
|
|
@ -754,13 +754,13 @@ PROPERTIES
|
|||
|
||||
## View Hudi catalogs
|
||||
|
||||
You can use [SHOW CATALOGS](../../sql-reference/sql-statements/data-manipulation/SHOW%20CATALOGS.md) to query all catalogs in the current StarRocks cluster:
|
||||
You can use [SHOW CATALOGS](../../sql-reference/sql-statements/data-manipulation/SHOW_CATALOGS.md) to query all catalogs in the current StarRocks cluster:
|
||||
|
||||
```SQL
|
||||
SHOW CATALOGS;
|
||||
```
|
||||
|
||||
You can also use [SHOW CREATE CATALOG](../../sql-reference/sql-statements/data-manipulation/SHOW%20CREATE%20CATALOG.md) to query the creation statement of an external catalog. The following example queries the creation statement of a Hudi catalog named `hudi_catalog_glue`:
|
||||
You can also use [SHOW CREATE CATALOG](../../sql-reference/sql-statements/data-manipulation/SHOW_CREATE_CATALOG.md) to query the creation statement of an external catalog. The following example queries the creation statement of a Hudi catalog named `hudi_catalog_glue`:
|
||||
|
||||
```SQL
|
||||
SHOW CREATE CATALOG hudi_catalog_glue;
|
||||
|
|
@ -770,7 +770,7 @@ SHOW CREATE CATALOG hudi_catalog_glue;
|
|||
|
||||
You can use one of the following methods to switch to a Hudi catalog and a database in it:
|
||||
|
||||
- Use [SET CATALOG](../../sql-reference/sql-statements/data-definition/SET%20CATALOG.md) to specify a Hudi catalog in the current session, and then use [USE](../../sql-reference/sql-statements/data-definition/USE.md) to specify an active database:
|
||||
- Use [SET CATALOG](../../sql-reference/sql-statements/data-definition/SET_CATALOG.md) to specify a Hudi catalog in the current session, and then use [USE](../../sql-reference/sql-statements/data-definition/USE.md) to specify an active database:
|
||||
|
||||
```SQL
|
||||
-- Switch to a specified catalog in the current session:
|
||||
|
|
@ -787,7 +787,7 @@ You can use one of the following methods to switch to a Hudi catalog and a datab
|
|||
|
||||
## Drop a Hudi catalog
|
||||
|
||||
You can use [DROP CATALOG](../../sql-reference/sql-statements/data-definition/DROP%20CATALOG.md) to drop an external catalog.
|
||||
You can use [DROP CATALOG](../../sql-reference/sql-statements/data-definition/DROP_CATALOG.md) to drop an external catalog.
|
||||
|
||||
The following example drops a Hudi catalog named `hudi_catalog_glue`:
|
||||
|
||||
|
|
@ -813,7 +813,7 @@ You can use one of the following syntaxes to view the schema of a Hudi table:
|
|||
|
||||
## Query a Hudi table
|
||||
|
||||
1. Use [SHOW DATABASES](../../sql-reference/sql-statements/data-manipulation/SHOW%20DATABASES.md) to view the databases in your Hudi cluster:
|
||||
1. Use [SHOW DATABASES](../../sql-reference/sql-statements/data-manipulation/SHOW_DATABASES.md) to view the databases in your Hudi cluster:
|
||||
|
||||
```SQL
|
||||
SHOW DATABASES FROM <catalog_name>
|
||||
|
|
@ -839,7 +839,7 @@ INSERT INTO default_catalog.olap_db.olap_tbl SELECT * FROM hudi_table
|
|||
|
||||
### Manual update
|
||||
|
||||
By default, StarRocks caches the metadata of Hudi and automatically updates the metadata in asynchronous mode to deliver better performance. Additionally, after some schema changes or table updates are made on a Hudi table, you can also use [REFRESH EXTERNAL TABLE](../../sql-reference/sql-statements/data-definition/REFRESH%20EXTERNAL%20TABLE.md) to manually update its metadata, thereby ensuring that StarRocks can obtain up-to-date metadata at its earliest opportunity and generate appropriate execution plans:
|
||||
By default, StarRocks caches the metadata of Hudi and automatically updates the metadata in asynchronous mode to deliver better performance. Additionally, after some schema changes or table updates are made on a Hudi table, you can also use [REFRESH EXTERNAL TABLE](../../sql-reference/sql-statements/data-definition/REFRESH_EXTERNAL_TABLE.md) to manually update its metadata, thereby ensuring that StarRocks can obtain up-to-date metadata at its earliest opportunity and generate appropriate execution plans:
|
||||
|
||||
```SQL
|
||||
REFRESH EXTERNAL TABLE <table_name>
|
||||
|
|
|
|||
|
|
@ -776,13 +776,13 @@ PROPERTIES
|
|||
|
||||
## View Iceberg catalogs
|
||||
|
||||
You can use [SHOW CATALOGS](../../sql-reference/sql-statements/data-manipulation/SHOW%20CATALOGS.md) to query all catalogs in the current StarRocks cluster:
|
||||
You can use [SHOW CATALOGS](../../sql-reference/sql-statements/data-manipulation/SHOW_CATALOGS.md) to query all catalogs in the current StarRocks cluster:
|
||||
|
||||
```SQL
|
||||
SHOW CATALOGS;
|
||||
```
|
||||
|
||||
You can also use [SHOW CREATE CATALOG](../../sql-reference/sql-statements/data-manipulation/SHOW%20CREATE%20CATALOG.md) to query the creation statement of an external catalog. The following example queries the creation statement of an Iceberg catalog named `iceberg_catalog_glue`:
|
||||
You can also use [SHOW CREATE CATALOG](../../sql-reference/sql-statements/data-manipulation/SHOW_CREATE_CATALOG.md) to query the creation statement of an external catalog. The following example queries the creation statement of an Iceberg catalog named `iceberg_catalog_glue`:
|
||||
|
||||
```SQL
|
||||
SHOW CREATE CATALOG iceberg_catalog_glue;
|
||||
|
|
@ -792,7 +792,7 @@ SHOW CREATE CATALOG iceberg_catalog_glue;
|
|||
|
||||
You can use one of the following methods to switch to an Iceberg catalog and a database in it:
|
||||
|
||||
- Use [SET CATALOG](../../sql-reference/sql-statements/data-definition/SET%20CATALOG.md) to specify an Iceberg catalog in the current session, and then use [USE](../../sql-reference/sql-statements/data-definition/USE.md) to specify an active database:
|
||||
- Use [SET CATALOG](../../sql-reference/sql-statements/data-definition/SET_CATALOG.md) to specify an Iceberg catalog in the current session, and then use [USE](../../sql-reference/sql-statements/data-definition/USE.md) to specify an active database:
|
||||
|
||||
```SQL
|
||||
-- Switch to a specified catalog in the current session:
|
||||
|
|
@ -809,7 +809,7 @@ You can use one of the following methods to switch to an Iceberg catalog and a d
|
|||
|
||||
## Drop an Iceberg catalog
|
||||
|
||||
You can use [DROP CATALOG](../../sql-reference/sql-statements/data-definition/DROP%20CATALOG.md) to drop an external catalog.
|
||||
You can use [DROP CATALOG](../../sql-reference/sql-statements/data-definition/DROP_CATALOG.md) to drop an external catalog.
|
||||
|
||||
The following example drops an Iceberg catalog named `iceberg_catalog_glue`:
|
||||
|
||||
|
|
@ -835,7 +835,7 @@ You can use one of the following syntaxes to view the schema of an Iceberg table
|
|||
|
||||
## Query an Iceberg table
|
||||
|
||||
1. Use [SHOW DATABASES](../../sql-reference/sql-statements/data-manipulation/SHOW%20DATABASES.md) to view the databases in your Iceberg cluster:
|
||||
1. Use [SHOW DATABASES](../../sql-reference/sql-statements/data-manipulation/SHOW_DATABASES.md) to view the databases in your Iceberg cluster:
|
||||
|
||||
```SQL
|
||||
SHOW DATABASES FROM <catalog_name>
|
||||
|
|
@ -851,7 +851,7 @@ You can use one of the following syntaxes to view the schema of an Iceberg table
|
|||
|
||||
## Create an Iceberg database
|
||||
|
||||
Similar to the internal catalog of StarRocks, if you have the [CREATE DATABASE](../../administration/privilege_item.md#catalog) privilege on an Iceberg catalog, you can use the [CREATE DATABASE](../../sql-reference/sql-statements/data-definition/CREATE%20DATABASE.md) statement to create databases in that Iceberg catalog. This feature is supported from v3.1 onwards.
|
||||
Similar to the internal catalog of StarRocks, if you have the [CREATE DATABASE](../../administration/privilege_item.md#catalog) privilege on an Iceberg catalog, you can use the [CREATE DATABASE](../../sql-reference/sql-statements/data-definition/CREATE_DATABASE.md) statement to create databases in that Iceberg catalog. This feature is supported from v3.1 onwards.
|
||||
|
||||
> **NOTE**
|
||||
>
|
||||
|
|
@ -879,7 +879,7 @@ The `prefix` varies based on the storage system you use:
|
|||
|
||||
## Drop an Iceberg database
|
||||
|
||||
Similar to the internal databases of StarRocks, if you have the [DROP](../../administration/privilege_item.md#database) privilege on an Iceberg database, you can use the [DROP DATABASE](../../sql-reference/sql-statements/data-definition/DROP%20DATABASE.md) statement to drop that Iceberg database. This feature is supported from v3.1 onwards. You can only drop empty databases.
|
||||
Similar to the internal databases of StarRocks, if you have the [DROP](../../administration/privilege_item.md#database) privilege on an Iceberg database, you can use the [DROP DATABASE](../../sql-reference/sql-statements/data-definition/DROP_DATABASE.md) statement to drop that Iceberg database. This feature is supported from v3.1 onwards. You can only drop empty databases.
|
||||
|
||||
> **NOTE**
|
||||
>
|
||||
|
|
@ -895,7 +895,7 @@ DROP DATABASE <database_name>;
|
|||
|
||||
## Create an Iceberg table
|
||||
|
||||
Similar to the internal databases of StarRocks, if you have the [CREATE TABLE](../../administration/privilege_item.md#database) privilege on an Iceberg database, you can use the [CREATE TABLE](../../sql-reference/sql-statements/data-definition/CREATE%20TABLE.md) or [CREATE TABLE AS SELECT (CTAS)](../../sql-reference/sql-statements/data-definition/CREATE%20TABLE%20AS%20SELECT.md) statement to create a table in that Iceberg database. This feature is supported from v3.1 onwards.
|
||||
Similar to the internal databases of StarRocks, if you have the [CREATE TABLE](../../administration/privilege_item.md#database) privilege on an Iceberg database, you can use the [CREATE TABLE](../../sql-reference/sql-statements/data-definition/CREATE_TABLE.md) or [CREATE TABLE AS SELECT (CTAS)](../../sql-reference/sql-statements/data-definition/CREATE_TABLE_AS_SELECT.md) statement to create a table in that Iceberg database. This feature is supported from v3.1 onwards.
|
||||
|
||||
> **NOTE**
|
||||
>
|
||||
|
|
@ -1084,7 +1084,7 @@ PARTITION (par_col1=<value> [, par_col2=<value>...])
|
|||
|
||||
## Drop an Iceberg table
|
||||
|
||||
Similar to the internal tables of StarRocks, if you have the [DROP](../../administration/privilege_item.md#table) privilege on an Iceberg table, you can use the [DROP TABLE](../../sql-reference/sql-statements/data-definition/DROP%20TABLE.md) statement to drop that Iceberg table. This feature is supported from v3.1 onwards.
|
||||
Similar to the internal tables of StarRocks, if you have the [DROP](../../administration/privilege_item.md#table) privilege on an Iceberg table, you can use the [DROP TABLE](../../sql-reference/sql-statements/data-definition/DROP_TABLE.md) statement to drop that Iceberg table. This feature is supported from v3.1 onwards.
|
||||
|
||||
> **NOTE**
|
||||
>
|
||||
|
|
|
|||
|
|
@ -83,13 +83,13 @@ PROPERTIES
|
|||
|
||||
## View JDBC catalogs
|
||||
|
||||
You can use [SHOW CATALOGS](../../sql-reference/sql-statements/data-manipulation/SHOW%20CATALOGS.md) to query all catalogs in the current StarRocks cluster:
|
||||
You can use [SHOW CATALOGS](../../sql-reference/sql-statements/data-manipulation/SHOW_CATALOGS.md) to query all catalogs in the current StarRocks cluster:
|
||||
|
||||
```SQL
|
||||
SHOW CATALOGS;
|
||||
```
|
||||
|
||||
You can also use [SHOW CREATE CATALOG](../../sql-reference/sql-statements/data-manipulation/SHOW%20CREATE%20CATALOG.md) to query the creation statement of an external catalog. The following example queries the creation statement of a JDBC catalog named `jdbc0`:
|
||||
You can also use [SHOW CREATE CATALOG](../../sql-reference/sql-statements/data-manipulation/SHOW_CREATE_CATALOG.md) to query the creation statement of an external catalog. The following example queries the creation statement of a JDBC catalog named `jdbc0`:
|
||||
|
||||
```SQL
|
||||
SHOW CREATE CATALOG jdbc0;
|
||||
|
|
@ -97,7 +97,7 @@ SHOW CREATE CATALOG jdbc0;
|
|||
|
||||
## Drop a JDBC catalog
|
||||
|
||||
You can use [DROP CATALOG](../../sql-reference/sql-statements/data-definition/DROP%20CATALOG.md) to drop a JDBC catalog.
|
||||
You can use [DROP CATALOG](../../sql-reference/sql-statements/data-definition/DROP_CATALOG.md) to drop a JDBC catalog.
|
||||
|
||||
The following example drops a JDBC catalog named `jdbc0`:
|
||||
|
||||
|
|
@ -107,13 +107,13 @@ DROP Catalog jdbc0;
|
|||
|
||||
## Query a table in a JDBC catalog
|
||||
|
||||
1. Use [SHOW DATABASES](../../sql-reference/sql-statements/data-manipulation/SHOW%20DATABASES.md) to view the databases in your JDBC-compatible cluster:
|
||||
1. Use [SHOW DATABASES](../../sql-reference/sql-statements/data-manipulation/SHOW_DATABASES.md) to view the databases in your JDBC-compatible cluster:
|
||||
|
||||
```SQL
|
||||
SHOW DATABASES <catalog_name>;
|
||||
```
|
||||
|
||||
2. Use [SET CATALOG](../../sql-reference/sql-statements/data-definition/SET%20CATALOG.md) to switch to the destination catalog in the current session:
|
||||
2. Use [SET CATALOG](../../sql-reference/sql-statements/data-definition/SET_CATALOG.md) to switch to the destination catalog in the current session:
|
||||
|
||||
```SQL
|
||||
SET CATALOG <catalog_name>;
|
||||
|
|
|
|||
|
|
@ -598,13 +598,13 @@ PROPERTIES
|
|||
|
||||
## View Paimon catalogs
|
||||
|
||||
You can use [SHOW CATALOGS](../../sql-reference/sql-statements/data-manipulation/SHOW%20CATALOGS.md) to query all catalogs in the current StarRocks cluster:
|
||||
You can use [SHOW CATALOGS](../../sql-reference/sql-statements/data-manipulation/SHOW_CATALOGS.md) to query all catalogs in the current StarRocks cluster:
|
||||
|
||||
```SQL
|
||||
SHOW CATALOGS;
|
||||
```
|
||||
|
||||
You can also use [SHOW CREATE CATALOG](../../sql-reference/sql-statements/data-manipulation/SHOW%20CREATE%20CATALOG.md) to query the creation statement of an external catalog. The following example queries the creation statement of a Paimon catalog named `paimon_catalog_fs`:
|
||||
You can also use [SHOW CREATE CATALOG](../../sql-reference/sql-statements/data-manipulation/SHOW_CREATE_CATALOG.md) to query the creation statement of an external catalog. The following example queries the creation statement of a Paimon catalog named `paimon_catalog_fs`:
|
||||
|
||||
```SQL
|
||||
SHOW CREATE CATALOG paimon_catalog_fs;
|
||||
|
|
@ -612,7 +612,7 @@ SHOW CREATE CATALOG paimon_catalog_fs;
|
|||
|
||||
## Drop a Paimon catalog
|
||||
|
||||
You can use [DROP CATALOG](../../sql-reference/sql-statements/data-definition/DROP%20CATALOG.md) to drop an external catalog.
|
||||
You can use [DROP CATALOG](../../sql-reference/sql-statements/data-definition/DROP_CATALOG.md) to drop an external catalog.
|
||||
|
||||
The following example drops a Paimon catalog named `paimon_catalog_fs`:
|
||||
|
||||
|
|
@ -638,13 +638,13 @@ You can use one of the following syntaxes to view the schema of a Paimon table:
|
|||
|
||||
## Query a Paimon table
|
||||
|
||||
1. Use [SHOW DATABASES](../../sql-reference/sql-statements/data-manipulation/SHOW%20DATABASES.md) to view the databases in your Paimon cluster:
|
||||
1. Use [SHOW DATABASES](../../sql-reference/sql-statements/data-manipulation/SHOW_DATABASES.md) to view the databases in your Paimon cluster:
|
||||
|
||||
```SQL
|
||||
SHOW DATABASES FROM <catalog_name>;
|
||||
```
|
||||
|
||||
2. Use [SET CATALOG](../../sql-reference/sql-statements/data-definition/SET%20CATALOG.md) to switch to the destination catalog in the current session:
|
||||
2. Use [SET CATALOG](../../sql-reference/sql-statements/data-definition/SET_CATALOG.md) to switch to the destination catalog in the current session:
|
||||
|
||||
```SQL
|
||||
SET CATALOG <catalog_name>;
|
||||
|
|
|
|||
|
|
@ -12,13 +12,13 @@ This topic guides you through querying data from external data sources by using
|
|||
- If you use the MySQL client to connect the StarRocks cluster, you go to `default_catalog` by default after connecting.
|
||||
- If you use JDBC to connect the StarRocks cluster, you can go directly to the destination database in the default catalog by specifying `default_catalog.db_name` when connecting.
|
||||
|
||||
2. (Optional) Execute the following statement to view all the catalogs and find the external catalog you have created. See [SHOW CATALOGS](../../sql-reference/sql-statements/data-manipulation/SHOW%20CATALOGS.md) to check the output of this statement.
|
||||
2. (Optional) Execute the following statement to view all the catalogs and find the external catalog you have created. See [SHOW CATALOGS](../../sql-reference/sql-statements/data-manipulation/SHOW_CATALOGS.md) to check the output of this statement.
|
||||
|
||||
```SQL
|
||||
SHOW CATALOGS;
|
||||
```
|
||||
|
||||
3. (Optional) Execute the following statement to view all the databases in the external catalog. See [SHOW DATABASES](../../sql-reference/sql-statements/data-manipulation/SHOW%20DATABASES.md) to check the output of this statement.
|
||||
3. (Optional) Execute the following statement to view all the databases in the external catalog. See [SHOW DATABASES](../../sql-reference/sql-statements/data-manipulation/SHOW_DATABASES.md) to check the output of this statement.
|
||||
|
||||
```SQL
|
||||
SHOW DATABASES FROM catalog_name;
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ This feature is supported from StarRocks v2.5.
|
|||
|
||||
## Limits
|
||||
|
||||
- File external tables must be created in databases within the [default_catalog](../data_source/catalog/default_catalog.md). You can run [SHOW CATALOGS](../sql-reference/sql-statements/data-manipulation/SHOW%20CATALOGS.md) to query catalogs created in the cluster.
|
||||
- File external tables must be created in databases within the [default_catalog](../data_source/catalog/default_catalog.md). You can run [SHOW CATALOGS](../sql-reference/sql-statements/data-manipulation/SHOW_CATALOGS.md) to query catalogs created in the cluster.
|
||||
- Only Parquet and ORC data files are supported.
|
||||
- You can only use file external tables to query data in the target data file. Data write operations such as INSERT, DELETE, and DROP are not supported.
|
||||
|
||||
|
|
@ -16,7 +16,7 @@ Before you create a file external table, you must configure your StarRocks clust
|
|||
|
||||
## Create a database (Optional)
|
||||
|
||||
After connecting to your StarRocks cluster, you can create a file external table in an existing database or create a new database to manage file external tables. To query existing databases in the cluster, run [SHOW DATABASES](../sql-reference/sql-statements/data-manipulation/SHOW%20DATABASES.md). Then you can run `USE <db_name>` to switch to the target database.
|
||||
After connecting to your StarRocks cluster, you can create a file external table in an existing database or create a new database to manage file external tables. To query existing databases in the cluster, run [SHOW DATABASES](../sql-reference/sql-statements/data-manipulation/SHOW_DATABASES.md). Then you can run `USE <db_name>` to switch to the target database.
|
||||
|
||||
The syntax for creating a database is as follows.
|
||||
|
||||
|
|
@ -272,4 +272,4 @@ SELECT * FROM t0;
|
|||
|
||||
## Manage file external tables
|
||||
|
||||
You can view the schema of the table using [DESC](../sql-reference/sql-statements/Utility/DESCRIBE.md) or drop the table using [DROP TABLE](../sql-reference/sql-statements/data-definition/DROP%20TABLE.md).
|
||||
You can view the schema of the table using [DESC](../sql-reference/sql-statements/Utility/DESCRIBE.md) or drop the table using [DROP TABLE](../sql-reference/sql-statements/data-definition/DROP_TABLE.md).
|
||||
|
|
|
|||
|
|
@ -384,7 +384,7 @@ A high-availability FE cluster requires at least THREE Follower FE nodes in the
|
|||
> **NOTE**
|
||||
>
|
||||
> - You can use the preceding command to add a single Follower FE nodes each time.
|
||||
> - If you want to add Observer FE nodes, execute `ALTER SYSTEM ADD OBSERVER "<fe_address>:<edit_log_port>"=`. For detailed instructions, see [ALTER SYSTEM - FE](../sql-reference/sql-statements/Administration/ALTER%20SYSTEM.md).
|
||||
> - If you want to add Observer FE nodes, execute `ALTER SYSTEM ADD OBSERVER "<fe_address>:<edit_log_port>"=`. For detailed instructions, see [ALTER SYSTEM - FE](../sql-reference/sql-statements/Administration/ALTER_SYSTEM.md).
|
||||
|
||||
3. Launch a terminal on the new FE instance, create a dedicated directory for metadata storage, navigate to the directory that stores the StarRocks FE deployment files, and modify the FE configuration file **fe/conf/fe.conf**. For more instructions, see [Step 1: Start the Leader FE node](#step-1-start-the-leader-fe-node). Basically, you can repeat the procedures in Step 1 **except for the command used to start the FE node**.
|
||||
|
||||
|
|
|
|||
|
|
@ -113,7 +113,7 @@ After the compatibility configuration and the availability test, you can downgra
|
|||
>
|
||||
> If you are downgrading StarRocks v3.0 to v2.5, you must follow these steps after the downgrade:
|
||||
>
|
||||
> 1. Run [ALTER SYSTEM CREATE IMAGE](../sql-reference/sql-statements/Administration/ALTER%20SYSTEM.md) to create a new image.
|
||||
> 1. Run [ALTER SYSTEM CREATE IMAGE](../sql-reference/sql-statements/Administration/ALTER_SYSTEM.md) to create a new image.
|
||||
> 2. Wait for the new image to be synchronized to all Follower FEs.
|
||||
>
|
||||
> If you do not run this command, some of the downgrade operations may fail. ALTER SYSTEM CREATE IMAGE is supported from v2.5.3 and later.
|
||||
|
|
|
|||
|
|
@ -187,7 +187,7 @@ After upgrading all BE and CN nodes, you can then upgrade the FE nodes. You must
|
|||
>
|
||||
> If you have downgraded your StarRocks cluster after upgrading it from v2.5 to v3.0, and again you upgrade it to v3.0, you must follow these steps in order to avoid metadata upgrade failure for some Follower FEs:
|
||||
>
|
||||
> 1. Run [ALTER SYSTEM CREATE IMAGE](../sql-reference/sql-statements/Administration/ALTER%20SYSTEM.md) to create a new image.
|
||||
> 1. Run [ALTER SYSTEM CREATE IMAGE](../sql-reference/sql-statements/Administration/ALTER_SYSTEM.md) to create a new image.
|
||||
> 2. Wait for the new image to be synchronized to all Follower FEs.
|
||||
>
|
||||
> You can check whether the image file has been synchronized by viewing the log file **fe.log** of the Leader FE. A record of log like "push image.* from subdir [] to other nodes. totally xx nodes, push successful xx nodes" suggests that the image file has been successfully synchronized.
|
||||
|
|
|
|||
|
|
@ -97,7 +97,7 @@ This error may occur during data loading into Primary Key tables. During data lo
|
|||
Patches to this bug (Primary Key tables do not support BE disk space re-balancing) is still under active development. Currently, you can fix it in either of the following two ways:
|
||||
|
||||
- Manually distribute data among disks. For example, copy the directory from the disk with a high space usage to a disk with a larger space.
|
||||
- If the data on these disks is not important, we recommend you delete the disks and modify the disk path. If this error persists, use [TRUNCATE TABLE](../sql-reference/sql-statements/data-definition/TRUNCATE%20TABLE.md) to clear data in the table to free up some space.
|
||||
- If the data on these disks is not important, we recommend you delete the disks and modify the disk path. If this error persists, use [TRUNCATE TABLE](../sql-reference/sql-statements/data-definition/TRUNCATE_TABLE.md) to clear data in the table to free up some space.
|
||||
|
||||
## Why does the error "Fe type:unknown ,is ready :false." occur when I start an FE during the cluster restart?
|
||||
|
||||
|
|
|
|||
|
|
@ -180,7 +180,7 @@ No, use functions to change "2021-10" to "2021-10-01" and then use "2021-10-01"
|
|||
|
||||
## Where can I query the size of a StarRocks database or table?
|
||||
|
||||
You can use the [SHOW DATA](../sql-reference/sql-statements/data-manipulation/SHOW%20DATA.md) command.
|
||||
You can use the [SHOW DATA](../sql-reference/sql-statements/data-manipulation/SHOW_DATA.md) command.
|
||||
|
||||
`SHOW DATA;` displays the data size and replicas of all tables in the current database.
|
||||
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
## 1. Does Broker Load support re-running load jobs that have been run successfully and are in the FINISHED state?
|
||||
|
||||
Broker Load does not support re-running load jobs that have been run successfully and are in the FINISHED state. Also, to prevent data loss and duplication, Broker Load does not allow reusing the labels of successfully run load jobs. You can use [SHOW LOAD](../../sql-reference/sql-statements/data-manipulation/SHOW%20LOAD.md) to view the history of load jobs and find the load job that you want to re-run. Then, you can copy the information of that load job and use the job information, except the label, to create another load job.
|
||||
Broker Load does not support re-running load jobs that have been run successfully and are in the FINISHED state. Also, to prevent data loss and duplication, Broker Load does not allow reusing the labels of successfully run load jobs. You can use [SHOW LOAD](../../sql-reference/sql-statements/data-manipulation/SHOW_LOAD.md) to view the history of load jobs and find the load job that you want to re-run. Then, you can copy the information of that load job and use the job information, except the label, to create another load job.
|
||||
|
||||
## 2. When I load data from HDFS by using Broker Load, what do I do if the date and time values loaded into the destination StarRocks table are 8 hours later than the date and time values from the source data file?
|
||||
|
||||
|
|
@ -14,7 +14,7 @@ The source data file has different column names than the destination StarRocks t
|
|||
|
||||
## 4. The Broker Load job does not report errors, but why am I unable to query the loaded data?
|
||||
|
||||
Broker Load is an asynchronous loading method. The load job may still fail even if the load statement does not return errors. After you run a Broker Load job, you can use [SHOW LOAD](../../sql-reference/sql-statements/data-manipulation/SHOW%20LOAD.md) to view the result and `errmsg` of the load job. Then, you can modify the job configuration and retry.
|
||||
Broker Load is an asynchronous loading method. The load job may still fail even if the load statement does not return errors. After you run a Broker Load job, you can use [SHOW LOAD](../../sql-reference/sql-statements/data-manipulation/SHOW_LOAD.md) to view the result and `errmsg` of the load job. Then, you can modify the job configuration and retry.
|
||||
|
||||
## 5. What do I do if the "failed to send batch" or "TabletWriter add batch with unknown id" error occurs?
|
||||
|
||||
|
|
@ -33,7 +33,7 @@ SET
|
|||
)
|
||||
```
|
||||
|
||||
The preceding example maps the `tmp_c1` and `tmp_c2` columns of the source data file onto the `name` and `id` columns of the destination StarRocks table, respectively. If you do not specify the `SET` clause, the column names specified in the `column_list` parameter are used to declare the column mapping. For more information, see [BROKER LOAD](../../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md).
|
||||
The preceding example maps the `tmp_c1` and `tmp_c2` columns of the source data file onto the `name` and `id` columns of the destination StarRocks table, respectively. If you do not specify the `SET` clause, the column names specified in the `column_list` parameter are used to declare the column mapping. For more information, see [BROKER LOAD](../../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md).
|
||||
|
||||
> **NOTICE**
|
||||
>
|
||||
|
|
|
|||
|
|
@ -40,7 +40,7 @@ Before you submit a load job, we recommend that you calculate the approximate am
|
|||
|
||||
## 3. What do I do if the "ETL_QUALITY_UNSATISFIED; msg:quality not good enough to cancel" error occurs?
|
||||
|
||||
Execute [SHOW LOAD](/sql-reference/sql-statements/data-manipulation/SHOW%20LOAD.md), and use the error URL in the returned execution result to view the error details.
|
||||
Execute [SHOW LOAD](/sql-reference/sql-statements/data-manipulation/SHOW_LOAD.md), and use the error URL in the returned execution result to view the error details.
|
||||
|
||||
Common data quality errors are as follows:
|
||||
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ Parameter description:
|
|||
- `alive_be_number`: the number of BE nodes alive.
|
||||
- `partition_number`: the number of partitions to be consumed.
|
||||
- `desired_concurrent_number`: the desired load task parallelism for a Routine Load job. The default value is `3`. You can set a higher value for this parameter to increase the actual load task parallelism.
|
||||
- If you have not created a Routine Load job, you need to set this parameter when using [CREATE ROUTINE LOAD](../../sql-reference/sql-statements/data-manipulation/ROUTINE%20LOAD.md) to create a Routine Load job.
|
||||
- If you have not created a Routine Load job, you need to set this parameter when using [CREATE ROUTINE LOAD](../../sql-reference/sql-statements/data-manipulation/ROUTINE_LOAD.md) to create a Routine Load job.
|
||||
- If you have already created a Routine Load job, you need to use [ALTER ROUTINE LOAD](../../sql-reference/sql-statements/data-manipulation/alter-routine-load.md) to modify this parameter.
|
||||
- `max_routine_load_task_concurrent_num`: the default maximum task parallelism for a Routine Load job. The default value is `5`. This parameter is a an FE dynamic parameter. For more information and the configuration method, see [Parameter configuration](../../administration/Configuration.md#loading-and-unloading).
|
||||
|
||||
|
|
@ -27,7 +27,7 @@ Therefore, when the number of partitions to be consumed and the number of BE nod
|
|||
|
||||
For example, the number of partitions to be consumed is `7`, the number of live BE nodes is `5`, and `max_routine_load_task_concurrent_num` is the default value `5`. At this time, if you need to increase the load task parallelism to the upper limit, you need to set `desired_concurrent_number` to `5` (the default value is `3`). Then, the actual task parallelism `min(5,7,5,5)` is computed to be `5`.
|
||||
|
||||
For more parameter descriptions, see [CREATE ROUTINE LOAD](../../sql-reference/sql-statements/data-manipulation/ROUTINE%20LOAD.md#example).
|
||||
For more parameter descriptions, see [CREATE ROUTINE LOAD](../../sql-reference/sql-statements/data-manipulation/ROUTINE_LOAD.md#example).
|
||||
|
||||
**Method 2: Increase the amount of data consumed by a Routine Load task from one or more partitions.**
|
||||
|
||||
|
|
@ -53,7 +53,7 @@ If the field `left_bytes` is less than `0`, it means that the amount of data con
|
|||
|
||||
**Cause analysis:** The consumer offset of the load job does not exist in the Kafka partition.
|
||||
|
||||
**Solution:** You can execute [SHOW ROUTINE LOAD](../../sql-reference/sql-statements/data-manipulation/SHOW%20ROUTINE%20LOAD.md) and check the latest consumer offset of the load job in the parameter `Progress`. Then, you can verify if the corresponding message exists in the Kafka partition. If it does not exist, it may be because
|
||||
**Solution:** You can execute [SHOW ROUTINE LOAD](../../sql-reference/sql-statements/data-manipulation/SHOW_ROUTINE_LOAD.md) and check the latest consumer offset of the load job in the parameter `Progress`. Then, you can verify if the corresponding message exists in the Kafka partition. If it does not exist, it may be because
|
||||
|
||||
- The consumer offset specified when the load job is created is an offset in the future.
|
||||
- The message at the specified consumer offset in the Kafka partition has been removed before being consumed by the load job. It is recommended to set a reasonable Kafka log cleaning policy and parameters, such as `log.retention.hours and log.retention.bytes`, based on the loading speed.
|
||||
|
|
@ -64,9 +64,9 @@ If the field `left_bytes` is less than `0`, it means that the amount of data con
|
|||
|
||||
**Solution:** You can troubleshoot and fix the issue by using error messages in the fields `ReasonOfStateChanged` and `ErrorLogUrls`.
|
||||
|
||||
- If it is caused by incorrect data format in the data source, you need to check the data format and fix the issue. After successfully fixing the issue, you can use [RESUME ROUTINE LOAD](../../sql-reference/sql-statements/data-manipulation/RESUME%20ROUTINE%20LOAD.md) to resume the paused load job.
|
||||
- If it is caused by incorrect data format in the data source, you need to check the data format and fix the issue. After successfully fixing the issue, you can use [RESUME ROUTINE LOAD](../../sql-reference/sql-statements/data-manipulation/RESUME_ROUTINE_LOAD.md) to resume the paused load job.
|
||||
|
||||
- If it is because that StarRocks cannot parse the data format in the data source, you need to adjust the threshold `max_error_number`. You can first execute [SHOW ROUTINE LOAD](../../sql-reference/sql-statements/data-manipulation/SHOW%20ROUTINE%20LOAD.md) to view the value of `max_error_number`, and then use [ALTER ROUTINE LOAD](../../sql-reference/sql-statements/data-manipulation/alter-routine-load.md) to increase the threshold. After modifying the threshold, you can use [RESUME ROUTINE LOAD](../../sql-reference/sql-statements/data-manipulation/RESUME%20ROUTINE%20LOAD.md) to resume the paused load job.
|
||||
- If it is because that StarRocks cannot parse the data format in the data source, you need to adjust the threshold `max_error_number`. You can first execute [SHOW ROUTINE LOAD](../../sql-reference/sql-statements/data-manipulation/SHOW_ROUTINE_LOAD.md) to view the value of `max_error_number`, and then use [ALTER ROUTINE LOAD](../../sql-reference/sql-statements/data-manipulation/alter-routine-load.md) to increase the threshold. After modifying the threshold, you can use [RESUME ROUTINE LOAD](../../sql-reference/sql-statements/data-manipulation/RESUME_ROUTINE_LOAD.md) to resume the paused load job.
|
||||
|
||||
## What do I do if the result of SHOW ROUTINE LOAD shows that the load job is in the `CANCELLED` state?
|
||||
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@ In v2.5 and earlier, Stream Load does not support skipping the first few rows of
|
|||
- In the load command or statement, use `-H "where: <column_name> != '<column_name>'"` to filter out the first few rows of the CSV file. `<column_name>` is any of the column names held in the first few rows. Note that StarRocks first transforms and then filters the source data. Therefore, if the column names in the first few rows fail to be transformed into their matching destination data types, `NULL` values are returned for them. This means the destination StarRocks table cannot contain columns that are set to `NOT NULL`.
|
||||
- In the load command or statement, add `-H "max_filter_ratio:0.01"` to set a maximum error tolerance that is 1% or lower but can tolerate a few error rows, thereby allowing StarRocks to ignore the data transformation failures in the first few rows. In this case, the Stream Load job can still succeed even if `ErrorURL` is returned to indicate error rows. Do not set `max_filter_ratio` to a large value. If you set `max_filter_ratio` to a large value, some important data quality issues may be missed.
|
||||
|
||||
From v3.0 onwards, Stream Load supports the `skip_header` parameter, which specifies whether to skip the first few rows of a CSV file. For more information,see [CSV parameters](../../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md#csv-parameters).
|
||||
From v3.0 onwards, Stream Load supports the `skip_header` parameter, which specifies whether to skip the first few rows of a CSV file. For more information,see [CSV parameters](../../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md#csv-parameters).
|
||||
|
||||
## 2. The data to be loaded into the partition column is not of standard DATE or INT type. For example, the data is in a format like 202106.00. How do I transform the data if I load it by using Stream Load?
|
||||
|
||||
|
|
|
|||
|
|
@ -346,7 +346,7 @@ PROPERTIES
|
|||
|
||||
You can use LOAD LABEL to load data from AWS S3.
|
||||
|
||||
The following examples load the data from all Parquet data files stored in the `s3a://test-bucket/test_brokerload_ingestion` path into the `test_ingestion_2` table in an existing database named `test_s3_db`. For detailed syntax and parameters, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md).
|
||||
The following examples load the data from all Parquet data files stored in the `s3a://test-bucket/test_brokerload_ingestion` path into the `test_ingestion_2` table in an existing database named `test_s3_db`. For detailed syntax and parameters, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md).
|
||||
|
||||
#### Instance profile-based authentication
|
||||
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@ StarRocks supports the following types of Azure Storage accounts:
|
|||
- Azure Data Lake Storage Gen1
|
||||
- Azure Data Lake Storagee Gen2
|
||||
|
||||
In this topic, Hive catalog, file external table, and Broker Load are used as examples to show how StarRocks integrates with Azure Storage by using these types of Azure Storage accounts. For information about the parameters in the examples, see [Hive catalog](../data_source/catalog/hive_catalog.md), [File external table](../data_source/file_external_table.md), and [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md).
|
||||
In this topic, Hive catalog, file external table, and Broker Load are used as examples to show how StarRocks integrates with Azure Storage by using these types of Azure Storage accounts. For information about the parameters in the examples, see [Hive catalog](../data_source/catalog/hive_catalog.md), [File external table](../data_source/file_external_table.md), and [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md).
|
||||
|
||||
## Blob Storage
|
||||
|
||||
|
|
@ -33,7 +33,7 @@ StarRocks supports using one of the following authentication methods to access B
|
|||
|
||||
#### External catalog
|
||||
|
||||
Configure `azure.blob.storage_account` and `azure.blob.shared_key` as follows in the [CREATE EXTERNAL CATALOG](../sql-reference/sql-statements/data-definition/CREATE%20EXTERNAL%20CATALOG.md) statement:
|
||||
Configure `azure.blob.storage_account` and `azure.blob.shared_key` as follows in the [CREATE EXTERNAL CATALOG](../sql-reference/sql-statements/data-definition/CREATE_EXTERNAL_CATALOG.md) statement:
|
||||
|
||||
```SQL
|
||||
CREATE EXTERNAL CATALOG hive_catalog_azure
|
||||
|
|
@ -48,7 +48,7 @@ PROPERTIES
|
|||
|
||||
#### File external table
|
||||
|
||||
Configure `azure.blob.storage_account`, `azure.blob.shared_key`, and the file path (`path`) as follows in the [CREATE EXTERNAL TABLE](../sql-reference/sql-statements/data-definition/CREATE%20TABLE.md) statement:
|
||||
Configure `azure.blob.storage_account`, `azure.blob.shared_key`, and the file path (`path`) as follows in the [CREATE EXTERNAL TABLE](../sql-reference/sql-statements/data-definition/CREATE_TABLE.md) statement:
|
||||
|
||||
```SQL
|
||||
CREATE EXTERNAL TABLE external_table_azure
|
||||
|
|
@ -68,7 +68,7 @@ PROPERTIES
|
|||
|
||||
#### Broker Load
|
||||
|
||||
Configure `azure.blob.storage_account`, `azure.blob.shared_key`, and the file path (`DATA INFILE`) as follows in the [LOAD LABEL](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md) statement:
|
||||
Configure `azure.blob.storage_account`, `azure.blob.shared_key`, and the file path (`DATA INFILE`) as follows in the [LOAD LABEL](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md) statement:
|
||||
|
||||
```SQL
|
||||
LOAD LABEL test_db.label000
|
||||
|
|
@ -88,7 +88,7 @@ WITH BROKER
|
|||
|
||||
#### External catalog
|
||||
|
||||
Configure `azure.blob.account_name`, `azure.blob.container_name`, and `azure.blob.sas_token` as follows in the [CREATE EXTERNAL CATALOG](../sql-reference/sql-statements/data-definition/CREATE%20EXTERNAL%20CATALOG.md) statement:
|
||||
Configure `azure.blob.account_name`, `azure.blob.container_name`, and `azure.blob.sas_token` as follows in the [CREATE EXTERNAL CATALOG](../sql-reference/sql-statements/data-definition/CREATE_EXTERNAL_CATALOG.md) statement:
|
||||
|
||||
```SQL
|
||||
CREATE EXTERNAL CATALOG hive_catalog_azure
|
||||
|
|
@ -104,7 +104,7 @@ PROPERTIES
|
|||
|
||||
#### File external table
|
||||
|
||||
Configure `azure.blob.account_name`, `azure.blob.container_name`, `azure.blob.sas_token`, and the file path (`path`) as follows in the [CREATE EXTERNAL TABLE](../sql-reference/sql-statements/data-definition/CREATE%20TABLE.md) statement:
|
||||
Configure `azure.blob.account_name`, `azure.blob.container_name`, `azure.blob.sas_token`, and the file path (`path`) as follows in the [CREATE EXTERNAL TABLE](../sql-reference/sql-statements/data-definition/CREATE_TABLE.md) statement:
|
||||
|
||||
```SQL
|
||||
CREATE EXTERNAL TABLE external_table_azure
|
||||
|
|
@ -125,7 +125,7 @@ PROPERTIES
|
|||
|
||||
#### Broker load
|
||||
|
||||
Configure `azure.blob.account_name`, `azure.blob.container_name`, `azure.blob.sas_token`, and the file path (`DATA INFILE`) as follows in the [LOAD LABEL](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md) statement:
|
||||
Configure `azure.blob.account_name`, `azure.blob.container_name`, `azure.blob.sas_token`, and the file path (`DATA INFILE`) as follows in the [LOAD LABEL](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md) statement:
|
||||
|
||||
```SQL
|
||||
LOAD LABEL test_db.label000
|
||||
|
|
@ -157,7 +157,7 @@ StarRocks supports using one of the following authentication methods to access D
|
|||
|
||||
#### External catalog
|
||||
|
||||
Configure `azure.adls1.use_managed_service_identity` as follows in the [CREATE EXTERNAL CATALOG](../sql-reference/sql-statements/data-definition/CREATE%20EXTERNAL%20CATALOG.md) statement:
|
||||
Configure `azure.adls1.use_managed_service_identity` as follows in the [CREATE EXTERNAL CATALOG](../sql-reference/sql-statements/data-definition/CREATE_EXTERNAL_CATALOG.md) statement:
|
||||
|
||||
```SQL
|
||||
CREATE EXTERNAL CATALOG hive_catalog_azure
|
||||
|
|
@ -171,7 +171,7 @@ PROPERTIES
|
|||
|
||||
#### File external table
|
||||
|
||||
Configure `azure.adls1.use_managed_service_identity` and the file path (`path`) as follows in the [CREATE EXTERNAL TABLE](../sql-reference/sql-statements/data-definition/CREATE%20TABLE.md) statement:
|
||||
Configure `azure.adls1.use_managed_service_identity` and the file path (`path`) as follows in the [CREATE EXTERNAL TABLE](../sql-reference/sql-statements/data-definition/CREATE_TABLE.md) statement:
|
||||
|
||||
```SQL
|
||||
CREATE EXTERNAL TABLE external_table_azure
|
||||
|
|
@ -190,7 +190,7 @@ PROPERTIES
|
|||
|
||||
#### Broker Load
|
||||
|
||||
Configure `azure.adls1.use_managed_service_identity` and the file path (`DATA INFILE`) as follows in the [LOAD LABEL](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md) statement:
|
||||
Configure `azure.adls1.use_managed_service_identity` and the file path (`DATA INFILE`) as follows in the [LOAD LABEL](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md) statement:
|
||||
|
||||
```SQL
|
||||
LOAD LABEL test_db.label000
|
||||
|
|
@ -209,7 +209,7 @@ WITH BROKER
|
|||
|
||||
#### External catalog
|
||||
|
||||
Configure `azure.adls1.oauth2_client_id`, `azure.adls1.oauth2_credential`, and `azure.adls1.oauth2_endpoint` as follows in the [CREATE EXTERNAL CATALOG](../sql-reference/sql-statements/data-definition/CREATE%20EXTERNAL%20CATALOG.md) statement:
|
||||
Configure `azure.adls1.oauth2_client_id`, `azure.adls1.oauth2_credential`, and `azure.adls1.oauth2_endpoint` as follows in the [CREATE EXTERNAL CATALOG](../sql-reference/sql-statements/data-definition/CREATE_EXTERNAL_CATALOG.md) statement:
|
||||
|
||||
```SQL
|
||||
CREATE EXTERNAL CATALOG hive_catalog_azure
|
||||
|
|
@ -225,7 +225,7 @@ PROPERTIES
|
|||
|
||||
#### File external table
|
||||
|
||||
Configure `azure.adls1.oauth2_client_id`, `azure.adls1.oauth2_credential`, `azure.adls1.oauth2_endpoint`, and the file path (`path`) as follows in the [CREATE EXTERNAL TABLE](../sql-reference/sql-statements/data-definition/CREATE%20TABLE.md) statement:
|
||||
Configure `azure.adls1.oauth2_client_id`, `azure.adls1.oauth2_credential`, `azure.adls1.oauth2_endpoint`, and the file path (`path`) as follows in the [CREATE EXTERNAL TABLE](../sql-reference/sql-statements/data-definition/CREATE_TABLE.md) statement:
|
||||
|
||||
```SQL
|
||||
CREATE EXTERNAL TABLE external_table_azure
|
||||
|
|
@ -246,7 +246,7 @@ PROPERTIES
|
|||
|
||||
#### Broker Load
|
||||
|
||||
Configure `azure.adls1.oauth2_client_id`, `azure.adls1.oauth2_credential`, `azure.adls1.oauth2_endpoint`, and the file path (`DATA INFILE`) as follows in the [LOAD LABEL](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md) statement:
|
||||
Configure `azure.adls1.oauth2_client_id`, `azure.adls1.oauth2_credential`, `azure.adls1.oauth2_endpoint`, and the file path (`DATA INFILE`) as follows in the [LOAD LABEL](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md) statement:
|
||||
|
||||
```SQL
|
||||
LOAD LABEL test_db.label000
|
||||
|
|
@ -288,7 +288,7 @@ Before you start, you need to make the following preparations:
|
|||
|
||||
#### External catalog
|
||||
|
||||
Configure `azure.adls2.oauth2_use_managed_identity`, `azure.adls2.oauth2_tenant_id`, and `azure.adls2.oauth2_client_id` as follows in the [CREATE EXTERNAL CATALOG](../sql-reference/sql-statements/data-definition/CREATE%20EXTERNAL%20CATALOG.md) statement:
|
||||
Configure `azure.adls2.oauth2_use_managed_identity`, `azure.adls2.oauth2_tenant_id`, and `azure.adls2.oauth2_client_id` as follows in the [CREATE EXTERNAL CATALOG](../sql-reference/sql-statements/data-definition/CREATE_EXTERNAL_CATALOG.md) statement:
|
||||
|
||||
```SQL
|
||||
CREATE EXTERNAL CATALOG hive_catalog_azure
|
||||
|
|
@ -304,7 +304,7 @@ PROPERTIES
|
|||
|
||||
#### File external table
|
||||
|
||||
Configure `azure.adls2.oauth2_use_managed_identity`, `azure.adls2.oauth2_tenant_id`, `azure.adls2.oauth2_client_id`, and the file path (`path`) as follows in the [CREATE EXTERNAL TABLE](../sql-reference/sql-statements/data-definition/CREATE%20TABLE.md) statement:
|
||||
Configure `azure.adls2.oauth2_use_managed_identity`, `azure.adls2.oauth2_tenant_id`, `azure.adls2.oauth2_client_id`, and the file path (`path`) as follows in the [CREATE EXTERNAL TABLE](../sql-reference/sql-statements/data-definition/CREATE_TABLE.md) statement:
|
||||
|
||||
```SQL
|
||||
CREATE EXTERNAL TABLE external_table_azure
|
||||
|
|
@ -325,7 +325,7 @@ PROPERTIES
|
|||
|
||||
#### Broker Load
|
||||
|
||||
Configure `azure.adls2.oauth2_use_managed_identity`, `azure.adls2.oauth2_tenant_id`, `azure.adls2.oauth2_client_id`, and the file path (`DATA INFILE`) as follows in the [LOAD LABEL](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md) statement:
|
||||
Configure `azure.adls2.oauth2_use_managed_identity`, `azure.adls2.oauth2_tenant_id`, `azure.adls2.oauth2_client_id`, and the file path (`DATA INFILE`) as follows in the [LOAD LABEL](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md) statement:
|
||||
|
||||
```SQL
|
||||
LOAD LABEL test_db.label000
|
||||
|
|
@ -346,7 +346,7 @@ WITH BROKER
|
|||
|
||||
#### External catalog
|
||||
|
||||
Configure `azure.adls2.storage_account` and `azure.adls2.shared_key` as follows in the [CREATE EXTERNAL CATALOG](../sql-reference/sql-statements/data-definition/CREATE%20EXTERNAL%20CATALOG.md) statement:
|
||||
Configure `azure.adls2.storage_account` and `azure.adls2.shared_key` as follows in the [CREATE EXTERNAL CATALOG](../sql-reference/sql-statements/data-definition/CREATE_EXTERNAL_CATALOG.md) statement:
|
||||
|
||||
```SQL
|
||||
CREATE EXTERNAL CATALOG hive_catalog_azure
|
||||
|
|
@ -361,7 +361,7 @@ PROPERTIES
|
|||
|
||||
#### File external table
|
||||
|
||||
Configure `azure.adls2.storage_account`, `azure.adls2.shared_key`, and the file path (`path`) as follows in the [CREATE EXTERNAL TABLE](../sql-reference/sql-statements/data-definition/CREATE%20TABLE.md) statement:
|
||||
Configure `azure.adls2.storage_account`, `azure.adls2.shared_key`, and the file path (`path`) as follows in the [CREATE EXTERNAL TABLE](../sql-reference/sql-statements/data-definition/CREATE_TABLE.md) statement:
|
||||
|
||||
```SQL
|
||||
CREATE EXTERNAL TABLE external_table_azure
|
||||
|
|
@ -381,7 +381,7 @@ PROPERTIES
|
|||
|
||||
#### Broker Load
|
||||
|
||||
Configure `azure.adls2.storage_account`, `azure.adls2.shared_key`, and the file path (`DATA INFILE`) as follows in the [LOAD LABEL](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md) statement:
|
||||
Configure `azure.adls2.storage_account`, `azure.adls2.shared_key`, and the file path (`DATA INFILE`) as follows in the [LOAD LABEL](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md) statement:
|
||||
|
||||
```SQL
|
||||
LOAD LABEL test_db.label000
|
||||
|
|
@ -403,7 +403,7 @@ Before you start, you need to create a service principal, create a role assignme
|
|||
|
||||
#### External catalog
|
||||
|
||||
Configure `azure.adls2.oauth2_client_id`, `azure.adls2.oauth2_client_secret`, and `azure.adls2.oauth2_client_endpoint` as follows in the [CREATE EXTERNAL CATALOG](../sql-reference/sql-statements/data-definition/CREATE%20EXTERNAL%20CATALOG.md) statement:
|
||||
Configure `azure.adls2.oauth2_client_id`, `azure.adls2.oauth2_client_secret`, and `azure.adls2.oauth2_client_endpoint` as follows in the [CREATE EXTERNAL CATALOG](../sql-reference/sql-statements/data-definition/CREATE_EXTERNAL_CATALOG.md) statement:
|
||||
|
||||
```SQL
|
||||
CREATE EXTERNAL CATALOG hive_catalog_azure
|
||||
|
|
@ -419,7 +419,7 @@ PROPERTIES
|
|||
|
||||
#### File external table
|
||||
|
||||
Configure `azure.adls2.oauth2_client_id`, `azure.adls2.oauth2_client_secret`, `azure.adls2.oauth2_client_endpoint`, and the file path (`path`) as follows in the [CREATE EXTERNAL TABLE](../sql-reference/sql-statements/data-definition/CREATE%20TABLE.md) statement:
|
||||
Configure `azure.adls2.oauth2_client_id`, `azure.adls2.oauth2_client_secret`, `azure.adls2.oauth2_client_endpoint`, and the file path (`path`) as follows in the [CREATE EXTERNAL TABLE](../sql-reference/sql-statements/data-definition/CREATE_TABLE.md) statement:
|
||||
|
||||
```SQL
|
||||
CREATE EXTERNAL TABLE external_table_azure
|
||||
|
|
@ -440,7 +440,7 @@ PROPERTIES
|
|||
|
||||
#### Broker Load
|
||||
|
||||
Configure `azure.adls2.oauth2_client_id`, `azure.adls2.oauth2_client_secret`, `azure.adls2.oauth2_client_endpoint`, and the file path (`DATA INFILE`) as follows in the [LOAD LABEL](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md) statement:
|
||||
Configure `azure.adls2.oauth2_client_id`, `azure.adls2.oauth2_client_secret`, `azure.adls2.oauth2_client_endpoint`, and the file path (`DATA INFILE`) as follows in the [LOAD LABEL](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md) statement:
|
||||
|
||||
```SQL
|
||||
LOAD LABEL test_db.label000
|
||||
|
|
|
|||
|
|
@ -25,7 +25,7 @@ StarRocks can authenticate to GCS in the following scenarios:
|
|||
- Query Parquet and ORC files in GCS.
|
||||
- Query [Hive](../data_source/catalog/hive_catalog.md), [Iceberg](../data_source/catalog/iceberg_catalog.md), [Hudi](../data_source/catalog/hudi_catalog.md), and [Delta Lake](../data_source/catalog/deltalake_catalog.md) tables in GCS.
|
||||
|
||||
In this topic, [Hive catalog](../data_source/catalog/hive_catalog.md), [file external table](../data_source/file_external_table.md), and [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md) are used as examples to show how StarRocks integrates with GCS in different scenarios. For information about `StorageCredentialParams` in the examples, see the "[Parameters](../integrations/authenticate_to_gcs.md#parameters)" section of this topic.
|
||||
In this topic, [Hive catalog](../data_source/catalog/hive_catalog.md), [file external table](../data_source/file_external_table.md), and [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md) are used as examples to show how StarRocks integrates with GCS in different scenarios. For information about `StorageCredentialParams` in the examples, see the "[Parameters](../integrations/authenticate_to_gcs.md#parameters)" section of this topic.
|
||||
|
||||
> **NOTE**
|
||||
>
|
||||
|
|
@ -33,7 +33,7 @@ In this topic, [Hive catalog](../data_source/catalog/hive_catalog.md), [file ext
|
|||
|
||||
### External catalog
|
||||
|
||||
Use the [CREATE EXTERNAL CATALOG](../sql-reference/sql-statements/data-definition/CREATE%20EXTERNAL%20CATALOG.md) statement to create a Hive catalog named `hive_catalog_gcs` as follows, in order to query files from GCS:
|
||||
Use the [CREATE EXTERNAL CATALOG](../sql-reference/sql-statements/data-definition/CREATE_EXTERNAL_CATALOG.md) statement to create a Hive catalog named `hive_catalog_gcs` as follows, in order to query files from GCS:
|
||||
|
||||
```SQL
|
||||
CREATE EXTERNAL CATALOG hive_catalog_gcs
|
||||
|
|
@ -47,7 +47,7 @@ PROPERTIES
|
|||
|
||||
### File external table
|
||||
|
||||
Use the [CREATE EXTERNAL TABLE](../sql-reference/sql-statements/data-definition/CREATE%20TABLE.md) statement to create a file external table named `external_table_gcs` as follows, in order to query a data file named `test_file_external_tbl` from GCS without any metastore:
|
||||
Use the [CREATE EXTERNAL TABLE](../sql-reference/sql-statements/data-definition/CREATE_TABLE.md) statement to create a file external table named `external_table_gcs` as follows, in order to query a data file named `test_file_external_tbl` from GCS without any metastore:
|
||||
|
||||
```SQL
|
||||
CREATE EXTERNAL TABLE external_table_gcs
|
||||
|
|
@ -66,7 +66,7 @@ PROPERTIES
|
|||
|
||||
### Broker load
|
||||
|
||||
Use the [LOAD LABEL](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md) statement to create a Broker Load job whose label is `test_db.label000`, in order to batch load data from GCS into the StarRocks table `target_table`:
|
||||
Use the [LOAD LABEL](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md) statement to create a Broker Load job whose label is `test_db.label000`, in order to batch load data from GCS into the StarRocks table `target_table`:
|
||||
|
||||
```SQL
|
||||
LOAD LABEL test_db.label000
|
||||
|
|
|
|||
|
|
@ -119,7 +119,7 @@ Join our [Slack channel](https://join.slack.com/t/starrocks/shared_invite/zt-z5z
|
|||
<NavBoxPart title="References">
|
||||
<NavBoxPartItem>
|
||||
|
||||
- [SQL reference](../sql-reference/sql-statements/account-management/ALTER%20USER)
|
||||
- [SQL reference](../sql-reference/sql-statements/account-management/ALTER_USER)
|
||||
- [Function reference](../sql-reference/sql-functions/date-time-functions/convert_tz)
|
||||
- [Data type](../sql-reference/sql-statements/data-types/TINYINT)
|
||||
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ import InsertPrivNote from '../assets/commonMarkdown/insertPrivNote.md'
|
|||
|
||||
StarRocks provides the loading method MySQL-based Broker Load to help you load a large amount of data from HDFS or cloud storage into StarRocks.
|
||||
|
||||
Broker Load runs in asynchronous loading mode. After you submit a load job, StarRocks asynchronously runs the job. You need to use the [SHOW LOAD](../sql-reference/sql-statements/data-manipulation/SHOW%20LOAD.md) statement or the `curl` command to check the result of the job.
|
||||
Broker Load runs in asynchronous loading mode. After you submit a load job, StarRocks asynchronously runs the job. You need to use the [SHOW LOAD](../sql-reference/sql-statements/data-manipulation/SHOW_LOAD.md) statement or the `curl` command to check the result of the job.
|
||||
|
||||
Broker Load supports single-table loads and multi-table loads. You can load one or multiple data files into one or multiple destination tables by running one Broker Load job. Broker Load ensures the transactional atomicity of each load job that is run to load multiple data files. Atomicity means that the loading of multiple data files in one load job must all succeed or fail. It never happens that the loading of some data files succeeds while the loading of the other files fails.
|
||||
|
||||
|
|
@ -18,11 +18,11 @@ In v2.4 and earlier, StarRocks depends on brokers to set up connections between
|
|||
|
||||
From v2.5 onwards, StarRocks no longer depends on brokers to set up connections between your StarRocks cluster and your external storage system when it runs a Broker Load job. Therefore, you no longer need to specify a broker in the load statement, but you still need to retain the `WITH BROKER` keyword. This is called "broker-free loading."
|
||||
|
||||
When your data is stored in HDFS, you may encounter situations where broker-free loading does not work. This can happen when your data is stored across multiple HDFS clusters or when you have configured multiple Kerberos users. In these situations, you can resort to using broker-based loading instead. To do this successfully, make sure that at least one independent broker group is deployed. For information about how to specify authentication configuration and HA configuration in these situations, see [HDFS](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md#hdfs).
|
||||
When your data is stored in HDFS, you may encounter situations where broker-free loading does not work. This can happen when your data is stored across multiple HDFS clusters or when you have configured multiple Kerberos users. In these situations, you can resort to using broker-based loading instead. To do this successfully, make sure that at least one independent broker group is deployed. For information about how to specify authentication configuration and HA configuration in these situations, see [HDFS](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md#hdfs).
|
||||
|
||||
> **NOTE**
|
||||
>
|
||||
> You can use the [SHOW BROKER](../sql-reference/sql-statements/Administration/SHOW%20BROKER.md) statement to check for brokers that are deployed in your StarRocks cluster. If no brokers are deployed, you can deploy brokers by following the instructions provided in [Deploy a broker](../deployment/deploy_broker.md).
|
||||
> You can use the [SHOW BROKER](../sql-reference/sql-statements/Administration/SHOW_BROKER.md) statement to check for brokers that are deployed in your StarRocks cluster. If no brokers are deployed, you can deploy brokers by following the instructions provided in [Deploy a broker](../deployment/deploy_broker.md).
|
||||
|
||||
## Supported data file formats
|
||||
|
||||
|
|
@ -67,7 +67,7 @@ The following figure shows the workflow of a Broker Load job.
|
|||
|
||||
### Create a multi-table load job
|
||||
|
||||
This topic uses CSV as an example to describe how to load multiple data files into multiple tables. For information about how to load data in other file formats and about the syntax and parameter descriptions for Broker Load, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md).
|
||||
This topic uses CSV as an example to describe how to load multiple data files into multiple tables. For information about how to load data in other file formats and about the syntax and parameter descriptions for Broker Load, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md).
|
||||
|
||||
Note that in StarRocks some literals are used as reserved keywords by the SQL language. Do not directly use these keywords in SQL statements. If you want to use such a keyword in an SQL statement, enclose it in a pair of backticks (`). See [Keywords](../sql-reference/sql-statements/keywords.md).
|
||||
|
||||
|
|
@ -152,7 +152,7 @@ PROPERTIES
|
|||
);
|
||||
```
|
||||
|
||||
In the preceding example, `StorageCredentialParams` represents a group of authentication parameters which vary depending on the authentication method you choose. For more information, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md#hdfs).
|
||||
In the preceding example, `StorageCredentialParams` represents a group of authentication parameters which vary depending on the authentication method you choose. For more information, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md#hdfs).
|
||||
|
||||
#### Load data from AWS S3
|
||||
|
||||
|
|
@ -181,7 +181,7 @@ WITH BROKER
|
|||
>
|
||||
> Broker Load supports accessing AWS S3 only according to the S3A protocol. Therefore, when you load data from AWS S3, you must replace `s3://` in the S3 URI you pass as the file path with `s3a://`.
|
||||
|
||||
In the preceding example, `StorageCredentialParams` represents a group of authentication parameters which vary depending on the authentication method you choose. For more information, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md#aws-s3).
|
||||
In the preceding example, `StorageCredentialParams` represents a group of authentication parameters which vary depending on the authentication method you choose. For more information, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md#aws-s3).
|
||||
|
||||
From v3.1 onwards, StarRocks supports directly loading the data of Parquet-formatted or ORC-formatted files from AWS S3 by using the INSERT command and the TABLE keyword, saving you from the trouble of creating an external table first. For more information, see [Load data using INSERT > Insert data directly from files in an external source using TABLE keyword](../loading/InsertInto.md#insert-data-directly-from-files-in-an-external-source-using-table-keyword).
|
||||
|
||||
|
|
@ -212,7 +212,7 @@ WITH BROKER
|
|||
>
|
||||
> Broker Load supports accessing Google GCS only according to the gs protocol. Therefore, when you load data from Google GCS, you must include `gs://` as the prefix in the GCS URI that you pass as the file path.
|
||||
|
||||
In the preceding example, `StorageCredentialParams` represents a group of authentication parameters which vary depending on the authentication method you choose. For more information, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md#google-gcs).
|
||||
In the preceding example, `StorageCredentialParams` represents a group of authentication parameters which vary depending on the authentication method you choose. For more information, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md#google-gcs).
|
||||
|
||||
#### Load data from other S3-compatible storage system
|
||||
|
||||
|
|
@ -237,7 +237,7 @@ WITH BROKER
|
|||
);
|
||||
```
|
||||
|
||||
In the preceding example, `StorageCredentialParams` represents a group of authentication parameters which vary depending on the authentication method you choose. For more information, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md#other-s3-compatible-storage-system).
|
||||
In the preceding example, `StorageCredentialParams` represents a group of authentication parameters which vary depending on the authentication method you choose. For more information, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md#other-s3-compatible-storage-system).
|
||||
|
||||
#### Load data from Microsoft Azure Storage
|
||||
|
||||
|
|
@ -274,7 +274,7 @@ WITH BROKER
|
|||
> - If your Data Lake Storage Gen2 allows access only via HTTP, use `abfs://` as the prefix, for example, `abfs://<container>@<storage_account>.dfs.core.windows.net/<file_name>`.
|
||||
> - If your Data Lake Storage Gen2 allows access only via HTTPS, use `abfss://` as the prefix, for example, `abfss://<container>@<storage_account>.dfs.core.windows.net/<file_name>`.
|
||||
|
||||
In the preceding example, `StorageCredentialParams` represents a group of authentication parameters which vary depending on the authentication method you choose. For more information, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md#microsoft-azure-storage).
|
||||
In the preceding example, `StorageCredentialParams` represents a group of authentication parameters which vary depending on the authentication method you choose. For more information, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md#microsoft-azure-storage).
|
||||
|
||||
#### Query data
|
||||
|
||||
|
|
@ -343,7 +343,7 @@ WITH BROKER
|
|||
);
|
||||
```
|
||||
|
||||
In the preceding examples, `StorageCredentialParams` represents a group of authentication parameters which vary depending on the authentication method you choose. For more information, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md#aws-s3).
|
||||
In the preceding examples, `StorageCredentialParams` represents a group of authentication parameters which vary depending on the authentication method you choose. For more information, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md#aws-s3).
|
||||
|
||||
### View a load job
|
||||
|
||||
|
|
@ -351,7 +351,7 @@ Broker Load allows you to view a lob job by using the SHOW LOAD statement or the
|
|||
|
||||
#### Use SHOW LOAD
|
||||
|
||||
For more information, see [SHOW LOAD](../sql-reference/sql-statements/data-manipulation/SHOW%20LOAD.md).
|
||||
For more information, see [SHOW LOAD](../sql-reference/sql-statements/data-manipulation/SHOW_LOAD.md).
|
||||
|
||||
#### Use curl
|
||||
|
||||
|
|
@ -394,7 +394,7 @@ The following table describes the parameters in `jobInfo`.
|
|||
|
||||
### Cancel a load job
|
||||
|
||||
When a load job is not in the **CANCELLED** or **FINISHED** stage, you can use the [CANCEL LOAD](../sql-reference/sql-statements/data-manipulation/CANCEL%20LOAD.md) statement to cancel the job.
|
||||
When a load job is not in the **CANCELLED** or **FINISHED** stage, you can use the [CANCEL LOAD](../sql-reference/sql-statements/data-manipulation/CANCEL_LOAD.md) statement to cancel the job.
|
||||
|
||||
For example, you can execute the following statement to cancel a load job, whose label is `label1`, in the database `test_db`:
|
||||
|
||||
|
|
@ -430,4 +430,4 @@ The [FE configuration item](../administration/Configuration.md#fe-configuration-
|
|||
|
||||
In StarRocks v2.4 and earlier, if the total number of Broker Load jobs that are submitted within a specific period of time exceeds the maximum number, excessive jobs are queued and scheduled based on their submission time.
|
||||
|
||||
Since StarRocks v2.5, if the total number of Broker Load jobs that are submitted within a specific period of time exceeds the maximum number, excessive jobs are queued and scheduled based on their priorities. You can specify a priority for a job by using the `priority` parameter at job creation. See [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md#opt_properties). You can also use [ALTER LOAD](../sql-reference/sql-statements/data-manipulation/ALTER%20LOAD.md) to modify the priority of an existing job that is in the **QUEUEING** or **LOADING** state.
|
||||
Since StarRocks v2.5, if the total number of Broker Load jobs that are submitted within a specific period of time exceeds the maximum number, excessive jobs are queued and scheduled based on their priorities. You can specify a priority for a job by using the `priority` parameter at job creation. See [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md#opt_properties). You can also use [ALTER LOAD](../sql-reference/sql-statements/data-manipulation/ALTER_LOAD.md) to modify the priority of an existing job that is in the **QUEUEING** or **LOADING** state.
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ import InsertPrivNote from '../assets/commonMarkdown/insertPrivNote.md'
|
|||
|
||||
StarRocks supports data transformation at loading.
|
||||
|
||||
This feature supports [Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md), [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md), and [Routine Load](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md) but does not support [Spark Load](../sql-reference/sql-statements/data-manipulation/SPARK%20LOAD.md).
|
||||
This feature supports [Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md), [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md), and [Routine Load](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md) but does not support [Spark Load](../sql-reference/sql-statements/data-manipulation/SPARK_LOAD.md).
|
||||
|
||||
<InsertPrivNote />
|
||||
|
||||
|
|
@ -144,7 +144,7 @@ curl --location-trusted -u <username>:<password> \
|
|||
>
|
||||
> If you choose Stream Load, you must use the `columns` parameter to temporarily name the columns of the data file to create a column mapping between the data file and the StarRocks table.
|
||||
|
||||
For detailed syntax and parameter descriptions, see [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md).
|
||||
For detailed syntax and parameter descriptions, see [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md).
|
||||
|
||||
#### Load data from an HDFS cluster
|
||||
|
||||
|
|
@ -166,7 +166,7 @@ WITH BROKER "broker1";
|
|||
>
|
||||
> If you choose Broker Load, you must use the `column_list` parameter to temporarily name the columns of the data file to create a column mapping between the data file and the StarRocks table.
|
||||
|
||||
For detailed syntax and parameter descriptions, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md).
|
||||
For detailed syntax and parameter descriptions, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md).
|
||||
|
||||
#### Load data from a Kafka cluster
|
||||
|
||||
|
|
@ -188,7 +188,7 @@ FROM KAFKA
|
|||
>
|
||||
> If you choose Routine Load, you must use the `COLUMNS` parameter to temporarily name the columns of the data file to create a column mapping between the data file and the StarRocks table.
|
||||
|
||||
For detailed syntax and parameter descriptions, see [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md).
|
||||
For detailed syntax and parameter descriptions, see [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md).
|
||||
|
||||
### Query data
|
||||
|
||||
|
|
@ -240,7 +240,7 @@ curl --location-trusted -u <username>:<password> \
|
|||
http://<fe_host>:<fe_http_port>/api/test_db/table1/_stream_load
|
||||
```
|
||||
|
||||
For detailed syntax and parameter descriptions, see [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md).
|
||||
For detailed syntax and parameter descriptions, see [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md).
|
||||
|
||||
#### Load data from an HDFS cluster
|
||||
|
||||
|
|
@ -259,7 +259,7 @@ LOAD LABEL test_db.label2
|
|||
WITH BROKER "broker1";
|
||||
```
|
||||
|
||||
For detailed syntax and parameter descriptions, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md).
|
||||
For detailed syntax and parameter descriptions, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md).
|
||||
|
||||
#### Load data from a Kafka cluster
|
||||
|
||||
|
|
@ -278,7 +278,7 @@ FROM KAFKA
|
|||
);
|
||||
```
|
||||
|
||||
For detailed syntax and parameter descriptions, see [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md).
|
||||
For detailed syntax and parameter descriptions, see [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md).
|
||||
|
||||
### Query data
|
||||
|
||||
|
|
@ -333,7 +333,7 @@ curl --location-trusted -u <username>:<password> \
|
|||
>
|
||||
> - Stream Load does not support `column_name = function(column_name)` but supports `column_name = function(column_name)`.
|
||||
|
||||
For detailed syntax and parameter descriptions, see [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md).
|
||||
For detailed syntax and parameter descriptions, see [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md).
|
||||
|
||||
#### Load data from an HDFS cluster
|
||||
|
||||
|
|
@ -356,7 +356,7 @@ WITH BROKER "broker1";
|
|||
>
|
||||
> You must first use the `column_list` parameter to temporarily name **all columns** of the data file, and then use the SET clause to temporarily name the new columns that you want to generate from the original columns of the data file. As shown in the preceding example, the only column of `file2.csv` is temporarily named as `date` in the `column_list` parameter, and then the `year=year(date)`, `month=month(date)`, and `day=day(date)` functions are invoked in the SET clause to generate three new columns, which are temporarily named as `year`, `month`, and `day`.
|
||||
|
||||
For detailed syntax and parameter descriptions, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md).
|
||||
For detailed syntax and parameter descriptions, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md).
|
||||
|
||||
#### Load data from a Kafka cluster
|
||||
|
||||
|
|
@ -378,7 +378,7 @@ FROM KAFKA
|
|||
>
|
||||
> In the `COLUMNS` parameter, you must first temporarily name **all columns** of the data file, and then temporarily name the new columns that you want to generate from the original columns of the data file. As shown in the preceding example, the only column of `file2.csv` is temporarily named as `date`, and then the `year=year(date)`, `month=month(date)`, and `day=day(date)` functions are invoked to generate three new columns, which are temporarily named as `year`, `month`, and `day`.
|
||||
|
||||
For detailed syntax and parameter descriptions, see [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md).
|
||||
For detailed syntax and parameter descriptions, see [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md).
|
||||
|
||||
### Query data
|
||||
|
||||
|
|
@ -438,7 +438,7 @@ WITH BROKER "broker1";
|
|||
>
|
||||
> In the preceding example, the `date` partition field in the specified file path is equivalent to the `event_date` column of `table1`. Therefore, you need to use the SET clause to map the `date` partition field onto the `event_date` column. If the partition field in the specified file path has the same name as a column of the StarRocks table, you do not need to use the SET clause to create a mapping.
|
||||
|
||||
For detailed syntax and parameter descriptions, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md).
|
||||
For detailed syntax and parameter descriptions, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md).
|
||||
|
||||
### Query data
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Continuously load data from Apache Flink®
|
||||
|
||||
StarRocks provides a self-developed connector named Flink connector for Apache Flink® (Flink connector for short) to help you load data into a StarRocks table by using Flink. The basic principle is to accumulate the data and then load it all at a time into StarRocks through [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD).
|
||||
StarRocks provides a self-developed connector named Flink connector for Apache Flink® (Flink connector for short) to help you load data into a StarRocks table by using Flink. The basic principle is to accumulate the data and then load it all at a time into StarRocks through [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD).
|
||||
|
||||
The Flink connector supports DataStream API, Table API & SQL, and Python API. It has a higher and more stable performance than [flink-connector-jdbc](https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/jdbc/) provided by Apache Flink®.
|
||||
|
||||
|
|
@ -103,13 +103,13 @@ In your Maven project's `pom.xml` file, add the Flink connector as a dependency
|
|||
| sink.connect.timeout-ms | No | 1000 | The timeout for establishing HTTP connection. Valid values: 100 to 60000. Unit: ms. |
|
||||
| sink.wait-for-continue.timeout-ms | No | 10000 | Supported since 1.2.7. The timeout for waiting response of HTTP 100-continue from the FE. Valid values: `3000` to `600000`. Unit: ms |
|
||||
| sink.ignore.update-before | No | true | Supported since version 1.2.8. Whether to ignore `UPDATE_BEFORE` records from Flink when loading data to Primary Key tables. If this parameter is set to false, the record is treated as a delete operation to StarRocks table. |
|
||||
| sink.properties.* | No | NONE | The parameters that are used to control Stream Load behavior. For example, the parameter `sink.properties.format` specifies the format used for Stream Load, such as CSV or JSON. For a list of supported parameters and their descriptions, see [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md). |
|
||||
| sink.properties.* | No | NONE | The parameters that are used to control Stream Load behavior. For example, the parameter `sink.properties.format` specifies the format used for Stream Load, such as CSV or JSON. For a list of supported parameters and their descriptions, see [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md). |
|
||||
| sink.properties.format | No | csv | The format used for Stream Load. The Flink connector will transform each batch of data to the format before sending them to StarRocks. Valid values: `csv` and `json`. |
|
||||
| sink.properties.row_delimiter | No | \n | The row delimiter for CSV-formatted data. |
|
||||
| sink.properties.column_separator | No | \t | The column separator for CSV-formatted data. |
|
||||
| sink.properties.max_filter_ratio | No | 0 | The maximum error tolerance of the Stream Load. It's the maximum percentage of data records that can be filtered out due to inadequate data quality. Valid values: `0` to `1`. Default value: `0`. See [Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md) for details. |
|
||||
| sink.properties.max_filter_ratio | No | 0 | The maximum error tolerance of the Stream Load. It's the maximum percentage of data records that can be filtered out due to inadequate data quality. Valid values: `0` to `1`. Default value: `0`. See [Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md) for details. |
|
||||
| sink.parallelism | No | NONE | The parallelism of the connector. Only available for Flink SQL. If not set, Flink planner will decide the parallelism. In the scenario of multi-parallelism, users need to guarantee data is written in the correct order. |
|
||||
| sink.properties.strict_mode | No | false | Specifies whether to enable the strict mode for Stream Load. It affects the loading behavior when there are unqualified rows, such as inconsistent column values. Valid values: `true` and `false`. Default value: `false`. See [Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md) for details. |
|
||||
| sink.properties.strict_mode | No | false | Specifies whether to enable the strict mode for Stream Load. It affects the loading behavior when there are unqualified rows, such as inconsistent column values. Valid values: `true` and `false`. Default value: `false`. See [Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md) for details. |
|
||||
|
||||
## Data type mapping between Flink and StarRocks
|
||||
|
||||
|
|
|
|||
|
|
@ -17,10 +17,10 @@ StarRocks v2.4 further supports overwriting data into a table by using INSERT OV
|
|||
## Precautions
|
||||
|
||||
- You can cancel a synchronous INSERT transaction only by pressing the **Ctrl** and **C** keys from your MySQL client.
|
||||
- You can submit an asynchronous INSERT task using [SUBMIT TASK](../sql-reference/sql-statements/data-manipulation/SUBMIT%20TASK.md).
|
||||
- You can submit an asynchronous INSERT task using [SUBMIT TASK](../sql-reference/sql-statements/data-manipulation/SUBMIT_TASK.md).
|
||||
- As for the current version of StarRocks, the INSERT transaction fails by default if the data of any rows does not comply with the schema of the table. For example, the INSERT transaction fails if the length of a field in any row exceeds the length limit for the mapping field in the table. You can set the session variable `enable_insert_strict` to `false` to allow the transaction to continue by filtering out the rows that mismatch the table.
|
||||
- If you execute the INSERT statement frequently to load small batches of data into StarRocks, excessive data versions are generated. It severely affects query performance. We recommend that, in production, you should not load data with the INSERT command too often or use it as a routine for data loading on a daily basis. If your application or analytic scenario demand solutions to loading streaming data or small data batches separately, we recommend you use Apache Kafka® as your data source and load the data via [Routine Load](../loading/RoutineLoad.md).
|
||||
- If you execute the INSERT OVERWRITE statement, StarRocks creates temporary partitions for the partitions which store the original data, inserts new data into the temporary partitions, and [swaps the original partitions with the temporary partitions](../sql-statements/data-definition/ALTER%20TABLE.md#use-a-temporary-partition-to-replace-current-partition). All these operations are executed in the FE Leader node. Hence, if the FE Leader node crashes while executing INSERT OVERWRITE command, the whole load transaction will fail, and the temporary partitions will be truncated.
|
||||
- If you execute the INSERT OVERWRITE statement, StarRocks creates temporary partitions for the partitions which store the original data, inserts new data into the temporary partitions, and [swaps the original partitions with the temporary partitions](../sql-statements/data-definition/ALTER_TABLE.md#use-a-temporary-partition-to-replace-current-partition). All these operations are executed in the FE Leader node. Hence, if the FE Leader node crashes while executing INSERT OVERWRITE command, the whole load transaction will fail, and the temporary partitions will be truncated.
|
||||
|
||||
## Preparation
|
||||
|
||||
|
|
@ -361,7 +361,7 @@ mysql> SELECT * FROM insert_generated_columns;
|
|||
|
||||
## Load data asynchronously using INSERT
|
||||
|
||||
Loading data with INSERT submits a synchronous transaction, which may fail because of session interruption or timeout. You can submit an asynchronous INSERT transaction using [SUBMIT TASK](../sql-reference/sql-statements/data-manipulation/SUBMIT%20TASK.md). This feature is supported since StarRocks v2.5.
|
||||
Loading data with INSERT submits a synchronous transaction, which may fail because of session interruption or timeout. You can submit an asynchronous INSERT transaction using [SUBMIT TASK](../sql-reference/sql-statements/data-manipulation/SUBMIT_TASK.md). This feature is supported since StarRocks v2.5.
|
||||
|
||||
- The following example asynchronously inserts the data from the source table to the target table `insert_wiki_edit`.
|
||||
|
||||
|
|
|
|||
|
|
@ -122,7 +122,7 @@ The following steps take a self-managed Kafka cluster as an example to demonstra
|
|||
| bufferflush.intervalms | NO | 300000 | Interval for sending a batch of data which controls the load latency. Range: [1000, 3600000]. |
|
||||
| connect.timeoutms | NO | 1000 | Timeout for connecting to the HTTP URL. Range: [100, 60000]. |
|
||||
| sink.properties.* | | | Stream Load parameters o control load behavior. For example, the parameter `sink.properties.format` specifies the format used for Stream Load, such as CSV or JSON. For a list of supported parameters and their descriptions, see [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM LOAD.md). |
|
||||
| sink.properties.format | NO | json | The format used for Stream Load. The Kafka connector will transform each batch of data to the format before sending them to StarRocks. Valid values: `csv` and `json`. For more information, see [CSV parameters**](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD#csv-parameters)和** [JSON parameters](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD#json-parameters). |
|
||||
| sink.properties.format | NO | json | The format used for Stream Load. The Kafka connector will transform each batch of data to the format before sending them to StarRocks. Valid values: `csv` and `json`. For more information, see [CSV parameters**](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD#csv-parameters)和** [JSON parameters](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD#json-parameters). |
|
||||
|
||||
## Limits
|
||||
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
import InsertPrivNote from '../assets/commonMarkdown/insertPrivNote.md'
|
||||
|
||||
[Primary Key tables](../table_design/table_types/primary_key_table.md) provided by StarRocks allow you to make data changes to StarRocks tables by running [Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md), [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md), or [Routine Load](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md) jobs. These data changes include inserts, updates, and deletions. However, Primary Key tables do not support changing data by using [Spark Load](../sql-reference/sql-statements/data-manipulation/SPARK%20LOAD.md) or [INSERT](../sql-reference/sql-statements/data-manipulation/insert.md).
|
||||
[Primary Key tables](../table_design/table_types/primary_key_table.md) provided by StarRocks allow you to make data changes to StarRocks tables by running [Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md), [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md), or [Routine Load](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md) jobs. These data changes include inserts, updates, and deletions. However, Primary Key tables do not support changing data by using [Spark Load](../sql-reference/sql-statements/data-manipulation/SPARK_LOAD.md) or [INSERT](../sql-reference/sql-statements/data-manipulation/insert.md).
|
||||
|
||||
StarRocks also supports partial updates and conditional updates.
|
||||
|
||||
|
|
@ -58,7 +58,7 @@ If you choose Routine Load, make sure that topics are created in your Apache Kaf
|
|||
|
||||
## Basic operations
|
||||
|
||||
This section provides examples of how to make data changes to a StarRocks table through loading. For detailed syntax and parameter descriptions, see [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md), [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md), and [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md).
|
||||
This section provides examples of how to make data changes to a StarRocks table through loading. For detailed syntax and parameter descriptions, see [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md), [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md), and [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md).
|
||||
|
||||
### UPSERT
|
||||
|
||||
|
|
|
|||
|
|
@ -392,4 +392,4 @@ INSERT INTO user_behavior_declared
|
|||
- This document only covered IAM user-based authentication. For other options please see [authenticate to AWS resources](../integrations/authenticate_to_aws_resources.md).
|
||||
- The [AWS CLI Command Reference](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3/index.html) covers the S3 URI in detail.
|
||||
- Learn more about [table design](../table_design/StarRocks_table_design.md).
|
||||
- Broker Load provides many more configuration and use options than those in the above examples, the details are in [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md)
|
||||
- Broker Load provides many more configuration and use options than those in the above examples, the details are in [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md)
|
||||
|
|
|
|||
|
|
@ -46,7 +46,7 @@ In asynchronous loading mode, after you submit a load job, StarRocks immediately
|
|||
|
||||
- If the result indicates a job creation failure, you can determine whether you need to retry the job based on the failure information.
|
||||
|
||||
StarRocks provides three loading methods that support asynchronous loading: [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md), [Routine Load](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md), and [Spark Load](../sql-reference/sql-statements/data-manipulation/SPARK%20LOAD.md).
|
||||
StarRocks provides three loading methods that support asynchronous loading: [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md), [Routine Load](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md), and [Spark Load](../sql-reference/sql-statements/data-manipulation/SPARK_LOAD.md).
|
||||
|
||||
The process of asynchronous loading is as follows:
|
||||
|
||||
|
|
@ -101,7 +101,7 @@ The workflow of a Routine job is described as follows:
|
|||
|
||||
## Loading methods
|
||||
|
||||
StarRocks provides five loading methods to help you load data in various business scenarios: [Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md), [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md), [Routine Load](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md), [Spark Load](../sql-reference/sql-statements/data-manipulation/SPARK%20LOAD.md), and [INSERT](../sql-reference/sql-statements/data-manipulation/insert.md).
|
||||
StarRocks provides five loading methods to help you load data in various business scenarios: [Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md), [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md), [Routine Load](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md), [Spark Load](../sql-reference/sql-statements/data-manipulation/SPARK_LOAD.md), and [INSERT](../sql-reference/sql-statements/data-manipulation/insert.md).
|
||||
|
||||
| Loading method | Data source | Business scenario | Data volume per load job | Data file format | Loading mode | Protocol |
|
||||
| ------------------ | -------------------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ----------------------------------------------- | ------------ | -------- |
|
||||
|
|
@ -136,7 +136,7 @@ The following figure provides an overview of various data sources supported by S
|
|||
|
||||
StarRocks provides parameters for you to limit the memory usage for each load job, thereby reducing memory consumption, especially in high concurrency scenarios. However, do not specify an excessively low memory usage limit. If the memory usage limit is excessively low, data may be frequently flushed from memory to disk because the memory usage for load jobs reaches the specified limit. We recommend that you specify a proper memory usage limit based on your business scenario.
|
||||
|
||||
The parameters that are used to limit memory usage vary for each loading method. For more information, see [Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md), [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md), [Routine Load](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md), [Spark Load](../sql-reference/sql-statements/data-manipulation/SPARK%20LOAD.md), and [INSERT](../sql-reference/sql-statements/data-manipulation/insert.md). Note that a load job usually runs on multiple BEs. Therefore, the parameters limit the memory usage of each load job on each involved BE rather than the total memory usage of the load job on all involved BEs.
|
||||
The parameters that are used to limit memory usage vary for each loading method. For more information, see [Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md), [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md), [Routine Load](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md), [Spark Load](../sql-reference/sql-statements/data-manipulation/SPARK_LOAD.md), and [INSERT](../sql-reference/sql-statements/data-manipulation/insert.md). Note that a load job usually runs on multiple BEs. Therefore, the parameters limit the memory usage of each load job on each involved BE rather than the total memory usage of the load job on all involved BEs.
|
||||
|
||||
StarRocks also provides parameters for you to limit the total memory usage of all load jobs that run on each individual BE. For more information, see the "[System configurations](../loading/Loading_intro.md#system-configurations)" section of this topic.
|
||||
|
||||
|
|
@ -148,7 +148,7 @@ When you load data, you can choose not to load the data from a specific field of
|
|||
|
||||
- If you have specified the `DEFAULT` keyword for the destination StarRocks table column mapping the source field when you create the StarRocks table, StarRocks automatically fills the specified default value into the destination column.
|
||||
|
||||
[Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md), [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md), [Routine Load](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md), and [INSERT](../sql-reference/sql-statements/data-manipulation/insert.md) support `DEFAULT current_timestamp`, `DEFAULT <default_value>`, and `DEFAULT (<expression>)`. [Spark Load](../sql-reference/sql-statements/data-manipulation/SPARK%20LOAD.md) supports only `DEFAULT current_timestamp` and `DEFAULT <default_value>`.
|
||||
[Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md), [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md), [Routine Load](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md), and [INSERT](../sql-reference/sql-statements/data-manipulation/insert.md) support `DEFAULT current_timestamp`, `DEFAULT <default_value>`, and `DEFAULT (<expression>)`. [Spark Load](../sql-reference/sql-statements/data-manipulation/SPARK_LOAD.md) supports only `DEFAULT current_timestamp` and `DEFAULT <default_value>`.
|
||||
|
||||
> **NOTE**
|
||||
>
|
||||
|
|
@ -160,13 +160,13 @@ When you load data, you can choose not to load the data from a specific field of
|
|||
>
|
||||
> If the destination column is defined as `NOT NULL`, the load fails.
|
||||
|
||||
For [Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md), [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md), [Routine Load](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md), and [Spark Load](../sql-reference/sql-statements/data-manipulation/SPARK%20LOAD.md), you can also specify the value you want to fill in the destination column by using the parameter that is used to specify column mapping.
|
||||
For [Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md), [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md), [Routine Load](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md), and [Spark Load](../sql-reference/sql-statements/data-manipulation/SPARK_LOAD.md), you can also specify the value you want to fill in the destination column by using the parameter that is used to specify column mapping.
|
||||
|
||||
For information about the usage of `NOT NULL` and `DEFAULT`, see [CREATE TABLE](../sql-reference/sql-statements/data-definition/CREATE%20TABLE.md).
|
||||
For information about the usage of `NOT NULL` and `DEFAULT`, see [CREATE TABLE](../sql-reference/sql-statements/data-definition/CREATE_TABLE.md).
|
||||
|
||||
### Set write quorum for data loading
|
||||
|
||||
If your StarRocks cluster has multiple data replicas, you can set different write quorum for tables, that is, how many replicas are required to return loading success before StarRocks can determine the loading task is successful. You can specify write quorum by adding the property `write_quorum` when you [CREATE TABLE](../sql-reference/sql-statements/data-definition/CREATE%20TABLE.md), or add this property to an existing table using [ALTER TABLE](../sql-reference/sql-statements/data-definition/ALTER%20TABLE.md). This property is supported from v2.5.
|
||||
If your StarRocks cluster has multiple data replicas, you can set different write quorum for tables, that is, how many replicas are required to return loading success before StarRocks can determine the loading task is successful. You can specify write quorum by adding the property `write_quorum` when you [CREATE TABLE](../sql-reference/sql-statements/data-definition/CREATE_TABLE.md), or add this property to an existing table using [ALTER TABLE](../sql-reference/sql-statements/data-definition/ALTER_TABLE.md). This property is supported from v2.5.
|
||||
|
||||
## System configurations
|
||||
|
||||
|
|
|
|||
|
|
@ -40,7 +40,7 @@ Routine Load now supports consuming CSV, JSON, and Avro (supported since v3.0.1)
|
|||
### Workflow
|
||||
|
||||
1. **Create a Routine Load job.**
|
||||
To load data from Kafka, you need to create a Routine Load job by running the [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md) statement. The FE parses the statement, and creates the job according to the properties you have specified.
|
||||
To load data from Kafka, you need to create a Routine Load job by running the [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md) statement. The FE parses the statement, and creates the job according to the properties you have specified.
|
||||
|
||||
2. **The FE splits the job into multiple load tasks.**
|
||||
|
||||
|
|
@ -56,20 +56,20 @@ Routine Load now supports consuming CSV, JSON, and Avro (supported since v3.0.1)
|
|||
|
||||
3. **Multiple load tasks run concurrently to consume the messages from multiple Kafka topic partitions, and load the data into StarRocks**
|
||||
|
||||
1. **The FE schedules and submits load tasks**: the FE schedules the load tasks in the queue on a timely basis, and assigns them to selected Coordinator BE nodes. The interval between load tasks is defined by the configuration item `max_batch_interval`. The FE distributes the load tasks evenly to all BE nodes. See [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md#example) for more information about `max_batch_interval`.
|
||||
1. **The FE schedules and submits load tasks**: the FE schedules the load tasks in the queue on a timely basis, and assigns them to selected Coordinator BE nodes. The interval between load tasks is defined by the configuration item `max_batch_interval`. The FE distributes the load tasks evenly to all BE nodes. See [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md#example) for more information about `max_batch_interval`.
|
||||
|
||||
2. The Coordinator BE starts the load task, consumes messages in partitions, parses and filters the data. A load task lasts until the pre-defined amount of messages are consumed or the pre-defined time limit is reached. The message batch size and time limit are defined in the FE configurations `max_routine_load_batch_size` and `routine_load_task_consume_second`. For detailed information, see [Configuration](../administration/Configuration.md). The Coordinator BE then distributes the messages to the Executor BEs. The Executor BEs write the messages to disks.
|
||||
|
||||
> **NOTE**
|
||||
>
|
||||
> StarRocks supports access to Kafka via a security authentication mechanism SASL_SSL, SASL or SSL, or without authentication. This topic takes connection to Kafka without authentication as an example. If you need to connect to Kafka via a security authentication mechanism, see [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md).
|
||||
> StarRocks supports access to Kafka via a security authentication mechanism SASL_SSL, SASL or SSL, or without authentication. This topic takes connection to Kafka without authentication as an example. If you need to connect to Kafka via a security authentication mechanism, see [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md).
|
||||
|
||||
4. **The FE generates new load tasks to load data continuously.**
|
||||
After the Executor BEs has written the data to disks, the Coordinator BE reports the result of the load task to the FE. Based on the result, the FE then generates new load tasks to load the data continuously. Or the FE retries the failed tasks to make sure the data loaded into StarRocks is neither lost nor duplicated.
|
||||
|
||||
## Create a Routine Load job
|
||||
|
||||
The following three examples describe how to consume CSV-format, JSON-format and Avro-format data in Kafka, and load the data into StarRocks by creating a Routine Load job. For detailed syntax and parameter descriptions, see [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md).
|
||||
The following three examples describe how to consume CSV-format, JSON-format and Avro-format data in Kafka, and load the data into StarRocks by creating a Routine Load job. For detailed syntax and parameter descriptions, see [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md).
|
||||
|
||||
### Load CSV-format data
|
||||
|
||||
|
|
@ -130,7 +130,7 @@ FROM KAFKA
|
|||
);
|
||||
```
|
||||
|
||||
After submitting the load job, you can execute the [SHOW ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/SHOW%20ROUTINE%20LOAD.md) statement to check the status of the load job.
|
||||
After submitting the load job, you can execute the [SHOW ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/SHOW_ROUTINE_LOAD.md) statement to check the status of the load job.
|
||||
|
||||
- **load job name**
|
||||
|
||||
|
|
@ -156,7 +156,7 @@ After submitting the load job, you can execute the [SHOW ROUTINE LOAD](../sql-re
|
|||
"property.kafka_default_offsets" = "OFFSET_BEGINNING"
|
||||
```
|
||||
|
||||
For detailed information, see [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md).
|
||||
For detailed information, see [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md).
|
||||
|
||||
- **Data mapping and transformation**
|
||||
|
||||
|
|
@ -182,13 +182,13 @@ After submitting the load job, you can execute the [SHOW ROUTINE LOAD](../sql-re
|
|||
|
||||
When there are many Kafka topic partitions and enough BE nodes, you can accelerate the loading by increasing the task concurrency.
|
||||
|
||||
To increase the actual load task concurrency, you can increase the desired load task concurrency `desired_concurrent_number` when you [create a routine load job](./RoutineLoad#Create a Routine%20Load%20job). You can also set the dynamic configuration item of FE `max_routine_load_task_concurrent_num` ( default maximum load task currency ) to a larger value. For more information about `max_routine_load_task_concurrent_num`, please see [FE configuration items](../administration/Configuration#fe-configuration-items).
|
||||
To increase the actual load task concurrency, you can increase the desired load task concurrency `desired_concurrent_number` when you create a routine load job. You can also set the dynamic configuration item of FE `max_routine_load_task_concurrent_num` ( default maximum load task currency ) to a larger value. For more information about `max_routine_load_task_concurrent_num`, please see [FE configuration items](../administration/Configuration#fe-configuration-items).
|
||||
|
||||
The actual task concurrency is defined by the minimum value among the number of BE nodes that are alive, the number of the pre-specified Kafka topic partitions, and the values of `desired_concurrent_number` and `max_routine_load_task_concurrent_num`.
|
||||
|
||||
In the example, the number of BE nodes that are alive is `5`, the number of the pre-specified Kafka topic partitions is `5`, and the value of `max_routine_load_task_concurrent_num` is `5`. To increase the actual load task concurrency, you can increase the `desired_concurrent_number` from the default value `3` to `5`.
|
||||
|
||||
For more about the properties, see [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md). For detailed instructions on accelerating the loading, see [Routine Load FAQ](../faq/loading/Routine_load_faq).
|
||||
For more about the properties, see [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md). For detailed instructions on accelerating the loading, see [Routine Load FAQ](../faq/loading/Routine_load_faq).
|
||||
|
||||
### Load JSON-format data
|
||||
|
||||
|
|
@ -250,7 +250,7 @@ FROM KAFKA
|
|||
);
|
||||
```
|
||||
|
||||
After submitting the load job, you can execute the [SHOW ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/SHOW%20ROUTINE%20LOAD.md) statement to check the status of the load job.
|
||||
After submitting the load job, you can execute the [SHOW ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/SHOW_ROUTINE_LOAD.md) statement to check the status of the load job.
|
||||
|
||||
- **Data format**
|
||||
|
||||
|
|
@ -375,7 +375,7 @@ FROM KAFKA
|
|||
>
|
||||
> You do not need to specify the `COLUMNS` parameter if the names and number of the fields in the Avro record completely match those of columns in the StarRocks table.
|
||||
|
||||
After submitting the load job, you can execute the [SHOW ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/SHOW%20ROUTINE%20LOAD.md) statement to check the status of the load job.
|
||||
After submitting the load job, you can execute the [SHOW ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/SHOW_ROUTINE_LOAD.md) statement to check the status of the load job.
|
||||
|
||||
#### Data types mapping
|
||||
|
||||
|
|
@ -414,9 +414,9 @@ The data type mapping between the Avro data fields you want to load and the Star
|
|||
|
||||
### Check a load job
|
||||
|
||||
Execute the [SHOW ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/SHOW%20ROUTINE%20LOAD.md) statement to check the status of the load job `example_tbl2_ordertest2`. StarRocks returns the execution state `State`, the statistical information (including the total rows consumed and the total rows loaded) `Statistics`, and the progress of the load job `progress`.
|
||||
Execute the [SHOW ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/SHOW_ROUTINE_LOAD.md) statement to check the status of the load job `example_tbl2_ordertest2`. StarRocks returns the execution state `State`, the statistical information (including the total rows consumed and the total rows loaded) `Statistics`, and the progress of the load job `progress`.
|
||||
|
||||
If the state of the load job is automatically changed to **PAUSED**, it is possibly because the number of error rows has exceeded the threshold. For detailed instructions on setting this threshold, see [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md). You can check the files `ReasonOfStateChanged` and `ErrorLogUrls` to identify and troubleshoot the problem. Having fixed the problem, you can then execute the [RESUME ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/RESUME%20ROUTINE%20LOAD.md) statement to resume the **PAUSED** load job.
|
||||
If the state of the load job is automatically changed to **PAUSED**, it is possibly because the number of error rows has exceeded the threshold. For detailed instructions on setting this threshold, see [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md). You can check the files `ReasonOfStateChanged` and `ErrorLogUrls` to identify and troubleshoot the problem. Having fixed the problem, you can then execute the [RESUME ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/RESUME_ROUTINE_LOAD.md) statement to resume the **PAUSED** load job.
|
||||
|
||||
If the state of the load job is **CANCELLED**, it is possibly because the load job encounters an exception (such as the table has been dropped). You can check the files `ReasonOfStateChanged` and `ErrorLogUrls` to identify and troubleshoot the problem. However, you cannot resume a **CANCELLED** load job.
|
||||
|
||||
|
|
@ -449,7 +449,7 @@ ReasonOfStateChanged:
|
|||
|
||||
### Check a load task
|
||||
|
||||
Execute the [SHOW ROUTINE LOAD TASK](../sql-reference/sql-statements/data-manipulation/SHOW%20ROUTINE%20LOAD%20TASK.md) statement to check the load tasks of the load job `example_tbl2_ordertest2`, such as how many tasks are currently running, the Kafka topic partitions that are consumed and the consumption progress `DataSourceProperties`, and the corresponding Coordinator BE node `BeId`.
|
||||
Execute the [SHOW ROUTINE LOAD TASK](../sql-reference/sql-statements/data-manipulation/SHOW_ROUTINE_LOAD_TASK.md) statement to check the load tasks of the load job `example_tbl2_ordertest2`, such as how many tasks are currently running, the Kafka topic partitions that are consumed and the consumption progress `DataSourceProperties`, and the corresponding Coordinator BE node `BeId`.
|
||||
|
||||
```SQL
|
||||
MySQL [example_db]> SHOW ROUTINE LOAD TASK WHERE JobName = "example_tbl2_ordertest2" \G
|
||||
|
|
@ -493,7 +493,7 @@ DataSourceProperties: {"0":2,"3":0}
|
|||
|
||||
## Pause a load job
|
||||
|
||||
You can execute the [PAUSE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/PAUSE%20ROUTINE%20LOAD.md) statement to pause a load job. The state of the load job will be **PAUSED** after the statement is executed. However, it has not stopped. You can execute the [RESUME ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/RESUME%20ROUTINE%20LOAD.md) statement to resume it. You can also check its status with the [SHOW ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/SHOW%20ROUTINE%20LOAD.md) statement.
|
||||
You can execute the [PAUSE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/PAUSE_ROUTINE_LOAD.md) statement to pause a load job. The state of the load job will be **PAUSED** after the statement is executed. However, it has not stopped. You can execute the [RESUME ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/RESUME_ROUTINE_LOAD.md) statement to resume it. You can also check its status with the [SHOW ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/SHOW_ROUTINE_LOAD.md) statement.
|
||||
|
||||
The following example pauses the load job `example_tbl2_ordertest2`:
|
||||
|
||||
|
|
@ -503,7 +503,7 @@ PAUSE ROUTINE LOAD FOR example_tbl2_ordertest2;
|
|||
|
||||
## Resume a load job
|
||||
|
||||
You can execute the [RESUME ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/RESUME%20ROUTINE%20LOAD.md) statement to resume a paused load job. The state of the load job will be **NEED_SCHEDULE** temporarily (because the load job is being re-scheduled), and then become **RUNNING**. You can check its status with the [SHOW ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/SHOW%20ROUTINE%20LOAD.md) statement.
|
||||
You can execute the [RESUME ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/RESUME_ROUTINE_LOAD.md) statement to resume a paused load job. The state of the load job will be **NEED_SCHEDULE** temporarily (because the load job is being re-scheduled), and then become **RUNNING**. You can check its status with the [SHOW ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/SHOW_ROUTINE_LOAD.md) statement.
|
||||
|
||||
The following example resumes the paused load job `example_tbl2_ordertest2`:
|
||||
|
||||
|
|
@ -513,7 +513,7 @@ RESUME ROUTINE LOAD FOR example_tbl2_ordertest2;
|
|||
|
||||
## Alter a load job
|
||||
|
||||
Before altering a load job, you must pause it with the [PAUSE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/PAUSE%20ROUTINE%20LOAD.md) statement. Then you can execute the [ALTER ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/alter-routine-load.md). After altering it, you can execute the [RESUME ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/RESUME%20ROUTINE%20LOAD.md) statement to resume it, and check its status with the [SHOW ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/SHOW%20ROUTINE%20LOAD.md) statement.
|
||||
Before altering a load job, you must pause it with the [PAUSE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/PAUSE_ROUTINE_LOAD.md) statement. Then you can execute the [ALTER ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/alter-routine-load.md). After altering it, you can execute the [RESUME ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/RESUME_ROUTINE_LOAD.md) statement to resume it, and check its status with the [SHOW ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/SHOW_ROUTINE_LOAD.md) statement.
|
||||
|
||||
Suppose the number of the BE nodes that are alive increases to `6` and the Kafka topic partitions to be consumed is `"0,1,2,3,4,5,6,7"`. If you want to increase the actual load task concurrency, you can execute the following statement to increase the number of desired task concurrency `desired_concurrent_number` to `6` (greater than or equal to the number of BE nodes that are alive), and specify the Kafka topic partitions and initial offsets.
|
||||
|
||||
|
|
@ -536,7 +536,7 @@ FROM kafka
|
|||
|
||||
## Stop a load job
|
||||
|
||||
You can execute the [STOP ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/STOP%20ROUTINE%20LOAD.md) statement to stop a load job. The state of the load job will be **STOPPED** after the statement is executed, and you cannot resume a stopped load job. You cannot check the status of a stopped load job with the [SHOW ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/SHOW%20ROUTINE%20LOAD.md) statement.
|
||||
You can execute the [STOP ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/STOP_ROUTINE_LOAD.md) statement to stop a load job. The state of the load job will be **STOPPED** after the statement is executed, and you cannot resume a stopped load job. You cannot check the status of a stopped load job with the [SHOW ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/SHOW_ROUTINE_LOAD.md) statement.
|
||||
|
||||
The following example stops the load job `example_tbl2_ordertest2`:
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Load data using Spark connector (recommended)
|
||||
|
||||
StarRocks provides a self-developed connector named StarRocks Connector for Apache Spark™ (Spark connector for short) to help you load data into a StarRocks table by using Spark. The basic principle is to accumulate the data and then load it all at a time into StarRocks through [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md). The Spark connector is implemented based on Spark DataSource V2. A DataSource can be created by using Spark DataFrames or Spark SQL. And both batch and structured streaming modes are supported.
|
||||
StarRocks provides a self-developed connector named StarRocks Connector for Apache Spark™ (Spark connector for short) to help you load data into a StarRocks table by using Spark. The basic principle is to accumulate the data and then load it all at a time into StarRocks through [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md). The Spark connector is implemented based on Spark DataSource V2. A DataSource can be created by using Spark DataFrames or Spark SQL. And both batch and structured streaming modes are supported.
|
||||
|
||||
> **NOTICE**
|
||||
>
|
||||
|
|
@ -99,7 +99,7 @@ Directly download the corresponding version of the Spark connector JAR from the
|
|||
| starrocks.write.retry.interval.ms | NO | 10000 | Supported since version 1.1.1. The interval to retry the Stream Load for the same batch of data if the load fails.|
|
||||
| starrocks.columns | NO | None | The StarRocks table column into which you want to load data. You can specify multiple columns, which must be separated by commas (,), for example, `"col0,col1,col2"`. |
|
||||
| starrocks.column.types | NO | None | Supported since version 1.1.1. Customize the column data types for Spark instead of using the defaults inferred from the StarRocks table and the [default mapping](#data-type-mapping-between-spark-and-starrocks). The parameter value is a schema in DDL format same as the output of Spark [StructType#toDDL](https://github.com/apache/spark/blob/master/sql/api/src/main/scala/org/apache/spark/sql/types/StructType.scala#L449) , such as `col0 INT, col1 STRING, col2 BIGINT`. Note that you only need to specify columns that need customization. One use case is to load data into columns of [BITMAP](#load-data-into-columns-of-bitmap-type) or [HLL](#load-data-into-columns-of-hll-type) type.|
|
||||
| starrocks.write.properties.* | NO | None | The parameters that are used to control Stream Load behavior. For example, the parameter `starrocks.write.properties.format` specifies the format of the data to be loaded, such as CSV or JSON. For a list of supported parameters and their descriptions, see [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md). |
|
||||
| starrocks.write.properties.* | NO | None | The parameters that are used to control Stream Load behavior. For example, the parameter `starrocks.write.properties.format` specifies the format of the data to be loaded, such as CSV or JSON. For a list of supported parameters and their descriptions, see [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md). |
|
||||
| starrocks.write.properties.format | NO | CSV | The file format based on which the Spark connector transforms each batch of data before the data is sent to StarRocks. Valid values: CSV and JSON. |
|
||||
| starrocks.write.properties.row_delimiter | NO | \n | The row delimiter for CSV-formatted data. |
|
||||
| starrocks.write.properties.column_separator | NO | \t | The column separator for CSV-formatted data. |
|
||||
|
|
|
|||
|
|
@ -79,7 +79,7 @@ The basic process of data pre-processing is as follows:
|
|||
|
||||
If you continue to load data through Broker process, you must ensure that Broker process are deployed in your StarRocks cluster.
|
||||
|
||||
You can use the [SHOW BROKER](../sql-reference/sql-statements/Administration/SHOW%20BROKER.md) statement to check for Broker that are deployed in your StarRocks cluster. If no Broker are deployed, you must deploy Broker by following the instructions provided in [Deploy Broker](../deployment/deploy_broker.md).
|
||||
You can use the [SHOW BROKER](../sql-reference/sql-statements/Administration/SHOW_BROKER.md) statement to check for Broker that are deployed in your StarRocks cluster. If no Broker are deployed, you must deploy Broker by following the instructions provided in [Deploy Broker](../deployment/deploy_broker.md).
|
||||
|
||||
### Configuring ETL Clusters
|
||||
|
||||
|
|
@ -157,7 +157,7 @@ PROPERTIES
|
|||
`PROPERTIES` inclueds parameters relating to the Apache Spark™ resource, as follows:
|
||||
> **Note**
|
||||
>
|
||||
> For detailed description of Apache Spark™ resource PROPERTIES, please see [CREATE RESOURCE](../sql-reference/sql-statements/data-definition/CREATE%20RESOURCE.md)
|
||||
> For detailed description of Apache Spark™ resource PROPERTIES, please see [CREATE RESOURCE](../sql-reference/sql-statements/data-definition/CREATE_RESOURCE.md)
|
||||
|
||||
- Spark related parameters:
|
||||
- `type`: Resource type, required, currently only supports `spark`.
|
||||
|
|
@ -185,7 +185,7 @@ PROPERTIES
|
|||
The above is a description of parameters for loading through Broker process. If you intend to load data without Broker process, the following should be noted.
|
||||
|
||||
- You do not need to specify `broker`.
|
||||
- If you need to configure user authentication, and HA for NameNode nodes, you need to configure the parameters in the hdfs-site.xml file in the HDFS cluster, see [broker_properties](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md#hdfs) for descriptions of parameters. and you need to move the **hdfs-site.xml** file under **$FE_HOME/conf** for each FE and **$BE_HOME/conf** for each BE.
|
||||
- If you need to configure user authentication, and HA for NameNode nodes, you need to configure the parameters in the hdfs-site.xml file in the HDFS cluster, see [broker_properties](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md#hdfs) for descriptions of parameters. and you need to move the **hdfs-site.xml** file under **$FE_HOME/conf** for each FE and **$BE_HOME/conf** for each BE.
|
||||
|
||||
> Note
|
||||
>
|
||||
|
|
|
|||
|
|
@ -62,7 +62,7 @@ The following figure shows the workflow of a Stream Load job.
|
|||
|
||||
### Create a load job
|
||||
|
||||
This section uses curl as an example to describe how to load the data of a CSV or JSON file from your local file system into StarRocks. For detailed syntax and parameter descriptions, see [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md).
|
||||
This section uses curl as an example to describe how to load the data of a CSV or JSON file from your local file system into StarRocks. For detailed syntax and parameter descriptions, see [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md).
|
||||
|
||||
Note that in StarRocks some literals are used as reserved keywords by the SQL language. Do not directly use these keywords in SQL statements. If you want to use such a keyword in an SQL statement, enclose it in a pair of backticks (`). See [Keywords](../sql-reference/sql-statements/keywords.md).
|
||||
|
||||
|
|
@ -114,7 +114,7 @@ curl --location-trusted -u <username>:<password> -H "label:123" \
|
|||
> **NOTE**
|
||||
>
|
||||
> - If you use an account for which no password is set, you need to input only `<username>:`.
|
||||
> - You can use [SHOW FRONTENDS](../sql-reference/sql-statements/Administration/SHOW%20FRONTENDS.md) to view the IP address and HTTP port of the FE node.
|
||||
> - You can use [SHOW FRONTENDS](../sql-reference/sql-statements/Administration/SHOW_FRONTENDS.md) to view the IP address and HTTP port of the FE node.
|
||||
|
||||
`example1.csv` consists of three columns, which are separated by commas (,) and can be mapped in sequence onto the `id`, `name`, and `score` columns of `table1`. Therefore, you need to use the `column_separator` parameter to specify the comma (,) as the column separator. You also need to use the `columns` parameter to temporarily name the three columns of `example1.csv` as `id`, `name`, and `score`, which are mapped in sequence onto the three columns of `table1`.
|
||||
|
||||
|
|
@ -177,7 +177,7 @@ curl -v --location-trusted -u <username>:<password> -H "strict_mode: true" \
|
|||
|
||||
> **NOTE**
|
||||
>
|
||||
> You can use [SHOW FRONTENDS](../sql-reference/sql-statements/Administration/SHOW%20FRONTENDS.md) to view the IP address and HTTP port of the FE node.
|
||||
> You can use [SHOW FRONTENDS](../sql-reference/sql-statements/Administration/SHOW_FRONTENDS.md) to view the IP address and HTTP port of the FE node.
|
||||
|
||||
`example2.json` consists of two keys, `name` and `code`, which are mapped onto the `id` and `city` columns of `table2`, as shown in the following figure.
|
||||
|
||||
|
|
@ -195,7 +195,7 @@ The mappings shown in the preceding figure are described as follows:
|
|||
>
|
||||
> In the preceding example, the value of `code` in `example2.json` is multiplied by 100 before it is loaded into the `id` column of `table2`.
|
||||
|
||||
For detailed mappings between `jsonpaths`, `columns`, and the columns of the StarRocks table, see the "Column mappings" section in [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md).
|
||||
For detailed mappings between `jsonpaths`, `columns`, and the columns of the StarRocks table, see the "Column mappings" section in [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md).
|
||||
|
||||
##### Query data
|
||||
|
||||
|
|
@ -213,7 +213,7 @@ MySQL [test_db]> SELECT * FROM table2;
|
|||
|
||||
### View a load job
|
||||
|
||||
After a load job is complete, StarRocks returns the result of the job in JSON format. For more information, see the "Return value" section in [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md).
|
||||
After a load job is complete, StarRocks returns the result of the job in JSON format. For more information, see the "Return value" section in [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md).
|
||||
|
||||
Stream Load does not allow you to query the result of a load job by using the SHOW LOAD statement.
|
||||
|
||||
|
|
@ -251,7 +251,7 @@ This section describes some system parameters that you need to configure if you
|
|||
>
|
||||
> **Average loading speed** in the preceding formula is the average loading speed of your StarRocks cluster. It varies depending on the disk I/O and the number of BEs in your StarRocks cluster.
|
||||
|
||||
Stream Load also provides the `timeout` parameter, which allows you to specify the timeout period of an individual load job. For more information, see [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md).
|
||||
Stream Load also provides the `timeout` parameter, which allows you to specify the timeout period of an individual load job. For more information, see [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md).
|
||||
|
||||
## Usage notes
|
||||
|
||||
|
|
|
|||
|
|
@ -483,4 +483,4 @@ curl --location-trusted -u <jack>:<123456> -H "label:streamload_txn_example1_tab
|
|||
|
||||
For information about the suitable application scenarios and supported data file formats of Stream Load and about how Stream Load works, see [Load data from a local file system or a streaming data source using HTTP PUT](../loading/StreamLoad.md).
|
||||
|
||||
For information about the syntax and parameters for creating Stream Load jobs, see [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md).
|
||||
For information about the syntax and parameters for creating Stream Load jobs, see [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md).
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
import InsertPrivNote from '../assets/commonMarkdown/insertPrivNote.md'
|
||||
|
||||
StarRocks supports using one of the following methods to load huge amounts of data from cloud storage: [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md) and [INSERT](../sql-reference/sql-statements/data-manipulation/insert.md).
|
||||
StarRocks supports using one of the following methods to load huge amounts of data from cloud storage: [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md) and [INSERT](../sql-reference/sql-statements/data-manipulation/insert.md).
|
||||
|
||||
In v3.0 and earlier, StarRocks only supports Broker Load, which runs in asynchronous loading mode. After you submit a load job, StarRocks asynchronously runs the job. You can use `SELECT * FROM information_schema.loads` to query the job result. This feature is supported from v3.1 onwards. For more information, see the "[View a load job](#view-a-load-job)" section of this topic.
|
||||
|
||||
|
|
@ -14,7 +14,7 @@ Additionally, Broker Load supports data transformation at data loading and suppo
|
|||
|
||||
From v3.1 onwards, StarRocks supports directly loading the data of Parquet-formatted or ORC-formatted files from AWS S3 by using the INSERT command and the FILES keyword, saving you from the trouble of creating an external table first. For more information, see [INSERT > Insert data directly from files in an external source using FILES keyword](../loading/InsertInto.md#insert-data-directly-from-files-in-an-external-source-using-files).
|
||||
|
||||
This topic focuses on using [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md) to load data from cloud storage.
|
||||
This topic focuses on using [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md) to load data from cloud storage.
|
||||
|
||||
## Background information
|
||||
|
||||
|
|
@ -24,7 +24,7 @@ From v2.5 onwards, StarRocks no longer depends on brokers to set up connections
|
|||
|
||||
> **NOTE**
|
||||
>
|
||||
> You can use the [SHOW BROKER](../sql-reference/sql-statements/Administration/SHOW%20BROKER.md) statement to check for brokers that are deployed in your StarRocks cluster. If no brokers are deployed, you can deploy brokers by following the instructions provided in [Deploy a broker](../deployment/deploy_broker.md).
|
||||
> You can use the [SHOW BROKER](../sql-reference/sql-statements/Administration/SHOW_BROKER.md) statement to check for brokers that are deployed in your StarRocks cluster. If no brokers are deployed, you can deploy brokers by following the instructions provided in [Deploy a broker](../deployment/deploy_broker.md).
|
||||
|
||||
## Supported data file formats
|
||||
|
||||
|
|
@ -103,7 +103,7 @@ The following figure shows the workflow of a Broker Load job.
|
|||
|
||||
Note that Broker Load supports accessing AWS S3 according to the S3 or S3A protocol. Therefore, when you load data from AWS S3, you can include `s3://` or `s3a://` as the prefix in the S3 URI that you pass as the file path (`DATA INFILE`).
|
||||
|
||||
Also, note that the following examples use the CSV file format and the instance profile-based authentication method. For information about how to load data in other formats and about the authentication parameters that you need to configure when using other authentication methods, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md).
|
||||
Also, note that the following examples use the CSV file format and the instance profile-based authentication method. For information about how to load data in other formats and about the authentication parameters that you need to configure when using other authentication methods, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md).
|
||||
|
||||
### Load a single data file into a single table
|
||||
|
||||
|
|
@ -267,7 +267,7 @@ After you confirm that the load job is successful, you can use [SELECT](../sql-r
|
|||
|
||||
Note that Broker Load supports accessing Google GCS only according to the gs protocol. Therefore, when you load data from Google GCS, you must include `gs://` as the prefix in the GCS URI that you pass as the file path (`DATA INFILE`).
|
||||
|
||||
Also, note that the following examples use the CSV file format and the VM-based authentication method. For information about how to load data in other formats and about the authentication parameters that you need to configure when using other authentication methods, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md).
|
||||
Also, note that the following examples use the CSV file format and the VM-based authentication method. For information about how to load data in other formats and about the authentication parameters that you need to configure when using other authentication methods, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md).
|
||||
|
||||
### Load a single data file into a single table
|
||||
|
||||
|
|
@ -436,7 +436,7 @@ Note that when you load data from Azure Storage, you need to determine which pre
|
|||
- If your Data Lake Storage Gen2 allows access only via HTTP, use `abfs://` as the prefix, for example, `abfs://<container>@<storage_account>.dfs.core.windows.net/<file_name>`.
|
||||
- If your Data Lake Storage Gen2 allows access only via HTTPS, use `abfss://` as the prefix, for example, `abfss://<container>@<storage_account>.dfs.core.windows.net/<file_name>`.
|
||||
|
||||
Also, note that the following examples use the CSV file format, Azure Blob Storage, and the shared key-based authentication method. For information about how to load data in other formats and about the authentication parameters that you need to configure when using other Azure storage services and authentication methods, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md).
|
||||
Also, note that the following examples use the CSV file format, Azure Blob Storage, and the shared key-based authentication method. For information about how to load data in other formats and about the authentication parameters that you need to configure when using other Azure storage services and authentication methods, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md).
|
||||
|
||||
### Load a single data file into a single table
|
||||
|
||||
|
|
@ -598,7 +598,7 @@ After you confirm that the load job is successful, you can use [SELECT](../sql-r
|
|||
|
||||
## Load data from an S3-compatible storage system
|
||||
|
||||
The following examples use the CSV file format and the MinIO storage system. For information about how to load data in other formats, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md).
|
||||
The following examples use the CSV file format and the MinIO storage system. For information about how to load data in other formats, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md).
|
||||
|
||||
### Load a single data file into a single table
|
||||
|
||||
|
|
@ -873,7 +873,7 @@ For information about the fields in the return results, see [Information Schema
|
|||
|
||||
## Cancel a load job
|
||||
|
||||
When a load job is not in the **CANCELLED** or **FINISHED** stage, you can use the [CANCEL LOAD](../sql-reference/sql-statements/data-manipulation/CANCEL%20LOAD.md) statement to cancel the job.
|
||||
When a load job is not in the **CANCELLED** or **FINISHED** stage, you can use the [CANCEL LOAD](../sql-reference/sql-statements/data-manipulation/CANCEL_LOAD.md) statement to cancel the job.
|
||||
|
||||
For example, you can execute the following statement to cancel a load job, whose label is `label1`, in the database `test_db`:
|
||||
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
import InsertPrivNote from '../assets/commonMarkdown/insertPrivNote.md'
|
||||
|
||||
StarRocks supports using the [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md) to load large amounts of data from HDFS into StarRocks.
|
||||
StarRocks supports using the [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md) to load large amounts of data from HDFS into StarRocks.
|
||||
|
||||
Broker Load runs in asynchronous loading mode. After you submit a load job, StarRocks asynchronously runs the job. You can use `SELECT * FROM information_schema.loads` to query the job result. This feature is supported from v3.1 onwards. For more information, see the "[View a load job](#view-a-load-job)" section of this topic.
|
||||
|
||||
|
|
@ -18,11 +18,11 @@ In v2.4 and earlier, StarRocks depends on brokers to set up connections between
|
|||
|
||||
From v2.5 onwards, StarRocks no longer depends on brokers to set up connections between your StarRocks cluster and your external storage system when it runs a Broker Load job. Therefore, you no longer need to specify a broker in the load statement, but you still need to retain the `WITH BROKER` keyword. This is called "broker-free loading."
|
||||
|
||||
When your data is stored in HDFS, you may encounter situations where broker-free loading does not work. This can happen when your data is stored across multiple HDFS clusters or when you have configured multiple Kerberos users. In these situations, you can resort to using broker-based loading instead. To do this successfully, make sure that at least one independent broker group is deployed. For information about how to specify authentication configuration and HA configuration in these situations, see [HDFS](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md#hdfs).
|
||||
When your data is stored in HDFS, you may encounter situations where broker-free loading does not work. This can happen when your data is stored across multiple HDFS clusters or when you have configured multiple Kerberos users. In these situations, you can resort to using broker-based loading instead. To do this successfully, make sure that at least one independent broker group is deployed. For information about how to specify authentication configuration and HA configuration in these situations, see [HDFS](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md#hdfs).
|
||||
|
||||
> **NOTE**
|
||||
>
|
||||
> You can use the [SHOW BROKER](../sql-reference/sql-statements/Administration/SHOW%20BROKER.md) statement to check for brokers that are deployed in your StarRocks cluster. If no brokers are deployed, you can deploy brokers by following the instructions provided in [Deploy a broker](../deployment/deploy_broker.md).
|
||||
> You can use the [SHOW BROKER](../sql-reference/sql-statements/Administration/SHOW_BROKER.md) statement to check for brokers that are deployed in your StarRocks cluster. If no brokers are deployed, you can deploy brokers by following the instructions provided in [Deploy a broker](../deployment/deploy_broker.md).
|
||||
|
||||
## Supported data file formats
|
||||
|
||||
|
|
@ -97,7 +97,7 @@ The following figure shows the workflow of a Broker Load job.
|
|||
|
||||
## Create a load job
|
||||
|
||||
Note that the following examples use the CSV format and the simple authentication method. For information about how to load data in other formats, how to specify HA configurations, and about the authentication parameters that you need to configure when using the Kerberos authentication method, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md).
|
||||
Note that the following examples use the CSV format and the simple authentication method. For information about how to load data in other formats, how to specify HA configurations, and about the authentication parameters that you need to configure when using the Kerberos authentication method, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md).
|
||||
|
||||
### Load a single data file into a single table
|
||||
|
||||
|
|
@ -363,7 +363,7 @@ For information about the fields in the return results, see [Information Schema
|
|||
|
||||
## Cancel a load job
|
||||
|
||||
When a load job is not in the **CANCELLED** or **FINISHED** stage, you can use the [CANCEL LOAD](../sql-reference/sql-statements/data-manipulation/CANCEL%20LOAD.md) statement to cancel the job.
|
||||
When a load job is not in the **CANCELLED** or **FINISHED** stage, you can use the [CANCEL LOAD](../sql-reference/sql-statements/data-manipulation/CANCEL_LOAD.md) statement to cancel the job.
|
||||
|
||||
For example, you can execute the following statement to cancel a load job, whose label is `label1`, in the database `test_db`:
|
||||
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ Note the following points:
|
|||
|
||||
- In actual business scenarios, both qualified and unqualified rows may contain `NULL` values. If the destination columns do not allow `NULL` values, StarRocks reports errors and filters out the rows that contain `NULL` values.
|
||||
|
||||
- The maximum percentage of unqualified rows that can be filtered out for a [Stream Load](../../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md), [Broker Load](../../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md), [Routine Load](../../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md), or [Spark Load](../../sql-reference/sql-statements/data-manipulation/SPARK%20LOAD.md) job is controlled by an optional job property `max_filter_ratio`. [INSERT](../../sql-reference/sql-statements/data-manipulation/insert.md) does not support setting the `max_filter_ratio` property.
|
||||
- The maximum percentage of unqualified rows that can be filtered out for a [Stream Load](../../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md), [Broker Load](../../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md), [Routine Load](../../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md), or [Spark Load](../../sql-reference/sql-statements/data-manipulation/SPARK_LOAD.md) job is controlled by an optional job property `max_filter_ratio`. [INSERT](../../sql-reference/sql-statements/data-manipulation/insert.md) does not support setting the `max_filter_ratio` property.
|
||||
|
||||
For example, you want to load four rows that hold `\N` (`\N` denotes a `NULL` value), `abc`, `2000`, and `1` values respectively in a column from a CSV-formatted data file into a StarRocks table, and the data type of the destination StarRocks table column is TINYINT [-128, 127].
|
||||
|
||||
|
|
@ -57,7 +57,7 @@ If strict mode is enabled, StarRocks loads only the rows that hold `\N` or `1` a
|
|||
|
||||
## Set strict mode
|
||||
|
||||
If you run a [Stream Load](../../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md), [Broker Load](../../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md), [Routine Load](../../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md), or [Spark Load](../../sql-reference/sql-statements/data-manipulation/SPARK%20LOAD.md) job to load data, use the `strict_mode` parameter to set strict mode for the load job. Valid values are `true` and `false`. The default value is `false`. The value `true` enables strict mode, and the value `false` disables strict mode.
|
||||
If you run a [Stream Load](../../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md), [Broker Load](../../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md), [Routine Load](../../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md), or [Spark Load](../../sql-reference/sql-statements/data-manipulation/SPARK_LOAD.md) job to load data, use the `strict_mode` parameter to set strict mode for the load job. Valid values are `true` and `false`. The default value is `false`. The value `true` enables strict mode, and the value `false` disables strict mode.
|
||||
|
||||
If you execute [INSERT](../../sql-reference/sql-statements/data-manipulation/insert.md) to load data, use the `enable_insert_strict` session variable to set strict mode. Valid values are `true` and `false`. The default value is `true`. The value `true` enables strict mode, and the value `false` disables strict mode.
|
||||
|
||||
|
|
@ -72,7 +72,7 @@ curl --location-trusted -u <username>:<password> \
|
|||
http://<fe_host>:<fe_http_port>/api/<database_name>/<table_name>/_stream_load
|
||||
```
|
||||
|
||||
For detailed syntax and parameters about Stream Load, see [STREAM LOAD](../../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md).
|
||||
For detailed syntax and parameters about Stream Load, see [STREAM LOAD](../../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md).
|
||||
|
||||
### Broker Load
|
||||
|
||||
|
|
@ -93,7 +93,7 @@ PROPERTIES
|
|||
)
|
||||
```
|
||||
|
||||
The preceding code snippet uses HDFS as an example. For detailed syntax and parameters about Broker Load, see [BROKER LOAD](../../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md).
|
||||
The preceding code snippet uses HDFS as an example. For detailed syntax and parameters about Broker Load, see [BROKER LOAD](../../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md).
|
||||
|
||||
### Routine Load
|
||||
|
||||
|
|
@ -110,7 +110,7 @@ FROM KAFKA
|
|||
)
|
||||
```
|
||||
|
||||
The preceding code snippet uses Apache Kafka® as an example. For detailed syntax and parameters about Routine Load, see [CREATE ROUTINE LOAD](../../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md).
|
||||
The preceding code snippet uses Apache Kafka® as an example. For detailed syntax and parameters about Routine Load, see [CREATE ROUTINE LOAD](../../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md).
|
||||
|
||||
### Spark Load
|
||||
|
||||
|
|
@ -132,7 +132,7 @@ PROPERTIES
|
|||
)
|
||||
```
|
||||
|
||||
The preceding code snippet uses HDFS as an example. For detailed syntax and parameters about Spark Load, see [SPARK LOAD](../../sql-reference/sql-statements/data-manipulation/SPARK%20LOAD.md).
|
||||
The preceding code snippet uses HDFS as an example. For detailed syntax and parameters about Spark Load, see [SPARK LOAD](../../sql-reference/sql-statements/data-manipulation/SPARK_LOAD.md).
|
||||
|
||||
### INSERT
|
||||
|
||||
|
|
|
|||
|
|
@ -44,7 +44,7 @@ The type of a subscription is defined when a consumer connects to it, and the ty
|
|||
|
||||
## Create a Routine Load job
|
||||
|
||||
The following examples describe how to consume CSV-formatted messages in Pulsar, and load the data into StarRocks by creating a Routine Load job. For detailed instruction and reference, see [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md).
|
||||
The following examples describe how to consume CSV-formatted messages in Pulsar, and load the data into StarRocks by creating a Routine Load job. For detailed instruction and reference, see [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md).
|
||||
|
||||
```SQL
|
||||
CREATE ROUTINE LOAD load_test.routine_wiki_edit_1 ON routine_wiki_edit
|
||||
|
|
@ -69,7 +69,7 @@ FROM PULSAR
|
|||
);
|
||||
```
|
||||
|
||||
When Routine Load is created to consume data from Pulsar, most input parameters except for `data_source_properties` are the same as consuming data from Kafka . For descriptions about parameters except data_source_properties `data_source_properties` , see [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md).
|
||||
When Routine Load is created to consume data from Pulsar, most input parameters except for `data_source_properties` are the same as consuming data from Kafka . For descriptions about parameters except data_source_properties `data_source_properties` , see [CREATE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md).
|
||||
|
||||
The parameters related to `data_source_properties` and their descriptions are as follows:
|
||||
|
||||
|
|
@ -91,7 +91,7 @@ Routine Load supports the following custom parameters for Pulsar.
|
|||
|
||||
### Check a load job
|
||||
|
||||
Execute the [SHOW ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation//SHOW%20ROUTINE%20LOAD.md) statement to check the status of the load job `routine_wiki_edit_1`. StarRocks returns the execution state `State`, the statistical information (including the total rows consumed and the total rows loaded) `Statistics`, and the progress of the load job `progress`.
|
||||
Execute the [SHOW ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation//SHOW_ROUTINE_LOAD.md) statement to check the status of the load job `routine_wiki_edit_1`. StarRocks returns the execution state `State`, the statistical information (including the total rows consumed and the total rows loaded) `Statistics`, and the progress of the load job `progress`.
|
||||
|
||||
When you check a Routine Load job that consumes data from Pulsar, most returned parameters except for `progress` are the same as consuming data from Kafka. `progress` refers to backlog, that is the number of unacked messages in a partition.
|
||||
|
||||
|
|
@ -121,7 +121,7 @@ ReasonOfStateChanged:
|
|||
|
||||
### Check a load task
|
||||
|
||||
Execute the [SHOW ROUTINE LOAD TASK](../sql-reference/sql-statements/data-manipulation/SHOW%20ROUTINE%20LOAD%20TASK.md) statement to check the load tasks of the load job `routine_wiki_edit_1`, such as how many tasks are running, the Kafka topic partitions that are consumed and the consumption progress `DataSourceProperties`, and the corresponding Coordinator BE node `BeId`.
|
||||
Execute the [SHOW ROUTINE LOAD TASK](../sql-reference/sql-statements/data-manipulation/SHOW_ROUTINE_LOAD_TASK.md) statement to check the load tasks of the load job `routine_wiki_edit_1`, such as how many tasks are running, the Kafka topic partitions that are consumed and the consumption progress `DataSourceProperties`, and the corresponding Coordinator BE node `BeId`.
|
||||
|
||||
```SQL
|
||||
MySQL [example_db]> SHOW ROUTINE LOAD TASK WHERE JobName = "routine_wiki_edit_1" \G
|
||||
|
|
@ -129,7 +129,7 @@ MySQL [example_db]> SHOW ROUTINE LOAD TASK WHERE JobName = "routine_wiki_edit_1"
|
|||
|
||||
## Alter a load job
|
||||
|
||||
Before altering a load job, you must pause it by using the [PAUSE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/PAUSE%20ROUTINE%20LOAD.md) statement. Then you can execute the [ALTER ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/alter-routine-load.md). After altering it, you can execute the [RESUME ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/RESUME%20ROUTINE%20LOAD.md) statement to resume it, and check its status by using the [SHOW ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/SHOW%20ROUTINE%20LOAD.md) statement.
|
||||
Before altering a load job, you must pause it by using the [PAUSE ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/PAUSE_ROUTINE_LOAD.md) statement. Then you can execute the [ALTER ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/alter-routine-load.md). After altering it, you can execute the [RESUME ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/RESUME_ROUTINE_LOAD.md) statement to resume it, and check its status by using the [SHOW ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/SHOW_ROUTINE_LOAD.md) statement.
|
||||
|
||||
When Routine Load is used to consume data from Pulsar, most returned parameters except for `data_source_properties` are the same as consuming data from Kafka.
|
||||
|
||||
|
|
|
|||
|
|
@ -23,17 +23,17 @@ mysql -h <fe_ip> -P<fe_query_port> -uroot
|
|||
|
||||
## Step 2: Create a database
|
||||
|
||||
Create a database named `sr_hub` by referring to [CREATE DATABASE](../sql-reference/sql-statements/data-definition/CREATE%20DATABASE.md).
|
||||
Create a database named `sr_hub` by referring to [CREATE DATABASE](../sql-reference/sql-statements/data-definition/CREATE_DATABASE.md).
|
||||
|
||||
```SQL
|
||||
CREATE DATABASE IF NOT EXISTS sr_hub;
|
||||
```
|
||||
|
||||
You can view all databases in this StarRocks instance by executing [SHOW DATABASES](../sql-reference/sql-statements/data-manipulation/SHOW%20DATABASES.md) SQL.
|
||||
You can view all databases in this StarRocks instance by executing [SHOW DATABASES](../sql-reference/sql-statements/data-manipulation/SHOW_DATABASES.md) SQL.
|
||||
|
||||
## Step 3: Create a table
|
||||
|
||||
Run `USE sr_hub` to switch to the `sr_hub` database and create a table named `sr_member` by referring to [CREATE TABLE](../sql-reference/sql-statements/data-definition/CREATE%20TABLE.md).
|
||||
Run `USE sr_hub` to switch to the `sr_hub` database and create a table named `sr_member` by referring to [CREATE TABLE](../sql-reference/sql-statements/data-definition/CREATE_TABLE.md).
|
||||
|
||||
```SQL
|
||||
USE sr_hub;
|
||||
|
|
@ -63,7 +63,7 @@ DISTRIBUTED BY HASH(city_code);
|
|||
> - The columns of the table exactly correspond to the fields of data that you will be loading into StarRocks in the tutorial on [loading and querying data](../quick_start/Import_and_query.md).
|
||||
> - To guarantee the high performance **in the production environment**, we strongly recommend that you strategize the data partitioning plan for the table by using the `PARTITION BY` clause. See [Design partitioning and bucketing rules](../table_design/Data_distribution.md#design-partitioning-and-bucketing-rules) for more instructions.
|
||||
|
||||
After the table is created, you can check the details of the table using the DESC statement, and view all the tables in the database by executing [SHOW TABLES](../sql-reference/sql-statements/data-manipulation/SHOW%20TABLES.md). Tables in StarRocks support schema changes. You can see [ALTER TABLE](../sql-reference/sql-statements/data-definition/ALTER%20TABLE) for more information.
|
||||
After the table is created, you can check the details of the table using the DESC statement, and view all the tables in the database by executing [SHOW TABLES](../sql-reference/sql-statements/data-manipulation/SHOW_TABLES.md). Tables in StarRocks support schema changes. You can see [ALTER TABLE](../sql-reference/sql-statements/data-definition/ALTER_TABLE) for more information.
|
||||
|
||||
## What to do next
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# System variables
|
||||
|
||||
StarRocks provides many system variables that can be set and modified to suit your requirements. This section describes the variables supported by StarRocks. You can view the settings of these variables by running the [SHOW VARIABLES](../sql-reference/sql-statements/Administration/SHOW%20VARIABLES.md) command on your MySQL client. You can also use the [SET](../sql-reference/sql-statements/Administration/SET.md) command to dynamically set or modify variables. You can make these variables take effect globally on the entire system, only in the current session, or only in a single query statement.
|
||||
StarRocks provides many system variables that can be set and modified to suit your requirements. This section describes the variables supported by StarRocks. You can view the settings of these variables by running the [SHOW VARIABLES](../sql-reference/sql-statements/Administration/SHOW_VARIABLES.md) command on your MySQL client. You can also use the [SET](../sql-reference/sql-statements/Administration/SET.md) command to dynamically set or modify variables. You can make these variables take effect globally on the entire system, only in the current session, or only in a single query statement.
|
||||
|
||||
The variables in StarRocks refer to the variable sets in MySQL, but **some variables are only compatible with the MySQL client protocol and do not function on the MySQL database**.
|
||||
|
||||
|
|
|
|||
|
|
@ -358,7 +358,7 @@ Release date: July 29, 2022
|
|||
- The Primary Key table supports complete DELETE WHERE syntax. For more information, see [DELETE](../sql-reference/sql-statements/data-manipulation/DELETE.md#delete-data-by-primary-key).
|
||||
- The Primary Key table supports persistent primary key indexes. You can choose to persist the primary key index on disk rather than in memory, significantly reducing memory usage. For more information, see [Primary Key table](../table_design/table_types/primary_key_table.md).
|
||||
- Global dictionary can be updated during real-time data ingestion,optimizing query performance and delivering 2X query performance for string data.
|
||||
- The CREATE TABLE AS SELECT statement can be executed asynchronously. For more information, see [CREATE TABLE AS SELECT](../sql-reference/sql-statements/data-definition/CREATE%20TABLE%20AS%20SELECT.md).
|
||||
- The CREATE TABLE AS SELECT statement can be executed asynchronously. For more information, see [CREATE TABLE AS SELECT](../sql-reference/sql-statements/data-definition/CREATE_TABLE_AS_SELECT.md).
|
||||
- Support the following resource group-related features:
|
||||
- Monitor resource groups: You can view the resource group of the query in the audit log and obtain the metrics of the resource group by calling APIs. For more information, see [Monitor and Alerting](../administration/Monitor_and_Alert.md#monitor-and-alerting).
|
||||
- Limit the consumption of large queries on CPU, memory, and I/O resources: You can route queries to specific resource groups based on the classifiers or by configuring session variables. For more information, see [Resource group](../administration/resource_group.md).
|
||||
|
|
@ -380,7 +380,7 @@ Release date: July 29, 2022
|
|||
- Equivalence comparison operators that contain functions can use Zone Map indexes when these operators are pushed down to scan operators.
|
||||
- Optimized Apache Hive™ external tables.
|
||||
- When Apache Hive™ tables are stored in Parquet, ORC, or CSV format, schema changes caused by ADD COLUMN or REPLACE COLUMN on Hive can be synchronized to StarRocks when you execute the REFRESH statement on the corresponding Hive external table. For more information, see [Hive external table](../data_source/External_table.md#hive-external-table).
|
||||
- `hive.metastore.uris` can be modified for Hive resources. For more information, see [ALTER RESOURCE](../sql-reference/sql-statements/data-definition/ALTER%20RESOURCE.md).
|
||||
- `hive.metastore.uris` can be modified for Hive resources. For more information, see [ALTER RESOURCE](../sql-reference/sql-statements/data-definition/ALTER_RESOURCE.md).
|
||||
- Optimized the performance of Apache Iceberg external tables. A custom catalog can be used to create an Iceberg resource. For more information, see [Apache Iceberg external table](../data_source/External_table.md#apache-iceberg-external-table).
|
||||
- Optimized the performance of Elasticsearch external tables. Sniffing the addresses of the data nodes in an Elasticsearch cluster can be disabled. For more information, see [Elasticsearch external table](../data_source/External_table.md#elasticsearch-external-table).
|
||||
- When the sum() function accepts a numeric string, it implicitly converts the numeric string.
|
||||
|
|
@ -399,7 +399,7 @@ Fixed the following bugs:
|
|||
|
||||
- StarGo, a cluster management tool, can deploy, start, upgrade, and roll back clusters and manage multiple clusters. For more information, see [Deploy StarRocks with StarGo](../administration/stargo.md).
|
||||
- The pipeline engine is enabled by default when you upgrade StarRocks to version 2.3 or deploy StarRocks. The pipeline engine can improve the performance of simple queries in high concurrency scenarios and complex queries. If you detect significant performance regressions when using StarRocks 2.3, you can disable the pipeline engine by executing the `SET GLOBAL` statement to set `enable_pipeline_engine` to `false`.
|
||||
- The [SHOW GRANTS](../sql-reference/sql-statements/account-management/SHOW%20GRANTS.md) statement is compatible with the MySQL syntax and displays the privileges assigned to a user in the form of GRANT statements.
|
||||
- The [SHOW GRANTS](../sql-reference/sql-statements/account-management/SHOW_GRANTS.md) statement is compatible with the MySQL syntax and displays the privileges assigned to a user in the form of GRANT statements.
|
||||
- It is recommended that the memory_limitation_per_thread_for_schema_change ( BE configuration item) use the default value 2 GB, and data is written to disk when data volume exceeds this limit. Therefore, if you have previously set this parameter to a larger value, it is recommended that you set it to 2 GB, otherwise a schema change task may take up a large amount of memory.
|
||||
|
||||
### Upgrade notes
|
||||
|
|
|
|||
|
|
@ -189,7 +189,7 @@ Release date: October 20, 2022
|
|||
|
||||
- Supports viewing the schema of external data via DESC. For more information, see [DESC](../sql-reference/sql-statements/Utility/DESCRIBE.md).
|
||||
|
||||
- Supports granting a specific role or IMPERSONATE permission to a user via GRANT and revoking them via REVOKE, and supports executing an SQL statement with IMPERSONATE permission via EXECUTE AS. For more information, see [GRANT](../sql-reference/sql-statements/account-management/GRANT.md), [REVOKE](../sql-reference/sql-statements/account-management/REVOKE.md), and [EXECUTE AS](../sql-reference/sql-statements/account-management/EXECUTE%20AS.md).
|
||||
- Supports granting a specific role or IMPERSONATE permission to a user via GRANT and revoking them via REVOKE, and supports executing an SQL statement with IMPERSONATE permission via EXECUTE AS. For more information, see [GRANT](../sql-reference/sql-statements/account-management/GRANT.md), [REVOKE](../sql-reference/sql-statements/account-management/REVOKE.md), and [EXECUTE AS](../sql-reference/sql-statements/account-management/EXECUTE_AS.md).
|
||||
|
||||
- Supports FDQN access: now you can use domain name or the combination of hostname and port as the unique identification of a BE or an FE node. This prevents access failures caused by changing IP addresses. For more information, see [Enable FQDN Access](../administration/enable_fqdn.md).
|
||||
|
||||
|
|
|
|||
|
|
@ -160,7 +160,7 @@ Release date: June 14, 2023
|
|||
|
||||
### New features
|
||||
|
||||
- Inactive materialized views can be manually activated using `ALTER MATERIALIZED VIEW <mv_name> ACTIVE`. You can use this SQL command to activate materialized views whose base tables were dropped and then recreated. For more information, see [ALTER MATERIALIZED VIEW](../sql-reference/sql-statements/data-definition/ALTER%20MATERIALIZED%20VIEW.md). [#24001](https://github.com/StarRocks/starrocks/pull/24001)
|
||||
- Inactive materialized views can be manually activated using `ALTER MATERIALIZED VIEW <mv_name> ACTIVE`. You can use this SQL command to activate materialized views whose base tables were dropped and then recreated. For more information, see [ALTER MATERIALIZED VIEW](../sql-reference/sql-statements/data-definition/ALTER_MATERIALIZED_VIEW.md). [#24001](https://github.com/StarRocks/starrocks/pull/24001)
|
||||
- StarRocks can automatically set an appropriate number of tablets when you create a table or add a partition, eliminating the need for manual operations. For more information, see [Determine the number of tablets](../table_design/Data_distribution.md#determine-the-number-of-tablets). [#10614](https://github.com/StarRocks/starrocks/pull/10614)
|
||||
|
||||
### Improvements
|
||||
|
|
@ -221,7 +221,7 @@ Added a metric to monitor the tablet status of Primary Key tables:
|
|||
- Added the `ErrorStateTabletNum` column to the output of `SHOW PROC '/statistic/'` to display the number of **err_state** tablets.
|
||||
- Added the `ErrorStateTablets` column to the output of `SHOW PROC '/statistic/<db_id>/'` to display the IDs of **err_state** tablets.
|
||||
|
||||
For more information, see [SHOW PROC](../sql-reference/sql-statements/Administration/SHOW%20PROC.md).
|
||||
For more information, see [SHOW PROC](../sql-reference/sql-statements/Administration/SHOW_PROC.md).
|
||||
|
||||
### Improvements
|
||||
|
||||
|
|
@ -255,7 +255,7 @@ Release date: April 4, 2023
|
|||
- Optimized the type inference logic. If a query like `SELECT sum(CASE WHEN XXX);` contains a constant `0`, such as `SELECT sum(CASE WHEN k1 = 1 THEN v1 ELSE 0 END) FROM test;`, pre-aggregation is automatically enabled to accelerate the query. [#19474](https://github.com/StarRocks/starrocks/pull/19474)
|
||||
- Supports using `SHOW CREATE VIEW` to view the creation statement of a materialized view. [#19999](https://github.com/StarRocks/starrocks/pull/19999)
|
||||
- Supports transmitting packets that are 2 GB or larger in size for a single bRPC request between BE nodes. [#20283](https://github.com/StarRocks/starrocks/pull/20283) [#20230](https://github.com/StarRocks/starrocks/pull/20230)
|
||||
- Supports using [SHOW CREATE CATALOG](../sql-reference/sql-statements/data-manipulation/SHOW%20CREATE%20CATALOG.md) to query the creation statement of an external catalog.
|
||||
- Supports using [SHOW CREATE CATALOG](../sql-reference/sql-statements/data-manipulation/SHOW_CREATE_CATALOG.md) to query the creation statement of an external catalog.
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
|
|
@ -335,7 +335,7 @@ Release date: February 5, 2023
|
|||
- Asynchronous materialized views created based on external catalogs support query rewrite. [#11116](https://github.com/StarRocks/starrocks/issues/11116) [#15791](https://github.com/StarRocks/starrocks/issues/15791)
|
||||
- Allows users to specify a collection period for automatic CBO statistics collection, which prevents cluster performance jitter caused by automatic full collection. [#14996](https://github.com/StarRocks/starrocks/pull/14996)
|
||||
- Added Thrift server queue. Requests that cannot be processed immediately during INSERT INTO SELECT can be pending in the Thrift server queue, preventing requests from being rejected. [#14571](https://github.com/StarRocks/starrocks/pull/14571)
|
||||
- Deprecated the FE parameter `default_storage_medium`. If `storage_medium` is not explicitly specified when users create a table, the system automatically infers the storage medium of the table based on BE disk type. For more information, see description of `storage_medium` in [CREATE TABLE](../sql-reference/sql-statements/data-definition/CREATE%20VIEW.md). [#14394](https://github.com/StarRocks/starrocks/pull/14394)
|
||||
- Deprecated the FE parameter `default_storage_medium`. If `storage_medium` is not explicitly specified when users create a table, the system automatically infers the storage medium of the table based on BE disk type. For more information, see description of `storage_medium` in [CREATE TABLE](../sql-reference/sql-statements/data-definition/CREATE_VIEW.md). [#14394](https://github.com/StarRocks/starrocks/pull/14394)
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
|
|
@ -365,15 +365,15 @@ Release date: January 22, 2023
|
|||
- Supports creating materialized views based on Hive, Hudi, Iceberg catalogs, and materialized views. For more information, see [Materialized view](../using_starrocks/Materialized_view.md). [#11116](https://github.com/StarRocks/starrocks/issues/11116) [#11873](https://github.com/StarRocks/starrocks/pull/11873)
|
||||
- Supports conditional updates for tables that use the Primary Key table. For more information, see [Change data through loading](../loading/Load_to_Primary_Key_tables.md). [#12159](https://github.com/StarRocks/starrocks/pull/12159)
|
||||
- Supports [Query Cache](../using_starrocks/query_cache.md), which stores intermediate computation results of queries, improving the QPS and reduces the average latency of highly-concurrent, simple queries. [#9194](https://github.com/StarRocks/starrocks/pull/9194)
|
||||
- Supports specifying the priority of Broker Load jobs. For more information, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md) [#11029](https://github.com/StarRocks/starrocks/pull/11029)
|
||||
- Supports specifying the number of replicas for data loading for StarRocks native tables. For more information, see [CREATE TABLE](../sql-reference/sql-statements/data-definition/CREATE%20TABLE.md). [#11253](https://github.com/StarRocks/starrocks/pull/11253)
|
||||
- Supports specifying the priority of Broker Load jobs. For more information, see [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md) [#11029](https://github.com/StarRocks/starrocks/pull/11029)
|
||||
- Supports specifying the number of replicas for data loading for StarRocks native tables. For more information, see [CREATE TABLE](../sql-reference/sql-statements/data-definition/CREATE_TABLE.md). [#11253](https://github.com/StarRocks/starrocks/pull/11253)
|
||||
- Supports [query queues](../administration/query_queues.md). [#12594](https://github.com/StarRocks/starrocks/pull/12594)
|
||||
- Supports isolating compute resources occupied by data loading, thereby limiting the resource consumption of data loading tasks. For more information, see [Resource group](../administration/resource_group.md). [#12606](https://github.com/StarRocks/starrocks/pull/12606)
|
||||
- Supports specifying the following data compression algorithms for StarRocks native tables: LZ4, Zstd, Snappy, and Zlib. For more information, see [Data compression](../table_design/data_compression.md). [#10097](https://github.com/StarRocks/starrocks/pull/10097) [#12020](https://github.com/StarRocks/starrocks/pull/12020)
|
||||
- Supports [user-defined variables](../reference/user_defined_variables.md). [#10011](https://github.com/StarRocks/starrocks/pull/10011)
|
||||
- Supports [lambda expression](../sql-reference/sql-functions/Lambda_expression.md) and the following higher-order functions: [array_map](../sql-reference/sql-functions/array-functions/array_map.md), [array_sum](../sql-reference/sql-functions/array-functions/array_sum.md), and [array_sortby](../sql-reference/sql-functions/array-functions/array_sortby.md). [#9461](https://github.com/StarRocks/starrocks/pull/9461) [#9806](https://github.com/StarRocks/starrocks/pull/9806) [#10323](https://github.com/StarRocks/starrocks/pull/10323) [#14034](https://github.com/StarRocks/starrocks/pull/14034)
|
||||
- Provides the QUALIFY clause that filters the results of [window functions](../sql-reference/sql-functions/Window_function.md). [#13239](https://github.com/StarRocks/starrocks/pull/13239)
|
||||
- Supports using the result returned by the uuid() and uuid_numeric() functions as the default value of a column when you create a table. For more information, see [CREATE TABLE](../sql-reference/sql-statements/data-definition/CREATE%20TABLE.md). [#11155](https://github.com/StarRocks/starrocks/pull/11155)
|
||||
- Supports using the result returned by the uuid() and uuid_numeric() functions as the default value of a column when you create a table. For more information, see [CREATE TABLE](../sql-reference/sql-statements/data-definition/CREATE_TABLE.md). [#11155](https://github.com/StarRocks/starrocks/pull/11155)
|
||||
- Supports the following functions: [map_size](../sql-reference/sql-functions/map-functions/map_size.md), [map_keys](../sql-reference/sql-functions/map-functions/map_keys.md), [map_values](../sql-reference/sql-functions/map-functions/map_values.md), [max_by](../sql-reference/sql-functions/aggregate-functions/max_by.md), [sub_bitmap](../sql-reference/sql-functions/bitmap-functions/sub_bitmap.md), [bitmap_to_base64](../sql-reference/sql-functions/bitmap-functions/bitmap_to_base64.md), [host_name](../sql-reference/sql-functions/utility-functions/host_name.md), and [date_slice](../sql-reference/sql-functions/date-time-functions/date_slice.md). [#11299](https://github.com/StarRocks/starrocks/pull/11299) [#11323](https://github.com/StarRocks/starrocks/pull/11323) [#12243](https://github.com/StarRocks/starrocks/pull/12243) [#11776](https://github.com/StarRocks/starrocks/pull/11776) [#12634](https://github.com/StarRocks/starrocks/pull/12634) [#14225](https://github.com/StarRocks/starrocks/pull/14225)
|
||||
|
||||
### Improvements
|
||||
|
|
@ -385,7 +385,7 @@ Release date: January 22, 2023
|
|||
- Asynchronous materialized views support multiple async refresh mechanisms. For more information, see [Materialized view](../using_starrocks/Materialized_view.md#enable-query-rewrite-based-on-async-materialized-views). [#12712](https://github.com/StarRocks/starrocks/pull/12712) [#13171](https://github.com/StarRocks/starrocks/pull/13171) [#13229](https://github.com/StarRocks/starrocks/pull/13229) [#12926](https://github.com/StarRocks/starrocks/pull/12926)
|
||||
- The efficiency of refreshing materialized views is improved. [#13167](https://github.com/StarRocks/starrocks/issues/13167)
|
||||
- Optimized the following aspects of data loading:
|
||||
- Optimized loading performance in multi-replica scenarios by supporting the "single leader replication" mode. Data loading gains a one-fold performance lift. For more information about "single leader replication", see `replicated_storage` in [CREATE TABLE](../sql-reference/sql-statements/data-definition/CREATE%20TABLE.md). [#10138](https://github.com/StarRocks/starrocks/pull/10138)
|
||||
- Optimized loading performance in multi-replica scenarios by supporting the "single leader replication" mode. Data loading gains a one-fold performance lift. For more information about "single leader replication", see `replicated_storage` in [CREATE TABLE](../sql-reference/sql-statements/data-definition/CREATE_TABLE.md). [#10138](https://github.com/StarRocks/starrocks/pull/10138)
|
||||
- Broker Load and Spark Load no longer need to depend on brokers for data loading when only one HDFS cluster or one Kerberos user is configured. However, if you have multiple HDFS clusters or multiple Kerberos users, you still need to deploy a broker. For more information, see [Load data from HDFS or cloud storage](../loading/BrokerLoad.md) and [Bulk load using Apache Spark™](../loading/SparkLoad.md). [#9049](https://github.com/starrocks/starrocks/pull/9049) [#9228](https://github.com/StarRocks/starrocks/pull/9228)
|
||||
- Optimized the performance of Broker Load when a large number of small ORC files are loaded. [#11380](https://github.com/StarRocks/starrocks/pull/11380)
|
||||
- Reduced the memory usage when you load data into Primary Key tables.
|
||||
|
|
|
|||
|
|
@ -21,7 +21,7 @@ Release date: September 12, 2023
|
|||
- The values returned by the RESTful API action `show_data` for cloud-native tables are incorrect. [#29473](https://github.com/StarRocks/starrocks/pull/29473)
|
||||
- If the [shared-data cluster](../deployment/shared_data/azure.md) stores data in Azure Blob Storage and a table is created, the FE fails to start after the cluster is rolled back to version 3.0. [#29433](https://github.com/StarRocks/starrocks/pull/29433)
|
||||
- A user has no permission when querying a table in the Iceberg catalog even if the user is granted permission on that table. [#29173](https://github.com/StarRocks/starrocks/pull/29173)
|
||||
- The `Default` field values returned by the [SHOW FULL COLUMNS](../sql-reference/sql-statements/Administration/SHOW%20FULL%20COLUMNS.md) statement for columns of the [BITMAP](../sql-reference/sql-statements/data-types/BITMAP.md) or [HLL](../sql-reference/sql-statements/data-types/HLL.md) data type are incorrect. [#29510](https://github.com/StarRocks/starrocks/pull/29510)
|
||||
- The `Default` field values returned by the [SHOW FULL COLUMNS](../sql-reference/sql-statements/Administration/SHOW_FULL_COLUMNS.md) statement for columns of the [BITMAP](../sql-reference/sql-statements/data-types/BITMAP.md) or [HLL](../sql-reference/sql-statements/data-types/HLL.md) data type are incorrect. [#29510](https://github.com/StarRocks/starrocks/pull/29510)
|
||||
- Modifying the FE dynamic parameter `max_broker_load_job_concurrency` using the `ADMIN SET FRONTEND CONFIG` command does not take effect.
|
||||
- The FE may fail to start when a materialized view is being refreshed while its refresh strategy is being modified. [#29964](https://github.com/StarRocks/starrocks/pull/29964) [#29720](https://github.com/StarRocks/starrocks/pull/29720)
|
||||
- The error `unknown error` is returned when `select count(distinct(int+double)) from table_name` is executed. [#29691](https://github.com/StarRocks/starrocks/pull/29691)
|
||||
|
|
@ -182,7 +182,7 @@ Release date: April 28, 2023
|
|||
|
||||
- [Preview] Supports Presto/Trino compatible dialect. Presto/Trino's SQL can be automatically rewritten into StarRocks' SQL pattern. For more information, see [the system variable](../reference/System_variable.md) `sql_dialect`.
|
||||
- [Preview] Supports [JDBC catalogs](../data_source/catalog/jdbc_catalog.md).
|
||||
- Supports using [SET CATALOG](../sql-reference/sql-statements/data-definition/SET%20CATALOG.md) to manually switch between catalogs in the current session.
|
||||
- Supports using [SET CATALOG](../sql-reference/sql-statements/data-definition/SET_CATALOG.md) to manually switch between catalogs in the current session.
|
||||
|
||||
#### Privileges and security
|
||||
|
||||
|
|
@ -198,7 +198,7 @@ Release date: April 28, 2023
|
|||
|
||||
#### SQL reference
|
||||
|
||||
- Added the following privilege-related SQL statements: [SET DEFAULT ROLE](../sql-reference/sql-statements/account-management/SET_DEFAULT_ROLE.md), [SET ROLE](../sql-reference/sql-statements/account-management/SET%20ROLE.md), [SHOW ROLES](../sql-reference/sql-statements/account-management/SHOW%20ROLES.md), and [SHOW USERS](../sql-reference/sql-statements/account-management/SHOW%20USERS.md).
|
||||
- Added the following privilege-related SQL statements: [SET DEFAULT ROLE](../sql-reference/sql-statements/account-management/SET_DEFAULT_ROLE.md), [SET ROLE](../sql-reference/sql-statements/account-management/SET_ROLE.md), [SHOW ROLES](../sql-reference/sql-statements/account-management/SHOW_ROLES.md), and [SHOW USERS](../sql-reference/sql-statements/account-management/SHOW_USERS.md).
|
||||
- Added the following semi-structured data analysis functions: [map_apply](../sql-reference/sql-functions/map-functions/map_apply.md), [map_from_arrays](../sql-reference/sql-functions/map-functions/map_from_arrays.md).
|
||||
- [array_agg](../sql-reference/sql-functions/array-functions/array_agg.md) supports ORDER BY.
|
||||
- Window functions [lead](../sql-reference/sql-functions/Window_function.md#lead) and [lag](../sql-reference/sql-functions/Window_function.md#lag) support IGNORE NULLS.
|
||||
|
|
@ -215,10 +215,10 @@ Release date: April 28, 2023
|
|||
|
||||
#### Storage engine and data ingestion
|
||||
|
||||
- Supports more CSV parameters for data ingestion, including SKIP_HEADER, TRIM_SPACE, ENCLOSE, and ESCAPE. See [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md), [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md), and [ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE%20ROUTINE%20LOAD.md).
|
||||
- Supports more CSV parameters for data ingestion, including SKIP_HEADER, TRIM_SPACE, ENCLOSE, and ESCAPE. See [STREAM LOAD](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md), [BROKER LOAD](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md), and [ROUTINE LOAD](../sql-reference/sql-statements/data-manipulation/CREATE_ROUTINE_LOAD.md).
|
||||
- The primary key and sort key are decoupled in [Primary Key tables](../table_design/table_types/primary_key_table.md). The sort key can be separately specified in `ORDER BY` when you create a table.
|
||||
- Optimized the memory usage of data ingestion into Primary Key tables in scenarios such as large-volume ingestion, partial updates, and persistent primary indexes.
|
||||
- Supports creating asynchronous INSERT tasks. For more information, see [INSERT](../loading/InsertInto.md#load-data-asynchronously-using-insert) and [SUBMIT TASK](../sql-reference/sql-statements/data-manipulation/SUBMIT%20TASK.md). [#20609](https://github.com/StarRocks/starrocks/issues/20609)
|
||||
- Supports creating asynchronous INSERT tasks. For more information, see [INSERT](../loading/InsertInto.md#load-data-asynchronously-using-insert) and [SUBMIT TASK](../sql-reference/sql-statements/data-manipulation/SUBMIT_TASK.md). [#20609](https://github.com/StarRocks/starrocks/issues/20609)
|
||||
|
||||
#### Materialized view
|
||||
|
||||
|
|
@ -226,20 +226,20 @@ Release date: April 28, 2023
|
|||
- Supports rewrite of View Delta Join, Outer Join, and Cross Join.
|
||||
- Optimized SQL rewrite of Union with partition.
|
||||
- Improved materialized view building capabilities: supporting CTE, select *, and Union.
|
||||
- Optimized the information returned by [SHOW MATERIALIZED VIEWS](../sql-reference/sql-statements/data-manipulation/SHOW%20MATERIALIZED%20VIEW.md).
|
||||
- Optimized the information returned by [SHOW MATERIALIZED VIEWS](../sql-reference/sql-statements/data-manipulation/SHOW_MATERIALIZED_VIEW.md).
|
||||
- Supports adding MV partitions in batches, which improves the efficiency of partition addition during materialized view building. [#21167](https://github.com/StarRocks/starrocks/pull/21167)
|
||||
|
||||
#### Query engine
|
||||
|
||||
- All operators are supported in the pipeline engine. Non-pipeline code will be removed in later versions.
|
||||
- Improved [Big Query Positioning](../administration/monitor_manage_big_queries.md) and added big query log. [SHOW PROCESSLIST](../sql-reference/sql-statements/Administration/SHOW%20PROCESSLIST.md) supports viewing CPU and memory information.
|
||||
- Improved [Big Query Positioning](../administration/monitor_manage_big_queries.md) and added big query log. [SHOW PROCESSLIST](../sql-reference/sql-statements/Administration/SHOW_PROCESSLIST.md) supports viewing CPU and memory information.
|
||||
- Optimized Outer Join Reorder.
|
||||
- Optimized error messages in the SQL parsing stage, providing more accurate error positioning and clearer error messages.
|
||||
|
||||
#### Data Lake Analytics
|
||||
|
||||
- Optimized metadata statistics collection.
|
||||
- Supports using [SHOW CREATE TABLE](../sql-reference/sql-statements/data-manipulation/SHOW%20CREATE%20TABLE.md) to view the creation statements of the tables that are managed by an external catalog and are stored in Apache Hive™, Apache Iceberg, Apache Hudi, or Delta Lake.
|
||||
- Supports using [SHOW CREATE TABLE](../sql-reference/sql-statements/data-manipulation/SHOW_CREATE_TABLE.md) to view the creation statements of the tables that are managed by an external catalog and are stored in Apache Hive™, Apache Iceberg, Apache Hudi, or Delta Lake.
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
|
|
@ -257,7 +257,7 @@ Release date: April 28, 2023
|
|||
### Behavior Change
|
||||
|
||||
- The new role-based access control (RBAC) system supports the previous privileges and roles. However, the syntax of related statements such as [GRANT](../sql-reference/sql-statements/account-management/GRANT.md) and [REVOKE](../sql-reference/sql-statements/account-management/REVOKE.md) is changed.
|
||||
- Renamed SHOW MATERIALIZED VIEW as [SHOW MATERIALIZED VIEWS](../sql-reference/sql-statements/data-manipulation/SHOW%20MATERIALIZED%20VIEW.md).
|
||||
- Renamed SHOW MATERIALIZED VIEW as [SHOW MATERIALIZED VIEWS](../sql-reference/sql-statements/data-manipulation/SHOW_MATERIALIZED_VIEW.md).
|
||||
- Added the following [Reserved keywords](../sql-reference/sql-statements/keywords.md): AUTO_INCREMENT, CURRENT_ROLE, DEFERRED, ENCLOSE, ESCAPE, IMMEDIATE, PRIVILEGES, SKIP_HEADER, TRIM_SPACE, VARBINARY.
|
||||
|
||||
### Upgrade Notes
|
||||
|
|
@ -280,6 +280,6 @@ StarRocks upgrades the BDB library in v3.0. However, BDBJE cannot be rolled back
|
|||
|
||||
The new RBAC privilege system is used by default after you upgrade to v3.0. You can only downgrade to v2.5.
|
||||
|
||||
After a downgrade, run [ALTER SYSTEM CREATE IMAGE](../sql-reference/sql-statements/Administration/ALTER%20SYSTEM.md) to create a new image and wait for the new image to be synchronized to all follower FEs. If you do not run this command, some of the downgrade operations may fail. This command is supported from 2.5.3 and later.
|
||||
After a downgrade, run [ALTER SYSTEM CREATE IMAGE](../sql-reference/sql-statements/Administration/ALTER_SYSTEM.md) to create a new image and wait for the new image to be synchronized to all follower FEs. If you do not run this command, some of the downgrade operations may fail. This command is supported from 2.5.3 and later.
|
||||
|
||||
For details about the differences between the privilege system of v2.5 and v3.0, see "Upgrade notes" in [Privileges supported by StarRocks](../administration/privilege_item.md).
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ Release date: September 25, 2023
|
|||
## New Features
|
||||
|
||||
- The aggregate function [group_concat](../sql-reference/sql-functions/string-functions/group_concat.md) supports the DISTINCT keyword and the ORDER BY clause. [#28778](https://github.com/StarRocks/starrocks/pull/28778)
|
||||
- [Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md), [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md), [Kafka Connector](../loading/Kafka-connector-starrocks.md), [Flink Connector](../loading/Flink-connector-starrocks.md), and [Spark Connector](../loading/Spark-connector-starrocks.md) support partial updates in column mode on a Primary Key table. [#28288](https://github.com/StarRocks/starrocks/pull/28288)
|
||||
- [Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md), [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md), [Kafka Connector](../loading/Kafka-connector-starrocks.md), [Flink Connector](../loading/Flink-connector-starrocks.md), and [Spark Connector](../loading/Spark-connector-starrocks.md) support partial updates in column mode on a Primary Key table. [#28288](https://github.com/StarRocks/starrocks/pull/28288)
|
||||
- Data in partitions can be automatically cooled down over time. (This feature is not supported for [list partitioning](../table_design/list_partitioning.md).) [#29335](https://github.com/StarRocks/starrocks/pull/29335) [#29393](https://github.com/StarRocks/starrocks/pull/29393)
|
||||
|
||||
## Improvements
|
||||
|
|
@ -22,11 +22,11 @@ Fixed the following issues:
|
|||
- After a follower FE is restarted, CpuCores statistics are not up-to-date, resulting in query performance degradation. [#28472](https://github.com/StarRocks/starrocks/pull/28472) [#30434](https://github.com/StarRocks/starrocks/pull/30434)
|
||||
- The execution cost of the [to_bitmap()](../sql-reference/sql-functions/bitmap-functions/to_bitmap.md) function is incorrectly calculated. As a result, an inappropriate execution plan is selected for the function after materialized views are rewritten. [#29961](https://github.com/StarRocks/starrocks/pull/29961)
|
||||
- In certain use cases of the shared-data architecture, after a follower FE is restarted, queries submitted to the follower FE return an error that reads "Backend node not found. Check if any backend node is down". [#28615](https://github.com/StarRocks/starrocks/pull/28615)
|
||||
- If data is continuously loaded into a table that is being altered by using the [ALTER TABLE](../sql-reference/sql-statements/data-definition/ALTER%20TABLE.md) statement, an error "Tablet is in error state" may be thrown. [#29364](https://github.com/StarRocks/starrocks/pull/29364)
|
||||
- If data is continuously loaded into a table that is being altered by using the [ALTER TABLE](../sql-reference/sql-statements/data-definition/ALTER_TABLE.md) statement, an error "Tablet is in error state" may be thrown. [#29364](https://github.com/StarRocks/starrocks/pull/29364)
|
||||
- Modifying the FE dynamic parameter `max_broker_load_job_concurrency` using the `ADMIN SET FRONTEND CONFIG` command does not take effect. [#29964](https://github.com/StarRocks/starrocks/pull/29964) [#29720](https://github.com/StarRocks/starrocks/pull/29720)
|
||||
- BEs crash if the time unit in the [date_diff()](../sql-reference/sql-functions/date-time-functions/date_diff.md) function is a constant but the dates are not constants. [#29937](https://github.com/StarRocks/starrocks/issues/29937)
|
||||
- In the shared-data architecture, automatic partitioning does not take effect after asynchronous load is enabled. [#29986](https://github.com/StarRocks/starrocks/issues/29986)
|
||||
- If users create a Primary Key table by using the [CREATE TABLE LIKE](../sql-reference/sql-statements/data-definition/CREATE%20TABLE%20LIKE.md) statement, an error `Unexpected exception: Unknown properties: {persistent_index_type=LOCAL}` is thrown. [#30255](https://github.com/StarRocks/starrocks/pull/30255)
|
||||
- If users create a Primary Key table by using the [CREATE TABLE LIKE](../sql-reference/sql-statements/data-definition/CREATE_TABLE_LIKE.md) statement, an error `Unexpected exception: Unknown properties: {persistent_index_type=LOCAL}` is thrown. [#30255](https://github.com/StarRocks/starrocks/pull/30255)
|
||||
- Restoring Primary Key tables causes metadata inconsistency after BEs are restarted. [#30135](https://github.com/StarRocks/starrocks/pull/30135)
|
||||
- If users load data into a Primary Key table on which truncate operations and queries are concurrently performed, an error "java.lang.NullPointerException" is thrown in certain cases. [#30573](https://github.com/StarRocks/starrocks/pull/30573)
|
||||
- If predicate expressions are specified in materialized view creation statements, the refresh results of those materialized views are incorrect. [#29904](https://github.com/StarRocks/starrocks/pull/29904)
|
||||
|
|
@ -45,13 +45,13 @@ Fixed the following issues:
|
|||
- If a user specifies which database is to be connected by default and the user only has permissions on tables in the database but does not have permissions on the database, an error stating that the user does not have permissions on the database is thrown. [#29767](https://github.com/StarRocks/starrocks/pull/29767)
|
||||
- The values returned by the RESTful API action `show_data` for cloud-native tables are incorrect. [#29473](https://github.com/StarRocks/starrocks/pull/29473)
|
||||
- BEs crash if queries are canceled while the [array_agg()](../sql-reference/sql-functions/array-functions/array_agg.md) function is being run. [#29400](https://github.com/StarRocks/starrocks/issues/29400)
|
||||
- The `Default` field values returned by the [SHOW FULL COLUMNS](../sql-reference/sql-statements/Administration/SHOW%20FULL%20COLUMNS.md) statement for columns of the [BITMAP](../sql-reference/sql-statements/data-types/BITMAP.md) or [HLL](../sql-reference/sql-statements/data-types/HLL.md) data type are incorrect. [#29510](https://github.com/StarRocks/starrocks/pull/29510)
|
||||
- The `Default` field values returned by the [SHOW FULL COLUMNS](../sql-reference/sql-statements/Administration/SHOW_FULL_COLUMNS.md) statement for columns of the [BITMAP](../sql-reference/sql-statements/data-types/BITMAP.md) or [HLL](../sql-reference/sql-statements/data-types/HLL.md) data type are incorrect. [#29510](https://github.com/StarRocks/starrocks/pull/29510)
|
||||
- If the [array_map()](../sql-reference/sql-functions/array-functions/array_map.md) function in queries involves multiple tables, the queries fail due to pushdown strategy issues. [#29504](https://github.com/StarRocks/starrocks/pull/29504)
|
||||
- Queries against ORC-formatted files fail because the bugfix ORC-1304 ([apache/orc#1299](https://github.com/apache/orc/pull/1299)) from Apache ORC is not merged. [#29804](https://github.com/StarRocks/starrocks/pull/29804)
|
||||
|
||||
### Behavior Change
|
||||
|
||||
For a newly deployed StarRocks v3.1 cluster, you must have the USAGE privilege on the destination external catalog if you want to run [SET CATALOG](../sql-reference/sql-statements/data-definition/SET%20CATALOG.md) to switch to that catalog. You can use [GRANT](../sql-reference/sql-statements/account-management/GRANT.md) to grant the required privileges.
|
||||
For a newly deployed StarRocks v3.1 cluster, you must have the USAGE privilege on the destination external catalog if you want to run [SET CATALOG](../sql-reference/sql-statements/data-definition/SET_CATALOG.md) to switch to that catalog. You can use [GRANT](../sql-reference/sql-statements/account-management/GRANT.md) to grant the required privileges.
|
||||
|
||||
For a v3.1 cluster upgraded from an earlier version, you can run SET CATALOG with inherited privilege.
|
||||
|
||||
|
|
@ -104,8 +104,8 @@ Release date: August 7, 2023
|
|||
|
||||
- Upgraded automatic partitioning to [expression partitioning](../table_design/expression_partitioning.md). Users only need to use a simple partition expression (either a time function expression or a column expression) to specify a partitioning method at table creation, and StarRocks will automatically create partitions based on the data characteristics and the rule defined in the partition expression during data loading. This method of partition creation is suitable for most scenarios and is more flexible and user-friendly.
|
||||
- Supports [list partitioning](../table_design/list_partitioning.md). Data is partitioned based on a list of values predefined for a particular column, which can accelerate queries and manage clearly categorized data more efficiently.
|
||||
- Added a new table named `loads` to the `Information_schema` database. Users can query the results of [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md) and [Insert](../sql-reference/sql-statements/data-manipulation/insert.md) jobs from the `loads` table.
|
||||
- Supports logging the unqualified data rows that are filtered out by [Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md), [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER%20LOAD.md), and [Spark Load](../sql-reference/sql-statements/data-manipulation/SPARK%20LOAD.md) jobs. Users can use the `log_rejected_record_num` parameter in their load job to specify the maximum number of data rows that can be logged.
|
||||
- Added a new table named `loads` to the `Information_schema` database. Users can query the results of [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md) and [Insert](../sql-reference/sql-statements/data-manipulation/insert.md) jobs from the `loads` table.
|
||||
- Supports logging the unqualified data rows that are filtered out by [Stream Load](../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md), [Broker Load](../sql-reference/sql-statements/data-manipulation/BROKER_LOAD.md), and [Spark Load](../sql-reference/sql-statements/data-manipulation/SPARK_LOAD.md) jobs. Users can use the `log_rejected_record_num` parameter in their load job to specify the maximum number of data rows that can be logged.
|
||||
- Supports [random bucketing](../table_design/Data_distribution.md#choose-bucketing-columns). With this feature, users do not need to configure bucketing columns at table creation, and StarRocks will randomly distribute the data loaded into it to buckets. Using this feature together with the capability of automatically setting the number of buckets (`BUCKETS`) that StarRocks has provided since v2.5.7, users no longer need to consider bucket configurations, and table creation statements are greatly simplified. In big data and high performance-demanding scenarios, however, we recommend that users continue using hash bucketing, because this way they can use bucket pruning to accelerate queries.
|
||||
- Supports using the table function FILES() in [INSERT INTO](../loading/InsertInto.md) to directly load the data of Parquet- or ORC-formatted data files stored in AWS S3. The FILES() function can automatically infer the table schema, which relieves the need to create external catalogs or file external tables before data loading and therefore greatly simplifies the data loading process.
|
||||
- Supports [generated columns](../sql-reference/sql-statements/generated_columns.md). With the generated column feature, StarRocks can automatically generate and store the values of column expressions and automatically rewrite queries to improve query performance.
|
||||
|
|
@ -114,9 +114,9 @@ Release date: August 7, 2023
|
|||
|
||||
#### SQL reference
|
||||
|
||||
- Added the following storage volume-related statements: [CREATE STORAGE VOLUME](../sql-reference/sql-statements/Administration/CREATE%20STORAGE%20VOLUME.md), [ALTER STORAGE VOLUME](../sql-reference/sql-statements/Administration/ALTER%20STORAGE%20VOLUME.md), [DROP STORAGE VOLUME](../sql-reference/sql-statements/Administration/DROP%20STORAGE%20VOLUME.md), [SET DEFAULT STORAGE VOLUME](../sql-reference/sql-statements/Administration/SET%20DEFAULT%20STORAGE%20VOLUME.md), [DESC STORAGE VOLUME](../sql-reference/sql-statements/Administration/DESC%20STORAGE%20VOLUME.md), [SHOW STORAGE VOLUMES](../sql-reference/sql-statements/Administration/SHOW%20STORAGE%20VOLUMES.md).
|
||||
- Added the following storage volume-related statements: [CREATE STORAGE VOLUME](../sql-reference/sql-statements/Administration/CREATE_STORAGE_VOLUME.md), [ALTER STORAGE VOLUME](../sql-reference/sql-statements/Administration/ALTER_STORAGE_VOLUME.md), [DROP STORAGE VOLUME](../sql-reference/sql-statements/Administration/DROP_STORAGE_VOLUME.md), [SET DEFAULT STORAGE VOLUME](../sql-reference/sql-statements/Administration/SET_DEFAULT_STORAGE_VOLUME.md), [DESC STORAGE VOLUME](../sql-reference/sql-statements/Administration/DESC_STORAGE_VOLUME.md), [SHOW STORAGE VOLUMES](../sql-reference/sql-statements/Administration/SHOW_STORAGE_VOLUMES.md).
|
||||
|
||||
- Supports altering table comments using [ALTER TABLE](../sql-reference/sql-statements/data-definition/ALTER%20TABLE.md). [#21035](https://github.com/StarRocks/starrocks/pull/21035)
|
||||
- Supports altering table comments using [ALTER TABLE](../sql-reference/sql-statements/data-definition/ALTER_TABLE.md). [#21035](https://github.com/StarRocks/starrocks/pull/21035)
|
||||
|
||||
- Added the following functions:
|
||||
|
||||
|
|
|
|||
|
|
@ -44,7 +44,7 @@ Create a database named `bitmapdb` and a table named `bitmap`. Use Stream Load t
|
|||
);
|
||||
```
|
||||
|
||||
2. Use [Stream Load](../../../sql-reference/sql-statements/data-manipulation/STREAM%20LOAD.md) to import JSON data into `bitmap_table`.
|
||||
2. Use [Stream Load](../../../sql-reference/sql-statements/data-manipulation/STREAM_LOAD.md) to import JSON data into `bitmap_table`.
|
||||
|
||||
Suppose there is a JSON file named **simpledata**. This file has the following content and `userid` is a Base64-encoded string.
|
||||
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
## Description
|
||||
|
||||
Reads a data file from cloud storage. FILES() accesses cloud storage with the path-related properties of the file, infers the table schema of the data in the file, and returns the data rows. You can directly query the data rows using [SELECT](../../sql-statements/data-manipulation/SELECT.md), load the data rows into an existing table using [INSERT](../../sql-statements/data-manipulation/insert.md), or create a new table and load the data rows into it using [CREATE TABLE AS SELECT](../../sql-statements/data-definition/CREATE%20TABLE%20AS%20SELECT.md). This feature is supported from v3.1.0 onwards.
|
||||
Reads a data file from cloud storage. FILES() accesses cloud storage with the path-related properties of the file, infers the table schema of the data in the file, and returns the data rows. You can directly query the data rows using [SELECT](../../sql-statements/data-manipulation/SELECT.md), load the data rows into an existing table using [INSERT](../../sql-statements/data-manipulation/insert.md), or create a new table and load the data rows into it using [CREATE TABLE AS SELECT](../../sql-statements/data-definition/CREATE_TABLE_AS_SELECT.md). This feature is supported from v3.1.0 onwards.
|
||||
|
||||
Currently, the FILES() function supports the following data sources and file formats:
|
||||
|
||||
|
|
|
|||
|
|
@ -51,4 +51,4 @@ select catalog();
|
|||
|
||||
## See also
|
||||
|
||||
[SET CATALOG](../../sql-statements/data-definition/SET%20CATALOG.md): Switches to a destination catalog.
|
||||
[SET CATALOG](../../sql-statements/data-definition/SET_CATALOG.md): Switches to a destination catalog.
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
## Description
|
||||
|
||||
This statement is used to set configuration items for the cluster (Currently, only FE dynamic configuration items can be set using this command). You can view these configuration items using the [ADMIN SHOW FRONTEND CONFIG](ADMIN%20SET%20CONFIG.md) command.
|
||||
This statement is used to set configuration items for the cluster (Currently, only FE dynamic configuration items can be set using this command). You can view these configuration items using the [ADMIN SHOW FRONTEND CONFIG](ADMIN_SET_CONFIG.md) command.
|
||||
|
||||
The configurations will be restored to the default values in the `fe.conf` file after the FE restarts. Therefore, we recommend that you also modify the configuration items in `fe.conf` to prevent the loss of modifications.
|
||||
|
||||
|
|
@ -4,7 +4,7 @@
|
|||
|
||||
Displays the configuration of the current cluster (Currently, only FE configuration items can be displayed). For detailed description of these configuration items, see [Configuration](../../../administration/Configuration.md#fe-configuration-items).
|
||||
|
||||
If you want to set or modify a configuration item, use [ADMIN SET CONFIG](ADMIN%20SET%20CONFIG.md).
|
||||
If you want to set or modify a configuration item, use [ADMIN SET CONFIG](ADMIN_SET_CONFIG.md).
|
||||
|
||||
## Syntax
|
||||
|
||||
|
|
@ -20,10 +20,10 @@ ALTER RESOURCE GROUP resource_group_name
|
|||
| **Parameter** | **Description** |
|
||||
| ------------------- | ------------------------------------------------------------ |
|
||||
| resource_group_name | Name of the resource group to be altered. |
|
||||
| ADD | Add classifiers to the resource group. See [CREATE RESOURCE GROUP - Parameters](../Administration/CREATE%20RESOURCE%20GROUP.md) for more information on how to define a classifier. |
|
||||
| DROP | Drop classifiers from the resource group via classifier IDs. You can check the ID of a classifier via [SHOW RESOURCE GROUP](../Administration/SHOW%20RESOURCE%20GROUP.md) statement. |
|
||||
| ADD | Add classifiers to the resource group. See [CREATE RESOURCE GROUP - Parameters](../Administration/CREATE_RESOURCE_GROUP.md) for more information on how to define a classifier. |
|
||||
| DROP | Drop classifiers from the resource group via classifier IDs. You can check the ID of a classifier via [SHOW RESOURCE GROUP](../Administration/SHOW_RESOURCE_GROUP.md) statement. |
|
||||
| DROP ALL | Drop all classifiers from the resource group. |
|
||||
| WITH | Modify the resource limits of the resource group. See [CREATE RESOURCE GROUP - Parameters](../Administration/CREATE%20RESOURCE%20GROUP.md) for more information on how to set resource limits. |
|
||||
| WITH | Modify the resource limits of the resource group. See [CREATE RESOURCE GROUP - Parameters](../Administration/CREATE_RESOURCE_GROUP.md) for more information on how to set resource limits. |
|
||||
|
||||
## Examples
|
||||
|
||||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
## Description
|
||||
|
||||
Alters the credential properties, comment, or status (`enabled`) of a storage volume. For more about the properties of a storage volume, see [CREATE STORAGE VOLUME](./CREATE%20STORAGE%20VOLUME.md). This feature is supported from v3.1.
|
||||
Alters the credential properties, comment, or status (`enabled`) of a storage volume. For more about the properties of a storage volume, see [CREATE STORAGE VOLUME](./CREATE_STORAGE_VOLUME.md). This feature is supported from v3.1.
|
||||
|
||||
> **CAUTION**
|
||||
>
|
||||
|
|
@ -25,7 +25,7 @@ ALTER STORAGE VOLUME [ IF EXISTS ] <storage_volume_name>
|
|||
| storage_volume_name | The name of the storage volume to alter. |
|
||||
| COMMENT | The comment on the storage volume. |
|
||||
|
||||
For detailed information on the properties that can be altered or added, see [CREATE STORAGE VOLUME - PROPERTIES](./CREATE%20STORAGE%20VOLUME.md#properties).
|
||||
For detailed information on the properties that can be altered or added, see [CREATE STORAGE VOLUME - PROPERTIES](./CREATE_STORAGE_VOLUME.md#properties).
|
||||
|
||||
## Examples
|
||||
|
||||
|
|
@ -49,8 +49,8 @@ Query OK, 0 rows affected (0.00 sec)
|
|||
|
||||
## Relevant SQL statements
|
||||
|
||||
- [CREATE STORAGE VOLUME](./CREATE%20STORAGE%20VOLUME.md)
|
||||
- [DROP STORAGE VOLUME](./DROP%20STORAGE%20VOLUME.md)
|
||||
- [SET DEFAULT STORAGE VOLUME](./SET%20DEFAULT%20STORAGE%20VOLUME.md)
|
||||
- [DESC STORAGE VOLUME](./DESC%20STORAGE%20VOLUME.md)
|
||||
- [SHOW STORAGE VOLUMES](./SHOW%20STORAGE%20VOLUMES.md)
|
||||
- [CREATE STORAGE VOLUME](./CREATE_STORAGE_VOLUME.md)
|
||||
- [DROP STORAGE VOLUME](./DROP_STORAGE_VOLUME.md)
|
||||
- [SET DEFAULT STORAGE VOLUME](./SET_DEFAULT_STORAGE_VOLUME.md)
|
||||
- [DESC STORAGE VOLUME](./DESC_STORAGE_VOLUME.md)
|
||||
- [SHOW STORAGE VOLUMES](./SHOW_STORAGE_VOLUMES.md)
|
||||
|
|
@ -53,7 +53,7 @@ Manages FE, BE, CN, Broker nodes, and metadata snapshots in a cluster.
|
|||
ALTER SYSTEM ADD BACKEND "<be_host>:<heartbeat_service_port>"[, ...]
|
||||
```
|
||||
|
||||
You can check the status of the new BE by executing [SHOW BACKENDS](../Administration/SHOW%20BACKENDS.md).
|
||||
You can check the status of the new BE by executing [SHOW BACKENDS](../Administration/SHOW_BACKENDS.md).
|
||||
|
||||
- Drop a BE node.
|
||||
|
||||
|
|
@ -71,7 +71,7 @@ Manages FE, BE, CN, Broker nodes, and metadata snapshots in a cluster.
|
|||
ALTER SYSTEM DECOMMISSION BACKEND "<be_host>:<heartbeat_service_port>"[, ...]
|
||||
```
|
||||
|
||||
Unlike dropping a BE node, which is removing it forcibly from the cluster, decommissioning a BE means removing it safely. It is an asynchronous operation. When a BE is decommissioned, the data on the BE is first migrated to other BEs, and then the BE is removed from the cluster. Data loading and query will not be affected during the data migration. You can check whether the operation is successful using [SHOW BACKENDS](../Administration/SHOW%20BACKENDS.md). If the operation is successful, the decommissioned BE will not be returned. If the operation fails, the BE will still be online. You can manually cancel the operation using [CANCEL DECOMMISSION](../Administration/CANCEL%20DECOMMISSION.md).
|
||||
Unlike dropping a BE node, which is removing it forcibly from the cluster, decommissioning a BE means removing it safely. It is an asynchronous operation. When a BE is decommissioned, the data on the BE is first migrated to other BEs, and then the BE is removed from the cluster. Data loading and query will not be affected during the data migration. You can check whether the operation is successful using [SHOW BACKENDS](../Administration/SHOW_BACKENDS.md). If the operation is successful, the decommissioned BE will not be returned. If the operation fails, the BE will still be online. You can manually cancel the operation using [CANCEL DECOMMISSION](../Administration/CANCEL_DECOMMISSION.md).
|
||||
|
||||
| **Parameter** | **Required** | **Description** |
|
||||
| ---------------------- | ------------ | ------------------------------------------------------------------------------------------ |
|
||||
|
|
@ -86,7 +86,7 @@ Manages FE, BE, CN, Broker nodes, and metadata snapshots in a cluster.
|
|||
ALTER SYSTEM ADD COMPUTE NODE "<cn_host>:<heartbeat_service_port>"[, ...]
|
||||
```
|
||||
|
||||
You can check the status of the new CN by executing [SHOW COMPUTE NODES](../Administration/SHOW%20COMPUTE%20NODES.md).
|
||||
You can check the status of the new CN by executing [SHOW COMPUTE NODES](../Administration/SHOW_COMPUTE_NODES.md).
|
||||
|
||||
- Drop a CN node.
|
||||
|
||||
|
|
@ -107,7 +107,7 @@ Manages FE, BE, CN, Broker nodes, and metadata snapshots in a cluster.
|
|||
ALTER SYSTEM ADD BROKER <broker_name> "<broker_host>:<broker_ipc_port>"[, ...]
|
||||
```
|
||||
|
||||
You can add multiple Broker nodes with one SQL. Each `<broker_host>:<broker_ipc_port>` pair represents one Broker node. And they share a common `broker_name`. You can check the status of the new Broker node by executing [SHOW BROKER](../Administration/SHOW%20BROKER.md).
|
||||
You can add multiple Broker nodes with one SQL. Each `<broker_host>:<broker_ipc_port>` pair represents one Broker node. And they share a common `broker_name`. You can check the status of the new Broker node by executing [SHOW BROKER](../Administration/SHOW_BROKER.md).
|
||||
|
||||
- Drop Broker nodes.
|
||||
|
||||
|
|
@ -183,8 +183,8 @@ Query OK, 0 rows affected (0.05 sec)
|
|||
|
||||
## Relevant SQL statements
|
||||
|
||||
- [ALTER STORAGE VOLUME](./ALTER%20STORAGE%20VOLUME.md)
|
||||
- [DROP STORAGE VOLUME](./DROP%20STORAGE%20VOLUME.md)
|
||||
- [SET DEFAULT STORAGE VOLUME](./SET%20DEFAULT%20STORAGE%20VOLUME.md)
|
||||
- [DESC STORAGE VOLUME](./DESC%20STORAGE%20VOLUME.md)
|
||||
- [SHOW STORAGE VOLUMES](./SHOW%20STORAGE%20VOLUMES.md)
|
||||
- [ALTER STORAGE VOLUME](./ALTER_STORAGE_VOLUME.md)
|
||||
- [DROP STORAGE VOLUME](./DROP_STORAGE_VOLUME.md)
|
||||
- [SET DEFAULT STORAGE VOLUME](./SET_DEFAULT_STORAGE_VOLUME.md)
|
||||
- [DESC STORAGE VOLUME](./DESC_STORAGE_VOLUME.md)
|
||||
- [SHOW STORAGE VOLUMES](./SHOW_STORAGE_VOLUMES.md)
|
||||
|
|
@ -14,7 +14,7 @@ DELETE SQLBLACKLIST <sql_index_number>
|
|||
|
||||
## Parameter
|
||||
|
||||
`sql_index_number`: the index number of the SQL regular expression in the blacklist. Separate multiple index numbers with a comma (,) and a space. You can obtain the index number using [SHOW SQLBLACKLIST](../Administration/SHOW%20SQLBLACKLIST.md).
|
||||
`sql_index_number`: the index number of the SQL regular expression in the blacklist. Separate multiple index numbers with a comma (,) and a space. You can obtain the index number using [SHOW SQLBLACKLIST](../Administration/SHOW_SQLBLACKLIST.md).
|
||||
|
||||
## Examples
|
||||
|
||||
|
|
@ -51,8 +51,8 @@ IsDefault: false
|
|||
|
||||
## Relevant SQL statements
|
||||
|
||||
- [CREATE STORAGE VOLUME](./CREATE%20STORAGE%20VOLUME.md)
|
||||
- [ALTER STORAGE VOLUME](./ALTER%20STORAGE%20VOLUME.md)
|
||||
- [DROP STORAGE VOLUME](./DROP%20STORAGE%20VOLUME.md)
|
||||
- [SET DEFAULT STORAGE VOLUME](./SET%20DEFAULT%20STORAGE%20VOLUME.md)
|
||||
- [SHOW STORAGE VOLUMES](./SHOW%20STORAGE%20VOLUMES.md)
|
||||
- [CREATE STORAGE VOLUME](./CREATE_STORAGE_VOLUME.md)
|
||||
- [ALTER STORAGE VOLUME](./ALTER_STORAGE_VOLUME.md)
|
||||
- [DROP STORAGE VOLUME](./DROP_STORAGE_VOLUME.md)
|
||||
- [SET DEFAULT STORAGE VOLUME](./SET_DEFAULT_STORAGE_VOLUME.md)
|
||||
- [SHOW STORAGE VOLUMES](./SHOW_STORAGE_VOLUMES.md)
|
||||
|
|
@ -7,7 +7,7 @@ Drops a storage volume. Dropped storage volumes cannot be referenced anymore. Th
|
|||
> **CAUTION**
|
||||
>
|
||||
> - Only users with the DROP privilege on a specific storage volume can perform this operation.
|
||||
> - The default storage volume and the built-in storage volume `builtin_storage_volume` cannot be dropped. You can use [DESC STORAGE VOLUME](./DESC%20STORAGE%20VOLUME.md) to check whether a storage volume is the default storage volume.
|
||||
> - The default storage volume and the built-in storage volume `builtin_storage_volume` cannot be dropped. You can use [DESC STORAGE VOLUME](./DESC_STORAGE_VOLUME.md) to check whether a storage volume is the default storage volume.
|
||||
> - Storage volumes that are referenced by existing databases or cloud-native tables cannot be dropped.
|
||||
|
||||
## Syntax
|
||||
|
|
@ -33,8 +33,8 @@ Query OK, 0 rows affected (0.01 sec)
|
|||
|
||||
## Relevant SQL statements
|
||||
|
||||
- [CREATE STORAGE VOLUME](./CREATE%20STORAGE%20VOLUME.md)
|
||||
- [ALTER STORAGE VOLUME](./ALTER%20STORAGE%20VOLUME.md)
|
||||
- [SET DEFAULT STORAGE VOLUME](./SET%20DEFAULT%20STORAGE%20VOLUME.md)
|
||||
- [DESC STORAGE VOLUME](./DESC%20STORAGE%20VOLUME.md)
|
||||
- [SHOW STORAGE VOLUMES](./SHOW%20STORAGE%20VOLUMES.md)
|
||||
- [CREATE STORAGE VOLUME](./CREATE_STORAGE_VOLUME.md)
|
||||
- [ALTER STORAGE VOLUME](./ALTER_STORAGE_VOLUME.md)
|
||||
- [SET DEFAULT STORAGE VOLUME](./SET_DEFAULT_STORAGE_VOLUME.md)
|
||||
- [DESC STORAGE VOLUME](./DESC_STORAGE_VOLUME.md)
|
||||
- [SHOW STORAGE VOLUMES](./SHOW_STORAGE_VOLUMES.md)
|
||||
|
|
@ -15,7 +15,7 @@ KILL [ CONNECTION | QUERY ] <processlist_id>
|
|||
| **Parameter** | **Description** |
|
||||
| ------------------------ | ------------------------------------------------------------ |
|
||||
| Modifier:<ul><li>CONNECTION</li><li>QUERY</li></ul> | <ul><li>With a `CONNECTION` modifier, the KILL statement terminates the connection associated with the given `processlist_id`, after terminating any statement the connection is executing.</li><li>With a `QUERY` modifier, the KILL statement terminates the statement the connection is currently executing, but leaves the connection itself intact.</li><li>If no modifier is present, the default is `CONNECTION`.</li></ul> |
|
||||
| processlist_id | The ID of the thread you want to terminate. You can get the IDs of the threads that are being executed using [SHOW PROCESSLIST](../Administration/SHOW%20PROCESSLIST.md). |
|
||||
| processlist_id | The ID of the thread you want to terminate. You can get the IDs of the threads that are being executed using [SHOW PROCESSLIST](../Administration/SHOW_PROCESSLIST.md). |
|
||||
|
||||
## Examples
|
||||
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
## Description
|
||||
|
||||
Sets the specified system variables or user-defined variables for StarRocks. You can view the system variables of StarRocks using [SHOW VARIABLES](../Administration/SHOW%20VARIABLES.md). For details about system variables, see [System Variables](../../../reference/System_variable.md). For details about user-defined variables, see [User-defined variables](../../../reference/user_defined_variables.md).
|
||||
Sets the specified system variables or user-defined variables for StarRocks. You can view the system variables of StarRocks using [SHOW VARIABLES](../Administration/SHOW_VARIABLES.md). For details about system variables, see [System Variables](../../../reference/System_variable.md). For details about user-defined variables, see [User-defined variables](../../../reference/user_defined_variables.md).
|
||||
|
||||
## Syntax
|
||||
|
||||
|
|
|
|||
|
|
@ -33,8 +33,8 @@ Query OK, 0 rows affected (0.01 sec)
|
|||
|
||||
## Relevant SQL statements
|
||||
|
||||
- [CREATE STORAGE VOLUME](./CREATE%20STORAGE%20VOLUME.md)
|
||||
- [ALTER STORAGE VOLUME](./ALTER%20STORAGE%20VOLUME.md)
|
||||
- [DROP STORAGE VOLUME](./DROP%20STORAGE%20VOLUME.md)
|
||||
- [DESC STORAGE VOLUME](./DESC%20STORAGE%20VOLUME.md)
|
||||
- [SHOW STORAGE VOLUMES](./SHOW%20STORAGE%20VOLUMES.md)
|
||||
- [CREATE STORAGE VOLUME](./CREATE_STORAGE_VOLUME.md)
|
||||
- [ALTER STORAGE VOLUME](./ALTER_STORAGE_VOLUME.md)
|
||||
- [DROP STORAGE VOLUME](./DROP_STORAGE_VOLUME.md)
|
||||
- [DESC STORAGE VOLUME](./DESC_STORAGE_VOLUME.md)
|
||||
- [SHOW STORAGE VOLUMES](./SHOW_STORAGE_VOLUMES.md)
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue