starrocks/be/test
Binglin Chang 8d83da4fdb
[Enhancement] Add scripting ability to BE (#20351)
This PR adds scripting ability to BE, as a way to increase system observability, making testing, debugging and online hot fix issues easier. The script engine uses wren script language and wrenbind17 binding util(it's a small OO language). 
Currently, only a small faction of BE objects & methods are bonded, mainly focusing on StorageEngine(Tablets, Rowsets, TabletUpates). 
Will add more binding for rocksdb metadata, thread/stack dumps, memory/cpu profiling utils in the future.

Sample uses cases:

```
mysql> admin execute on 10004 '
    '> var ts = StorageEngine.get_tablet_infos(-1,-1)
    '> for (t in ts) {
    '>     System.print("%(t.tablet_id) %(t.data_size)")
    '> }
    '> ';
Query OK, 0 rows affected (0.01 sec)
10009 0
10011 0
10013 0
...

mysql> admin execute on 10004 '
    '> var t = StorageEngine.get_tablet(11005)
    '> System.print("info: %(t.updates().toPB().toString())")
    '> ';
Query OK, 0 rows affected (0.00 sec)
info: {"versions":[{"version":{"major":"3","minor":"1"},"creationTime":"1679897711","rowsets":[2],"compaction":{"startVersion":{"major":"3","minor":"0"},"inputs":[0,1],"outputs":[2]}},{"version":{"major":"4","minor":"0"},"creationTime":"1679900665","rowsets":[2,3],"deltas":[3]},{"version":{"major":"5","minor":"0"},"creationTime":"1679900666","rowsets":[2,3,4],"deltas":[4]},{"version":{"major":"6","minor":"0"},"creationTime":"1679900667","rowsets":[2,3,4,5],"deltas":[5]},{"version":{"major":"7","minor":"0"},"creationTime":"1679900667","rowsets":[2,3,4,5,6],"deltas":[6]},{"version":{"major":"8","minor":"0"},"creationTime":"1679900667","rowsets":[2,3,4,5,6,7],"deltas":[7]},{"version":{"major":"9","minor":"0"},"creationTime":"1679900667","rowsets":[2,3,4,5,6,7,8],"deltas":[8]},{"version":{"major":"10","minor":"0"},"creationTime":"1679900668","rowsets":[2,3,4,5,6,7,8,9],"deltas":[9]},{"version":{"major":"11","minor":"0"},"creationTime":"1679900668","rowsets":[2,3,4,5,6,7,8,9,10],"deltas":[10]},{"version":{"major":"12","minor":"0"},"creationTime":"1679900668","rowsets":[2,3,4,5,6,7,8,9,10,11],"deltas":[11]},{"version":{"major":"13","minor":"0"},"creationTime":"1679900668","rowsets":[2,3,4,5,6,7,8,9,10,11,12],"deltas":[12]},{"version":{"major":"13","minor":"1"},"creationTime":"1679900708","rowsets":[13],"compaction":{"startVersion":{"major":"13","minor":"0"},"inputs":[2,3,4,5,6,7,8,9,10,11,12],"outputs":[13]}}],"applyVersion":{"major":"13","minor":"1"},"nextRowsetId":14,"nextLogId":"28"}


mysql> admin execute on 10004 '
    '> for (dir in StorageEngine.get_data_dirs()) {
    '>     System.print("dir: %(dir.path()) rocksdb: %(dir.get_meta().sst_file_size())")
    '>     var result = dir.get_meta().compact().toString()
    '>     System.print("compact %(result) size: %(dir.get_meta().sst_file_size())")
    '> }
    '> ';
Query OK, 0 rows affected (0.00 sec)
dir: /home/decster/projects/starrocks/localrun/be/storage rocksdb: 156484
compact OK size: 12708

```

Signed-off-by: Binglin Chang <decstery@gmail.com>
2023-03-31 09:37:28 +08:00
..
block_cache Revert "[Enhancement] Import starcache submodule to support DLA block cache. (#20164)" (#20422) 2023-03-28 15:11:06 +08:00
column Optimize the performance of bitmap_contains on cross join for non-pipeline engine (#20653) 2023-03-30 18:11:28 +08:00
common [License] Change Elastic License to Apache License 2.0 (#14744) 2022-12-06 10:50:55 -08:00
exec [Enhancement] Support query iceberg tables with binary columns (#19118) 2023-03-29 17:00:23 +08:00
exprs [BugFix] array_agg supports upgrade to 3.0 (#20597) 2023-03-29 16:28:14 +00:00
formats [Enhancement] Support query iceberg tables with binary columns (#19118) 2023-03-29 17:00:23 +08:00
fs [Refactor] Reduce head object requests in lake tablet GC part 2/2 (#19736) 2023-03-20 17:10:12 +08:00
geo [Refactor] Fix ut compiling problems with Clang (#12894) 2022-11-02 20:33:41 -07:00
gutil [License] Change Elastic License to Apache License 2.0 (#14744) 2022-12-06 10:50:55 -08:00
http [UT] Fix metrics coredump (#15974) 2022-12-29 19:34:03 +08:00
io [Feature] Implements partition spiller writer (#20079) 2023-03-29 10:51:25 +08:00
runtime [Enhancement] Add scripting ability to BE (#20351) 2023-03-31 09:37:28 +08:00
serde [Refactor] Rename VectorizedField to Field and VectorizedSchema to Schema (#16862) 2023-01-26 11:19:09 -08:00
service [Feature] Support schema changes to the lake table of primary key (#16850) 2023-01-30 16:46:36 +08:00
simd [Refactor] PrimitiveType -> LogicalType follow-ups (#17778) 2023-02-14 12:13:29 -08:00
storage [Feature] Cleanup the expired and overcapacity binlog (#18672) 2023-03-30 09:15:53 -07:00
types [Enhancement] Constructor of BitmapValue support shallow copy (#20617) 2023-03-30 14:56:13 +08:00
util [Enhancement] Move io coalesce input stream below cache input stream (zero-copy feature) (#19584) 2023-03-16 16:39:20 -07:00
CMakeLists.txt [Enhancement] Constructor of BitmapValue support shallow copy (#20617) 2023-03-30 14:56:13 +08:00
test_main.cpp [Refactor] Remove the namespace vectorized (#15429) 2022-12-21 06:06:23 +08:00