We need to status of progress, when creating big tables/projections . After a bit googling I cannot see any monitoring scripts. Vertica documantation mention in about monitoring but It is not sufficient. So I typed my own script with trial and error. You can use this scripts to monitor progress of creation projection. Plesae inform me if you discover a problem with this scripts. In my cases this scripts run well.
Case1 : create a new superprojection or projection from an existing table. in my tracing I get 2 long running phases, so you will see 2 rows result, each row corresponds one phase. first step 1 reaches %100 and then step2 starts. After step 2 reaches %100, create projection will finish with success.
eg:
create projection public.monitor_projection_del_p
as select * from public.monitor_projection_del order by date, privateAddress,region;
select start_refresh();
(SELECT 'step 1' as 'step',
round(((eep.counter_value)/ <anchor_table_row_nums>)*100,2) as progress
FROM execution_engine_profiles eep, projection_refreshes qp
where operator_name='StorageUnion'
--and operator_name='ValExpr' --and operator_name='Sort' --and operator_name='TopK' --use anyone of this operator_name, result should be same
and counter_name='rows produced'
and eep.is_executing
and counter_value != 0
and eep.transaction_id=qp.transaction_id
and eep.session_id= qp.session_id
and qp.is_executing
limit 1) --projection_refreshes tables returns 2 rows because of k-safety=1 . one of them is <projection_name_b0> and other one is should be <projection_name_b1> . they are body projections and comlately have the same rows therefore have same storage on disk. with using limit 1 I get only one projction's information. It doesn't matter which one, both are same on all conditions.
union all
select 'step 2' as 'step',
round(sum(progress)/2,2) as progress -- projection_refreshes tables returns 2 rows because of k-safety=1 . and sum(progress) reaches tottaly 200. so I will devide 2 to get "%100" return value from script.
from
(
SELECT eep.counter_value/<anchor_table_row_nums>*100 as progress
FROM execution_engine_profiles eep, projection_refreshes qp
where operator_name='DataTarget'
and counter_name='written rows'
and counter_tag like '%isDV=f%'
and eep.is_executing
and counter_value != 0
and eep.transaction_id=qp.transaction_id
and eep.session_id= qp.session_id
and qp.is_executing
) a;
Case 2: create table as select (CTAS) statement. in my tracing I get 3 long running phases, so you will see three rows result, each row corresponds one phase. first step 1 reaches %100 and then step2 starts. After step 2 reaches %100, step 3 starts . After step 3 reaches %100, create projection will finish with success.
eg: Örnek:
create table public.monitor_projection_test as select * from (select * from public.netflow limit 500000000) a order by date,message;
SELECT 'step 1' as 'step',
round(((eep.counter_value)/<anchor_table_row_nums>)*100,2) as 'progress %'
FROM execution_engine_profiles eep, QUERY_PROFILES qp
where eep.session_id='gdatalab_db_node0003-212021:0x3ba7c4' --“create projection sesson”s id
and operator_name='StorageUnion'
and counter_name='rows produced'
and eep.is_executing
and counter_value != 0
and eep.transaction_id=qp.transaction_id
and eep.session_id= qp.session_id
and qp.is_executing
union all
SELECT 'step 2' as 'step',
round(((eep.counter_value)/ <anchor_table_row_nums>)*100,2) as 'progress %'
FROM execution_engine_profiles eep, QUERY_PROFILES qp
where eep.session_id='gdatalab_db_node0003-212021:0x3ba7c4' --“create projection sesson”s id
and operator_name='ValExpr' --and operator_name='Sort' --and operator_name='TopK' --use anyone of this operator_name, result should be same
and counter_name='rows produced'
and eep.is_executing
and counter_value != 0
and eep.transaction_id=qp.transaction_id
and eep.session_id= qp.session_id
and qp.is_executing
union all
select 'step 3' as 'step',
round(sum(progress),2) as 'progress %'
from
(
SELECT
eep.counter_value/<anchor_table_row_nums>*100 as 'progress'
FROM execution_engine_profiles eep, QUERY_PROFILES qp
where eep.session_id='gdatalab_db_node0003-212021:0x3ba7c4' --“create projection sesson”s id
and operator_name='DataTarget'
and counter_name='written rows'
and counter_tag like '%isDV=f%'
and eep.is_executing
and counter_value != 0
and eep.transaction_id=qp.transaction_id
and eep.session_id= qp.session_id
and qp.is_executing
order by counter_value desc
) a;