Offload entry root slice to QE #385

Ray-Eldath · 2024-02-28T11:45:21Z

For some queries, Sort, (Finalized) Aggregate and Join are executed on QD, which increase workload on a single point. This PR alleviates this by offloading these operators to a QE. Specifically, it checks whether the root slice (the slice of the Plan before it meets the first Motion) contains these operators in should_offload_entry_to_qe_plan_walker, then it checks whether the offloading can be performed safely in safe_to_offload_entry_to_qe_rte_walker by walking the range table. This implementation specifically does not rely on RelOptInfo or anything postgres-optimizer-specific so that we can port this to Orca in the future.

src/backend/utils/misc/guc_gp.c

src/test/regress/sql/offload_entry_to_qe.sql

src/test/regress/expected/offload_entry_to_qe.out

src/backend/optimizer/plan/createplan.c

avamingli

LGTM.

src/backend/optimizer/plan/createplan.c

src/test/regress/sql/offload_entry_to_qe.sql

src/backend/optimizer/plan/planner.c

src/test/regress/sql/offload_entry_to_qe.sql

src/backend/optimizer/plan/createplan.c

src/test/regress/sql/offload_entry_to_qe.sql

For some queries, Sort, (Finalized) Aggregate and Join are executed on QD, which increase workload on a single point. This PR alleviates this by offloading these operators to a QE. Specifically, it checks whether the root slice (the slice of the Plan before it meets the first Motion) contains these operators in should_offload_entry_to_qe_plan_walker, then it checks whether the offloading can be performed safely in safe_to_offload_entry_to_qe_rte_walker by walking the range table. This implementation specifically does not rely on RelOptInfo or anything postgres-optimizer-specific so that we can port this to Orca in the future.

Ray-Eldath · 2024-03-18T10:43:42Z

@my-ship-it Maybe merge this as well? :)

For some queries, Sort, (Finalized) Aggregate and Join are executed on QD, which increase workload on a single point. This PR alleviates this by offloading these operators to a QE. Specifically, it checks whether the root slice (the slice of the Plan before it meets the first Motion) contains these operators in should_offload_entry_to_qe_plan_walker, then it checks whether the offloading can be performed safely in safe_to_offload_entry_to_qe_rte_walker by walking the range table. This implementation specifically does not rely on RelOptInfo or anything postgres-optimizer-specific so that we can port this to Orca in the future.

Ray-Eldath force-pushed the offload-entry-to-qe/modify branch 3 times, most recently from eb7f7fc to e5e074f Compare February 29, 2024 07:01

avamingli reviewed Mar 1, 2024

View reviewed changes

src/backend/utils/misc/guc_gp.c Outdated Show resolved Hide resolved

src/test/regress/sql/offload_entry_to_qe.sql Outdated Show resolved Hide resolved

avamingli reviewed Mar 1, 2024

View reviewed changes

src/test/regress/expected/offload_entry_to_qe.out Outdated Show resolved Hide resolved

Ray-Eldath force-pushed the offload-entry-to-qe/modify branch 3 times, most recently from e9ae87e to ec724c3 Compare March 1, 2024 08:44

avamingli reviewed Mar 4, 2024

View reviewed changes

src/test/regress/expected/offload_entry_to_qe.out Outdated Show resolved Hide resolved

src/backend/optimizer/plan/createplan.c Outdated Show resolved Hide resolved

Ray-Eldath force-pushed the offload-entry-to-qe/modify branch 3 times, most recently from 6229fcb to 58a5e51 Compare March 6, 2024 03:42

avamingli reviewed Mar 8, 2024

View reviewed changes