Sat May 28, 2022 4:25 pm
Login Register Lost Password? Contact Us


Implicit SORT on index write

Comments and questions related to the Enterprise Control Language

Mon Nov 22, 2021 5:59 pm Change Time Zone

Hello everyone,

I was analyzing some workunits and saw that one of the most intensive activity in the graph was the implicit sort done by the index build. My question is, this SORT is implicitly DISTRIBUTEd and LOCAL or not? If it isn't, can I SORT the dataset by the index key using DISTRIBUTED and LOCAL and then call the BUILD function with the SORTED option? If I do this, the resulting index will have the same performance as the current one?
eduardo.costa
 
Posts: 1
Joined: Fri Feb 14, 2020 7:27 pm

Mon Nov 29, 2021 9:36 pm Change Time Zone

eduardo.
I was analyzing some workunits and saw that one of the most intensive activity in the graph was the implicit sort done by the index build. My question is, this SORT is implicitly DISTRIBUTEd and LOCAL or not?
If you're building a standard INDEX, then it will be a global SORT, because that is how the data must be sorted to create the INDEX. You have hit on one of the primary reasons why the Thor cluster was created -- to build INDEXes on a massively parallel platform, so this operation has been highly optimized for many years now. If you need to have local operations (where each INDEX file part is a local index), then you need to use the DISTRIBUTED option on the BUILD and INDEX definitions.

If it isn't, can I SORT the dataset by the index key using DISTRIBUTED and LOCAL and then call the BUILD function with the SORTED option? If I do this, the resulting index will have the same performance as the current one?
You can try doing something like that, but I doubt that you'll find any performance difference in the BUILD. And, if you're not careful, you may end up with an INDEX that doesn't work. :)

HTH,

Richard
rtaylor
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 1619
Joined: Wed Oct 26, 2011 7:40 pm


Return to ECL

Who is online

Users browsing this forum: Bing [Bot] and 1 guest

cron