Sun Apr 22, 2018 6:20 pm
Login Register Lost Password? Contact Us


Query Time

Comments or questions specific to the features of ECL Watch

Wed Apr 04, 2018 3:46 pm Change Time Zone

Hello!

Is there an easy way today to get the "time period" of a result?
For example, if I open ECL Watch on a workunit, and go to the Outputs tab, I'd love to see the "time period" for each result (say as a separate column).
By "time period" I mean from start to finish, vs. adding up each second spent on each node when running ECL code. So "2s time period" could be actually "6s total time spent" with 3 nodes spending each 2s.
Now I don't see it (today) but maybe it's somewhere else?

If it's not available today, is there a way to calculate it? I was thinking of getting the graph of the WU (WUGetGraph.json?) and just go bottom-up (from leaf to root) and add up the TimeMaxLocalExecute to get a good approx. of "time period" for a given leaf (i.e. result). Is that the right approach?
I understand with this approach I may get more seconds than the total number of seconds spent by cluster if there's more than 1 result.


Thanks!
lpezet
 
Posts: 51
Joined: Wed Sep 10, 2014 3:14 am

Thu Apr 05, 2018 8:10 am Change Time Zone

Luke,

The Timers tab displays a number of timings for the workunit, including the total TimeElapsed, which tells you the total time it took for your whole Thor to do the job. The individual graph timings are also on a whole-Thor basis, not per-node.

So, why would you want timings on a per-node basis? What problem are you trying to solve?

HTH,

Richard
rtaylor
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 1353
Joined: Wed Oct 26, 2011 7:40 pm

Thu Apr 05, 2018 2:27 pm Change Time Zone

Hi Richard!

If I just generate 1 OUTPUT I understand the Total Cluster Time would be the time it took for that one OUTPUT.
But say I OUTPUT 3 different results. I'd like to know how long it took (in seconds or so) for each OUTPUT to complete.
In the screenshot, and just to be more visual in my explanations, I'd like to know how long each output here took, not just how many results they have. For example, I'd like to know that Query3A took 65 seconds, Query3B took 100 seconds and Query3C took 555 seconds. Total Cluster Time will tell me it's 555 seconds only and Timers are...well, cryptic at best as they are right now. (NB: Don't get me wrong, I use Timers a lot, to figure out where my queries spent the most time for example. They just don't provide what what I'm asking for here directly, that's all).
Since this "elapsed time" for each output is not there in ECL Watch, I'm guessing that information is not provided today (through any of the ECL Watch API endpoint) and some calculations are needed. Am I wrong?

Screen Shot 2018-04-05 at 8.22.17 AM.png
Screen Shot 2018-04-05 at 8.22.17 AM.png (106.01 KiB) Viewed 39 times
lpezet
 
Posts: 51
Joined: Wed Sep 10, 2014 3:14 am

Thu Apr 05, 2018 2:42 pm Change Time Zone

Luke,

In this case, the timing for any single OUTPUT could be computed by going through the graphs backwards,starting with the OUTPUT activity and adding up the time each graph that contributed to that result took.

Or the simple way would be to run the job again, commenting out the OUTPUTs you don't want to see, and then the total time is the total time for the single OUTPUT. :)

HTH,

Richard
rtaylor
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 1353
Joined: Wed Oct 26, 2011 7:40 pm

Mon Apr 09, 2018 4:30 pm Change Time Zone

Thanks Richard!
lpezet
 
Posts: 51
Joined: Wed Sep 10, 2014 3:14 am


Return to ECL Watch

Who is online

Users browsing this forum: No registered users and 1 guest

cron