Thu Aug 18, 2022 10:20 am
Login Register Lost Password? Contact Us

Please Note: The HPCC Systems forums are moving to Stack Overflow. We invite you to post your questions on Stack Overflow utilizing the tag hpcc-ecl ( This legacy forum will be active and monitored during our transition to Stack Overflow but will become read only beginning September 1, 2022.

Node-wise Workload Distribution

Topics related to recommendations or questions on the design for HPCC Systems clusters

Wed Apr 11, 2018 12:31 pm Change Time Zone

Hi guys, I have 1 master node and 2 slave nodes.

Processes running on Master :
mydafilesrv ( pid 2252 ) is running ...
myeclagent ( pid 3533 ) is running ...
myesp ( pid 5159 ) is running ...
mysasha ( pid 6796 ) is running ...
mythor ( pid 21232 ) is running with 2 slave process(es) ...

Processes running on Slave :
mydafilesrv ( pid 2096 ) is running ...
mydali ( pid 3349 ) is running ...
myeclccserver ( pid 4779 ) is running ...

Initially, the CPU utilization for all 3 nodes is 0% (100% idle state).

When I start running a dataGeneration ECL script, the CPU utilization of master node reaches 40-50% however CPU utilization on Slaves continues to remain 0%.
Even the network utilization on all 3 nodes is similar and the disk utilization for just the master seems very high.

I am confused. I expected the slave processes to be doing all the data generation and writing it to the disk work. But low-level metrics don't indicate that. Can someone please share some insight on the same?

Only HPCC related user processes are running on these 3 instances.
I am consistently seeing eclagent to be the top process on master.
I am periodically seeing daserver & thorslave to be the top process on slave
Posts: 13
Joined: Wed Mar 28, 2018 5:13 pm

Wed Apr 11, 2018 1:29 pm Change Time Zone


ECL Agent is also known as hThor. So without seeing your actual code, this sounds like the job is running on hThor, not Thor.

Even if you target your Thor cluster, ECL Agent will "hijack" jobs when the code is really simple. If you have targeted your Thor, and think your code is complex enough that it should be running on Thor, then please post it.


Community Advisory Board Member
Community Advisory Board Member
Posts: 1619
Joined: Wed Oct 26, 2011 7:40 pm

Wed Apr 11, 2018 4:14 pm Change Time Zone

You are spot on @RTaylor.

I have changed my target to thor and now I see different metrics. Thanks for the help.
Posts: 13
Joined: Wed Mar 28, 2018 5:13 pm

Return to Clustering

Who is online

Users browsing this forum: No registered users and 1 guest