HPCC on Azure
Hello
We are running HPCC on Azure and I am wondering if anyone can recommend a way to test internode communication speeds on a multinode THOR cluster?
We need to run this from time to time to track if we are seeing any significant skews in terms of speed due to sudden changes in network speed.
Any suggestions would be appreciated.
David
We are running HPCC on Azure and I am wondering if anyone can recommend a way to test internode communication speeds on a multinode THOR cluster?
We need to run this from time to time to track if we are seeing any significant skews in terms of speed due to sudden changes in network speed.
Any suggestions would be appreciated.
David
- David Dasher
- Posts: 56
- Joined: Tue Feb 18, 2014 9:17 am
Hi David,
I'm still checking with the HPCC development team, but on my level I just normally use the Graphs on each ECL process to identify any hot spots or skew. As long as your nodes are relatively homogeneous, there shouldn't be any significant speed difference between them. But you can also look at the cluster processes in the ECL watch Topology section and monitor the logs.
Regards,
Bob
I'm still checking with the HPCC development team, but on my level I just normally use the Graphs on each ECL process to identify any hot spots or skew. As long as your nodes are relatively homogeneous, there shouldn't be any significant speed difference between them. But you can also look at the cluster processes in the ECL watch Topology section and monitor the logs.
Regards,
Bob
- bforeman
- Community Advisory Board Member
- Posts: 1006
- Joined: Wed Jun 29, 2011 7:13 pm
Hi Bob
Many thanks for your reply, I will take a good look.
We are mainly looking for a tool that can profile how fast the data is coming off disk for HPCC and also what kind of internode network speed between the Thor nodes.
Kind regards
David
Many thanks for your reply, I will take a good look.
We are mainly looking for a tool that can profile how fast the data is coming off disk for HPCC and also what kind of internode network speed between the Thor nodes.
Kind regards
David
- David Dasher
- Posts: 56
- Joined: Tue Feb 18, 2014 9:17 am
Hi David,
I forgot to mention, there is also a Timings section for every workunit you submit. Just click on the ECL Watch tab and then look for the Timings link. The timings page is VERY detailed
Regards,
Bob
I forgot to mention, there is also a Timings section for every workunit you submit. Just click on the ECL Watch tab and then look for the Timings link. The timings page is VERY detailed

Regards,
Bob
- bforeman
- Community Advisory Board Member
- Posts: 1006
- Joined: Wed Jun 29, 2011 7:13 pm
4 posts
• Page 1 of 1
Who is online
Users browsing this forum: No registered users and 1 guest