HPCC on Azure

[David Dasher]

Hello

We are running HPCC on Azure and I am wondering if anyone can recommend a way to test internode communication speeds on a multinode THOR cluster?

We need to run this from time to time to track if we are seeing any significant skews in terms of speed due to sudden changes in network speed.

Any suggestions would be appreciated.

David

[bforeman]

Hi David,

I'm still checking with the HPCC development team, but on my level I just normally use the Graphs on each ECL process to identify any hot spots or skew. As long as your nodes are relatively homogeneous, there shouldn't be any significant speed difference between them. But you can also look at the cluster processes in the ECL watch Topology section and monitor the logs.

Regards,

Bob

[David Dasher]

Hi Bob

Many thanks for your reply, I will take a good look.

We are mainly looking for a tool that can profile how fast the data is coming off disk for HPCC and also what kind of internode network speed between the Thor nodes.

Kind regards

David

[bforeman]

Hi David,

I forgot to mention, there is also a Timings section for every workunit you submit. Just click on the ECL Watch tab and then look for the Timings link. The timings page is VERY detailed

Regards,

Bob