How many queries can Roxie handle simultaneously?

Mon Feb 09, 2015 6:09 pm

I'm placing this under CLOUD because the person who asked me the question, wants to configure and deploy an HPCC System on AWS.

The answer,below, was given my an in-house expert.

Answer:

Assuming you have configured your roxie as an N way roxie – the data will be spread across all N nodes. Again configuration dependent – there will often be 2 or more copies spread across the N nodes.

Each roxie farmer can take some number of threads – again this is configuration based – and depends upon things such as memory each farmer needs. I have come across 40 threads per farmer.

Additionally it is common to have a farmer on every node fronted with some kind of virtual IP. On our prod roxies we often had 100 nodes each with at least a dozen threads allowing for 1200 threads simultaneously.

Of course all of this is dependend upon the configuration (and query)

Mon Feb 09, 2015 6:21 pm

Tim,

The answer is of course, the number of threads per Roxie node times the number of nodes in the Roxie. I've heard of 30 threads before (not 40) but this is configuration-based, so YMMV.

So, with 30 threads per node and a 100-node Roxie, that would be 3000 simultaneous queries. A 20-node Roxie with 40 threads per node gets you 800 simultaneous queries. ...

HTH,

Richard