Performing Stress Test on Roxie

[abaruchi]

Hi Guys,

I would like to know if some of you have an script or something to perform a stress test in a Roxie Cluster. The servers we just received is to run Roxie Queries only and we would like to check its performance.
I was thinking in deploy a Roxie Query with some mocked data and run several queries per second using some script (bash or python or anything else) in order to stress the cluster and try to reach its limit. However, if someone have any other idea or have done this before, I really appreciate any help.

Thanks in advance!

[rtaylor]

Artur,

The way I would approach this would be to create your query and the data you intend to place on that Roxie. Before you put that Roxie into production do your stress testing.

Roxie is configured with the number of threads each Roxie Server process will run. That number specifies the number of concurrent queries each node is capable of handling. If that number is 30, then each node can handle 30 simultaneous queries. And if it's a 10-node Roxie, that means the Roxie cluster can handle 300 simultaneous queries. And if your average response time for that query is 1/4 of a second (as some of our really complex queries do), then the Roxie could handle 1200 of those queries per second (or if your query only takes 1/10 of a second, that would be 3000/second total -- YMMV).

So to stress that 10-node Roxie, using that 1/4-second query, you would need to send more than 1200 queries per second. That suggests to me using Thor.

You would need to set up a Thor job that uses SOAPCALL to run that Roxie query. A simple PROJECT through a large dataset would do as the test case, calling SOAPCALL once for each record in that test dataset, each record containing "randomized" query parameters to run (so you're not just repeating the same query each time).

HTH,

Richard