Thor on Amazon with on-premise Roxie

[arunarav]

We would like to create some proof-of-concepts on varying dataset size (ranging from 300 GB to 10 TB).

In order to get a quick ramp-up, we are looking to leverage the cloud. My understanding is that Thor is available on Amazon AWS but Roxie is not yet available.

Request some insight into the following questions:

(1) Is this a feasible option (ie Thor on Amazon with on-premise Roxie?)

(2) What are the suggestions around using Thor on Amazon with on-premise Roxie? What network and infrastructure would be required to demonstrate ETL on Thor with low latency querying on Roxie?

(3) I read on the blog that a thumb rule for estimating Thor disk space is usually about 3 times the size of the aggregated input data. How should we go about estimating the # of Roxie nodes?

Thanks

[bforeman]

I just did a quick check, and it looks like Roxie nodes are now available on the one-click THOR AWS cluster. That would be your best bet.

The rule of thumb for the ratio of THOR to ROXIE clusters is that it be evenly divisible. Typically in a 400-node production cluster you will see a 100 node Roxie.

Hope this helps,

Bob

[arunarav]

Bob: Thanks for the update. Will there be an equivalent of one-click Thor for Roxie? Could you kindly point me to the documentation to get Roxie instances going on AWS?

Thanks
-Arun

[bforeman]

Once you log into AWS and then Launch Cluster, you should see the entry field for adding any number of Roxie nodes that you need. On the one-click interface they are created together. Rule of thumb is that your THOR nodes should be a multiple of your Roxie nodes, for example if you create 10 THOR nodes, 5 Roxie nodes would be a good choice, and adding 2 support nodes you come up with 17 nodes total created by the one-click.