Thor on Amazon with on-premise Roxie
We would like to create some proof-of-concepts on varying dataset size (ranging from 300 GB to 10 TB).
In order to get a quick ramp-up, we are looking to leverage the cloud. My understanding is that Thor is available on Amazon AWS but Roxie is not yet available.
Request some insight into the following questions:
(1) Is this a feasible option (ie Thor on Amazon with on-premise Roxie?)
(2) What are the suggestions around using Thor on Amazon with on-premise Roxie? What network and infrastructure would be required to demonstrate ETL on Thor with low latency querying on Roxie?
(3) I read on the blog that a thumb rule for estimating Thor disk space is usually about 3 times the size of the aggregated input data. How should we go about estimating the # of Roxie nodes?
Thanks
In order to get a quick ramp-up, we are looking to leverage the cloud. My understanding is that Thor is available on Amazon AWS but Roxie is not yet available.
Request some insight into the following questions:
(1) Is this a feasible option (ie Thor on Amazon with on-premise Roxie?)
(2) What are the suggestions around using Thor on Amazon with on-premise Roxie? What network and infrastructure would be required to demonstrate ETL on Thor with low latency querying on Roxie?
(3) I read on the blog that a thumb rule for estimating Thor disk space is usually about 3 times the size of the aggregated input data. How should we go about estimating the # of Roxie nodes?
Thanks
- arunarav
- Posts: 20
- Joined: Fri Sep 21, 2012 5:34 pm
I just did a quick check, and it looks like Roxie nodes are now available on the one-click THOR AWS cluster. That would be your best bet.
The rule of thumb for the ratio of THOR to ROXIE clusters is that it be evenly divisible. Typically in a 400-node production cluster you will see a 100 node Roxie.
Hope this helps,
Bob
The rule of thumb for the ratio of THOR to ROXIE clusters is that it be evenly divisible. Typically in a 400-node production cluster you will see a 100 node Roxie.
Hope this helps,
Bob
- bforeman
- Community Advisory Board Member
- Posts: 1006
- Joined: Wed Jun 29, 2011 7:13 pm
Bob: Thanks for the update. Will there be an equivalent of one-click Thor for Roxie? Could you kindly point me to the documentation to get Roxie instances going on AWS?
Thanks
-Arun
Thanks
-Arun
- arunarav
- Posts: 20
- Joined: Fri Sep 21, 2012 5:34 pm
Once you log into AWS and then Launch Cluster, you should see the entry field for adding any number of Roxie nodes that you need. On the one-click interface they are created together. Rule of thumb is that your THOR nodes should be a multiple of your Roxie nodes, for example if you create 10 THOR nodes, 5 Roxie nodes would be a good choice, and adding 2 support nodes you come up with 17 nodes total created by the one-click.
- bforeman
- Community Advisory Board Member
- Posts: 1006
- Joined: Wed Jun 29, 2011 7:13 pm
4 posts
• Page 1 of 1
Who is online
Users browsing this forum: No registered users and 1 guest