Configuring multi-node HPCC System in AWS
Hi,
As part of study i was asked to set up a multi-node system on Amazon instances,in such a way that instance1 will be of AWS account1 and instance2 will be of AWS account2.
Is it possible to configure multi-node system on AWS instances that are of different account(here between instance1 and instance2)?
Can anyone provide a clarification?
Thanks,
Anjali
As part of study i was asked to set up a multi-node system on Amazon instances,in such a way that instance1 will be of AWS account1 and instance2 will be of AWS account2.
Is it possible to configure multi-node system on AWS instances that are of different account(here between instance1 and instance2)?
Can anyone provide a clarification?
Thanks,
Anjali
- Anjali
- Posts: 20
- Joined: Wed Feb 25, 2015 11:52 am
Anjali,
So, my real question is: Why?
Richard
So you have two user accounts on AWS. You want to ramp up a single HPCC environment where the nodes are billed to two separate AWS accounts? How do you expect to do that? The nodes must talk to each other to work together.As part of study i was asked to set up a multi-node system on Amazon instances,in such a way that instance1 will be of AWS account1 and instance2 will be of AWS account2.
So, my real question is: Why?
Richard
- rtaylor
- Community Advisory Board Member
- Posts: 1619
- Joined: Wed Oct 26, 2011 7:40 pm
Hi,
Thanks for your reply
Yes,that is the scenario i have given.We are a study group with our own AWS free accounts.Just for knowledge purpose we are trying to set up a cluster between our free instances.
So is it possible to configure the multi node system between instances of different account?is it possible to have a connect between these free instances?
Thanks,
Anjali
Thanks for your reply

Yes,that is the scenario i have given.We are a study group with our own AWS free accounts.Just for knowledge purpose we are trying to set up a cluster between our free instances.
So is it possible to configure the multi node system between instances of different account?is it possible to have a connect between these free instances?
Thanks,
Anjali
- Anjali
- Posts: 20
- Joined: Wed Feb 25, 2015 11:52 am
It is possible to have an HPCC System whose nodes are on different accounts. The setup process doesn't ask any information about the AWS accounts. I do have one caveat. If you want the HPCC System to be fast, the instances should be in the same AWS Placement Group, but it isn't a requirement. I don't know if you can have the same placement group in two accounts -- probably not.
With that said, the key to making all this work is getting the programs that setup the HPCC System on all instances and making sure that all instances have the correct ports setup in their security group. You can get all the programs for setting up the HPCC System on github at https://github.com/tlhumphrey2/EasyFastHPCCoAWS.
The document in this github repository, EasyFastHPCCoAWS.pdf, tells you how to setup an HPCC System using the programs and the CloudFormation template in the repository. You won't be able to use CloudFormation since your HPCC System will be made instances from different accounts. But, you can use the programs.
With that said, the key to making all this work is getting the programs that setup the HPCC System on all instances and making sure that all instances have the correct ports setup in their security group. You can get all the programs for setting up the HPCC System on github at https://github.com/tlhumphrey2/EasyFastHPCCoAWS.
The document in this github repository, EasyFastHPCCoAWS.pdf, tells you how to setup an HPCC System using the programs and the CloudFormation template in the repository. You won't be able to use CloudFormation since your HPCC System will be made instances from different accounts. But, you can use the programs.
- tlhumphrey2
- Posts: 260
- Joined: Mon May 07, 2012 6:23 pm
Another alternative.
You can have multiple HPCC nodes on the same instance. And, if the instance type is one with multiple cores, lots of memory and disk space, the HPCC System can run fairly fast.
You can have multiple HPCC nodes on the same instance. And, if the instance type is one with multiple cores, lots of memory and disk space, the HPCC System can run fairly fast.
- tlhumphrey2
- Posts: 260
- Joined: Mon May 07, 2012 6:23 pm
Hi,
Thank you for your valuable comments..
I hope the git hub link that you have shared can help me to sort out this.Let me try once:)
Thanks,
Anjali
Thank you for your valuable comments..
I hope the git hub link that you have shared can help me to sort out this.Let me try once:)
Thanks,
Anjali
- Anjali
- Posts: 20
- Joined: Wed Feb 25, 2015 11:52 am
The following document is the best place to start: http://cdn.hpccsystems.com/pdf/RunningHPCCwithinAWS_EC2_Beta1.pdf.
It gives the basics. The github repository I sited is more advanced and therefore leaves out details you need to know.
It gives the basics. The github repository I sited is more advanced and therefore leaves out details you need to know.
- tlhumphrey2
- Posts: 260
- Joined: Mon May 07, 2012 6:23 pm
8 posts
• Page 1 of 1
Who is online
Users browsing this forum: No registered users and 1 guest