Sat Aug 18, 2018 9:53 pm
Login Register Lost Password? Contact Us


Configuring multi-node HPCC System in AWS

Questions or comments related to Cloud Computing and the HPCC Systems Instant Cloud for AWS

Tue Oct 13, 2015 8:31 am Change Time Zone

Hi,

As part of study i was asked to set up a multi-node system on Amazon instances,in such a way that instance1 will be of AWS account1 and instance2 will be of AWS account2.

Is it possible to configure multi-node system on AWS instances that are of different account(here between instance1 and instance2)?


Can anyone provide a clarification?


Thanks,
Anjali
Anjali
 
Posts: 20
Joined: Wed Feb 25, 2015 11:52 am

Tue Oct 13, 2015 6:26 pm Change Time Zone

Anjali,
As part of study i was asked to set up a multi-node system on Amazon instances,in such a way that instance1 will be of AWS account1 and instance2 will be of AWS account2.
So you have two user accounts on AWS. You want to ramp up a single HPCC environment where the nodes are billed to two separate AWS accounts? How do you expect to do that? The nodes must talk to each other to work together.

So, my real question is: Why?

Richard
rtaylor
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 1370
Joined: Wed Oct 26, 2011 7:40 pm

Wed Oct 14, 2015 5:07 am Change Time Zone

Hi,

Thanks for your reply :)

Yes,that is the scenario i have given.We are a study group with our own AWS free accounts.Just for knowledge purpose we are trying to set up a cluster between our free instances.

So is it possible to configure the multi node system between instances of different account?is it possible to have a connect between these free instances?

Thanks,
Anjali
Anjali
 
Posts: 20
Joined: Wed Feb 25, 2015 11:52 am

Wed Oct 14, 2015 2:48 pm Change Time Zone

It is possible to have an HPCC System whose nodes are on different accounts. The setup process doesn't ask any information about the AWS accounts. I do have one caveat. If you want the HPCC System to be fast, the instances should be in the same AWS Placement Group, but it isn't a requirement. I don't know if you can have the same placement group in two accounts -- probably not.

With that said, the key to making all this work is getting the programs that setup the HPCC System on all instances and making sure that all instances have the correct ports setup in their security group. You can get all the programs for setting up the HPCC System on github at https://github.com/tlhumphrey2/EasyFastHPCCoAWS.

The document in this github repository, EasyFastHPCCoAWS.pdf, tells you how to setup an HPCC System using the programs and the CloudFormation template in the repository. You won't be able to use CloudFormation since your HPCC System will be made instances from different accounts. But, you can use the programs.
tlhumphrey2
 
Posts: 250
Joined: Mon May 07, 2012 6:23 pm

Wed Oct 14, 2015 3:05 pm Change Time Zone

Another alternative.
You can have multiple HPCC nodes on the same instance. And, if the instance type is one with multiple cores, lots of memory and disk space, the HPCC System can run fairly fast.
tlhumphrey2
 
Posts: 250
Joined: Mon May 07, 2012 6:23 pm

Thu Oct 15, 2015 5:00 am Change Time Zone

Hi,

Thank you for your valuable comments..

I hope the git hub link that you have shared can help me to sort out this.Let me try once:)

Thanks,
Anjali
Anjali
 
Posts: 20
Joined: Wed Feb 25, 2015 11:52 am

Thu Oct 15, 2015 10:17 am Change Time Zone

The following document is the best place to start: http://cdn.hpccsystems.com/pdf/RunningHPCCwithinAWS_EC2_Beta1.pdf.

It gives the basics. The github repository I sited is more advanced and therefore leaves out details you need to know.
tlhumphrey2
 
Posts: 250
Joined: Mon May 07, 2012 6:23 pm

Thu Oct 15, 2015 11:54 am Change Time Zone

Sure...Thanks a lot :)
Anjali
 
Posts: 20
Joined: Wed Feb 25, 2015 11:52 am


Return to Cloud

Who is online

Users browsing this forum: No registered users and 1 guest

cron