CfnCluster ("cloud formation cluster") is a framework that deploys and maintains high performance computing clusters on Amazon Web Services (AWS). Developed by AWS, CfnCluster facilitates both quick start proof of concepts (POCs) and production deployments. CfnCluster supports many different types of clustered applications and can easily be extended to support different frameworks. Download CfnCluster today to see how CfnCluster's command line interface leverages AWS CloudFormation templates and other AWS cloud services.
The current working version is CfnCluster-1.3. The CLI is written in Python and uses BOTO for AWS actions. You can install the CLI with the following commands, depending on your OS.
$ sudo pip install cfncluster
$ sudo easy_install cfncluster
Windows support is experimental!!
Install the following packages:
Once installed, you should update the Environment Variables to have the Python install directory and Python Scripts directory in the PATH, for example:
Now it should be possible to run the following within a command prompt window:
C:\> easy_install CfnCluster
To upgrade an older version of CfnCluster, you can use either of the following commands, depening on how it was originally installed:
$ sudo pip install --upgrade cfncluster
$ sudo easy_install -U cfncluster
Remember when upgrading to check that the exiting config is compatible with the latest version installed.
Once installed you will need to setup some initial config. The easiest way to do this is below:
$ cfncluster configure
This configure wizard will prompt you for everything you need to create your cluster. You will first be prompted for your cluster template name, which is the logical name of the template you will create a cluster from.
Cluster Template [mycluster]:
Next, you will be prompted for your AWS Access & Secret Keys. Enter the keys for an IAM user with administrative privledges. These can also be read from your environment variables or the aws CLI config.
AWS Access Key ID : AWS Secret Access Key ID :
Now, you will be presented with a list of valid AWS region identifiers. Choose the region in which you'd like your cluster to run.
Acceptable Values for AWS Region ID: us-east-1 cn-north-1 ap-northeast-1 eu-west-1 ap-southeast-1 ap-southeast-2 us-west-2 us-gov-west-1 us-west-1 eu-central-1 sa-east-1 AWS Region ID :
Choose a descriptive name for your VPC. Typically, this will something like
VPC Name [myvpc]:
Next, you will need to choose a keypair that already exists in EC2 in order to log into your master instance. If you do not already have a keypair, refer to the EC2 documentation on EC2 Key Pairs.
Acceptable Values for Key Name: keypair1 keypair-test production-key Key Name :
Choose the VPC ID in which you'd like your cluster launched into.
Acceptable Values for VPC ID: vpc-1kd24879 vpc-blk4982d VPC ID :
Finally, choose the subnet in which you'd like your master server to run in.
Acceptable Values for Master Subnet ID: subnet-9k284a6f subnet-1k01g357 subnet-b921nv04 Master Subnet ID :
Next, a simple cluster launches into a VPC and uses an existing subnet which supports public IP's i.e. the route table for the subnet is
0.0.0.0/0 => igw-xxxxxx. The VPC must have
DNS Resolution = yes and
DNS Hostnames = yes. It should also have DHCP options with the correct
domain-name for the region, as defined in the docs: VPC DHCP Options.
Once all of those settings contain valid values, you can launch the cluster by repeating the command that was used at the start.
$ cfncluster create mycluster
Once the cluster reaches the "CREATE_COMPLETE" status, you can connect using your normal SSH client/settings. For more details on connecting to EC2 instances, check the EC2 User Guide.