cfncluster uses the file ~/.cfncluster/config by default for all configuration parameters.
You can see an example configuration file site-packages/cfncluster/examples/config
Configuration is defined in multiple sections. Required sections are "global", "aws", one "cluster", and one "subnet".
A section starts with the section name in brackets, followed by parameters and configuration.
[global]
cluster_template = default
update_check = true
sanity_check = true
Global configuration options related to cfncluster.
[global]
The name of the cluster section used for the cluster.
See the Cluster Definition.
cluster_template = default
Attempts to validate that resources defined in parameters actually exist.
sanity_check = true
This is the AWS credentials section (required). These settings apply to all clusters.
If not defined, boto will attempt to use a) enviornment or b) EC2 IAM role.
[aws]
aws_access_key_id = #your_aws_access_key_id
aws_secret_access_key = #your_secret_access_key
# Defaults to us-east-1 if not defined in enviornment or below
aws_region_name = #region
You can define one or more clusters for different types of jobs or workloads.
Each cluster has it's own configuration based on your needs.
The format is [cluster <clustername>].
[cluster default]
Overrides the path to the cloudformation template used to create the cluster
Defaults to https://s3.amazonaws.com/cfncluster-<aws_region_name>/templates/cfncluster-<version>.cfn.json.
template_url = https://s3.amazonaws.com/cfncluster-us-east-1/templates/cfncluster.cfn.json
The EC2 instance type used for the cluster compute nodes.
Defaults to t2.micro for default template.
compute_instance_type = t2.micro
The EC2 instance type use for the master node.
This defaults to t2.micro for default template.
master_instance_type = t2.micro
The inital number of EC2 instances to launch as compute nodes in the cluster.
The default is 2 for default template.
initial_queue_size = 2
The maximum number of EC2 instances that can be launched in the cluster.
This defaults to 10 for the default template.
max_queue_size = 10
Boolean flag to set autoscaling group to maintain initial size.
If set to true, the Auto Scaling group will never have fewer members than the value of initial_queue_size. It will still allow the cluster to scale up to the value of max_queue_size.
Setting to false allows the Auto Scaling group to scale down to 0 members, so resources will not sit idle when they aren't needed.
Defaults to false for the default template.
maintain_initial_size = false
Scheduler to be used with the cluster. Valid options are sge, openlava, or torque.
Defaults to sge for the default template.
scheduler = sge
Type of cluster to launch i.e. ondemand or spot
Defaults to ondemand for the default template.
cluster_type = ondemand
If cluster_type is set to spot, the maximum spot price for the ComputeFleet.
spot_price = 0.00
Specify S3 resource which cfncluster nodes will be granted read-only access
See working with S3 for details on format.
Defaults to NONE for the default template.
s3_read_resource = NONE
Specify S3 resource which cfncluster nodes will be granted read-write access
See working with S3 for details on format.
Defaults to NONE for the default template.
s3_read_write_resource = NONE
URL to a preinstall script. This is executed before any of the boot_as_* scripts are run
Can be specified in "http://hostname/path/to/script.sh" or "s3://bucketname/path/to/script.sh" format.
Defaults to NONE for the default template.
pre_install = NONE
Quoted list of arguments to be passed to preinstall script
Defaults to NONE for the default template.
pre_install_args = NONE
URL to a postinstall script. This is executed after any of the boot_as_* scripts are run
Can be specified in "http://hostname/path/to/script.sh" or "s3://bucketname/path/to/script.sh" format.
Defaults to NONE for the default template.
post_install = NONE
Arguments to be passed to postinstall script
Defaults to NONE for the default template.
post_install_args = NONE
HTTP(S) proxy server, typically http://x.x.x.x:8080
Defaults to NONE for the default template.
proxy_server = NONE
Cluster placement group. This placement group must already exist.
Defaults to NONE for the default template.
placement_group = NONE
Cluster placment logic. This enables the whole cluster or only compute to use the placement group.
Defaults to cluster in the default template.
placement = cluster
If instance store volumes exist, this is the path/mountpoint they will be mounted on.
Defaults to /scratch in the default template.
ephemeral_dir = /scratch
Encrypted ephemeral drives. In-memory keys, non-recoverable.
Defaults to false in default template.
encrypted_ephemeral = false
MasterServer root volume size in GB. (AMI must support growroot)
Defaults to 10 in default template.
master_root_volume_size = 10
ComputeFleet root volume size in GB. (AMI must support growroot)
Defaults to 10 in default template.
compute_root_volume_size = 10
CloudWatch Logs Log Group name
Defaults to NONE in the default template.
cwl_log_group = NONE
Existing EC2 IAM Role that will be attached to all instances in the cluster.
Defaults to NONE in the default template.
ec2_iam_role = NONE
Extra JSON that will be merged into the dna.json used by Chef.
Defaults to {} in the default template.
extra_json = {}
Settings section relating to EBS volume mounted on the master.
See EBS Section.
ebs_settings = custom
Settings section relation to scaling
See Scaling Section.
scaling_settings = custom
VPC Configuration Settings:
[vpc public]
vpc_id = vpc-xxxxxx
master_subnet_id = subnet-xxxxxx
ID of an existing subnet you want to provision the Master server into.
master_subnet_id = subnet-xxxxxx
CIDR formatted IP range in which to allow SSH access from.
This is only used when cfncluster creates the security group.
Defaults to 0.0.0.0/0 in the default template.
ssh_from = 0.0.0.0/0
Additional VPC security group Id for all instances.
Defaults to NONE in the default template.
additional_sg = sg-xxxxxx
ID of an existing subnet you want to provision the compute nodes into.
compute_subnet_id = subnet-xxxxxx
If you wish for cfncluster to create a compute subnet, this is the CIDR that.
compute_subnet_cidr = 10.0.100.0/24
Define whether or not to assign public IP addresses to EC2 instances.
Set to false if operating in a private VPC.
Defaults to true.
use_public_ips = true
Use an existing security group for all instances.
Defaults to NONE in the default template.
vpc_security_group_id = sg-xxxxxx
EBS Volume configuration settings for the volume mounted on the master node and shared via NFS to compute nodes.
[ebs custom]
ebs_snapshot_id = snap-xxxxx
volume_type = io1
volume_iops = 200
Id of EBS snapshot if using snapshot as source for volume.
Defaults to NONE for default template.
ebs_snapshot_id = snap-xxxxx
The API name for the type of volume you wish to launch.
Defaults to gp2 for default template.
volume_type = io1
Size of volume to be created (if not using a snapshot).
Defaults to 20GB for default template.
volume_size = 20
Whether or not the volume should be encrypted (should not be used with snapshots).
Defaults to false for default template.
encrypted = false
EBS Volume Id of an existing volume that will be attached to the MasterServer.
Defaults to NONE for default template.
ebs_volume_id = vol-xxxxxx
Settings which define how the compute nodes scale.
[scaling custom]
scaling_period = 60
scaling_cooldown = 300
Threshold for triggering CloudWatch ScaleUp action.
Defaults to 1 for default template.
scaling_threshold = 1
Number of instances to add when called CloudWatch ScaleUp action.
Defaults to 1 for default template.
scaling_adjustment = 1
Threshold for triggering CloudWatch ScaleUp2 action.
Defaults to 200 for default template.
scaling_threshold2 = 200
Number of instances to add when called CloudWatch ScaleUp2 action
Defaults to 20 for default template.
scaling_adjustment2 = 20
Period to measure ScalingThreshold.
Defaults to 60 for default template.
scaling_period = 60
Number of periods to measure ScalingThreshold.
Defaults to 2 for default template.
scaling_evaluation_periods = 2
Amount of time in seconds to wait before attempting further scaling actions.
Defaults to 300 for the default template.
scaling_cooldown = 300