Cluster Section

These attributes are for cluster-wide settings and can be specified within a [cluster] block inside the template.

Autoscale
If true, nodes in a nodearray will be added or deleted based on
demand.
CycleCloudVersion
Sets nodes in the cluster to be started with images and cookbooks to
match the given version of CycleCloud. Using a wildcard like 5.5.x
will use the latest version in the 5.5 release.
MaxCoreCount
The maximum number of cores to allocate for this cluster. Valid values
are any positive number. To ensure that the cluster never exceeds 100
cores you would specify a value of 100. Note that MaxCount and
MaxCoreCount can be used together, in which case the lower effective
constraint will take effect.
MaxCount
The maximum number of instances to allocate for this cluster. Valid
values are any positive number. To ensure that the cluster never
exceeds 10 instances you would specify a value of 10. Note that
MaxCount and MaxCoreCount can be used together, in which case the
lower effective constraint will take effect.
ParentName
The name of the parent cluster if you are using a cluster-of-clusters
configuration for massive scale. Valid values are any cluster name
that is running and set up as a parent cluster.
Password
Ignored as of CycleCloud 1.10. For the same behavior, set the node
level configuration attribute ‘cyclecloud.shared_user.password’
instead.
Username
Ignored as of CycleCloud 1.10. For the same behavior, set the node
level configuration attribute ‘cyclecloud.shared_user.name’ instead.

Node / NodeArray Section

These attributes are for node-specific settings and can be specified within a [node] or [nodearray] block within a template.

Note

You can apply these settings to all nodes via node inheritance specifying the attributes inside a [node defaults] section.

AwaitInstallation
If CycleCloud should wait for for the node to finish installing
software before being ready to use. This should be set to true when
using non-registry custom images that are Jetpack based and thus
support sending installation complete signals.
AwaitInstallationTimeout
How long CycleCloud should wait (in minutes) for the node to finish
installing software before erroring. This defaults to 30 minutes and
should only be changed if the length of software installation is known
to be greater than the default timeout.
BlackboardLocker
The name of the blackboard locker you want to use for this node. Valid
values are any locker name for your account. This attribute is
automatically detected and set for you, so specifying this attribute
manually should almost never need to happen.
CapacitySet
The name of the cloud-provider-specific capacity pool that this node
is coming from.
ChefRepoLocker
The name of the Chef repo locker you want to use for this node. Valid
values are any locker name for your account. This attribute is
automatically detected and set for you, so specifying this attribute
manually should almost never need to happen.
ChefRepoVersion
The version of your Chef cookbooks to use. When providing your own
Chef cookbooks, this attribute can be set to use a version other than
the default ‘latest’. Any string is valid so long as it defines an
actual chef version in one of your lockers.
ClusterInit
The name of the cluster-init that you wish to use. If not specified
the name ‘default’ is assumed. Valid values are any string. If the
cluster-init is not found, then no cluster-init will be used.
ClusterInitLocker
The name of the cluster-init locker you want to use for this node.
Valid values are any locker name for your account. This attribute is
automatically detected and set for you, so specifying this attribute
manually should almost never need to happen.
CommonChefRepoVersion
The version of the Cycle Computing-provided chef repository if the
default (stable) version is not wanted. If this attribute is not set
than the system default will be used. In general, this setting should
only be set if you know specific version you want to use or test.
Configuration
Settings used to configure the node on bootup (as name-value pairs).
CoreCount
The number of cores that this node is considered to have. Defines the
number of cores possessed by this node for the purposes of
autoscaling. This value overrides the default value for the cloud
resource type.
Credentials
The name of the credentials you wish to use to start your node. If not
specified (default), the default credentials for your account will be
used. This attribute is used if you have more than one set of
credentials set up for your CycleCloud account. For example, if you
have a ‘prod’ account and a ‘dev’ account set up, you can tell all
nodes in a cluster to start in ‘dev’ by specifying
Credentials=dev.
DisableAutomaticEphemeral
Indicates that no local (ephemeral) disks should be attached to this
node. If this attribute is set to true, you will have to attach your
own ephemeral disks, have them configured on the image, or use no
ephemeral disks at all. CycleCloud will not automatically generate
and attach them.
Fixed
If false, these nodes are deleted when the node is terminated. This
indicates that nodes represent only the current intention for
instances and may be expanded into fewer large nodes or contracted.
FixedPublicIp
A persistent public IP address to assign to this instance. If you want
to attach a public-facing IP to give the node a consistent public IP
address you can specify the address here. Valid values are a standard
IPv4 address, for example: 192.134.5.19. You typically must
reserve the specific address from your cloud provider before
specifying it on a node.
Image
Standard image to start this instance. This specifies the operating
system that will run on the machine. For a list of CycleCloud images,
please see the Image reference section of the documentation.
ImageId
Provider-specific id to use to start this instance. This defines the
operating system that will run on the machine. For example, ami-
abcd1234
. This is intended for public non-CycleCloud images or
custom user-defined images.
ImageName
Name of a standard image to start this instance. Note: this is the
name of the image, not the label referenced by ImageName.
ImageVersion
The version specification of the image. This matches a specific
version (eg, 1.2.3) or a version pattern (eg, 1.2.x). This is
not typically recommended for standard CycleCloud images, because the
correct image for your cluster is chosen automatically. Note that this
is not the version of the operating system, which is usually
included in the image name itself.
IpForwardingEnabled
If instances are allowed to send packets whose source IP address does
not match the IP address of the instance sending the packet. Default:
False.
IsArray
True if this node is a nodearray.
IsReturnProxy
If true, this node will be used as a proxy to access the controlling
CycleCloud instance. The node with this enabled will be configured to
allow access back to the controlling CycleServer instance.
Additionally, all nodes in the cluster will automatically be
configured to direct their CycleServer traffic to this node. Only one
node in a cluster may have this setting enabled; otherwise the cluster
will not start. If this setting is enabled it requires the KeyPair and
KeyPairLocation settings to also be defined. Currently this setting is
only supported for linux nodes.
IsSpotInstance
Set automatically if a bid price is set on this node.
KeyPair
The name of the keypair to use when starting the node. This keypair
will be used as the root login. If you followed the default
instructions for CycleCloud this keypair is likely named ‘cyclecloud’.
The available keypairs are typically available from your cloud
provider console.
KeyPairLocation
The location of the keypair on your local machine. This is used to log
into the remote machine after it has been started. Valid values are
any path, for example: /home/users/test.user/.ssh/cyclecloud.pem.
Note: This attribute is not required to start an instance, but it is
required to connect to that instance using the connect command.
Locker
The name of a storage configuration locker to use for this node. This
attribute is automatically and detected and set for you based on the
credentials in use by the node. If you specify a value this
effectively sets both the ClusterInitLocker and ChefRepoLocker
settings.
MachineType
Provider-specific name for instance to start (eg, m1.large). Consult
your cloud provider documentation for the available machine types.
ManagedServices
The list of services that this node has enabled.
NodeIndex
The position of this node in the nodes for its array, zero-based.
Preemptible
The node is allowed to be preempted (terminated). The node will be run
at a much lower cost, but can be terminated by the cloud provider at
any time if the resources are needed
PricingModel
The type of cloud-provider-specific billing being used for this node.
Provider
The cloud provider that will be used for this node, given the
credentials.
ReturnPath.BrokerPort
The port for the AMQP connection to CycleCloud from the perspective of
this node when not using the automatic return proxy feature.
ReturnPath.Hostname
The hostname or IP address this node uses to connect to CycleCloud
when not using the automatic return proxy feature. This setting needs
to be specified if CycleCloud’s own hostname would not be valid from
this node, such as using a bastion server to control access from cloud
instances back to CycleCloud.
ReturnPath.WebserverPort
The port for the HTTP/HTTPS connection to CycleCloud from the
perspective of this node when not using the automatic return proxy
feature.
ReturnProxyAddress
The Return Proxy network address that the cluster should communicate
with. If not specified, it will default to ‘ ‘ which indicates that
the cluster nodes should communicate with the Return Proxy using the
proxy’s private network address. For situations where the private
network is not accessible to the cluster (on separate Cloud Providers
or different regions) this should be set to ‘ ‘. This tells the
cluster to use the public network.
ReturnProxyTimeout
The duration (in seconds) to wait for the Return Proxy to be
established before considering it failed.
ScriptFile
The path to a script that will be run on the instance on first start.
TagList
The list of tags to apply to the node, if any. Note: tags are limited
to lower case letters, numbers and ‘-‘
TargetState
The target state to move this resource to.
Template
The name of the template that this node’s settings came from.
Username
The username of an administrative user to log into the node.
Zone
The zone or data center you want to start your instance in. If not
specified, one will be chosen for you. Placing all nodes in the same
zone reduces latency between nodes. However, if a zone goes down, all
nodes in this zone may be lost. Also, availability of some machine
types may be constrained by zone.

NodeArray Extensions

In addition, [nodearray] elements support extra attributes.

ApplyAfter
The next time nodes should be added to meet the target count.
ApplyOnce
If true, autoscaling happens one time, not continuously.
InitialCoreCount
The number of cores to allocate for this node array when the cluster
starts. Valid values are any positive number. If not specified 0 (no
nodes) is assumed. This is a one-time operation, and instances that
are allocated for this and then shutdown are not replaced. Note: This
cannot be specified together with InitialCount.
InitialCount
The number of nodes to allocate for this node array when the cluster
starts. Valid values are any positive number. If not specified 0 is
assumed. This is a one-time operation, and instances that are
allocated for this and then shutdown are not replaced. Note: This
cannot be specified together with InitialCoreCount.
LastApplied
The last time nodes were added to meet the target count.
MaxCoreCount
The maximum number of cores to allocate for this node array. Valid
values are any positive number. To ensure that the cluster never
exceed 100 cores of a given node array you would specify a value of
100. Note that MaxCount and MaxCoreCount can be used together, in
which case the lower effective constraint will take effect.
MaxCount
The maximum number of instances to allocate for this node array. Valid
values are any positive number. To ensure that the cluster never
exceed 10 instances of a given node array you would specify a value of
10. Note that MaxCount and MaxCoreCount can be used together, in which
case the lower effective constraint will take effect.
RequestSet
The name of the autoscale request this nodearray came from. All
nodearrays in a given request set are replaced by the nodearrays the
next time that autoscale request is made. By default, the autoscale
request is ‘standard’.
TargetCoreCount
How many cores to allocate. CycleCloud will attempt to keep this many
cores running in the cloud. Note: This cannot be specified together
with TargetCount.
TargetCount
How many instances to allocate. CycleCloud will attempt to keep this
many instances running in the cloud. Note: This cannot be specified
together with TargetCoreCount.
TargetExpiration
The time after which nodes should not be added to meet any target
counts.

AWS Extensions

Some settings take advantage of features of a specific cloud provider and are thus specified with a ”vendor prefix”. Note: These are specified in the template as listed, but actually create a subrecord on the Cloud.Node record called AWS.

AWS
AWS-specific settings.
AWS.AvailabilityZoneGroup
The user-specified name for a group of spot instances that are in the
same availability zone.
AWS.EBSOptimized
Whether the instance is optimized for EBS I/O.
AWS.InstanceProfileArn
The Amazon Resource Name (ARN) of the IAM instance profile to
associate with the instances.
AWS.InstanceProfileName
The name of the IAM Instance Profile (IIP) to associate with the
instances.
AWS.Kernel
The ID of the kernel. Note: Amazon recommends that you use PV-GRUB
instead of kernels and RAM disks.
AWS.LaunchGroup
The user-specified name for a group of spot instances that launch
together and terminate together.
AWS.Monitoring
Whether to enable CloudWatch monitoring on the instance.
AWS.RamDisk
The ID of the RAM disk. Note: Amazon recommends that you use PV-GRUB
instead of kernels and RAM disks.
AWS.RootDevice
The name of the root device, if different than what is reported on the
image.
AWS.RootDeviceIops
The number of IOPS to assign to the root device.
AWS.RootDeviceSize
The size of the root device.
AWS.RootDeviceType
The type of root device to use (‘standard’, ‘gp2’, ‘io1’).
AWS.SpotRequestType
The type of spot-instance request (‘one-time’ or ‘persistent’).
AWS.Virtualization
Whether to use ‘hvm’ (the default) or ‘pvm’ virtualization, for
instance types that support both. Note: this only applies when
selecting images by name, if both HVM and PVM images are available.

Some older attributes are AWS-specific but do not have the AWS. prefix:

BidDuration
Indicates the maximum amount of time (in minutes) to keep a spot
request open. If defined, this value will be used instead of the
default of 20 minutes. If the specified time has elapsed without the
spot request being fulfilled the request will be automatically
cancelled.
BidPrice
Indicates the maximum amount you are willing to pay for spot
instances. If defined, this indicates nodes should be created as a bid
for spot instances using the Amazon Spot Market. Valid values are any
decimal, for example 0.01 means $0.01 (1 cent in USD). If the value is
too low, you will not be granted the machine until the value of that
machine goes below your bid price.For more information, see Amazon
EC2 Spot Instances
.
BidPricePerCore
Indicates the maximum amount you are willing to pay per-core for spot
instances. If defined, this indicates that nodes should be created as
a bid for spot instances using the Amazon Spot Market. Like
BidPrice, this is the maximum amount of money (in dollars) you are
willing to spend, but specified per CPU core hour instead of per
instance. Valid values are any decimal, for example 0.01 means $0.01
(1 cent in USD). This attribute allows you to specify bid prices more
consistently across instance types like c1.medium and c1.xlarge, which
differ only in the number of cores available.
ExtendedRequest
True if this request will wait an extended period of time to be
fulfilled.
PlacementGroup
The placement group to put a node in. All nodes in a placement group
must be in the same region. Placement groups ensure that all nodes
within the placement group are physically located near each other,
making placement groups ideal for latency-sensitive workflows such as
MPI jobs. Placement groups are only available for cluster compute
instances. Valid values are any placement group name (which have to be
created in the AWS console beforehand), for example test-placement-
group
. For more information on placement groups see: http://docs.aws
.amazon.com/AWSEC2/latest/UserGuide/using_cluster_computing.html.
SecurityGroups
A list of AWS EC2 security groups to apply to the node. If not
specified the default security group will be used; otherwise, valid
values are a comma-separated list of security groups. For example:
sg-1234abcd, sg-98761234.
SubnetId
The VPC subnet in which to launch this instance, if defined. If your
cluster is going to run inside an Amazon VPC, you will need to specify
the SubnetId to start the instance in. Valid values are any subnet ID,
for example: subnet-1234abcd. You can view available subnets in
the AWS VPC console.
Tenancy
Specifies the degree to which this instance can be shared with other
accounts. For Amazon Web Services, the two accepted values are
“default” or “dedicated”. If not specified, “default” will be used.
There may be additional charges for using this feature.
TerminateOnShutdown
Indicates instances should terminate themselves when they are
shutdown. If true, when an instance attempts to shut itself down
(shutdown -h, poweroff, etc), the instance will go into a terminated
state (if false, it goes into a ‘stopped’ state). This value shouldn’t
need to be updated unless you have a specific case where terminating
an instance is not the correct behavior. Only applies to on-demand AWS
nodes.
TerminationProtection
Indicates instances should not be allowed to be terminated as normal.
If true, termination requests on the instance will fail leaving the
instance in a running state. To terminate the instance, Termination
Proection will have to be disabled via the AWS console before making a
termination request. This attribute should be used to prevent the
accidental termination of important instances. Only applies to on-
demand AWS nodes.

Status Attributes

These attributes are set automatically by CycleCloud and are available for informational purposes only.

InstallationStatus
Status of node’s software installation.
InstanceId
The id of the instance started for this node.
InstanceName
The name of the instance started for this node, if any.
Last
The previous state of various transient attributes. This is set when a
node is terminated and retains its value until the node is started and
terminated again.
SpotRequestId
The id of the spot request started for this node, if any.

Node Status Attributes

Nodes report on their status with the following attributes. Note that these cannot be set directly.

ActivePhases
The cluster initialization steps that are currently being executed for
this node.
PhaseFailed
If true, the initialization process failed.
PhaseMap
Information on the cluster initialization steps for this node.
State
The state this node is in currently.
Status
The overall summary of this’s node progress. The status options are
Off (no instance is active or being acquired), Acquiring (getting
an instance from the cloud provider), Preparing (configuring the
instance and installing software), and Ready (instance is up and
running). If any phase fails during starting or terminating the node,
the status is Failed.
StatusMessage
The description of this node’s current status.