CycleCloud supports automatically attaching volumes (disks) to your nodes for additional storage space. For example, to create a 100GB volume, add the following to your [[node]] element in your cluster template:
[[[volume example-vol]]] Size = 100
This volume will be created when the instance is started, and deleted when the instance is terminated. If you want to preserve the data on the volume even after the instance is terminated, make it a persistent volume:
[[[volume example-vol]]] Size = 100 Persistent = true
This volume will be created the first time the instance is started, but will not be deleted when the instance is terminated. Instead, it will be kept and re-attached to the instance the next time the node is started. Persistent volumes are not deleted until the cluster is deleted.
To use Premium Storage for the disk (or the equivalent cloud provider SSD storage), use SSD = true:
[[[volume example-vol]]] Size = 100 Persistent = true SSD = true
Azure SSD will round up to the next size for pricing. For example, if you create a disk size of 100GB, you will be charged at the 128GB rate. Please see Azure Pricing Details for more information.
When your cluster is deleted, all persistent volumes are deleted as well! If you want your storage to persist longer than your cluster, you must attach a preexisting volume by id.
For Linux-based operating systems, you can control what device to attach the volume to, using the Device attribute:
[[[volume example-vol]]] Size = 100 Device = /dev/sdk
If you do not specify a device, CycleCloud will automatically pick a device that is not in use. The specific device chosen depends on the cloud provider configuration and the image.
Specifying a volume attaches the device(s) to your instance, but does not mount and format the device. If you prefer to have the volumes mounted and formatted when the node is started, set the optional attribute Mount to the name of the mountpoint configuration you wish to use with that volume:
[[[volume reference-data]]] Size = 100 Mount = data # The name of the mountpoint to use with this volume
The mountpoint named data is then defined in the configuration section on the node:
[[[configuration cyclecloud.mounts.data]]] mountpoint = /mount fs_type = ext4
The above configuration specifies that you are configuring a cyclecloud.mountpoint named data using all volumes which include Mount = data. This volume would be formatted with the ext4 filesystem and would appear at /mount.
By defining volumes with a Mountpoint attribute, the device names will be automatically assigned and used for a given mountpoint. You can, however, customize a mountpoint with your own device names if there is a need. For example:
[[node master]] [[[configuration cyclecloud.mounts.data]]] mountpoint = /data Azure.LUN=0
In Azure, devices are assigned using Logical Unit Numbers (LUN). The devices parameter is used to manually specify each underlying device that is part of the mountpoint configuration.
In most cases, Azure CycleCloud will automatically assign devices for you. Specifying devices manually is advanced usage, and useful in cases where the AMI you are using for your node has volumes that will be automatically attached because their attachment was baked into the image. Specifying the devices by hand can also be useful when the ordering of devices has special meaning.
A volume named boot has special meaning.
For AWS, use device=/dev/sdc in place of LUN.
Google Cloud has the capability to resize the boot volume by defining the size attribute:
[[[volume boot]]] Size = 50
The previous example was a fairly simple: mounting a single, pre-formatted snapshot to a node. However, more advanced mounting can take place, including RAIDing multiple devices together, encrypting, and formatting new filesystems. As an example, the following will describes how to RAID several EBS volumes together and encrypt them before mounting them as a single device on a node:
[[node master]] .... [[[volume vol1]]] VolumeId = vol-1234abcd Mount = giant [[[volume vol2]]] VolumeId = vol-5678abcd Mount = giant [[[volume vol3]]] VolumeId = vol-abcd1234 Mount = giant [[[configuration cyclecloud.mounts.giant]]] mountpoint = /mnt/giant fs_type = xfs raid_level = 0 encryption.bits = 256 encryption.key = "0123456789abcdef9876543210"
The above example shows there are three EBS volumes that should be attached to the node named master, and that their mountpoint is named giant. The configuration for the mountpoint says that these three volumes should be RAIDed together using raid_level = 0 for RAID0, formatted using the xfs filesystem, and the resulting device should be mounted at /mnt/giant. Furthermore, the device should have block level encryption using 256-bit AES with an encryption key as defined in the template. See below for a full list of available mountpoint configuration options and their meanings.
Ephemeral Storage and Mounting
By default, ephemeral devices for a node will be automatically attached (see volume section above) and then RAIDed with using RAID0 and mounted to /mnt. This is the suggested way of using ephemeral devices within CycleCloud, however you can override the default behavior if necessary. All ephemeral devices are automatically assigned a mountpoint of ephemeral, so you can use this default behavior to customize the mountpoint as follows:
[[[configuration cyclecloud.mounts.ephemeral]]] mountpoint = /mnt/ephemeral fs_type = ext4 raid_level = 1
The above configuration will instruct CycleCloud to combine all the ephemeral device using RAID1, format them using ext4, then mount them at the alternative location of /mnt/ephemeral, which is different from the default of /mnt.
You can manually define ephemeral volumes using the following syntax, although it is not required in most cases as reasonable defaults are already in place:
[[node master]] [[[volume ephemeral0]]] Ephemeral = true Mount = ephemeral [[[volume ephemeral1]]] Ephemeral = true Mount = ephemeral
If you do not want any ephemeral automatically mapped for you, meaning you will either use no ephemeral storage or will rely on another form of attaching/mounting, you can set the DisableAutomaticEphemeral to true:
[[node master]] DisableAutomaticEphemeral = true # No ephemeral disks will be automatically attached to this instance
Mounting Configuration Options
|mountpoint||The place where the device(s) will be mounted after any additional configuration is applied. If a mountpoint is not specified, the name of the mount will be used as part of the mountpoint. For example, if your mount was named ‘data’, the mountpoint would default to ‘/media/data’.|
|options||Any non-default options to use when mounting the device.|
|fs_type||The filesystem to use when formatting and/or mounting. Available options are: ext3, ext4, xfs. CentOS 5 will use ext3 by default, and all other OSes will use ext4.|
|size||The size of the filesystem to create when formatting the device(s). Omitting this parameter will use all the space on the device. Size can be specified using M for megabytes (e.g. 150M for 150MB) G for gigabytes (e.g. 200G for 20GB), or percentages (e.g. 100% to use all of the available space).|
|disabled||If true, the mountpoint will not be created. Useful for quick toggling of mounts for testing and to disable automatic ephemeral mounting. Default: false.|
|raid_level||The type of RAID configuration to use when multiple devices/volumes are being used. Defaults to a value of 0, meaning RAID0, but other raid levels can be used such as 1 or 10.|
|raid_device_symlink||When a raid device is created, specifying this attribute will create a symbolic link to the raid device. By default, this attribute is not set and therefore no symlink is created. This should be set in cases where you need access to the underlying raid device.|
|devices||This is a list of devices that should compose the mountpoint. In general, this parameter shouldn’t need to be specified (as CycleCloud will set this for you based on [[[volume]]] sections), but you can manually specify the devices if so desired.|
|vg_name||Devices are configured on Linux using the Logical Volume Manager (LVM). The volume group name will be automatically assigned, but in cases where a specific name is used, this attribute can be set. The default is set to cyclecloud-vgX, where X is an automatically assigned number.|
|lv_name||Devices are configured on Linux using the Logical Volume Manager (LVM). This value is automatically assigned and does not need specification, but if you want to use a custom logical volume name, it can be specified using this attribute. Defaults to lv0.|
|order||By specifying an order, you can control the order in which mountpoints are mounted. The default order value for all mountpoints is 1000, except for ‘ephemeral’ which is 0 (ephemeral is always mounted first by default). You can override this behaviour on a case-by-case basis as needed.|
|encryption.bits||The number of bits to use when encrypting the filesystem. Standard values are 128 or 256 bit AES encryption. This value is required if encryption is desired.|
|encryption.key||The encryption key to use when encrypting the filesystem. If omitted, a random 2048 bit key will be generated. The automatically generated key is useful for when you are encrypting disks that do not persist between reboots (e.g. encrypting ephemeral devices).|
|encryption.name||The name of the encrypted filesystem, used when saving encryption keys. Defaults to cyclecloud_cryptX, where X is an automatically generated number.|
|encryption.key_path||The location of the file the key will be written on disk to. Defaults to /root/cyclecloud_cryptX.key, where X is a automatically generated number.|
Mounting Configuration Defaults
There are times when many different mountpoints are defined, and specifying the same options over and over can become tedious. The following options will allow you to set the system defaults for various mountpoints, and will be used unless otherwise specified:
|cyclecloud.mount_defaults.fs_type||The filesystem type to use for mounts, if not otherwise specified. Default: ext3/ext4 (depending on the platform).|
|cyclecloud.mount_defaults.size||The default filesystem size to use, if not otherwise specified. Default: 50GB.|
|cyclecloud.mount_defaults.raid_level||The default raid level to use if multiple devices are assigned to the mountpoint. Default: 0 (RAID0).|
|cyclecloud.mount_defaults.encryption.bits||The default encryption level unless otherwise specified. Default: undefined.|
Configuring NFS Mounts and Exports
CycleCloud provides built-in support for exporting, mounting, and configuring simple NFS filesystems.
Creating an NFS Export
To export a directory from a node as a shared NFS filesystem, provide a mount configuration section with type=nfs and an export path. For example:
[[[configuration cyclecloud.exports.nfs_data]]] type = nfs export_path = /mnt/exports/nfs_data
The above configuration cyclecloud.exports.nfs_data specifies that you are configuring directory /mnt/exports/nfs_data to be exported as an NFS filesystem named nfs_data. The attributes within the configuration section describe the exported filesystem properties.
Mounting an NFS Filesystem
To mount an existing NFS filesystem:
[[[configuration cyclecloud.mounts.nfs_data]]] type = nfs mountpoint = /mnt/exports/nfs_data export_path = /mnt/exports/data
The export_path is the path on the server, and the mountpoint is the path to mount the share on the client. The mounted NFS filesystem may be exported from a node in the same CycleCloud cluster, exported from a node in another CycleCloud cluster, or a separate NFS filesystem that allows simple mounts. If the filesystem is exported from a node in the local cluster, then CycleCloud will use search to discover the address automatically. If the filesystem is exported from a different CycleCloud cluster, then the mount configuration may specify attribute cluster_name to instruct CycleCloud to search the cluster with that name:
[[[configuration cyclecloud.mounts.other_cluster_fs]]] type = nfs mountpoint = /mnt/exports/other_cluster_fs export_path = /mnt/exports/data cluster_name = filesystem_cluster
Finally, to specify the location of the filesystem explicitly (required for mounting non-CycleCloud filesystems), the mount configuration may specify attribue address with the hostname or IP of the filesystem. For example:
[[[configuration cyclecloud.mounts.external_filer]]] type = nfs mountpoint = /mnt/exports/external_filer address = 188.8.131.52
Mounting an Amazon Elastic File System
Amazon offers an Elastic File System (EFS) service which provides NFS-compatible storage to EC2 instances. This is supported in CycleCloud similarly to NFS when the type of the mount is set to efs. An EFS mount requires either a filesystem_id, which is the ID of the EFS filesystem, or an address, which is the fully-qualified domain name of the EFS endpoint. For example, using the filesystem_id:
[[[configuration cyclecloud.mounts.efs]]] type = efs filesystem_id = fs-f00cf6b8
Using the address:
[[[configuration cyclecloud.mounts.efs]]] type = efs address = us-east-1e.fs-f00cf6b8.efs.us-east-1.amazonaws.com
Other settings such as mountpoint are the same as for the NFS type.
Disabling NFS Mounts
CycleCloud NFS mounts may be disabled by setting the disabled attribute to true.
Even the default shares may be disabled this way. For example:
[[[configuration cyclecloud.mounts.shared]]] disabled = true
Export Configuration Options
|type||REQUIRED The type attribute must be set to nfs for all NFS exports. This is required in order to differentiate from other shared filesystem types.|
|export_path||The local path to export as an NFS filesystem. If the directory does not exist already, it will be created.|
|owner||The user account that should own the exported directory.|
|group||The group of the user that should own the exported directory.|
|mode||The default filesystem permissions on the exported directory.|
|network||The network interface on which the directory is exported. Defaults to all: *.|
|sync||Synchronous/asynchronous export option. Defaults to true.|
|writable||The ro/rw export option for the filesystem. Defaults to true.|
|options||Any non-default options to use when exporting the filesystem.|
Mount Configuration Options
|type||REQUIRED The type attribute must be set to nfs for all NFS mounts. This is required in order to differentiate from volume mounts and other shared filesystem types.|
|export_path||The location of the export on the NFS filer. If an export_path is not specified, the mountpoint of the mount will be used as the export_path.|
|mountpoint||The location where the filesystem will be mounted after any additional configuration is applied. If the directory does not already exist, it will be created.|
|cluster_name||The name of the CycleCloud cluster which exports the filesystem. If not set, the node’s local cluster is assumed.|
|address||The explicit hostname or IP address of the filesystem. If not set, search will attempt to find the filesystem in a CycleCloud cluster.|
|options||Any non-default options to use when mounting the filesystem.|
|disabled||If set to true, the node will not mount the filesystem.|