CycleCloud manages compute resources that can be acquired on demand, for example, virtual machines or instances in the cloud. These resources are called nodes and are grouped into clusters. Nodes in a cluster are logically related, but each node can be configured completely independently. A node specifies the cloud provider, operating system, and machine type to use, the region or location to run in, and the software to install on the node when it starts.
Clusters make use of resources from a cloud provider account that is configured in CycleCloud. This includes the credentials used to authenticate with the provider. In addition, CycleCloud makes use of scripts and packages kept in a cloud storage locker to configure nodes on startup.
One important feature of CycleCloud is the ability to autoscale nodes as needed to meet variable demand. Clusters can contain a node array for this purpose, which is a node definition used to create as many nodes as needed. Each node array can be set to hit a certain target capacity (measured by either instance count or core count). Nodes are started as needed to hit this count, and when nodes are idle for too long, they will automatically shutdown. The target can be set directly by the user. When used with supported HPC schedulers (such as Grid Engine, HTCondor, Jupiter, etc), the target core count is set according to the backlog of jobs in the queue.
To run a cluster with the appropriate configuration, CycleCloud uses Cluster Init Projects. The project contains a set of executable scripts and installation packages that are uploaded into a subdirectory of a bucket or container configured as the locker for the node. When the nodes boot, they run each script in turn as the root user under Linux or Administrator under Windows. Cluster Init Projects are highly configurable and scalable, making complicated jobs easy for any user to set up and run.
In This Guide
- Cluster Init Projects
- Common Cookbooks Reference
- The Anatomy of Jetpack
- Auto-Scale Plugins and API
- Creating a Custom Alert for CycleServer
- CycleCloud Settings
- User Creation and Role-Based Access Control
- Event Logging
- Template Customization
- Using Custom Machine Images with CycleCloud
- Cloud Provider Reference
- Node Configuration Reference
- Transfer Listeners
- Scheduler Integration
- Performance Tuning