In CycleCloud, clusters and nodes are orchestrated resources, which means that they follow a specific sequence of steps to get to an intended state (the target state). The orchestration process is designed to deal with external operations that must be done for a resource, any of which can fail, and any of which can take a significant amount of time. For instance, when you start a node, the end result should be a compute instance or virtual machine running in the cloud provider, configured according to the node. In order to reach that result, the target state is set to Started, and the node transitions through a sequence of specific states to reach that target state. Each type of resource has its own sequence of states, and these are intended to reflect key points in the lifecycle during which appropriate plugins can be run.
Each state has a set of phases that all have to complete successfully before the state is done. Unlike states, which are fixed for the resource type, there is a phase for every phase plugin that is loaded. For example, the Configuration state for a node may consist of a phase to tag the instance, as well as one to attach any volumes. Developers can add new phases simply by adding a new phase plugin and specifying which type and state it should be executed in. Phases can be either specific to the cloud provider in use, or general to all nodes. By default, all phases in a state run concurrently.
CycleCloud tracks the status of each phase separately on the resource itself. When a resource finishes a phase, the status for that phase on that resource is set to completed. When all the phases in a state finish for a given resource, the state itself is finished, and the resource moves on to the next state in line. If the next state is the last state (ie, the target state), then the resource is completed, because its target state and its actual state match.
CycleCloud ships with a standard set of plugins to define the orchestration process for specific cloud providers, as well as shared features. Developers can also customize the orchestration process.
Each node in CycleCloud goes through the following states to get to the Started state:
- Determines if this node can be started, as well as sets some initial attributes (eg Provider)
- Acquires the instance from the cloud provider
- Waits for the instance to be ready
- Sets any cloud-specific settings on this instance that require an instance to be ready in the CSP
- Directly installing any custom software on this node, or waiting for auto-installed software to finish
- Instance has fully started