https://docs.cyclecomputing.com/wp-content/uploads/2018/04/create_csp_account.png

Enter a name for your cloud provider setup. Use a descriptive name, such as “My AWS Account”. From the drop-down, select Amazon Web Services, then enter your Default Region, Access Key, and Secret Key.

Enter a Default Bucket name to use for storing configuration and application data for your cluster. If it does not already exist, the bucket will be created. Please remember to follow S3 bucket naming conventions, and note that bucket names must be unique. For more information, please visit the AWS Bucket Restrictions documentation.

If this AWS account will be your main cloud provider for CycleCloud, check the “Set Default” option. Once you have completed setting the parameters for your AWS account, click Save to continue.

Amazon Web Services

CycleCloud has run on Amazon Web Services (AWS) since 2008. CycleCloud extends the functionality of core AWS services such as EC2 (Elastic Compute Cloud), S3 (Simple Storage Service), VPC (Virtual Private Cloud), and many others to offer an accessible and agile solution to a variety of computing challenges.

To use Amazon Web Services with CycleCloud, you will need the following:

AWS Console

The AWS Console provides an interface for managing the cloud services and resources. You can login using your root AWS credentials, but a better practice is to generate individual user accounts with a specific role created with the appropriate access policies.

Security Recommendation: Avoid using the root account credentials for anything but user management. We recommend you create an IAM Profile that contains an appropriate policy and role for CycleCloud access.

IAM Profile

  • Log into your AWS console
  • Click IAM under Security, Identity & Compliance, then Policies in the sidebar
  • Click Create Policy, and select Create Your Own Policy
  • Give your policy a name and description
  • Copy the template from here and paste it into the Policy Document window.

Note: For more secure environments, please contact info@cyclecomputing.com.

Add a Role

  • Click Roles from the IAM Dashboard
  • Create New Role
  • Select Amazon EC2
  • Choose the Policy you created in the previous step, then click Next Step
  • Give the role a name and description
  • Click Create Role

Access Keys

CycleCloud uses AWS access keys for authentication. We recommend creating a CycleCloud IAM user and then generating user access keys. More information on key generation best practices can be found in the AWS documentation.

To generate access keys:

  • Log into the AWS Console
  • Select IAM service
  • Select Users in the sidebar and click on “Create New Users”
  • Add cyclecloud and leave “generate access keys” selected
  • Click on “Download Credentials” to save to disk

Creating a Key Pair

You will need to create a public/private key pair to access the nodes started by CycleCloud. This key pair will allow you to SSH into the nodes as the root user. For more information on creating key pairs, consult the AWS Key Pair documentation.

To create a keypair from the AWS Console:

  • Sign in to the AWS Console
  • Select EC2 service
  • Click the “Create Key Pair” button
  • Name the key pair “cyclecloud” (you can use a different name, but the default configuration assumes a keypair named cyclecloud)
  • When asked to save the key to your computer, save it as “cyclecloud.pem” inside the ~/.ssh directory (you can use a different name or path, but this will require additional configuration)
  • You may have to alter the permissions on this key so that SSH will be able to use it. You can modify the permissions as follows: chmod 600 ~/.ssh/cyclecloud.pem

To generate your own SSH key:

# Ensure your .ssh directory exists
$> mkdir ­p ~/.ssh
# Generate the key pair
$> ssh­keygen ­f ~/.ssh/cyclecloud ­t rsa ­b 2048
# Do not enter a passphrase to allow CycleCloud's automated processes to work.
# Rename the private key to have a .pem extension
$> mv ~/.ssh/cyclecloud ~/.ssh/cyclecloud­private.pem

Once you have generated the SSH key pair:

Importing Your Key

  • Log in to AWS console
  • Go to the EC2 Dashboard
  • Select Key Pairs
  • Import the public key “cyclecloud.pub”

Warning

Use phraseless SSH key. If you enter a passphrase when generating your SSH key, the return proxy feature of CycleCloud will not work, which may cause node reporting and autoscaling to fail.

Security Groups

AWS Network Security Groups are used to control inbound and outbound access to instances. CycleCloud will start nodes in the default security group unless otherwise specified. To use a security group other than the default, you can edit the template files in ~/.cycle to reflect the desired security group.

You will need to edit the default group to open several ports, which will allow the nodes within a cluster to talk to one another.

  • In your EC2 Dashboard, click on Security Groups in the left menu
  • Select “default” from the list of security groups (if this is a new AWS account, default should be the only item in the list)
  • Select the “Inbound” tab from the bottom of the screen
  • Add the following rules:
Rule Port Range Source Description
Custom TCP Rule 22 Anywhere Opens port 22 so you can SSH into your nodes.
Custom TCP Rule 8652 Anywhere Optional – opens port 8652 to allow CycleCloud to poll Ganglia data from the clusters.
Custom TCP Rule 1 – 65535 default (typing “default” will bring up your Security Group ID) Allows TCP communication on all ports between nodes in the default security group.
Custom UDP Rule 1 – 65535 default (typing “default” will bring up your Security Group ID) Allows UDP communication on all ports between nodes in the default security group.
RDP 3389 Anywhere Opens port 3389 to allow RDP into Windows nodes.

Note

Should you wish to start a cluster that includes CycleServer (such as the standard Condor cluster), you may want to include the following rule for port 8443. Please note that this will require SSL configured with a valid domain and certificates. Additional configuration information can be found in our Installation Guide.

Rule Port Range Source Description
HTTPS 8443 | Anywhere | Opens port 8443 so you can use default HTTPS to access CycleCloud.
  • Click Save

https://docs.cyclecomputing.com/wp-content/uploads/2018/04/aws_inbound_rules.png