Creating and managing EC2s using Python: Creating an EC2 instance and accessing it via SSH

In my endeavours to see a valid, up to date tutorial on using and managing EC2s using Python3 and Boto3 I failed to find much out there.

Understanding the EC2 creation paradigm

In order to create an EC2 instance in boto3, you have to understand what are the key elements.

The first key element to creating an ec2 instance in code is making sure the security group is configured, using this code snippet:

# Add a rule to the security group to allow SSH
ec2.authorize_security_group_ingress(
    GroupId=security_group_id,
    IpPermissions=[
        {
            'IpProtocol': 'tcp',
            'FromPort': 22,
            'ToPort': 22,
            'IpRanges': [
                {
                    'CidrIp': '0.0.0.0/0'
                }
            ]
        }
    ]
)

Next, we want to make sure we can create a PEM for programmatic access to the instance:

# Create a new key pair
key_pair_name = 'my_key_pair'
response = ec2.create_key_pair(KeyName=key_pair_name)
private_key = response['KeyMaterial']

# Save the private key to a file
with io.StringIO(private_key) as f:
    with open(f'{key_pair_name}.pem', 'w') as key_file:
        key_file.write(f.getvalue())

With these two key parts, we can now create the instance:

# Create a new EC2 instance
response = ec2.run_instances(
    ImageId='ami-08cd358d745620807',
    MinCount=1,
    MaxCount=1,
    InstanceType='t2.micro',
    KeyName='my_key_pair',
    SecurityGroupIds=[security_group_id]
)

Full example

Here is a full example, putting together the snippets above into a working example:


import boto3
import paramiko
import io
import base64

# Create an EC2 client
ec2 = boto3.client('ec2')

# Create a new key pair
key_pair_name = 'my_key_pair'
response = ec2.create_key_pair(KeyName=key_pair_name)
private_key = response['KeyMaterial']

# Save the private key to a file
with io.StringIO(private_key) as f:
    with open(f'{key_pair_name}.pem', 'w') as key_file:
        key_file.write(f.getvalue())


# Create a security group
security_group_name = 'my_security_group'
response = ec2.create_security_group(GroupName=security_group_name, Description='This is my security group.')
security_group_id = response['GroupId']

# Add a rule to the security group to allow SSH
ec2.authorize_security_group_ingress(
    GroupId=security_group_id,
    IpPermissions=[
        {
            'IpProtocol': 'tcp',
            'FromPort': 22,
            'ToPort': 22,
            'IpRanges': [
                {
                    'CidrIp': '0.0.0.0/0'
                }
            ]
        }
    ]
)



# Create a new EC2 instance
response = ec2.run_instances(
    ImageId='ami-08cd358d745620807',
    MinCount=1,
    MaxCount=1,
    InstanceType='t2.micro',
    KeyName='my_key_pair',
    SecurityGroupIds=[security_group_id]
)

# Get the instance ID
instance_id = response['Instances'][0]['InstanceId']

# Wait for the instance to start
ec2.get_waiter('instance_running').wait(InstanceIds=[instance_id])

# wait for the instance to be reachable
ec2.get_waiter('instance_status_ok').wait(InstanceIds=[instance_id])


# Get the public IP of the instance
instance = ec2.describe_instances(InstanceIds=[instance_id])
public_ip = instance['Reservations'][0]['Instances'][0]['PublicIpAddress']

# Connect to the EC2 instance using paramiko
key = paramiko.RSAKey.from_private_key_file(f'{key_pair_name}.pem')
client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect(hostname=public_ip, username='ec2-user', pkey=key)

# Execute a command on the EC2 instance
stdin, stdout, stderr = client.exec_command('ls -l')
result = stdout.read().decode()
print(result)

# Close the connection
client.close()

Conclusion

To round up what we have learnt is the following:

The full example above is a Python script that uses the boto3 and paramiko libraries to create an EC2 instance, connect to it over SSH, and execute a command.

The first section creates an EC2 client object, which is used to interact with the AWS EC2 service.

The second section creates a new key pair, which is used to securely SSH into the EC2 instance. The private key is saved to a file.

The third section creates a security group, which is used to control access to the EC2 instance. A rule is added to allow incoming SSH traffic.

The fourth section creates a new EC2 instance and specifies the security group created in the previous section.

The fifth section waits for the instance to start and retrieves its public IP address.

The sixth section connects to the EC2 instance using the Paramiko library and the private key saved to a file.

The seventh section executes an ‘ls -l’ command on the EC2 instance using the SSH connection. The result is printed to the console.

The final section closes the SSH connection to the EC2 instance.

In conclusion this tutorial was designed to show you a brief introduction into creating EC2 instances, and configuring them with Boto3 and the AWS SDK in Python. Boto3 allows developers to replace both IAAC solutions like Terraform, in addition to other configuration solutions like Ansible.

I hope you have been able to copy and paste the concepts here to use in your own projects.

For more tutorials please check out the tag links below: