Kubernetes The Hard Way IaC — Part I
Provisioning the compute resource for k8s setup in AWS using Pulumi
Kubernetes The Hard Way IaC — Part I
Provisioning the compute resource for k8s setup in AWS using Pulumi
In this series, we are going to set up the Kubernetes cluster in the hard way using Infrastructure as Code tools Pulumi and Ansible in the AWS cloud environment by following the works of Kelsey Hightower’s Kubernetes-the-hard-way and Prabhat Sharma’s Kubernetes-the-hard-way-aws guides.
Pulumi is a modern Infrastructure as Code platform that allows you to use familiar programming languages to create, deploy, and manage public cloud infrastructures. Ansible is a configuration management tool, that helps you to install and manage software on existing cloud infrastructure.
Let’s begin by creating the cloud resources in AWS using Pulumi in Python language.
i. Getting .ssh key content
Reading SSH keys and these keys will be used later to create an EC2 key pair for SSH access to the instances.
file_path = os.path.expanduser("~/.ssh/")
public_key = open(f"{file_path}/id_rsa.pub").read()
private_key = pulumi.Output.secret(open(f"{file_path}/id_rsa").read())
ii. Creating a VPC
VPCs provide network isolation, security, and control over the cloud resources, allowing to define the network architecture of an application. The below code creates a Virtual Private Cloud (VPC) with a CIDR block of 10.0.0.0/16 and enables DNS support.
vpc = ec2.Vpc(
"vpc",
cidr_block="10.0.0.0/16",
enable_dns_hostnames=True,
enable_dns_support=True,
tags={"Name": "kubernetes-the-hard-way"},
)
iii. Creating a Subnet in an Availability Zone
Availability Zone is a data center or isolated location within a region. AWS has multiple AZs within each region, physically separated to provide redundancy and resiliency. A subnet is a range of IP addresses allocated from the VPC range. And subnets are associated with specific Availability Zones (AZs) in a region.
It creates a subnet within this VPC range using the first availability zone returned by get_availability_zones() . The below CIDR range of the subnet 10.0.0.0/16 can host up to 251 ec2 instances.
availability_zone = get_availability_zones().names[0]
subnet = ec2.Subnet(
"subnet",
availability_zone=availability_zone,
vpc_id=vpc.id,
cidr_block="10.0.1.0/24",
tags={"Name": "kubernetes"},
)
iv. Setting up an Internet gateway and Route table
Internet gateway provisions the resources within the VPC to access the internet. Route tables help to determine where network traffic from your subnet or gateway is directed. A route is added to the route table to direct all traffic (0.0.0.0/0) to the Internet Gateway.
internet_gateway = ec2.InternetGateway(
"internet-gateway", vpc_id=vpc.id, tags={"Name": "kubernetes"}
)
route_table = ec2.RouteTable("route-table", vpc_id=vpc.id)
route_table_association = ec2.RouteTableAssociation(
"route-table-association", route_table_id=route_table.id, subnet_id=subnet.id
)
route = ec2.Route(
"route",
route_table_id=route_table.id,
destination_cidr_block="0.0.0.0/0",
gateway_id=internet_gateway.id,
)
v. Configuring Security group rules
Egress rules are used to control the outbound traffic; traffic generated by AWS resources (such as ec2 instances) and sent out from VPC to the internet or the other networks. Below egress rules allow services to connect to any IP address, on any port, using any protocol (“-1” means all protocols)
Ingress rules are used to control the inbound traffic; traffic from the outside world entering into the AWS network. Below ingress rules allows all protocols & all ports from 10.0.0.0/16 and 10.200.0.0/16 for internal communication and access for ssh-22, https-443, and kube-api-server-6443
security_group = ec2.SecurityGroup(
"kubernetes",
vpc_id=vpc.id,
description="Kubernetes security group",
tags={"Name": "kubernetes"},
egress=[
{"protocol": "-1", "from_port": 0, "to_port": 0, "cidr_blocks": ["0.0.0.0/0"]},
],
ingress=[
{
"protocol": "-1",
"from_port": 0,
"to_port": 0,
"cidr_blocks": ["10.0.0.0/16", "10.200.0.0/16"],
},
{
"protocol": "tcp",
"from_port": 22,
"to_port": 22,
"cidr_blocks": ["0.0.0.0/0"],
},
{
"protocol": "tcp",
"from_port": 6443,
"to_port": 6443,
"cidr_blocks": ["0.0.0.0/0"],
},
{
"protocol": "tcp",
"from_port": 443,
"to_port": 443,
"cidr_blocks": ["0.0.0.0/0"],
},
{
"protocol": "icmp",
"from_port": -1,
"to_port": -1,
"cidr_blocks": ["0.0.0.0/0"],
},
],
)
vi. Load balancer and target group
Load balancer (NLB) is used to distribute incoming network traffic across multiple targets, in this case, it will be directed to IP addresses, and attaching the target group instructs the load balancer to distribute traffic to the IP addresses specified target_id within the target group specified by target_group.arn. The below snippet receives the income request and forwards it to the 6443 port to the controller instances where kube-api-server listens.
load_balancer = lb.LoadBalancer(
"loadBalancer",
internal=False,
name="kubernetes",
subnets=[subnet.id],
load_balancer_type="network",
)
target_group = lb.TargetGroup(
"targetGroup",
name="kubernetes",
protocol="TCP",
port=6443,
target_type="ip",
vpc_id=vpc.id,
)
for i in range(3):
lb.TargetGroupAttachment(
f"target{i}", target_group_arn=target_group.arn, target_id=f"10.0.1.1{i}"
)
listener = lb.Listener(
"listener",
load_balancer_arn=load_balancer.arn,
protocol="TCP",
port=443,
default_actions=[{"type": "forward", "target_group_arn": target_group.arn}],
)
vii. Get the latest Amazon machine image
Below snippet of code retrieves the most recent Ubuntu 20.04 AMI owned by Amazon.
instance_image = pulumi.Output.from_input(
ec2.get_ami(
most_recent=True,
owners=["099720109477"],
filters=[
{
"name": "name",
"values": ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"],
},
{
"name": "root-device-type",
"values": ["ebs"],
},
{
"name": "architecture",
"values": ["x86_64"],
},
],
)
)
viii. Instance creation
The below helper function is used to create six EC2 instances (three workers and three controllers) in the subnet, with the security group attached to it. The instances are configured with user data for identifying their roles and private IP addresses.
def create_instance(name: str):
res = name.split(sep="-")
name = f"{res[0]}-{res[1]}"
private_ip = (
f"10.0.1.2{res[1]}" if res[0].lower() == "worker" else f"10.0.1.1{res[1]}"
)
user_data = (
f"name=worker-{res[1]}|pod-cidr=10.200.{res[1]}.0/24"
if res[0].lower() == "worker"
else name
)
return ec2.Instance(
name,
ami=instance_image.id,
instance_type=instance_type,
key_name=key_pair.key_name,
vpc_security_group_ids=[security_group.id],
subnet_id=subnet.id,
associate_public_ip_address=True,
user_data=user_data,
private_ip=private_ip,
tags={"Name": name},
availability_zone=availability_zone,
ebs_block_devices=[
{
"device_name": "/dev/sda1",
"volume_size": 20,
"delete_on_termination": True,
}
],
)
ix. Bringing up the resources
The following command brings up the above-specified resources in the AWS cloud infrastructure.
pulumi up -y -s dev
Originally published on Medium
🌟 🌟 🌟 The source code for this blog post can be found here 🌟🌟🌟
References:
[1] https://github.com/prabhatsharma/kubernetes-the-hard-way-aws
[2] https://github.com/kelseyhightower/kubernetes-the-hard-way