Mount AWS EBS on EC2 automatically with cloud-init
AWS EBS is always used as data disk for AWS EC2. However, before we can use the EBS, we have to manually mount all the disks according the below link.
What if we can mount all the disks automatically during EC2 initialization process? It would be quite useful when we need to launch tons of EC2 instances at the same time.
In fact, we can accomplish this by adding cloud-init config in EC2 user data. When launching new EC2 instances, AWS provide a metadata called user data that we can use to set instructions we want to execute in the instances. It can be written in shell script format or in cloud-init directives. The detail is in below link.
Here, I would choose cloud-init config since its directives would make the whole config structure more understandable.
#cloud-config
output: {all: '| tee -a /var/log/cloud-init-output.log'}bootcmd:
- ln -sf bash /bin/shruncmd:
- EBS_DEVICE=( $(lsblk | grep -e disk | awk '{sub("G","",$4)} {if ($4+0 > 50) print $1}') )
- for i in "${!EBS_DEVICE[@]}"; do mkfs -t ext4 /dev/"${EBS_DEVICE[$i]}"; done
- for i in "${!EBS_DEVICE[@]}"; do mkdir -p "/data$(expr $i + 1)"; done
- for i in "${!EBS_DEVICE[@]}"; do mount /dev/"${EBS_DEVICE[$i]}" /data$(expr $i + 1); done
- for i in "${!EBS_DEVICE[@]}"; do echo -e "\"UUID=\"$(lsblk -o +uuid /dev/"${EBS_DEVICE[$i]}" | grep "${EBS_DEVICE[$i]}" | awk '{print $8}')\" \t /data$(expr $i + 1) \t ext4 \t defaults \t 0 \t 0\"" >> /etc/fstab; done
When we want to use cloud-init, the #cloud-config line at the top is required. Then we specify the output of cloud-init procedure at line 2.
At line 4 and line 7, you can see two directives: bootcmd and runcmd . Both of them are used to specify the shell command, the difference is bootcmd will run on every boot and runcmd only runs during the first boot. There's on thing worth noting, bootcmd runs before runcmd during boot process.
We change the default /bin/sh from dash to bash every boot at line 5. At line 8, we retrieve the EBS device names that size is larger than 50G (You are free to change the filter to fit your need). From 9 to 12, we build the file system, create mount folder (in sequence data1, data2, …), mount the disks and rewrite /etc/fstab. Please be noted that all the mount steps should only run once in first boot. The automatic mount will be handled by /etc/fstab in future reboot.
For more detail about cloud-init, you can refer to the below documentation.
After finishing the script, we could add the user data when we want to create new EC2 instances in AWS web console and AWS cli.
You can also make use of this config along with some IaC tools such as AWS CloudFormation or Terraform to automate the whole infrastructure building process.
Thanks for reading and feel free to share your thoughts!
