How-To: Make Ubuntu Autoinstall ISO with Cloud-init

Introduction

This post will guide you through the process of creating an Ubuntu 20.04 (or newer) autoinstall ISO by modifying the default installer ISO. The install configuration will be done using cloud-init cloud-config method that is now used for the Ubuntu server installer.

Note: I am using the Ubuntu server image for the autoinstall. I will show how to (auto)install a desktop of your choice on top of the server install. Starting with server is my preferred way to do “Desktop” installs since the server image is usually much “less problematic” on new and varied hardware configurations. I will also include NVIDIA driver and NetworkManager configuration in the autoinstall.

Prior to Ubuntu 20.04 the Ubuntu server installer used the Debian installer with pre-seed files for configuration. (Preseed installs are still possible with the Desktop installer.) With the 20.04 release a new install mechanism was introduced using “cloud-init” and “curtin” with the Ubuntu subiquity install program. It is a more modern and direct way of doing the install by loading a prebuilt root file system image (a cloud image!) directly onto the install disk and using the cloud-init cloud-config files, user-data and meta-data, with the installer for interactive install. As you may guess from the name, cloud-init is also a common method of configuring cloud instance VM’s for all of the major cloud vendors.

At the initial launch of Ubuntu 20.04 they did make available a “legacy server” install image that still used the Debian installer and pre-seed file. I wrote a blog post describing how to create an autoinstall ISO with that. Note: Auto-Install Ubuntu with Custom Preseed ISO. It is possible to still use that if you have a good pressed that you need to use but, it is probably best to learn the “new way”.

There is some [documentation on the Ubuntu site “Automated Server Installs” for doing this but it is not very complete or detailed. However, you should read through that. You might also want to look through some of the documentation for cloud-init and curtin that I linked earlier to go deeper into the general mechanism.

---

Before I start describing how to create the autoinstall ISO I want to point out a wonderfully useful script for rapidly iterating on experiments with the user-data configuration YAML file you will need to create. Ubuntu Autoinstall Generator

This could get you started putting an ISO image together. Below I will go through the procedure by hand to allow for more detailed editing of the ISO configuration files.

---

Outline for creating the autoinstall ISO

• Step 0) Prerequisites and Creating the “user-data” YAML file(s)
• Step 1) Create a working directory and get a “fresh” copy of the base install image
• Step 2) Unpack and Copy the install ISO file system to your working directory
• Step 3) Edit the ISO grub.cfg, loopback.cfg and isolinux/txt.cfg files
• Step 4) Add your custom “user-data” file(s) to the install ISO
• Step 5) Generate a the new autoinstall ISO image

Step 0) Prerequisites and Creating the “user-data” YAML file(s)

I’m building the ISO’s on an Ubuntu 20.04 host system. The packages you will need are,

• 7z
• wget or curl
• genisoimage (or mkisofs
• cloud-init

You may have most of those already installed except for cloud-init. cloud-init is a package that is installed on the installer itself (and generally by default in instance images you would launch in the cloud). I am adding it here only for one purpose, to run a validation check on “user-data” YAML files against the JSON schema for “cloud-config”. This can save you some time by catching mistakes in your YAML config files before you build an image with them and create an ISO for testing.

sudo apt-get install cloud-init

The validation command would look like this,

cloud-init devel schema --config-file user-data

That should catch syntax errors (stuff not itn the schema) and typos or misconfigured YAML (see [this] post for a nice tutorial on YAML).

The cloud-init mechanism we will use is via “cloud-config”. For this you will need to create to files,

• meta-data – which will be an empty file in our case but would contain vendor specific information when launching on some cloud service.
• user-data – This is the YAML file that will contain the autoinstall directives.

meta-data file is easy. It’s just an empty file,

touch meta-data

Creating the user-data will take some work and experimentation. Read the (not great) documentation from Ubuntu. This long Ubuntu discourse thread may also be helpful.

The basic structure is like this,

user-data

#cloud-config
autoinstall:
  version: 1
  identity:
    hostname: ubuntu-server
    password: "$6$exDY1mhS4KUYCE/2$zmn9ToZwTKLhCw.b4/b.ZRTIZM30JZ4QrOQ2aOXJ8yk96xpcCof0kxKwuX1kqLG/ygbJ1f8wxED22bTL4F46P0"
    username: ubuntu
  interactive-sections:
    - storage

The crypted password there is ubuntu

The first line must be #cloud-config That user-data file will provide the user “ubuntu” with password “ubuntu” to the installer and ask it to return to interactive mode for the storage setup.

I recommend that you start by checking the validation of that file

cloud-init devel schema --config-file test-user-data

Valid cloud-config: test-user-data

then try the script Ubuntu Autoinstall Generator with this file. Then use the generated ISO to do an install on a virtual machine. I used VirtualBox for this. You may need to do this cycle many many times before you get a working user-data file that does what you want!

After downloading the linked script you could create an ISO with something like ( use –help with the command for some documentation),

bash ubuntu-autoinstall-generator.sh -a -u user-data -d first-test.iso 

I will include a more complete user-data file near the end of this post.

The script used above is very nice and will save you many hours when working up and testing a user-data config file. It may be all you need. However, if you would like to make a more user friendly ISO image you can follow the simple steps below for more customization.

Step 1) Create a working directory and get a “fresh” copy of the base install image

Make a working directory,

mkdir Build-ISO
cd Build-ISO

Get a fresh copy of the Ubuntu “daily” server install image. This will have all of the most recent updates which will save you time and may be less buggy than using one of the numbered releases (like 20.04.3)

wget -N https://cdimage.ubuntu.com/ubuntu-server/focal/daily-live/current/focal-live-server-amd64.iso

Step 2) Unpack the install ISO file system

Make a directory for the source files and unpack ISO

mkdir source-files

7z -y x focal-live-server-amd64.iso -osource-files

Notice there is no space between -o and the name of the directory where the file systems is being unpacked.

Remove the unneeded ‘[BOOT]’ dir

rm -rf source-files/[BOOT]/

Step 3) Edit the ISO grub.cfg and isolinux/txt.cfg files

Copy the files boot/grub/grub.cfg and and isolinux/txt.cfg to your working directory

cp source-files/boot/grub/grub.cfg .
cp source-files/isolinux/txt.cfg .

Edit these files for the ISO you will create.

boot/grub/grub.cfg changes

set timeout=10
menuentry "Autoinstall Server (HWE Kernel, NVIDIA, NetworkManager)" {
	set gfxpayload=keep
	linux	/casper/hwe-vmlinuz   quiet autoinstall ds=nocloud;s=/cdrom/server/ ---
	initrd	/casper/hwe-initrd
}

I’ve made two changes; increased the timeout from 5 seconds to 10 and add the above “grub menu stanza” as the first menu entry and edited it’s text label.

I just copied one of the other stanzas. Set it to use the HWE (HardWare Enhanced kernel) which is better to use if you are on very new hardware. (Like I usually have at Puget Systems). And, added the text,

autoinstall ds=nocloud;s=/cdrom/server/

# PLEASE LOOK IN THE COMMENTS where I have a correction there should be a backlash before the ;
# our blog system is ripping it out of the post!!

before the “—“.

That text is telling the kernel to us the cloud-init “NoCloud” data source in the directory /cdrom/server/ That’s where we will put our meta-data and user-data yaml config files.

I named the directory “server” for the set of config files. You could add another stanza and put config files in a directory called gnome-desktop that install the Gnome package on top of server 🙂

Note: the “” before the “;” is important! it escapes the ; so that grub doesn’t ignore everything past it. THERE SHOULD BE A BACKSLASH in the “” our blog software is ripping it out!!

isolinux/txt.cfg changes

default autoinstall-server
label autoinstall-server
  menu label ^Autoinstall Server (HWE Kernel, NVIDIA, NetworkManager)
  kernel /casper/hwe-vmlinuz
  append   initrd=/casper/hwe-initrd quiet autoinstall ds=nocloud;s=/cdrom/server/ ---

These changes are similar to those in grun.cfg. We have set the default install to be our server install, set use of the HWE kernel,wrote an appropriate label, and added the ds information.

Now copy these back to there original source-files directories,

cp grub.cfg source-files/boot/grub/

cp txt.cfg source-files/isolinux/

Step 4) Add your custom “user-data” file(s) to the install ISO

In step 0) we looked at a process for creating and debugging a user-data file. The files I have created above were for use with the user-data file that listed after the next step. You are free to use and modify that. However, you come up with your user-data file we now need to put it in the right place. (don’t forget the empty meta-data file too.)

Make the “server” directory in source-files

mkdir source-files/server

copy in user-data and meta-data

cp user-data source-files/server/

touch source-files/server/meta-data

Step 5) Generate a the new autoinstall ISO image

cd source-files

genisoimage -quiet -D -r -V "ubuntu-autoinstall-testing" -cache-inodes -J -l -joliet-long -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -eltorito-alt-boot -e boot/grub/efi.img -no-emul-boot -o  ../autoinstall-testing.iso .

This will create an ISO image named autoinstall-testing.iso in your working directory with a volume label ubuntu-autoinstall-testing. Don’t for get the “.” at the end, that is telling genisoimage to use the current directory as source for the ISO image.

Note: if you would like to remove the “integrity” check at the start of install do, (in the source-files directory)

echo > md5sum.txt

Sample “user-data” file

The following user-data file is what I have been experimenting with while writing this post. Hopefully it will give you some ideas for creating your own config file.

user-data

!!THIS WHOLE SECTION IS BEING CORRUPTED BY OUT WEB SERVER!!

Here is a Gist with a couple of user-data files that you can try

https://gist.github.com/dbkinghorn/c236aea31d76028b2b6ccdf6d3c6f07

Note: I have a line commented out in the packages: section that would install the default Ubuntu Gnome3 desktop. You could uncomment that line and save the resulting user-data file (along with an empty meta-data file) in a directory called “desktop” in the source-files directory. Then add another stanza to boot/grub/grub.cfg and isolinux/txt.cfg similar to what we did did for “server”

menuentry "Autoinstall Gnome3 Desktop (HWE Kernel, NVIDIA, NetworkManager)" {
	set gfxpayload=keep
	linux	/casper/hwe-vmlinuz   quiet autoinstall ds=nocloud;s=/cdrom/desktop/ ---
	initrd	/casper/hwe-initrd
}

label autoinstall-desktop
  menu label ^Autoinstall Gnome3 Desktop (HWE Kernel, NVIDIA, NetworkManager)
  kernel /casper/hwe-vmlinuz
  append   initrd=/casper/hwe-initrd quiet autoinstall ds=nocloud;s=/cdrom/desktop/ ---

You would then be able to generate the ISO with two autoinstall options to choose from. You could add more too! You could install other desktops this way too including my favorite MATE (ubuntu-mate-desktop)

Conclusion

I have just scratched the surface of what can be done and am still experimenting with different user-data configurations. (I’m still working on it) I hope that the explanations, resource links, and example in this post will save you some time and frustration when building your own autoinstall ISO’s!

Happy computing! –dbk @dbkinghorn

---

Why Choose Puget Systems?