Create custom VM images (and use them via CLI/Terraform)

GitHub Pages: Here

The goal of this tutorial is to describe how to create a custom VM image from an existing VM, and provision a new VM from this custom image using CLI and Terraform. Here we cover the use of the Azure VM image gallery, which is created to host the VM images to be used in a subscription. We do not cover here the creation of VM images to be published in the Azure Marketplace.

Content of this git folder:

DISCLAIMER: This document is work-in-progress and my personal experience performing this task.

Definitions

A Virtual Machine (VM) image is a snapshot of a VM. It contains its operating system, applications, configurations, etc. that was captured at a given moment. With this image, new VMs can be created that will be similar to that original VM.

In Azure, one can configure a VM image to be either generalized or specialized. VM image generalization is a process to remove machine and user specific information from the VM (e.g. Unique System Identifiers, computer name, user specific data, etc).

There are also different types of images:

In Azure, VM image definitions consist of three main fields:

Image definitions can also contain: recommended vCPUs, recommended memory, description, end of life date, and release notes.

A VM image also has a version, which allows one to track changes and manage updates to the VM image over time.

Example of VM image identification: microsoft-dsvm:ubuntu-hpc:1804:18.04.2021051701

In the Azure marketplace website you can find more info (such as pointers to websites describing the VM images): https://azuremarketplace.microsoft.com/en-us/marketplace/apps

Basic commands and examples to list images

To obtain the list of all VM images (this command execution takes a while):

az vm image list --all --output table > bigtable.txt

To get the Ubuntu VM images for HPC:

az vm image list --publisher microsoft-dsvm --offer ubuntu-hpc --output table --all

Architecture    Offer       Publisher       Sku                Urn                                                           Version
--------------  ----------  --------------  -----------------  ------------------------------------------------------------  ----------------
x64             ubuntu-hpc  microsoft-dsvm  1804               microsoft-dsvm:ubuntu-hpc:1804:18.04.2021051701               18.04.2021051701
x64             ubuntu-hpc  microsoft-dsvm  1804               microsoft-dsvm:ubuntu-hpc:1804:18.04.2021110101               18.04.2021110101
x64             ubuntu-hpc  microsoft-dsvm  1804               microsoft-dsvm:ubuntu-hpc:1804:18.04.2021120101               18.04.2021120101
x64             ubuntu-hpc  microsoft-dsvm  1804               microsoft-dsvm:ubuntu-hpc:1804:18.04.2022061601               18.04.2022061601
x64             ubuntu-hpc  microsoft-dsvm  1804               microsoft-dsvm:ubuntu-hpc:1804:18.04.2022121201               18.04.2022121201
.
.
.

Create VM image from existing VM

Here we assume one has a VM already provisioned called myoriginalvm in a resource group myoriginalrg.

Create resource group:

az group create --name mygalleryrg --location eastus

Create gallery:

az sig create --resource-group mygalleryrg --gallery-name mygallery

Get original VM id (this will get a string like "/subscriptions/xyz/resourceGroups/.../myoriginalvm"):

ORIGINALVMID=$(az vm get-instance-view -g myoriginalrg -n myoriginalvm --query id)

The example below creates the VM image definition, containing publisher, offer, SKU, for a linux, and using the VM image generalization process.

az sig image-definition create --resource-group mygalleryrg \
                               --gallery-name mygallery \
                               --gallery-image-definition myimagedef \
                               --publisher myimagepub \
                               --offer myimageoffer \
                               --sku myimagesku \
                               --os-type Linux \
                               --os-state Generalized \
                               --hyper-v-generation V2

The following command shows that the definition is there:

az sig image-definition list --gallery-name mygallery \
                             --resource-group mygalleryrg \
                             --output table

We need to generalize the VM before creating the generalized VM image.

Deprovision the VM by using the Azure VM agent to delete machine-specific files and data.

sudo waagent -deprovision+user -force

Turn off the VM (release its resources), while keeping the VM state stored:

az vm deallocate --resource-group myoriginalrg --name myoriginalvm

Mark VM as generalized

az vm generalize --resource-group myoriginalrg --name myoriginalvm

Then we create an actual VM image version:

az sig image-version create \
   --resource-group mygalleryrg \
   --gallery-name mygallery \
   --gallery-image-definition myimagedef \
   --gallery-image-version 1.0.0 \
   --target-regions "eastus" \
   --managed-image "/subscriptions/<subscriptionid>/resourceGroups/myoriginalrg/providers/Microsoft.Compute/virtualMachines/myoriginalvm"

Create VM from custom VM image using CLI

az group create --name myResourceGroup --location eastus

az vm create --resource-group myResourceGroup \
    --name mynewvm \
    --image "/subscriptions/<Subscription ID>/resourceGroups/mygalleryrg/providers/Microsoft.Compute/galleries/mygallery/images/myimagedef"

Create VM from custom VM image using Terraform

Inside this git folder you can find a subfolder called terra, which contains code to provision a VM in azure using terraform. The code came from: https://learn.microsoft.com/en-us/azure/virtual-machines/linux/quick-create-terraform

Inside the original main.tf, you would find something like:

  source_image_reference {
    publisher = "Canonical"
    offer     = "0001-com-ubuntu-server-jammy"
    sku       = "22_04-lts-gen2"
    version   = "latest"
  }

The link below shows how to define the source image id in terraform: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/linux_virtual_machine.html#source_image_id

To use the custom VM image, main.tf was modified to use the VM image from the gallery defined above:

  source_image_id = "/subscriptions/<Subscription ID>/resourceGroups/mygalleryrg/providers/Microsoft.Compute/galleries/mygallery/images/myimagedef/versions/1.0.0"

Then:

terraform init -upgrade
terraform plan -out main.tfplan
terraform apply main.tfplan

References