{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Virtualization\n",
" \n",
"\n",
"- Introduction to operating system virtualization\n",
"\n",
"- Moderm Virtualization types\n",
"\n",
"- KVM virtualization\n",
"\n",
"- VM CLI management tool: ```virsh```\n",
"\n",
"- VM deployment\n",
"\n",
"- VM cloning\n",
"\n",
"- Virtual drive access\n",
"\n",
"- VM resizing\n",
"\n",
"- VM removal\n",
"\n",
"- VM cloud image installation\n",
"\n",
"
\n",
"References:\n",
"
\n",
"\n",
"[KVM and QEMU. Explains well how the KVM module and QEMU work together](https://cloudbuilder.in/blogs/2014/03/09/kvm-and-qemu)\n",
"\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Prerequisites for the practical exercises\n",
"\n",
"\n",
"Please create directory KVM:\n",
"\n",
"```bash\n",
"mkdir KVM\n",
"cd KVM\n",
"```\n",
"\n",
"and start downloading kvm1 VM and Rocky 10 cloud distribution in the directory:\n",
"```bash\n",
"wget http://capone.rutgers.edu/coursefiles/kvm1.tgz\n",
"wget https://dl.rockylinux.org/pub/rocky/10/images/x86_64/Rocky-10-GenericCloud-Base.latest.x86_64.qcow2\n",
"```\n",
"\n",
"***"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## What is virtualization\n",
"\n",
"\n",
"\n",
"\n",
"In computing, a virtualization is the facility that allows multiple operating systems (VMs) to run simultaneously on a computer in a safe and efficient manner.\n",
"\n",
"\n",
"\n",
"Virtualization purposes:\n",
"- Application development and testing on a separate system,\n",
"- Server consolidation on one platform,\n",
"- Virtual appliances (download a VM applience for specific application or service)\n",
"- Prototyping and setting VMs to run on a remote server or/and cloud.\n",
"- Multiple Operating systems on a desktop.\n",
"\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Openstack computational cloud JetStream 2\n",
"\n",
"\n",
"\n",
"Openstack based cluster at Indiana University. The compute nodes are running Ubuntu VMs in KVM.\n",
"\n",
"***"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Virtualization challenges and solutions. \n",
"\n",
"The virtualization implies sharing of CPU, RAM and I/O between the virtual machines (VMs). The traditional x86 platform was not designed for this. The modern hardware technology, however, allows presenting itself with multiple instances to help with virtualization.\n",
"\n",
"- CPU virtualization challenges: how to share the privileged mode (Ring 0)? Hardware emulation and hardware assisted virtualization.\n",
"\n",
"- Memory virtualization challenges involve sharing the physical system memory and dynamically allocating it to virtual machines.\n",
"\n",
"- Device and I/O virtualization involves managing the routing of I/O requests between virtual devices and the shared physical hardware.\n",
" Single Root I/O Virtualization (SR-IOV) is a technology that allows a physical PCIe device to present itself multiple times through the PCIe bus. This technology enables multiple virtual instances of the device with separate resources. \n",
"\n",
"- Video virtualization relies on assigning vGPU or MIG (multi instance gpu) to VMs for video processing, then IOMMU.\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The most common virtualization types on x86 platform\n",
"\n",
"- Operating system virtualization.\n",
"The system runs with a single Kernel; applications run within 'containers' like on different operating systems.\n",
"Example: Solaris zones, LXC linux containers, Docker and Podman containers.\n",
"\n",
"- Hardware Emulation (Full virtualization).\n",
"A hypervisor presents an emulated hardware to unmodified guest operating systems.\n",
"Example: VMware desktop/server, VirtualBox, QEMU.\n",
"\t\n",
"- A hardware assist virtualization on modern x86_64 platforms, such as ```AMD-V``` and ```Intel-VT```.\n",
"Example: KVM, VirtualBox, VMware ESX, Hyper-V\n",
"\n",
"```Proxmox``` is a very popular Debian based hypervisor platform. It provides all the above virtualization types via KVM and LXC.\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Virtualization with KVM\n",
"\n",
"QEMU alone can provide full hardware emulation and system call trapping (Full virtualization).\n",
"
\n",
"QEMU + KVM (Kernel Virtual Machine module) is a full virtualization solution for Linux on x86 hardware containing virtualization extensions (Intel VT or AMD-V).\n",
"\n",
"\n",
"```{figure} img/KVM_QEMU.png\n",
"---\n",
"height: 300px\n",
"name: KVM diagram\n",
"---\n",
"KVM virtualization outline.\n",
"```\n",
"\n",
"A virtual machine (VM) essentially consists of two parts:\n",
"- XML configuration file: ```/etc/libvirt/qemu/vm.xml```\n",
"- Disk partition or image file, by default: ```/var/lib/libvirt/images/vm.qcow2```\n",
"\n",
"To see if the processor supports hardware virtualization: \n",
"```bash\n",
" egrep -c '(vmx|svm)' /proc/cpuinfo\n",
"```\n",
"\n",
"If 0 it means that your CPU doesn't support hardware virtualization.
\n",
"If 1 or more it does - but you still need to make sure that virtualization is enabled in the BIOS.\n",
"\n",
"Verify that the BIOS has virtualization enabled\n",
"```bash\n",
" sudo apt install cpu-checker\n",
" kvm-ok\n",
"```\n",
"If it shows\n",
"\n",
"INFO: Your CPU supports KVM extensions\n",
"INFO: KVM is disabled by your BIOS\n",
"\n",
"\n",
"the Intel Virtualization Technology needs to be enabled in the BIOS.\n",
"On the other hand, if you see the outbut below, the Intel-VT is enabled.\n",
"\n",
"INFO: Your CPU supports KVM extensions\n",
"INFO: /dev/kvm exists\n",
"KVM acceleration can be used\n",
"\n",
"\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## KVM setup (Exercise)\n",
"\n",
"Install KVM virtual director:\n",
"```bash\n",
"sudo -s\n",
"apt update\n",
"apt install virt-manager\n",
"```\n",
"Correct the ownership of the VM image directory:\n",
"```\n",
"chown hostadm:hostadm /var/lib/libvirt/images\n",
"```\n",
"Add user hostadm to group libvirt:\n",
"```\n",
"usermod -a -G libvirt hostadm\n",
"usermod -a -G libvirt-qemu hostadm\n",
"```\n",
"Install kvm packages:\n",
"```\n",
"apt install qemu-kvm qemu-system\n",
"```\n",
"Restart libvirtd service: \n",
"```\n",
"systemctl restart libvirtd \n",
"```\n",
"Logout from the desktop, then login again.\n",
"\n",
"\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Management user interface, virsh (Exercise)\n",
"\n",
"Command ```virsh```is a CLI alternative to the GUI based ```virt-manager```.\n",
"Both ```virsh``` and ```virt-manager``` are parts of libvirt programming interface (API).\n",
"
\n",
"```virsh``` can be used, for example, to see the list of running VMs:\n",
"\n",
"```bash\n",
"virsh -c qemu:///system list\n",
"```\n",
"\n",
"To eliminate typing `-c qemu:///system` in the command option every time, we can setup environment variable,\n",
" LIBVIRT_DEFAULT_URI. \n",
"\n",
"On the desktop, edit file .bashrc in the home directory of hostadm, and add the following line in the end of the file:\n",
"
\n",
"\n",
"```\n",
"export LIBVIRT_DEFAULT_URI='qemu:///system'\n",
"```\n",
"\n",
"\n",
"This tells the libvirt commands what hypervisor to communicate with, specifically, qemu on the local system.\n",
"\n",
"\n",
"Run command\n",
"```bash\n",
"source .bashrc\n",
"```\n",
"\n",
"Enable virtual network\n",
"```bash\n",
"virsh net-autostart default\n",
"virsh net-start default\n",
"```\n",
"\n",
"See the virtual network status:\n",
"```bash\n",
"virsh net-list --all\n",
"```\n",
"\n",
"\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Deployment of an Ubuntu 26.04 appliance VM (Exercise)\n",
"\n",
"Unless you have already done it, download a tar ball with Ubuntu 26.04 VM, `kvm1`, into directory KVM:\n",
"```bash\n",
"cd KVM\n",
"wget http://capone.rutgers.edu/coursefiles/kvm1.tgz\n",
"```\n",
"\n",
"Extract the archive content:\n",
"```bash\n",
"tar -zxvf kvm1.tgz\n",
"```\n",
"\n",
"Copy the xml file with VM configuration into directory ```/etc/libvirt/qemu``` as shown below; the qcow2 file with the VM root file system goes into ```/var/lib/libvirt/images``` directory.\n",
"Assign hostadm user and group ownership on the both, xml and img, files:\n",
"```bash\n",
"sudo cp kvm1.xml /etc/libvirt/qemu\n",
"sudo mv kvm1.qcow2 /var/lib/libvirt/images\n",
"sudo chown hostadm:hostadm /etc/libvirt/qemu/kvm1.xml\n",
"sudo chown hostadm:hostadm /var/lib/libvirt/images/kvm1.qcow2\n",
"```\n",
"To conserve space on your desktop, remove the kvm1 tarball:\n",
"```bash\n",
"rm kvm1.tgz\n",
"```\n",
"\n",
"Define the new VM configuration in KVM:\n",
"```bash\n",
"sudo virsh define /etc/libvirt/qemu/kvm1.xml\n",
"```\n",
"Start the new VM:\n",
"```bash\n",
"virsh start kvm1\n",
"```\n",
"Login to the new VM console as user hostadm with password unisys:\n",
"```bash\n",
"virsh console kvm1\n",
"```\n",
"Shutdown rocky with console command:\n",
"```bash\n",
"shutdown -h now\n",
"```\n",
"\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Basic virsh commands (Exercise)\n",
"\n",
"See the list of running VMs:\n",
"```bash\n",
"virsh list\n",
"```\n",
"List of all the VMs:\n",
"```bash\n",
"virsh list --all\n",
"```\n",
"Start a VM, kvm1 for example:\n",
"```bash\n",
"virsh start kvm1\n",
"```\n",
"\n",
"Shutdown the VM through virsh\n",
"```bash\n",
"virsh shutdown kvm1\n",
"```\n",
"Verify that it is down\n",
"```bash\n",
"virsh list --all\n",
"```\n",
"Start the VM again\n",
"```bash\n",
"virsh start kvm1\n",
"```\n",
"\n",
"To find out the IP address of the running VM:\n",
"```bash\n",
"virsh domifaddr kvm1\n",
"```\n",
"\n",
"Login to the VM console from the desktop terminal by using command virsh:\n",
"```bash\n",
"virsh console kvm1\n",
"```\n",
"To exit from the console, press\n",
"```bash\n",
"^]\n",
"```\n",
"\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Cloning VMs with virt-clone (Exercise)\n",
"\n",
"\n",
"Clone kvm1 to a new VM host, kvm3, by executing the command below:\n",
"```bash\n",
"virt-clone -o kvm1 -n kvm3 -f /home/hostadm/KVM/kvm3.qcow2\n",
"```\n",
"Check if the new VM is in the list, start it, then login to its console:\n",
"```bash\n",
"virsh list --all\n",
"virsh start kvm3\n",
"virsh console kvm3\n",
"```\n",
"If you see error like \n",
"```yaml\n",
"error: Failed to start domain 'kvm3'\n",
"error: Cannot access storage file '/home/hostadm/KVM/kvm3.qcow2' (as uid:64055, gid:64055): Permission denied\n",
"```\n",
"it means user ```libvirt-qemu``` can't read the home directory of user ```hostadm```.\n",
"Assign permission 755 on directory ```/home/hostadm```:\n",
"```bash\n",
"sudo chmod 755 /home/hostadm\n",
"```\n",
"\n",
"Fix the host name:\n",
"Login to the system as user hostadm, elevate privileges by running \n",
"```bash\n",
"sudo -s\n",
"```\n",
"To change the host name from kvm1 to kvm3,\n",
" edit file ```/etc/hostname``` and replace kvm1 by kvm3.\n",
"\n",
"Reset the machine ID by running the following commands on kvm3:\n",
"```bash\n",
"./vm_id_reset.sh\n",
"```\n",
"\n",
"Reboot the VM:\n",
"```bash\n",
"reboot\n",
"```\n",
"\n",
"It should come up with the correct hostname and IP address.\n",
"\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Accessing the virtual drive of a VM (Exercise)\n",
"\n",
"If a VM fails to boot for some reason, you may need to access the virtual drive, analyse the system logs, and fix the configuration.\n",
"Below is the procedure how to mount the qcow2 disk image.\n",
"\n",
"\n",
"Shutdown kvm3\n",
"```bash\n",
"virsh shutdown kvm3 \n",
"```\n",
"Load nbd kernel module for two partitions. It allows to create a mountable block device from the qcow2 file:\n",
"```bash\n",
"sudo -s\n",
"cd KVM\n",
"modprobe nbd max_part=2\n",
"qemu-nbd --connect=/dev/nbd0 /home/hostadm/KVM/kvm3.qcow2\n",
"```\n",
"Create a mounting point, ```/mnt/vm```, and mount the root partition of the drive:\n",
"```bash\n",
"mkdir -p /mnt/vm\n",
"```\n",
"Check the partitions that show up on device /dev/nbd0:\n",
"```bash\n",
"fdisk -l /dev/nbd0\n",
"```\n",
"Mount the partition that shows up with thee Linux filesystem, for example /dev/nbd0p1:\n",
"```bash\n",
"mount /dev/nbd0p1 /mnt/vm\n",
"```\n",
"Now you should be able to access the virtual drive content in directory /mnt/vm\n",
"```bash\n",
"cd /mnt/vm\n",
"ls \n",
"cat etc/hostname\n",
"```\n",
"Unmount and disconnect the drive:\n",
"```bash\n",
"cd\n",
"umount /mnt/vm\n",
"qemu-nbd --disconnect /dev/nbd0 \n",
"```\n",
"\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {
"jp-MarkdownHeadingCollapsed": true
},
"source": [
"## Increase the disk size of a VM.\n",
"\n",
"The following steps are involved in the file system increase of a VM:\n",
"- Increase the virtual disk size of qcow2.\n",
"- On the VM, grow the partition size.\n",
"- On the VM, resize the file system size.\n",
"\n",
"### Increase the virtual disk size of qcow2.\n",
"Install package ```libguestfs-tools```:\n",
"```bash\n",
"sudo apt install libguestfs-tools\n",
"```\n",
"Make sure kvm3 is down. Otherwise shutdown kvm3:\n",
"```bash\n",
"virsh shutdown kvm3\n",
"```\n",
"Check the virtual and the system disk sizes:\n",
"```bash\n",
"qemu-img info KVM/kvm3.qcow2\n",
"```\n",
"\n",
"Check the partitions in the kvm3 qcow2 file:\n",
"```bash\n",
"sudo virt-filesystems --long -h --all -a KVM/kvm3.qcow2\n",
"```\n",
"\n",
"We need to expand the virtual disk.\n",
"\n",
"Expand the virtual disk to 10 Gbytes:\n",
"```bash\n",
"qemu-img resize KVM/kvm3.qcow2 10G\n",
"```\n",
"\n",
"Run the info command again to verify that the virtual size was increased accordingly:\n",
"```bash\n",
"qemu-img info KVM/kvm3.qcow2\n",
"```\n",
"\n",
"### On the VM, grow the partition size and resize the file system.\n",
"Start kvm3, login to the console.\n",
"\n",
"Check the size of the virtual disk:\n",
"```bash\n",
"lsblk\n",
"```\n",
"\n",
"Check the size of the root file system:\n",
"\n",
"```bash\n",
"df -h\n",
"```\n",
"It is still about 4 GB.\n",
"\n",
"Install the tools for modifying the partition size:\n",
"```bash\n",
"sudo apt install cloud-guest-utils\n",
"```\n",
"Expand the root partition:\n",
"```bash\n",
"sudo growpart /dev/vda 1\n",
"```\n",
"Expand the ext4 file system:\n",
"```bash\n",
"sudo resize2fs /dev/vda1\n",
"```\n",
"\n",
"Check the size of the root file system:\n",
"\n",
"```bash\n",
"df -h\n",
"```\n",
"\n",
"Note, ```resize2fs``` may not work for file systems different from ext4. For example, for ```xfs``` one should use command ```xfs_growfs```.\n",
"***"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Delete VM kvm3 (Exercise)\n",
"\n",
"To delete a VM, you need, first, to shutdown the VM, then run command virsh undefine for the VM:\n",
"```bash\n",
"virsh undefine kvm3\n",
"```\n",
"If it gives an error with nvram file, try it with\n",
"```bash\n",
"virsh undefine kvm3 --nvram\n",
"```\n",
"\n",
"Now you can delete the VM disk images:\n",
"```bash\n",
"rm KVM/kvm3.qcow2\n",
"```\n",
"\n",
"Check what other VMs are registered with your hypervisor:\n",
"```bash\n",
"virsh list --all\n",
"```\n",
"\n",
"
\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Deployment of a Rocky cloud base image (Exercise)\n",
"\n",
"Assuming you already have downloaded the Rocky 10 base image in directory KVM.\n",
"\n",
"Step into directory KVM:\n",
"```bash\n",
"cd KVM\n",
"```\n",
"\n",
"Convert the Rocky 10 cloud image into the virtual disk for rocky10 VM.\n",
"```bash\n",
"sudo qemu-img convert -f qcow2 -O qcow2 \\\n",
" Rocky-10-GenericCloud-Base.latest.x86_64.qcow2 \\\n",
" /var/lib/libvirt/images/rocky10.qcow2\n",
"```\n",
"\n",
"Deploy rocky10 with the command below. The new VM will have 1 CPU core, 1 GB of RAM:\n",
"```c\n",
"virt-install \\\n",
"--name rocky10 \\\n",
"--memory 1024 \\\n",
"--vcpus 1 \\\n",
"--os-variant rocky9-unknown \\\n",
"--disk /var/lib/libvirt/images/rocky10.qcow2 \\\n",
"--virt-type kvm \\\n",
"--graphics none \\\n",
"--import \\\n",
"--network network=default,model=virtio \\\n",
"--cloud-init \\\n",
"--boot uefi\n",
"```\n",
"Make sure rocky10 is down, \n",
"```bash\n",
"virsh list \n",
"```\n",
"and change the root password:\n",
"```bash\n",
"sudo virt-customize -a /var/lib/libvirt/images/rocky10.qcow2 \\\n",
"--root-password password:coolpass\n",
"```\n",
"\n",
"***\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Customize rocky10\n",
"\n",
"Start rocky10 and login to the console:\n",
"```bash\n",
"virsh start rocky10\n",
"virsh console rocky10\n",
"```\n",
"Login as root.\n",
"\n",
"Install nano editor:\n",
"```bash\n",
"dnf install nano\n",
"```\n",
"\n",
"Open new file /etc/hostname with an editor:\n",
"```bash\n",
"nano /etc/hostname\n",
"```\n",
"Place ```rocky10``` in the file. Save the file.\n",
"\n",
"Create account hostadm:\n",
"```bash\n",
"adduser hostadm\n",
"passwd hostadm\n",
"```\n",
"\n",
"Add user hostadm to group wheel:\n",
"```bash\n",
"usermod -a -G wheel hostadm\n",
"```\n",
"\n",
"Reboot the VM.\n",
"Make sure it boots up with the correct hostname, and user hostadm gets sudo privileges.\n",
"\n",
"Shutdown rocky with console command:\n",
"```bash\n",
"shutdown -h now\n",
"```\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.11"
}
},
"nbformat": 4,
"nbformat_minor": 4
}