{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "remove-cell",
     "remove_cell"
    ]
   },
   "outputs": [],
   "source": [
    "my_notebook_text = \"\"\"\n",
    "# Networking\n",
    "\n",
    "## Linux networking\n",
    "\n",
    "- Overview of network protocols and TCP/IP in Linux.\n",
    "- Network configuration in Linux.\n",
    "- Netplan on Ubuntu servers.\n",
    "- Network settings in Red Hat for NetworkManager.\n",
    "- Network troubleshooting tools: ping, traceroute, tcpdump.\n",
    "- Network calculation with <TT>ipcalc</TT>\n",
    "- Network applications and services.\n",
    "- Network traffic sniffing with tcpdump.\n",
    "- lighttpd server installation.\n",
    "- Port tunneling with SSH.\n",
    "<hr>\n",
    "\n",
    "## Computers on the intertnet\n",
    "\n",
    "- Network communication allows sending information, transfering files, and managing applications remotely between computers.\n",
    "- Networks are organized in a series of layers implemented on each machine.\n",
    "- The layers are independent and each layer offers a specific task to the neighboring layers above and below.\n",
    "- The communication between the layers is based on defined rules and procedures, called protocols.\n",
    "- Data is processed and passed through the layers from above to below, until the bottom layer is reached.\n",
    "- At the bottom layer, computers communicate with each other via sending and receiving network datagrams through - cables, switches and routers.\n",
    "\n",
    "![](img/computers_on_network.png)\n",
    "<hr>\n",
    "\n",
    "## TCP/IP 5 layer protocol suit\n",
    "\n",
    "\n",
    "![](img/tcp-ip.jpg)\n",
    "\n",
    "Each layer has its own task:\n",
    "\n",
    "\n",
    "|TCP/IP Protocol Layer | Tasks\n",
    "| :-- | :--\n",
    "|5. Application Layer: ssh, ftp, http ... | Data segmentation by the application into Message Transfer Unit (MTU) segments. Typical MTU is 1.5 Kbyte, which includes the data payload plus 40 bytes for TCP and IP headers.\n",
    "|4. Transport Layer: TCP, UDP, ports  | Connection management, quality, and ports to the applications\n",
    "|3. Internet Layer: IP addresses, subnets, routing | Addressing: each computer has a unique address on its network or subnet. Routing between networks.\n",
    "|2. Link Layer: Ethernet | Communication on the local network (subnet) through a switch. Media Access Card (MAC) address unique on the subnet. For example, 48:d2:4f:f4:d:fb, should be unique within a subnet. Address Resolution Protocol (ARP) matches the IP address to MAC on the subnet.\n",
    "|1. Physical Layer: NIC, hubs, switches, routers, cabling, wifi APs and cards | Electronic devices to send and receive signals between computers.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## OSI 7 layer Network Stack Model\n",
    "\n",
    "![](img/osi.jpg)\n",
    "\n",
    "|TCP/IP Protocol Layer | Task\n",
    "| :-- | :--\n",
    "|7. Application Layer: ssh, ftp, http ... | Data segmentation by the application into Message Transfer Unit (MTU) segments. Typical MTU is 1.5 Kbyte\n",
    "|6. Presentation Layer: SSL, libz, XDR  | Data encryption, compression, and encoding/decoding\n",
    "|5. Session Layer: Remote Procedure Call (RPC) | Responsible for opening, using and closing sessions. Remote procedures.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## TCP/IP and Kernel\n",
    "\n",
    "![](img/tcpip-kernel.jpg)\n",
    "\n",
    "The Link, Internet and Transport layers are implemented through kernel.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Packet Encapsulation\n",
    "\n",
    "Data is processed in form of packet encapsulation and transmitted/received in form of Ethernet frames.\n",
    "\n",
    "![](img/ip_packet.jpg)\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Hosts communication via TCP/IP\n",
    "\n",
    "![](img/host-host.jpg)\n",
    "\n",
    "![](img/host-tcpip.jpg)\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Wired connection checkup (Exercise)\n",
    "\n",
    "- To see whether your desktop is connected to a network switch and at what speed, use command  ```ethtool```.\n",
    "\n",
    "- On your desktop, install package <TT>ethtool</TT>:\n",
    "```bash\n",
    "apt install ethtool\n",
    "```\n",
    "- See your network interfaces with command  ```ip address show``` or shorter ```ip a```:\n",
    "\n",
    "```bash\n",
    "ip a\n",
    "```\n",
    "\n",
    "The main network interface, connected to the Ethernet, is <TT>enp0s3</TT>. Run command below:\n",
    "```bash\n",
    "/sbin/ethtool enp0s3\n",
    "```\n",
    "\n",
    "In the output, among various parameters, it doesn't show Speed and Duplex because the desktop is a VM in VDI.\n",
    "\n",
    "  - Speed: Unknown!\n",
    "  - Duplex: Unknown!\n",
    "  - Link detected: yes\n",
    "  \n",
    "If the desktop was installed on a bare metal hardware, we would see something like below:\n",
    "\n",
    "  - Speed: 1000Mb/s, \n",
    "  - Duplex: Full, \n",
    "  - Link detected: yes\n",
    "  \n",
    "Which means the cable is connected and the link speed is 1Gb/s speed, full duplex mode. \n",
    "  <hr>\n",
    "\n",
    "## Network IP configuration\n",
    "\n",
    "- Static configuration\n",
    "    - IP address, gateway, hostname, and Domain Name System (DNS) are configured on the host.\n",
    "    \n",
    "    \n",
    "    \n",
    "- Dynamic Host Configuration Protocol (DHCP)\n",
    "    - The settings are obtained from a DHCP server.\n",
    "    \n",
    "![](img/dhcp.gif)\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Environment for IP configuration on Ubuntu and RedHat\n",
    "\n",
    "|ifupdown and ifcfg scripts\t| ```NetworkManager``` service \t| Netplan ```networkd``` service Meaning\n",
    "|:---    | ---      | ---:        |  \n",
    "|Old Debian/Ubuntu: | Ubuntu desktop, Debian, Redhat: | Ubuntu server: |\n",
    "|File: ```/etc/network/interfaces``` |  Commands: ```nmtui, nmcli```     | File:```/etc/netplan/00-installer-config.yaml``` |\n",
    "|RedHat:                 | Redhat                 | Command: ```netplan apply``` |\n",
    "|File: ```/etc/sysconfig/network-scripts/ifcfg-ens3```|  File: ```/etc/NetworkManager/system-connections/<conn name>.nmconnection``` |\n",
    "|Command:```ifup ens3``` |  ```systemctl restart NetworkManager; nmcli con up <conn name>```\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "<HR>\n",
    "\n",
    "\n",
    "\n",
    "## Deploy a new VM for exercises\n",
    "\n",
    "Shutdown <TT>kvm1</TT> VM, and clone it into <TT>netplan</TT> VM:\n",
    "```bash\n",
    "virsh shutdown kvm1\n",
    "virt-clone  -o kvm1 -n netplan -f /home/hostadm/KVM/netplan.qcow2\n",
    "```\n",
    "Start netplan. Login to console of netplan:\n",
    "```bash\n",
    "virsh console netplan\n",
    "```\n",
    "Fix the hostname in file <TT>/etc/hostname</TT>.\n",
    "\n",
    "Reset the machine ID by running the following script on netplan:\n",
    "```bash\n",
    "./vm_id_reset.sh\n",
    "```\n",
    "Remove the SSH password authentication restriction configuration file:\n",
    "```bash\n",
    "rm /etc/ssh/sshd_config.d/60-cloudimg-settings.conf\n",
    "```\n",
    "\n",
    "Execute command reboot on netplan:\n",
    "```bash\n",
    "reboot\n",
    "```\n",
    "\n",
    "<hr>\n",
    "\n",
    "\n",
    "## Configuring network settings on Ubuntu server with Netplan (Exercises)\n",
    "\n",
    "On <TT>netplan</TT> VM check the IP address settings by running command below:\n",
    "```bash\n",
    "ip addr\n",
    "```\n",
    "All the commands below in this section need to be run as root. Elevate the user privileges to root:\n",
    "```bash\n",
    "sudo -s\n",
    "```\n",
    "\n",
    "Assign IP address <TT>192.168.122.222</TT> by modifying the content of ```/etc/netplan/00-installer-config.yaml``` as follows:\n",
    "\n",
    "```yaml\n",
    "---\n",
    "network:\n",
    "  version: 2\n",
    "  renderer: networkd\n",
    "  ethernets:\n",
    "    enp1s0:\n",
    "      dhcp4: false\n",
    "      addresses:\n",
    "        - 192.168.122.222/24\n",
    "      routes:\n",
    "        - to: default\n",
    "          via: 192.168.122.1\n",
    "      nameservers:\n",
    "        addresses: [192.168.122.1]\n",
    "```\n",
    "\n",
    "Install the Yaml syntax check tool:\n",
    "```bash\n",
    "apt install yamllint\n",
    "```\n",
    "\n",
    "Check the network configuration file for syntax correctness:\n",
    "```bash\n",
    "yamllint /etc/netplan/00-installer-config.yaml\n",
    "```\n",
    "If no errors found, have the new network settings applied:\n",
    "```bash\n",
    "netplan apply\n",
    "```\n",
    "Check the IP address again:\n",
    "```bash\n",
    "ip addr\n",
    "```\n",
    "It should show <TT>192.168.122.222</TT> for <TT>ens3</TT> interface now.\n",
    "\n",
    "See the gateway settings by running command \n",
    "```bash\n",
    "ip route show\n",
    "```\n",
    "It should show the \"default via 192.168.122.1\"\n",
    "\n",
    "Logout and disconnect from the console.\n",
    "<hr>\n",
    "\n",
    "## Clone rocky10 VM (Exercises)\n",
    "\n",
    "In case you already have ```rocky10-1``` VM from the package management class, skip the cloning steps below and proceed to the \n",
    " network configuration on ```rocky10-1```.\n",
    " \n",
    "Make sure rocky10 is down:\n",
    "```bash\n",
    "virsh list\n",
    "virsh stop rocky10\n",
    "```\n",
    "Clone rocky10 into new VM rocky10-1:\n",
    "```bash\n",
    "virt-clone -o rocky10 -n rocky10-1 -f /home/hostadm/KVM/rocky10-1.qcow2\n",
    "```\n",
    "Start the VM:\n",
    "```bash\n",
    "virsh start rocky10-1\n",
    "```\n",
    "Connect to the cnsole:\n",
    "```bash\n",
    "virsh console rocky10-1\n",
    "```\n",
    "Login to the system.\n",
    "\n",
    "You need to install editor `dbus-tools`:\n",
    "```bash\n",
    "sudo dnf install dbus-tools\n",
    "```\n",
    "\n",
    "Change the hostname to `rocky10-1` by editing file `/etc/hostname` with `nano`.\n",
    "\n",
    "Reset the machine ID:\n",
    "```bash\n",
    "sudo dnf install dbus-tools\n",
    "sudo rm -f /etc/machine-id\n",
    "sudo dbus-uuidgen --ensure=/etc/machine-id\n",
    "sudo dbus-uuidgen --ensure\n",
    "sudo dhclient -r enp1s0\n",
    "```\n",
    "\n",
    "Reboot the VM.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Network configurations on a Red Hat server (Exercises)\n",
    "\n",
    "On RedHat based systems, we can use ```nmcli``` to control the network settings in the NetworkManager service.\n",
    "\n",
    "Start rocky10-1 VMs:\n",
    "```bash\n",
    "virsh start rocky10-1\n",
    "```\n",
    "Login to console of rocky10-1:\n",
    "```bash\n",
    "virsh console rocky10-1\n",
    "```\n",
    "Execute command ip address show to read the IP address:\n",
    "```bash\n",
    "ip addr show enp1s0\n",
    "```\n",
    "Try pinging ```netplan``` by its IP address, for example:\n",
    "```bash\n",
    "ping -c 3 192.168.122.222\n",
    "```\n",
    "Become root on rocky10-1 VM\n",
    "```bash\n",
    "sudo -s\n",
    "```\n",
    "\n",
    "By using command ```nmcli``` check available and active connactions:\n",
    "```bash\n",
    "nmcli conn show\n",
    "```\n",
    "There is connection named \"Wired connection 1\" for interface enp1s0.\n",
    "\n",
    "See the settings for device enp1s0:\n",
    "```bash\n",
    "nmcli dev show enp1s0\n",
    "```\n",
    "\n",
    "With command ```nmcli```, we can create various connections with different settings for enp1s0 interface, then we can activate the needed connection, and de-activate non-needed.\n",
    "\n",
    "Let's create new connection named \"eth_static\" and assign it static IP address 192.168.122.126 and netmask /24:\n",
    "```bash\n",
    "nmcli con add type ethernet con-name eth_static ip4 192.168.122.126/24 gw4 192.168.122.1 ipv4.dns 192.168.122.1\n",
    "```\n",
    "Activate the connection:\n",
    "```bash\n",
    "nmcli con up eth_static\n",
    "```\n",
    "Check the connection status:\n",
    "```bash\n",
    "nmcli conn show\n",
    "```\n",
    "Check the new IP address:\n",
    "```bash\n",
    "ip addr show enp1s0\n",
    "```\n",
    "\n",
    "Change the IP address to 192.168.122.25 via ```nmcli``` command:\n",
    "```bash\n",
    "nmcli con modify eth_static ipv4.addresses 192.168.122.125/24\n",
    "```\n",
    "Activate eth_static connection:\n",
    "Activate the connection:\n",
    "```bash\n",
    "nmcli con up eth_static\n",
    "```\n",
    "Check the connection status:\n",
    "```bash\n",
    "nmcli conn show\n",
    "```\n",
    "Check the new IP address:\n",
    "```bash\n",
    "ip addr show enp1s0\n",
    "```\n",
    "\n",
    "**Alternative way of modifying the IP address settings for a connection.**\n",
    "\n",
    "When creating a new connection with ```nmcli``` command, a new file is created in directory \n",
    " /etc/NetworkManager/system-connections/.\n",
    "\n",
    "We can modify the settings in the file with an editor, then restart the NetworkManager and activate the connection, for example:\n",
    "```bash\n",
    "systemctl restart NetworkManager\n",
    "nmcli con up eth_static\n",
    "```\n",
    "\n",
    "**Revert back to the DHCP based connection**\n",
    "\n",
    "```bash\n",
    "nmcli conn show\n",
    "```\n",
    "\n",
    "We need to activate connection \"Wired connection 1\":\n",
    "```bash\n",
    "nmcli conn up \"Wired connection 1\"\n",
    "```\n",
    "<hr>\n",
    "\n",
    "## Internet Control Message Protocol (ICMP) protocol (Exercises)\n",
    "\n",
    "Implemented on hosts and gateways (routers) for:\n",
    "\n",
    "Reporting the status of datagram processing.\n",
    "Diagnostics of connection and routing.\n",
    "Reporting errors in the processing of a datagram.\n",
    "\n",
    "Example: ```ping```\n",
    "\n",
    "![](img/ping.jpg)\n",
    "\n",
    "Exercise: command ```ping```.\n",
    "```bash\n",
    "ping -c 5 capone.rutgers.edu\n",
    "ping -c 5 engsoft.rutgers.edu\n",
    "ping -c 5 google.com\n",
    "```\n",
    "If the systems respond, it means they are reachable on the network. Notice how varies the round-trip time and Time to Live (ttl) for packets between the hosts.\n",
    "\n",
    "Example: ```traceroute```\n",
    "\n",
    "\n",
    "![](img/traceroute.jpg)\n",
    "\n",
    "Time to leave (TTL) decrement. The TTL value of an IP packet represents the maximum number of IP routers that the packet can go through before being thrown away. You can expect each router in the Internet to decrement the TTL field by exactly one. The default TTL for traceroute on Linux is 30. It can be increased up to 255 maximum by using the -m command option.\n",
    "\n",
    "Exercise: command traceroute.\n",
    "\n",
    "```bash\n",
    "sudo apt install traceroute\n",
    "sudo /usr/sbin/traceroute -I capone.rutgers.edu\n",
    "sudo /usr/sbin/traceroute -I engsoft.rutgers.edu\n",
    "sudo /usr/sbin/traceroute -I google.com\n",
    "```\n",
    "This shows you all the gateways between the subnets your packet travels towards the destination. Notice the difference in the number of gateways between your desktop and the three remote hosts as well as the round trip times.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Subnets and routing\n",
    "\n",
    "Example: Host A can communicate with both the hosts - B and C. Hosts B and C can't communicate with each other.\n",
    "What is the problem?\n",
    "<br>\n",
    "Answer: the routing (gateway) is not defined on Host B.\n",
    "\n",
    "![](img/hosts-router.png)\n",
    "\n",
    "## Network calculation, gateway and routing\n",
    "\n",
    "- Computers connected to the same switch are on the same network/subnet and communicate with each other directly through the switch.\n",
    "- For computers located on the different networks/subnets, gateway/router is needed.\n",
    "- To define the local subnet, netmask is used.\n",
    "\n",
    "- Subnet mask defines the network and the host parts of the IP address\n",
    "\n",
    "Network address = Host address (logical AND) Netmask:\n",
    "\n",
    "\n",
    "\n",
    "| Address         | Decimal/Hexadecimal  | Binary | \n",
    "|:- | :-: | :- |                                  \n",
    "| IP address      | 192.168.5.10  |  ```11000000 10010100 00000101 00001010``` |  \n",
    "| Netmask         | 255.255.255.0 |  ```11111111 11111111 11111111 00000000``` |\n",
    "| Network address | 192.168.5.0   |  ```11000000 10010100 00000101 00000000``` |\n",
    "\n",
    "\n",
    "\n",
    "- The network address is the smallest address on the network:\n",
    "\n",
    "```11000000 10010100 00000101 00000000 = 192.168.5.0```\n",
    "\n",
    "- The broadcast address is the largest address on the network:\n",
    "\n",
    "```11000000 10010100 00000101 11111111 = 192.168.5.255```\n",
    "\n",
    "- Max number of hosts on the subnet: 254 = 256 - 2\n",
    "The gateway address should be on the same subnet with the host\n",
    "\n",
    "- Subnet mask consists of contiguous 1. The bytes filled with 1 become 255 in decimal notation.  The bytes ending with 0 can be converted from binary to decimal by the following technique:\n",
    "  \n",
    "  a_7 * 128 + a_6 * 64 + a_5 * 32 + a_4 * 16 + a_3 * 8 + a_2 * 4 + a_1 * 2 + a_0\n",
    "\n",
    "  where the coefficients, $a_7, a_6, a_5, a_4, a_3, a_2, a_1, a_0$, are the bits with values of 0 or 1.\n",
    "\n",
    "- It is easier to use ipcalculator for computing subnets.\n",
    "<hr>\n",
    "\n",
    "\n",
    "## IP calculator exercises\n",
    "\n",
    "Network IP calculations with ipcalc\n",
    "On the desktop, install ipcalc by using APT:\n",
    "```bash\n",
    "apt install ipcalc\n",
    "```\n",
    "Run ipcalc for network address 192.168.5.0 with subnet mask 255.255.255.0.\n",
    "```bash\n",
    "ipcalc 192.168.5.0/255.255.255.0\n",
    "```\n",
    "See the output for Address, Netmask, Network, HostMin, HostMax, Broadcast, Hosts/Net.\n",
    "Notice the same results for the same network and the different representation of the netmask:\n",
    "```bash\n",
    "ipcalc 192.168.5.0/24\n",
    "```\n",
    "Notice the same results for Netmask, Network, HostMin, HostMax, Broadcast, Hosts/Net if using the different IP addresses within the same network in ipcalc, for example:\n",
    "```bash\n",
    "ipcalc 192.168.5.15/24\n",
    "ipcalc 192.168.5.34/24\n",
    "```\n",
    "Run ipcalc for subnets (networks) 192.168.5.0/25 and 192.168.5.128/25:\n",
    "```bash\n",
    "ipcalc 192.168.5.0/25\n",
    "ipcalc 192.168.5.128/25\n",
    "```\n",
    "Notice the values for HostMin and HostMax in both the cases. By looking at the ranges [HostMin, HostMax], you can see, for example, that IP address 192.168.5.5 belongs to the first subnet and 192.168.5.250 to the second.\n",
    "You can verify that by running ipcalc on the IP addresses above and then comparing the Network values:\n",
    "```bash\n",
    "ipcalc 192.168.5.5/25\n",
    "ipcalc 192.168.5.250/25\n",
    "```\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Network applications: ssh, sftp, scp, rsync (Exercises)\n",
    "\n",
    "Make sure virtual machine netplan is running\n",
    "```bash\n",
    "virsh list\n",
    "```\n",
    "Figure out the IP address of netplan:\n",
    "```bash\n",
    "virsh domifaddr netplan\n",
    "```\n",
    "\n",
    "Place the IP address and netplan host name into file ```/etc/hosts``` on your desktop. \n",
    "netplan  IP is 192.168.122.222\n",
    "```bash\n",
    "nano /etc/hosts\n",
    "```\n",
    "Make entry in the file:  ```192.168.122.222   netplan```\n",
    "\n",
    "ssh to netplan as user hostadm:\n",
    "```bash\n",
    "ssh hostadm@netplan\n",
    "```\n",
    "\n",
    "Configure SSH for private/public key authentication.\n",
    "For SSH authentication, you can use either RSA or DSA public/private keys besides password. We'll be using RSA in the exercises below.\n",
    "To generate an RSA key pair, type the following command at a shell prompt on your desktop:\n",
    "```bash\n",
    "ssh-keygen -t rsa\n",
    "```\n",
    "Accept the default file location of ```~/.ssh/id_rsa```. Enter a passphrase different from your account password and confirm it by entering it again.\n",
    "The public key is written to ```~/.ssh/id_rsa.pub```. The private key is written to ```~/.ssh/id_rsa```.\n",
    "Never distribute your private key to anyone.\n",
    "The contents of ```~/.ssh/id_rsa.pub``` needs to be delivered onto the remote machine to which you want to connect, specifically netplan, into file ```~/.ssh/authorized_keys```\n",
    "To accomplish the transfer task, here you can use one of the commands for file transfer.\n",
    "Commands ```scp``` and ```sftp``` come with ssh. We use interactive ```sftp``` command here:\n",
    "```bash\n",
    "sftp  netplan\n",
    "Name (netplan:hostadm): hostadm\n",
    "sftp> cd .ssh\n",
    "sftp> lcd .ssh\n",
    "sftp> put id_rsa.pub authorized_keys\n",
    "sftp>  quit\n",
    "```\n",
    "Command ```cd``` in the sftp> shell above is for stepping into the directory, .ssh, on the remote host, netplan.\n",
    "Command ```lcd``` is for stepping into the directory, .ssh, on the local desktop.\n",
    "\n",
    "Now try to ssh to netplan. You should be prompted to enter your passphrase.\n",
    "\n",
    "The ```ssh-agent``` can be used to store your passphrase so that you do not have to enter it each time you make a ssh or scp connection.\n",
    "At a shell prompt on the desktop, type the following command:\n",
    "```bash\n",
    "exec ssh-agent $SHELL\n",
    "```\n",
    "Then type the command:\n",
    "```bash\n",
    "ssh-add\n",
    "```\n",
    "and enter your passphrase(s). If you have more than one key pair configured, you will be prompted for each one. When you log out, your passphrase(s) will be forgotten. You must execute these two commands each time you log in to a virtual console or open a terminal window.\n",
    "\n",
    "Run a remote command over ssh, for example:\n",
    "```bash\n",
    "ssh netplan \"uname -a\"\n",
    "```\n",
    "Beside ```sftp```, command ```scp``` can be used for copying files from host to host.\n",
    "In `scp`, the first argument is the <i>source_host:source_file</i>, the second is <i>destination_host:destination_dir</i>. If the source_host or the destination_host is local, it is absent in the argumernt. The local directory is signified by ```.```\n",
    "\n",
    "Copy file /etc/hosts from netplan VM to the current directory on the desktop by using scp command:\n",
    "```bash\n",
    "scp netplan:/etc/hosts .\n",
    "```\n",
    "\n",
    "Copy new file, somef.txt, from the local directory to the home directory on netplan VM.\n",
    "```bash\n",
    "touch somef.txt\n",
    "scp somef.txt netplan:/home/$USER\n",
    "```\n",
    "\n",
    "You can also copy a group of files, for example, somef.txt and /etc/hosts, by placing them in block {}:\n",
    "```bash\n",
    "scp {somef.txt,/etc/hosts} netplan:/home/$USER\n",
    "```\n",
    "\n",
    "Directories can be copied recursively with ```scp```. For example, to copy /etc directory recursively from the desktop to  the user home directory on netplan:\n",
    "```bash\n",
    "scp -r /etc netplan:\n",
    "```\n",
    "\n",
    "Syncronizing directories between remote hosts by using ```rsync```.\n",
    "This tool lets you copy files and directories between a local host and a remote host.\n",
    "Install ```rsync``` on both your desktop and netplan:\n",
    "```bash\n",
    "apt install rsync\n",
    "```\n",
    "Creat a directory tree and copy it over to netplan with rsync command.\n",
    "```bash\n",
    "mkdir -p dir1/dir2/dir3\n",
    "rsync  -avz dir1 netplan:/home/$USER\n",
    "```\n",
    "Option a stands for archive (preserve links and timestamps); <TT>v</TT> is for verbose and <TT>z</TT> is for data compression when sending-receiving.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Send/receive data with netcat (Exercises)\n",
    "\n",
    "Netcat is a very useful tool to connect to any TCP and UDP port on a remote host, and send/receive data.\n",
    "\n",
    "\n",
    "Start both, netplan and rocky10-1 VMs:\n",
    "```bash\n",
    "virsh start netplan\n",
    "virsh start rocky10-1\n",
    "```\n",
    "Start ```tmux``` on your desktop, and split the terminal into two panes.\n",
    "\n",
    "In one pane, login to console of netplan:\n",
    "```bash\n",
    "virsh console netplan\n",
    "```\n",
    "In the other pane of tmux, login to rocky10-1:\n",
    "```bash\n",
    "virsh console rocky10-1\n",
    "```\n",
    "\n",
    "Become root on both the VMs\n",
    "```bash\n",
    "sudo -s\n",
    "```\n",
    "\n",
    "| Step | On Ubuntu (server) | On rocky10-1 |\n",
    "|:- | :-: | -: |     \n",
    "|1.  |Install netcat: ```apt install netcat``` | ```dnf install nmap-ncat``` | \n",
    "|2.    | Start netcat as a server, listening on port UDP/8080: |  Connect to UDP/8080 port on netplan, 192.168.122.222: |  \n",
    "|      | ```netcat -l -u 8080``` | ```ncat 192.168.122.222 -u 8080``` | \n",
    "|3.    | When the session is over, type Ctrl-C to stop the netcat server.  |     Start typing text. Hit ENTER key. The text should appear on the both terminals. Press Ctrl+D to close the UDP connection. | \n",
    "|4.    | Start the server on TCP/8080 to write into a file, outputfile.txt |   Send file /etc/hosts to TCP/8080 on netplan VM via netcat |\n",
    "|   | ```netcat -l 8080 > outputfile.txt```  |  ```ncat 192.168.122.222  8080 < /etc/hosts``` | \n",
    "|5.  |  After the file is received, the connection closes itself. Check the content of the file: |  |\n",
    "|    | ```less outputfile.txt```  |  \n",
    "|6.  | Start ```netcat``` to receive a directory via ```tar``` command:   | Send ```/etc``` directory via ```tar``` and netcat: | \n",
    "|    | ```netcat -l 8080 | tar xf -```   |  ```tar cf - /etc | nc -w 20  192.168.122.222  8080``` |\n",
    "|7.  | Check if the directory tree has been transferred:  | |\n",
    "|    |  ```tree etc``` | |\n",
    " \n",
    "\n",
    "`netcat` provides away faster transfer of files and directories than `scp`, but it can be used only on a private network.\n",
    "<hr>\n",
    "\n",
    "## tcpdump command to see the traffic on NIC (Exercises)\n",
    "\n",
    "When you need to see all the network traffic on a network card, you can run the command for the network interface.\n",
    "It helps to identify the subnet your system is connected to, and see all the active traffic.\n",
    "\n",
    "On netplan VM, install tcpdump:\n",
    "```bash\n",
    "apt install tcpdump\n",
    "```\n",
    "Run command `tcpdump` on the ethernet interface:\n",
    "```bash\n",
    "tcpdump -v -n -i enp1s0\n",
    "```\n",
    "\n",
    "On rocky10-1 VM, start pingin the IP address of netplan VM:\n",
    "```bash\n",
    "ping 192.168.122.222\n",
    "```\n",
    "\n",
    "See the output in tcpdump on netplan VM.\n",
    "It should show some traffic related to echo request and reply from rocky VM:\n",
    "\n",
    "```\n",
    "    192.168.122.126 > 192.168.122.222: ICMP echo request ...\n",
    "    192.168.122.222 > 192.168.122.126: ICMP echo reply ...\n",
    "```\n",
    "\n",
    "```tcpdum``` is very noisy because it picks up all the packets going through interface enp1s0.\n",
    "We can filter the output by the IP, protocol, and port, for example:\n",
    "```bash\n",
    "tcpdump -v -n -i enp1s0 host 192.168.122.126 and proto \\\\icmp\n",
    "```\n",
    "The IP address of rocky10-1 is 192.168.122.126 here. In your case it maybe different.\n",
    "\n",
    "To see only the incoming packets, use the `src` IP address:\n",
    "```bash\n",
    "tcpdump -v -n -i enp1s0 src host 192.168.122.126 and proto \\\\icmp\n",
    "```\n",
    "\n",
    "Move to the pane with rocky10-1 in ```tmux```.\n",
    "\n",
    "SSH to netplan and start ```tcpdump``` with reading the traffic, coming from rocky10-1, on local port udp/8080:\n",
    "```bash\n",
    "sudo tcpdump -i enp1s0 src host 192.168.122.126 and dst port 8080 -A\n",
    "```\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Light web server, lighttpd, for file download (Exercises)\n",
    "\n",
    "- On netplan VM, install lighttpd:\n",
    "```bash\n",
    "apt install lighttpd\n",
    "```\n",
    "\n",
    "Add the following line to the server configuration file, ```/etc/lighttpd/lighttpd.conf```, right after the line with \n",
    "```server.port = 80```:\n",
    "```{admonition} add to lighttpd.conf \n",
    "    dir-listing.activate   = \"enable\"\n",
    "```\n",
    "Restart the service:\n",
    "```bash\n",
    "systemctl restart lighttpd\n",
    "```\n",
    "Populate files and directories in the server document-root directory, ```/var/www/html```:\n",
    "```bash\n",
    "cd /var/www/html\n",
    "apt download lua5.4\n",
    "dpkg -X lua5.4_5.4.8-1build1_amd64.deb LUA\n",
    "```\n",
    "Remove the index file from the directory:\n",
    "```bash\n",
    "rm index.lighttpd.html\n",
    "```\n",
    "\n",
    "- On the desktop, create new directory DOWNLOADS, then download file lua5.4_5.4.8-1build1_amd64.deb from the web server into the directory:\n",
    "```bash\n",
    "mkdir DOWNLOADS\n",
    "cd DOWNLOADS\n",
    "wget http://192.168.122.222/lua5.4_5.4.8-1build1_amd64.deb\n",
    "```\n",
    "\n",
    "Download the whole directory recursively:\n",
    "```bash\n",
    "wget --recursive -nH http://192.168.122.222/\n",
    "```\n",
    "\n",
    "***\n",
    "\n",
    "## SSH local port tunneling (Exercise).\n",
    "\n",
    "With SSH you can make a tcp port available from a private network to your local system.\n",
    "For example, we can map port tcp/80 from 192.168.122.222 to your localhost:8080 via SSH to the desktop.\n",
    "\n",
    "Add the port mapping option to the SSH command you usually use:\n",
    "```{admonition} add this to your ssh command you run to the desktop\n",
    "    -L 8080:192.168.122.222:80\n",
    "```\n",
    "\n",
    "In the browser on your laptop, navigate to port 8080 at localhost:\n",
    "```{admonition} in the browser\n",
    "http://localhost:8080/\n",
    "```\n",
    "\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "remove-cell",
     "remove_cell"
    ]
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Both GOOGLE_API_KEY and GEMINI_API_KEY are set. Using GOOGLE_API_KEY.\n"
     ]
    },
    {
     "ename": "ServerError",
     "evalue": "503 UNAVAILABLE. {'error': {'code': 503, 'message': 'This model is currently experiencing high demand. Spikes in demand are usually temporary. Please try again later.', 'status': 'UNAVAILABLE'}}",
     "output_type": "error",
     "traceback": [
      "\u001b[31m---------------------------------------------------------------------------\u001b[39m",
      "\u001b[31mServerError\u001b[39m                               Traceback (most recent call last)",
      "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[10]\u001b[39m\u001b[32m, line 58\u001b[39m\n\u001b[32m     54\u001b[39m     output_text = response.text\n\u001b[32m     55\u001b[39m     print(output_text)\n\u001b[32m     56\u001b[39m \n\u001b[32m     57\u001b[39m \u001b[38;5;28;01mexcept\u001b[39;00m APIError \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[32m---> \u001b[39m\u001b[32m58\u001b[39m     \u001b[38;5;28;01mraise\u001b[39;00m e\n",
      "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[10]\u001b[39m\u001b[32m, line 58\u001b[39m\n\u001b[32m     54\u001b[39m     output_text = response.text\n\u001b[32m     55\u001b[39m     print(output_text)\n\u001b[32m     56\u001b[39m \n\u001b[32m     57\u001b[39m \u001b[38;5;28;01mexcept\u001b[39;00m APIError \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[32m---> \u001b[39m\u001b[32m58\u001b[39m     \u001b[38;5;28;01mraise\u001b[39;00m e\n",
      "\u001b[36mFile \u001b[39m\u001b[32m~/miniconda3/lib/python3.13/site-packages/google/genai/models.py:6549\u001b[39m, in \u001b[36mModels.generate_content\u001b[39m\u001b[34m(self, model, contents, config)\u001b[39m\n\u001b[32m   6545\u001b[39m   parsed_config_to_call = _extra_utils.get_usage_header(\n\u001b[32m   6546\u001b[39m       parsed_config_to_call\n\u001b[32m   6547\u001b[39m   )\n\u001b[32m   6548\u001b[39m i += \u001b[32m1\u001b[39m\n\u001b[32m-> \u001b[39m\u001b[32m6549\u001b[39m response = \u001b[30;43mself\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43m_generate_content\u001b[39;49m\u001b[30;43m(\u001b[39;49m\n\u001b[32m   6550\u001b[39m \u001b[30;43m    \u001b[39;49m\u001b[30;43mmodel\u001b[39;49m\u001b[30;43m=\u001b[39;49m\u001b[30;43mmodel\u001b[39;49m\u001b[30;43m,\u001b[39;49m\u001b[30;43m \u001b[39;49m\u001b[30;43mcontents\u001b[39;49m\u001b[30;43m=\u001b[39;49m\u001b[30;43mcontents\u001b[39;49m\u001b[30;43m,\u001b[39;49m\u001b[30;43m \u001b[39;49m\u001b[30;43mconfig\u001b[39;49m\u001b[30;43m=\u001b[39;49m\u001b[30;43mparsed_config_to_call\u001b[39;49m\n\u001b[32m   6551\u001b[39m \u001b[30;43m\u001b[39;49m\u001b[30;43m)\u001b[39;49m\n\u001b[32m   6553\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m function_map:\n\u001b[32m   6554\u001b[39m   \u001b[38;5;28;01mbreak\u001b[39;00m\n",
      "\u001b[36mFile \u001b[39m\u001b[32m~/miniconda3/lib/python3.13/site-packages/google/genai/models.py:4977\u001b[39m, in \u001b[36mModels._generate_content\u001b[39m\u001b[34m(self, model, contents, config)\u001b[39m\n\u001b[32m   4974\u001b[39m request_dict = _common.convert_to_dict(request_dict)\n\u001b[32m   4975\u001b[39m request_dict = _common.encode_unserializable_types(request_dict)\n\u001b[32m-> \u001b[39m\u001b[32m4977\u001b[39m response = \u001b[30;43mself\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43m_api_client\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43mrequest\u001b[39;49m\u001b[30;43m(\u001b[39;49m\n\u001b[32m   4978\u001b[39m \u001b[30;43m    \u001b[39;49m\u001b[30;43m'\u001b[39;49m\u001b[30;43mpost\u001b[39;49m\u001b[30;43m'\u001b[39;49m\u001b[30;43m,\u001b[39;49m\u001b[30;43m \u001b[39;49m\u001b[30;43mpath\u001b[39;49m\u001b[30;43m,\u001b[39;49m\u001b[30;43m \u001b[39;49m\u001b[30;43mrequest_dict\u001b[39;49m\u001b[30;43m,\u001b[39;49m\u001b[30;43m \u001b[39;49m\u001b[30;43mhttp_options\u001b[39;49m\n\u001b[32m   4979\u001b[39m \u001b[30;43m\u001b[39;49m\u001b[30;43m)\u001b[39;49m\n\u001b[32m   4981\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m config \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;28mgetattr\u001b[39m(\n\u001b[32m   4982\u001b[39m     config, \u001b[33m'\u001b[39m\u001b[33mshould_return_http_response\u001b[39m\u001b[33m'\u001b[39m, \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[32m   4983\u001b[39m ):\n\u001b[32m   4984\u001b[39m   return_value = types.GenerateContentResponse(sdk_http_response=response)\n",
      "\u001b[36mFile \u001b[39m\u001b[32m~/miniconda3/lib/python3.13/site-packages/google/genai/_api_client.py:1650\u001b[39m, in \u001b[36mBaseApiClient.request\u001b[39m\u001b[34m(self, http_method, path, request_dict, http_options)\u001b[39m\n\u001b[32m   1640\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mrequest\u001b[39m(\n\u001b[32m   1641\u001b[39m     \u001b[38;5;28mself\u001b[39m,\n\u001b[32m   1642\u001b[39m     http_method: \u001b[38;5;28mstr\u001b[39m,\n\u001b[32m   (...)\u001b[39m\u001b[32m   1645\u001b[39m     http_options: Optional[HttpOptionsOrDict] = \u001b[38;5;28;01mNone\u001b[39;00m,\n\u001b[32m   1646\u001b[39m ) -> SdkHttpResponse:\n\u001b[32m   1647\u001b[39m   http_request = \u001b[38;5;28mself\u001b[39m._build_request(\n\u001b[32m   1648\u001b[39m       http_method, path, request_dict, http_options\n\u001b[32m   1649\u001b[39m   )\n\u001b[32m-> \u001b[39m\u001b[32m1650\u001b[39m   response = \u001b[30;43mself\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43m_request\u001b[39;49m\u001b[30;43m(\u001b[39;49m\u001b[30;43mhttp_request\u001b[39;49m\u001b[30;43m,\u001b[39;49m\u001b[30;43m \u001b[39;49m\u001b[30;43mhttp_options\u001b[39;49m\u001b[30;43m,\u001b[39;49m\u001b[30;43m \u001b[39;49m\u001b[30;43mstream\u001b[39;49m\u001b[30;43m=\u001b[39;49m\u001b[30;43;01mFalse\u001b[39;49;00m\u001b[30;43m)\u001b[39;49m\n\u001b[32m   1651\u001b[39m   response_body = (\n\u001b[32m   1652\u001b[39m       response.response_stream[\u001b[32m0\u001b[39m] \u001b[38;5;28;01mif\u001b[39;00m response.response_stream \u001b[38;5;28;01melse\u001b[39;00m \u001b[33m'\u001b[39m\u001b[33m'\u001b[39m\n\u001b[32m   1653\u001b[39m   )\n\u001b[32m   1654\u001b[39m   \u001b[38;5;28;01mreturn\u001b[39;00m SdkHttpResponse(headers=response.headers, body=response_body)\n",
      "\u001b[36mFile \u001b[39m\u001b[32m~/miniconda3/lib/python3.13/site-packages/google/genai/_api_client.py:1443\u001b[39m, in \u001b[36mBaseApiClient._request\u001b[39m\u001b[34m(self, http_request, http_options, stream)\u001b[39m\n\u001b[32m   1440\u001b[39m     retry = tenacity.Retrying(**retry_kwargs)\n\u001b[32m   1441\u001b[39m     \u001b[38;5;28;01mreturn\u001b[39;00m retry(\u001b[38;5;28mself\u001b[39m._request_once, http_request, stream)  \u001b[38;5;66;03m# type: ignore[no-any-return]\u001b[39;00m\n\u001b[32m-> \u001b[39m\u001b[32m1443\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[30;43mself\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43m_retry\u001b[39;49m\u001b[30;43m(\u001b[39;49m\u001b[30;43mself\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43m_request_once\u001b[39;49m\u001b[30;43m,\u001b[39;49m\u001b[30;43m \u001b[39;49m\u001b[30;43mhttp_request\u001b[39;49m\u001b[30;43m,\u001b[39;49m\u001b[30;43m \u001b[39;49m\u001b[30;43mstream\u001b[39;49m\u001b[30;43m)\u001b[39;49m\n",
      "\u001b[36mFile \u001b[39m\u001b[32m~/miniconda3/lib/python3.13/site-packages/tenacity/__init__.py:470\u001b[39m, in \u001b[36mRetrying.__call__\u001b[39m\u001b[34m(self, fn, *args, **kwargs)\u001b[39m\n\u001b[32m    468\u001b[39m retry_state = RetryCallState(retry_object=\u001b[38;5;28mself\u001b[39m, fn=fn, args=args, kwargs=kwargs)\n\u001b[32m    469\u001b[39m \u001b[38;5;28;01mwhile\u001b[39;00m \u001b[38;5;28;01mTrue\u001b[39;00m:\n\u001b[32m--> \u001b[39m\u001b[32m470\u001b[39m     do = \u001b[30;43mself\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43miter\u001b[39;49m\u001b[30;43m(\u001b[39;49m\u001b[30;43mretry_state\u001b[39;49m\u001b[30;43m=\u001b[39;49m\u001b[30;43mretry_state\u001b[39;49m\u001b[30;43m)\u001b[39;49m\n\u001b[32m    471\u001b[39m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(do, DoAttempt):\n\u001b[32m    472\u001b[39m         \u001b[38;5;28;01mtry\u001b[39;00m:\n",
      "\u001b[36mFile \u001b[39m\u001b[32m~/miniconda3/lib/python3.13/site-packages/tenacity/__init__.py:371\u001b[39m, in \u001b[36mBaseRetrying.iter\u001b[39m\u001b[34m(self, retry_state)\u001b[39m\n\u001b[32m    369\u001b[39m result = \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[32m    370\u001b[39m \u001b[38;5;28;01mfor\u001b[39;00m action \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mself\u001b[39m.iter_state.actions:\n\u001b[32m--> \u001b[39m\u001b[32m371\u001b[39m     result = \u001b[30;43maction\u001b[39;49m\u001b[30;43m(\u001b[39;49m\u001b[30;43mretry_state\u001b[39;49m\u001b[30;43m)\u001b[39;49m\n\u001b[32m    372\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m result\n",
      "\u001b[36mFile \u001b[39m\u001b[32m~/miniconda3/lib/python3.13/site-packages/tenacity/__init__.py:413\u001b[39m, in \u001b[36mBaseRetrying._post_stop_check_actions.<locals>.exc_check\u001b[39m\u001b[34m(rs)\u001b[39m\n\u001b[32m    411\u001b[39m retry_exc = \u001b[38;5;28mself\u001b[39m.retry_error_cls(fut)\n\u001b[32m    412\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m.reraise:\n\u001b[32m--> \u001b[39m\u001b[32m413\u001b[39m     \u001b[38;5;28;01mraise\u001b[39;00m \u001b[30;43mretry_exc\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43mreraise\u001b[39;49m\u001b[30;43m(\u001b[39;49m\u001b[30;43m)\u001b[39;49m\n\u001b[32m    414\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m retry_exc \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34;01mfut\u001b[39;00m\u001b[34;01m.\u001b[39;00m\u001b[34;01mexception\u001b[39;00m()\n",
      "\u001b[36mFile \u001b[39m\u001b[32m~/miniconda3/lib/python3.13/site-packages/tenacity/__init__.py:184\u001b[39m, in \u001b[36mRetryError.reraise\u001b[39m\u001b[34m(self)\u001b[39m\n\u001b[32m    182\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mreraise\u001b[39m(\u001b[38;5;28mself\u001b[39m) -> t.NoReturn:\n\u001b[32m    183\u001b[39m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m.last_attempt.failed:\n\u001b[32m--> \u001b[39m\u001b[32m184\u001b[39m         \u001b[38;5;28;01mraise\u001b[39;00m \u001b[30;43mself\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43mlast_attempt\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43mresult\u001b[39;49m\u001b[30;43m(\u001b[39;49m\u001b[30;43m)\u001b[39;49m\n\u001b[32m    185\u001b[39m     \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;28mself\u001b[39m\n",
      "\u001b[36mFile \u001b[39m\u001b[32m~/miniconda3/lib/python3.13/concurrent/futures/_base.py:449\u001b[39m, in \u001b[36mFuture.result\u001b[39m\u001b[34m(self, timeout)\u001b[39m\n\u001b[32m    447\u001b[39m     \u001b[38;5;28;01mraise\u001b[39;00m CancelledError()\n\u001b[32m    448\u001b[39m \u001b[38;5;28;01melif\u001b[39;00m \u001b[38;5;28mself\u001b[39m._state == FINISHED:\n\u001b[32m--> \u001b[39m\u001b[32m449\u001b[39m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[30;43mself\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43m__get_result\u001b[39;49m\u001b[30;43m(\u001b[39;49m\u001b[30;43m)\u001b[39;49m\n\u001b[32m    451\u001b[39m \u001b[38;5;28mself\u001b[39m._condition.wait(timeout)\n\u001b[32m    453\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m._state \u001b[38;5;129;01min\u001b[39;00m [CANCELLED, CANCELLED_AND_NOTIFIED]:\n",
      "\u001b[36mFile \u001b[39m\u001b[32m~/miniconda3/lib/python3.13/concurrent/futures/_base.py:401\u001b[39m, in \u001b[36mFuture.__get_result\u001b[39m\u001b[34m(self)\u001b[39m\n\u001b[32m    399\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m._exception \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m    400\u001b[39m     \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[32m--> \u001b[39m\u001b[32m401\u001b[39m         \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;28mself\u001b[39m._exception\n\u001b[32m    402\u001b[39m     \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[32m    403\u001b[39m         \u001b[38;5;66;03m# Break a reference cycle with the exception in self._exception\u001b[39;00m\n\u001b[32m    404\u001b[39m         \u001b[38;5;28mself\u001b[39m = \u001b[38;5;28;01mNone\u001b[39;00m\n",
      "\u001b[36mFile \u001b[39m\u001b[32m~/miniconda3/lib/python3.13/site-packages/tenacity/__init__.py:473\u001b[39m, in \u001b[36mRetrying.__call__\u001b[39m\u001b[34m(self, fn, *args, **kwargs)\u001b[39m\n\u001b[32m    471\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(do, DoAttempt):\n\u001b[32m    472\u001b[39m     \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[32m--> \u001b[39m\u001b[32m473\u001b[39m         result = \u001b[30;43mfn\u001b[39;49m\u001b[30;43m(\u001b[39;49m\u001b[30;43m*\u001b[39;49m\u001b[30;43margs\u001b[39;49m\u001b[30;43m,\u001b[39;49m\u001b[30;43m \u001b[39;49m\u001b[30;43m*\u001b[39;49m\u001b[30;43m*\u001b[39;49m\u001b[30;43mkwargs\u001b[39;49m\u001b[30;43m)\u001b[39;49m\n\u001b[32m    474\u001b[39m     \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mBaseException\u001b[39;00m:  \u001b[38;5;66;03m# noqa: B902\u001b[39;00m\n\u001b[32m    475\u001b[39m         retry_state.set_exception(sys.exc_info())  \u001b[38;5;66;03m# type: ignore[arg-type]\u001b[39;00m\n",
      "\u001b[36mFile \u001b[39m\u001b[32m~/miniconda3/lib/python3.13/site-packages/google/genai/_api_client.py:1420\u001b[39m, in \u001b[36mBaseApiClient._request_once\u001b[39m\u001b[34m(self, http_request, stream)\u001b[39m\n\u001b[32m   1412\u001b[39m   httpx_request = \u001b[38;5;28mself\u001b[39m._httpx_client.build_request(  \u001b[38;5;66;03m# type: ignore[union-attr]\u001b[39;00m\n\u001b[32m   1413\u001b[39m       method=http_request.method,\n\u001b[32m   1414\u001b[39m       url=http_request.url,\n\u001b[32m   (...)\u001b[39m\u001b[32m   1417\u001b[39m       timeout=http_request.timeout,\n\u001b[32m   1418\u001b[39m   )\n\u001b[32m   1419\u001b[39m   response = \u001b[38;5;28mself\u001b[39m._httpx_client.send(httpx_request, stream=stream)  \u001b[38;5;66;03m# type: ignore[union-attr]\u001b[39;00m\n\u001b[32m-> \u001b[39m\u001b[32m1420\u001b[39m \u001b[30;43merrors\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43mAPIError\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43mraise_for_response\u001b[39;49m\u001b[30;43m(\u001b[39;49m\u001b[30;43mresponse\u001b[39;49m\u001b[30;43m)\u001b[39;49m\n\u001b[32m   1421\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m HttpResponse(\n\u001b[32m   1422\u001b[39m     response.headers, response \u001b[38;5;28;01mif\u001b[39;00m stream \u001b[38;5;28;01melse\u001b[39;00m [response.text]\n\u001b[32m   1423\u001b[39m )\n",
      "\u001b[36mFile \u001b[39m\u001b[32m~/miniconda3/lib/python3.13/site-packages/google/genai/errors.py:155\u001b[39m, in \u001b[36mAPIError.raise_for_response\u001b[39m\u001b[34m(cls, response)\u001b[39m\n\u001b[32m    152\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[32m    153\u001b[39m   response_json = response.body_segments[\u001b[32m0\u001b[39m].get(\u001b[33m'\u001b[39m\u001b[33merror\u001b[39m\u001b[33m'\u001b[39m, {})\n\u001b[32m--> \u001b[39m\u001b[32m155\u001b[39m \u001b[30;43mcls\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43mraise_error\u001b[39;49m\u001b[30;43m(\u001b[39;49m\u001b[30;43mresponse\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43mstatus_code\u001b[39;49m\u001b[30;43m,\u001b[39;49m\u001b[30;43m \u001b[39;49m\u001b[30;43mresponse_json\u001b[39;49m\u001b[30;43m,\u001b[39;49m\u001b[30;43m \u001b[39;49m\u001b[30;43mresponse\u001b[39;49m\u001b[30;43m)\u001b[39;49m\n",
      "\u001b[36mFile \u001b[39m\u001b[32m~/miniconda3/lib/python3.13/site-packages/google/genai/errors.py:186\u001b[39m, in \u001b[36mAPIError.raise_error\u001b[39m\u001b[34m(cls, status_code, response_json, response)\u001b[39m\n\u001b[32m    184\u001b[39m   \u001b[38;5;28;01mraise\u001b[39;00m ClientError(status_code, response_json, response)\n\u001b[32m    185\u001b[39m \u001b[38;5;28;01melif\u001b[39;00m \u001b[32m500\u001b[39m <= status_code < \u001b[32m600\u001b[39m:\n\u001b[32m--> \u001b[39m\u001b[32m186\u001b[39m   \u001b[38;5;28;01mraise\u001b[39;00m ServerError(status_code, response_json, response)\n\u001b[32m    187\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[32m    188\u001b[39m   \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;28mcls\u001b[39m(status_code, response_json, response)\n",
      "\u001b[31mServerError\u001b[39m: 503 UNAVAILABLE. {'error': {'code': 503, 'message': 'This model is currently experiencing high demand. Spikes in demand are usually temporary. Please try again later.', 'status': 'UNAVAILABLE'}}"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import time\n",
    "import base64 \n",
    "from google import genai\n",
    "from google.genai.errors import APIError\n",
    "\n",
    "os.environ[\"GEMINI_API_KEY\"] = \"AQ.Ab8RN6K9G5S-h6IWgpUgfmHbIk7b8iZlWTuYptMFqOT4Ph6FnQ\"\n",
    "client = genai.Client()\n",
    "\n",
    "\n",
    "prompt_instructions_detailed5 = (\n",
    "    \"You are processing raw course text meant for Jupyter Notebook markdown cells.\\n\\n\"\n",
    "    \n",
    "    \"CRITICAL RULE: Do NOT delete, alter, or remove any original image links, image paths, \"\n",
    "    \"or image tags (e.g., lines starting with `![...](...)` or containing `<img ...>`). \"\n",
    "    \"Leave them exactly where they are in the original text sequence.\\n\\n\"\n",
    "    \n",
    "    \"INSTRUCTION:\\n\"\n",
    "    \"Rewrite the text to make it clear and university-appropriate. Make it clear and write WCAG 2.1 AA compliant. Use the following strict sequence \"\n",
    "    \"when dealing with image placements:\\n\\n\"\n",
    "    \n",
    "    \"1. First, insert the NEW Text-Based Structured Alternative ABOVE the image:\\n\"\n",
    "    \"   - For numerical data charts or grid plots: Provide a clean Markdown Data Table.\\n\"\n",
    "    \"   - For hardware architecture, network layouts, or OS Process lifecycles (like fork(), process states, execution threads): \"\n",
    "    \"Provide a high-level text flowchart inside a preformatted ```text code block.\\n\"\n",
    "    \"   - For Terminal Commands or Pipelines (like stdin/stdout piping, shell interactions): \"\n",
    "    \"Provide a vertical box-and-arrow sequence map illustrating data flow from one utility to the next.\\n\"\n",
    "    \"   - Keep diagram characters perfectly aligned and compatible with static HTML page exports.\\n\\n\"\n",
    "    \"   - CRITICAL RULE FOR VIRTUAL DESKTOP LOGIN CONTEXT:\\n\"\n",
    "    \"     If the text or diagram is about logging into the Virtual Desktop or using Ravada, \"\n",
    "    \"the text diagram sequence MUST start exactly with this workflow path:\\n\"\n",
    "    \"     [Web access to Ravada portal with authentication] ===> [Start VM] ===> [Access VM via SSH with provided port]\\n\\n\"\n",
    "    \n",
    "    \"2. Second, place the ORIGINAL untouched image link exactly as it was.\\n\\n\"\n",
    "    \n",
    "    \"3. Third, immediately BELOW the original image link, add the Comprehensive Detailed Description:\\n\"\n",
    "    \"Inside this caption block, you MUST start with the exact text '**Accessible Image Description:**'. Place it always under the image.\"\n",
    "    \"   - Overview: One sentence describing what the visual shows at a high level.\\n\"\n",
    "    \"   - Setup: Explain the primary components, labels, axes, or code pathways without hyper-granular specs.\\n\"\n",
    "    \"   - Key Takeaways: Core conceptual trends or execution results shown.\\n\\n\"\n",
    "\n",
    "    f\"Course Text:\\n{my_notebook_text}\"\n",
    ")\n",
    "prompt_instructions = prompt_instructions_detailed5\n",
    "\n",
    "try:\n",
    "    response = client.models.generate_content(\n",
    "        model='models/gemini-3.6-flash',\n",
    "        contents= prompt_instructions,\n",
    "        )\n",
    "\n",
    "    #print(response.text)\n",
    "\n",
    "    output_text = response.text\n",
    "    print(output_text)\n",
    "    \n",
    "except APIError as e:\n",
    "    raise e"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "# Networking\n",
    "\n",
    "## Linux Networking\n",
    "\n",
    "- Overview of network protocols and TCP/IP in Linux.\n",
    "- Network configuration in Linux.\n",
    "- Netplan configuration on Ubuntu servers.\n",
    "- Network settings in Red Hat via NetworkManager.\n",
    "- Network troubleshooting tools: `ping`, `traceroute`, `tcpdump`.\n",
    "- Network calculation using `ipcalc`.\n",
    "- Network applications and services.\n",
    "- Network traffic sniffing with `tcpdump`.\n",
    "- Installation and configuration of the `lighttpd` web server.\n",
    "- Port tunneling using SSH.\n",
    "\n",
    "---\n",
    "\n",
    "## Computers on the Internet\n",
    "\n",
    "Network communication enables data transmission, file transfer, and remote application management between interconnected computer systems. Networks are structured into independent, functional layers implemented on each system:\n",
    "\n",
    "- Each layer performs a specific set of tasks and provides services to adjacent layers (above and below).\n",
    "- Inter-layer communication adheres to defined rules and procedures known as **protocols**.\n",
    "- Data is processed sequentially through the layers from top to bottom before transmission.\n",
    "- At the bottom layer, physical systems communicate by transmitting and receiving network datagrams across hardware infrastructure, including cables, switches, and routers.\n",
    "\n",
    "```text\n",
    "[ Host Machine A: Application Layer ]\n",
    "                |\n",
    "                v\n",
    "[ Transport / Internet / Link Layers ]\n",
    "                |\n",
    "                v\n",
    "   ( Physical Infrastructure )\n",
    "   (  Cables / Switches / Routers  )\n",
    "                |\n",
    "                v\n",
    "[ Transport / Internet / Link Layers ]\n",
    "                |\n",
    "                v\n",
    "[ Host Machine B: Application Layer ]\n",
    "```\n",
    "\n",
    "![](img/computers_on_network.png)\n",
    "\n",
    "**Accessible Image Description:**\n",
    "- **Overview:** A network diagram illustrating inter-computer communication through connected infrastructure.\n",
    "- **Setup:** The diagram shows terminal nodes (computers) connected via physical cabling through intermediate networking hardware including network switches and routers.\n",
    "- **Key Takeaways:** Data flows down through protocol layers on the source node, across network hardware, and up through protocol layers on the destination node.\n",
    "\n",
    "---\n",
    "\n",
    "## TCP/IP 5-Layer Protocol Suite\n",
    "\n",
    "```text\n",
    "+-------------------------------------------------------------------+\n",
    "| 5. Application Layer: SSH, FTP, HTTP, SMTP                       |\n",
    "+-------------------------------------------------------------------+\n",
    "| 4. Transport Layer: TCP, UDP, Port Numbers                        |\n",
    "+-------------------------------------------------------------------+\n",
    "| 3. Internet Layer: IP Addresses, Subnets, Routing                 |\n",
    "+-------------------------------------------------------------------+\n",
    "| 2. Link Layer: Ethernet, MAC Addresses, Network Switch            |\n",
    "+-------------------------------------------------------------------+\n",
    "| 1. Physical Layer: Network Interface Cards (NIC), Cabling, Signal |\n",
    "+-------------------------------------------------------------------+\n",
    "```\n",
    "\n",
    "![](img/tcp-ip.jpg)\n",
    "\n",
    "**Accessible Image Description:**\n",
    "- **Overview:** Architectural diagram displaying the 5-layer TCP/IP protocol stack model.\n",
    "- **Setup:** Layers are arranged vertically from Layer 5 (Application) down to Layer 1 (Physical), identifying key protocols and hardware at each tier.\n",
    "- **Key Takeaways:** Each layer performs dedicated communication tasks, passing formatted data payloads to adjacent layers.\n",
    "\n",
    "### TCP/IP Protocol Layers and Tasks\n",
    "\n",
    "| TCP/IP Protocol Layer | Core Responsibilities |\n",
    "| :--- | :--- |\n",
    "| **5. Application Layer**<br>(`ssh`, `ftp`, `http`, etc.) | Data segmentation by the application into Maximum Transmission Unit (MTU) segments. Typical MTU is 1.5 KB, containing the data payload plus 40 bytes for TCP and IP headers. |\n",
    "| **4. Transport Layer**<br>(TCP, UDP, Port management) | Connection management, error control, traffic flow control, and port addressing for application delivery. |\n",
    "| **3. Internet Layer**<br>(IP addresses, subnets, routing) | Host addressing across network interfaces and packet routing across interconnected subnets. |\n",
    "| **2. Link Layer**<br>(Ethernet, MAC addressing) | Local subnet communication via switches. Uses unique Media Access Control (MAC) addresses (e.g., `48:d2:4f:f4:0d:fb`). The Address Resolution Protocol (ARP) maps IP addresses to MAC addresses on the local network segment. |\n",
    "| **1. Physical Layer**<br>(NIC, cabling, switches, routers, Wi-Fi APs) | Hardware devices responsible for converting frames into electronic, optical, or radio signals across physical transmission media. |\n",
    "\n",
    "---\n",
    "\n",
    "## OSI 7-Layer Network Stack Model\n",
    "\n",
    "```text\n",
    "+-------------------------------------------------------------------+\n",
    "| 7. Application Layer (Network Services & User Interfaces)         |\n",
    "+-------------------------------------------------------------------+\n",
    "| 6. Presentation Layer (Data Formatting, Encryption, Compression)  |\n",
    "+-------------------------------------------------------------------+\n",
    "| 5. Session Layer (Session Establishment, RPC Management)          |\n",
    "+-------------------------------------------------------------------+\n",
    "| 4. Transport Layer (End-to-End Reliability, Error Recovery)       |\n",
    "+-------------------------------------------------------------------+\n",
    "| 3. Network Layer (Logical Addressing, Routing Path Selection)     |\n",
    "+-------------------------------------------------------------------+\n",
    "| 2. Data Link Layer (Physical Addressing, Framing, Media Access)   |\n",
    "+-------------------------------------------------------------------+\n",
    "| 1. Physical Layer (Bit Transmission, Hardware Interfaces)         |\n",
    "+-------------------------------------------------------------------+\n",
    "```\n",
    "\n",
    "![](img/osi.jpg)\n",
    "\n",
    "**Accessible Image Description:**\n",
    "- **Overview:** Architectural diagram depicting the Open Systems Interconnection (OSI) 7-layer reference model.\n",
    "- **Setup:** A stacked vertical layout illustrating the Application, Presentation, Session, Transport, Network, Data Link, and Physical layers.\n",
    "- **Key Takeaways:** Demonstrates the theoretical 7-layer segmentation for network functions, highlighting upper-level processing layers (5–7) that handle application interaction, encoding, and session persistence.\n",
    "\n",
    "### OSI Upper Layers Summary\n",
    "\n",
    "| OSI Protocol Layer | Core Responsibilities |\n",
    "| :--- | :--- |\n",
    "| **7. Application Layer**<br>(`ssh`, `ftp`, `http`, etc.) | Data segmentation by applications into standard payload sizes. |\n",
    "| **6. Presentation Layer**<br>(SSL/TLS, `libz`, XDR) | Data encryption, compression, syntax formatting, and encoding/decoding. |\n",
    "| **5. Session Layer**<br>(Remote Procedure Calls - RPC) | Establishes, maintains, synchronizes, and terminates interactive sessions between systems. |\n",
    "\n",
    "---\n",
    "\n",
    "## TCP/IP and Kernel Integration\n",
    "\n",
    "```text\n",
    "+-------------------------------------------------------------------+\n",
    "| USER SPACE: Application Layer (HTTP, SSH, FTP)                    |\n",
    "+-------------------------------------------------------------------+\n",
    "==================== Operating System Boundary ======================\n",
    "+-------------------------------------------------------------------+\n",
    "| KERNEL SPACE: Transport Layer (TCP, UDP Protocol Engines)         |\n",
    "+-------------------------------------------------------------------+\n",
    "| KERNEL SPACE: Internet Layer (IP Routing, ICMP, Subnet Masking)   |\n",
    "+-------------------------------------------------------------------+\n",
    "| KERNEL SPACE: Link Layer (NIC Device Drivers, ARP, Frame Buffers) |\n",
    "+-------------------------------------------------------------------+\n",
    "```\n",
    "\n",
    "![](img/tcpip-kernel.jpg)\n",
    "\n",
    "**Accessible Image Description:**\n",
    "- **Overview:** Diagram showing where network stack components reside within Linux OS architecture.\n",
    "- **Setup:** Divides networking responsibilities between User Space (Application layer) and Kernel Space (Transport, Internet, and Link layers).\n",
    "- **Key Takeaways:** The Linux operating system kernel directly manages lower-level networking protocols, abstracting physical driver and network handling away from user applications.\n",
    "\n",
    "---\n",
    "\n",
    "## Packet Encapsulation\n",
    "\n",
    "Data is transmitted across networks using packet encapsulation, ultimately structured as physical Ethernet frames.\n",
    "\n",
    "```text\n",
    "+-------------------------------------------------------------------+\n",
    "| Ethernet Frame Header                                             |\n",
    "|   +-----------------------------------------------------------+   |\n",
    "|   | IP Packet Header                                          |   |\n",
    "|   |   +---------------------------------------------------+   |   |\n",
    "|   |   | TCP Segment Header                                |   |   |\n",
    "|   |   |   +-------------------------------------------+   |   |   |\n",
    "|   |   |   | User Application Payload / Data           |   |   |   |\n",
    "|   |   |   +-------------------------------------------+   |   |   |\n",
    "|   |   +---------------------------------------------------+   |   |\n",
    "|   +-----------------------------------------------------------+   |\n",
    "| Ethernet Frame Trailer (FCS Checksum)                             |\n",
    "+-------------------------------------------------------------------+\n",
    "```\n",
    "\n",
    "![](img/ip_packet.jpg)\n",
    "\n",
    "**Accessible Image Description:**\n",
    "- **Overview:** Conceptual diagram illustrating packet encapsulation.\n",
    "- **Setup:** Concentric nested headers showing user data wrapped inside TCP, IP, and Ethernet frame control structures.\n",
    "- **Key Takeaways:** As application data travels down the network stack, each layer prepends control metadata required for transport, logical addressing, and hardware frame delivery.\n",
    "\n",
    "---\n",
    "\n",
    "## Host-to-Host Communication via TCP/IP\n",
    "\n",
    "```text\n",
    "[ Host A Application ]                               [ Host B Application ]\n",
    "          |                                                    ^\n",
    "          v                                                    |\n",
    "[ Host A TCP/IP Stack ] === ( Physical Network Link ) ===> [ Host B TCP/IP Stack ]\n",
    "```\n",
    "\n",
    "![](img/host-host.jpg)\n",
    "\n",
    "**Accessible Image Description:**\n",
    "- **Overview:** High-level sequence diagram showing end-to-end communication between two host computers across a network link.\n",
    "- **Setup:** Illustrates logical peer-to-peer communications at matching protocol layers between Host A and Host B.\n",
    "- **Key Takeaways:** Protocols at corresponding layers logically interact, while physical data movement occurs vertically down through the sending host's stack and up the receiving host's stack.\n",
    "\n",
    "```text\n",
    "Host A (Sender)                                Host B (Receiver)\n",
    "+-------------------+                          +-------------------+\n",
    "| Application Layer | === Logical Channel ===> | Application Layer |\n",
    "| Transport Layer   |                          | Transport Layer   |\n",
    "| Internet Layer    |                          | Internet Layer    |\n",
    "| Link Layer        | === Physical Medium ===> | Link Layer        |\n",
    "+-------------------+                          +-------------------+\n",
    "```\n",
    "\n",
    "![](img/host-tcpip.jpg)\n",
    "\n",
    "**Accessible Image Description:**\n",
    "- **Overview:** Detailed layout of host-to-host TCP/IP interaction showing protocol abstraction across physical links.\n",
    "- **Setup:** Side-by-side comparative stack diagram mapping encapsulation on Host A and decapsulation on Host B.\n",
    "- **Key Takeaways:** Physical data transmission occurs through underlying network media, while user processes communicate via abstracted logical protocol channels.\n",
    "\n",
    "---\n",
    "\n",
    "## Wired Connection Checkup (Lab Exercise)\n",
    "\n",
    "To verify whether a network interface is physically connected to a switch and check its operational speed, use the `ethtool` command.\n",
    "\n",
    "1. Install `ethtool` on Ubuntu:\n",
    "```bash\n",
    "sudo apt update && sudo apt install ethtool\n",
    "```\n",
    "\n",
    "2. List active network interfaces using `ip address show` or its shorthand version:\n",
    "```bash\n",
    "ip a\n",
    "```\n",
    "\n",
    "3. Query interface properties for the primary Ethernet device (`enp0s3`):\n",
    "```bash\n",
    "/sbin/ethtool enp0s3\n",
    "```\n",
    "\n",
    "In virtualized environments (such as VDI or virtual machines), hardware parameter queries may return link detection without showing fixed hardware negotiation properties:\n",
    "- Speed: Unknown!\n",
    "- Duplex: Unknown!\n",
    "- Link detected: yes\n",
    "\n",
    "On bare-metal physical hardware, expected output includes hardware link negotiation details:\n",
    "- Speed: 1000Mb/s\n",
    "- Duplex: Full\n",
    "- Link detected: yes\n",
    "\n",
    "This output confirms that the network cable is connected, running at 1 Gbps full-duplex mode.\n",
    "\n",
    "---\n",
    "\n",
    "## Network IP Configuration Overview\n",
    "\n",
    "Network interface configurations generally use one of two methods:\n",
    "\n",
    "- **Static Configuration:** IP address, subnet mask, default gateway, hostname, and Domain Name System (DNS) servers are manually specified on the host.\n",
    "- **Dynamic Configuration (DHCP):** Network settings are leased automatically from a Dynamic Host Configuration Protocol server.\n",
    "\n",
    "```text\n",
    "[ Client Host ]                                   [ DHCP Server ]\n",
    "       |                                                 |\n",
    "       | --- 1. DHCPDISCOVER (Broadcast) --------------> |\n",
    "       |                                                 |\n",
    "       | <-- 2. DHCPOFFER (Unicast/Broadcast) ---------- |\n",
    "       |                                                 |\n",
    "       | --- 3. DHCPREQUEST (Broadcast) ---------------> |\n",
    "       |                                                 |\n",
    "       | <-- 4. DHCPACK (Unicast) ---------------------- |\n",
    "       v                                                 v\n",
    "```\n",
    "\n",
    "![](img/dhcp.gif)\n",
    "\n",
    "**Accessible Image Description:**\n",
    "- **Overview:** Sequence diagram showing the four-step Dynamic Host Configuration Protocol (DHCP) handshake process.\n",
    "- **Setup:** A vertical time sequence illustrating message exchanges between a client requesting configuration and a responding DHCP server.\n",
    "- **Key Takeaways:** Dynamic network configuration relies on the DHCP sequence (Discover, Offer, Request, Acknowledge) to assign IP addresses, subnet masks, and default gateways automatically.\n",
    "\n",
    "---\n",
    "\n",
    "## Network Configuration Tools Across Linux Distributions\n",
    "\n",
    "| Configuration Tool / Script | `NetworkManager` Service | Netplan (`systemd-networkd`) Service |\n",
    "| :--- | :--- | :--- |\n",
    "| **Traditional Debian / Red Hat** | **Desktop / Modern Enterprise Linux** | **Ubuntu Server Systems** |\n",
    "| File: `/etc/network/interfaces` (Debian/Ubuntu) | Commands: `nmtui`, `nmcli` | Configuration file:<br>`/etc/netplan/00-installer-config.yaml` |\n",
    "| File: `/etc/sysconfig/network-scripts/ifcfg-ens3` (Red Hat) | System File Path:<br>`/etc/NetworkManager/system-connections/<conn-name>.nmconnection` | Application Command:<br>`netplan apply` |\n",
    "| Command: `ifup ens3` | Reload Command:<br>`systemctl restart NetworkManager; nmcli con up <conn-name>` | |\n",
    "\n",
    "---\n",
    "\n",
    "## Deploying a Virtual Machine for Exercises\n",
    "\n",
    "1. Shut down the existing `kvm1` VM and clone it to create `netplan`:\n",
    "```bash\n",
    "virsh shutdown kvm1\n",
    "virt-clone -o kvm1 -n netplan -f /home/hostadm/KVM/netplan.qcow2\n",
    "```\n",
    "\n",
    "2. Start the new `netplan` VM and open its console:\n",
    "```bash\n",
    "virsh start netplan\n",
    "virsh console netplan\n",
    "```\n",
    "\n",
    "3. Update the host name inside `/etc/hostname`.\n",
    "\n",
    "4. Reset the machine ID by running the system script inside the VM:\n",
    "```bash\n",
    "./vm_id_reset.sh\n",
    "```\n",
    "\n",
    "5. Remove default cloud-init SSH restriction files if present:\n",
    "```bash\n",
    "rm /etc/ssh/sshd_config.d/60-cloudimg-settings.conf\n",
    "```\n",
    "\n",
    "6. Reboot the VM:\n",
    "```bash\n",
    "reboot\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## Configuring Network Settings on Ubuntu Server with Netplan\n",
    "\n",
    "1. Check existing IP parameters on the `netplan` VM:\n",
    "```bash\n",
    "ip addr\n",
    "```\n",
    "\n",
    "2. Switch to root privileges:\n",
    "```bash\n",
    "sudo -s\n",
    "```\n",
    "\n",
    "3. Configure static IP address `192.168.122.222` by editing `/etc/netplan/00-installer-config.yaml`:\n",
    "```yaml\n",
    "---\n",
    "network:\n",
    "  version: 2\n",
    "  renderer: networkd\n",
    "  ethernets:\n",
    "    enp1s0:\n",
    "      dhcp4: false\n",
    "      addresses:\n",
    "        - 192.168.122.222/24\n",
    "      routes:\n",
    "        - to: default\n",
    "          via: 192.168.122.1\n",
    "      nameservers:\n",
    "        addresses: [192.168.122.1]\n",
    "```\n",
    "\n",
    "4. Install and run YAML syntax validation tools:\n",
    "```bash\n",
    "apt install yamllint\n",
    "yamllint /etc/netplan/00-installer-config.yaml\n",
    "```\n",
    "\n",
    "5. Apply the network configuration:\n",
    "```bash\n",
    "netplan apply\n",
    "```\n",
    "\n",
    "6. Verify the updated configuration on interface `ens3` (or `enp1s0`):\n",
    "```bash\n",
    "ip addr\n",
    "```\n",
    "\n",
    "7. Confirm gateway route settings:\n",
    "```bash\n",
    "ip route show\n",
    "```\n",
    "*(Expected output includes: `default via 192.168.122.1`)*\n",
    "\n",
    "---\n",
    "\n",
    "## Cloning the `rocky10` VM\n",
    "\n",
    "*(If `rocky10-1` is already available from previous exercises, skip cloning and proceed to network configuration).*\n",
    "\n",
    "1. Stop `rocky10` and clone it:\n",
    "```bash\n",
    "virsh list\n",
    "virsh stop rocky10\n",
    "virt-clone -o rocky10 -n rocky10-1 -f /home/hostadm/KVM/rocky10-1.qcow2\n",
    "```\n",
    "\n",
    "2. Start `rocky10-1` and attach to the console:\n",
    "```bash\n",
    "virsh start rocky10-1\n",
    "virsh console rocky10-1\n",
    "```\n",
    "\n",
    "3. Install system tools and update the machine hostname to `rocky10-1`:\n",
    "```bash\n",
    "sudo dnf install -y dbus-tools\n",
    "```\n",
    "\n",
    "4. Reset unique system identifiers:\n",
    "```bash\n",
    "sudo rm -f /etc/machine-id\n",
    "sudo dbus-uuidgen --ensure=/etc/machine-id\n",
    "sudo dbus-uuidgen --ensure\n",
    "sudo dhclient -r enp1s0\n",
    "```\n",
    "\n",
    "5. Reboot the VM:\n",
    "```bash\n",
    "reboot\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## Network Configurations on Red Hat Enterprise Linux / Rocky Linux\n",
    "\n",
    "Red Hat systems utilize `nmcli` to command the `NetworkManager` service.\n",
    "\n",
    "1. Log into the console of `rocky10-1`:\n",
    "```bash\n",
    "virsh start rocky10-1\n",
    "virsh console rocky10-1\n",
    "```\n",
    "\n",
    "2. Read IP interface information:\n",
    "```bash\n",
    "ip addr show enp1s0\n",
    "```\n",
    "\n",
    "3. Test initial connectivity to `netplan`:\n",
    "```bash\n",
    "ping -c 3 192.168.122.222\n",
    "```\n",
    "\n",
    "4. Elevate privileges to root:\n",
    "```bash\n",
    "sudo -s\n",
    "```\n",
    "\n",
    "5. List active NetworkManager connections:\n",
    "```bash\n",
    "nmcli conn show\n",
    "```\n",
    "\n",
    "6. Display device properties for `enp1s0`:\n",
    "```bash\n",
    "nmcli dev show enp1s0\n",
    "```\n",
    "\n",
    "7. Create a static network profile named `eth_static`:\n",
    "```bash\n",
    "nmcli con add type ethernet con-name eth_static ip4 192.168.122.126/24 gw4 192.168.122.1 ipv4.dns 192.168.122.1\n",
    "```\n",
    "\n",
    "8. Activate `eth_static`:\n",
    "```bash\n",
    "nmcli con up eth_static\n",
    "nmcli conn show\n",
    "ip addr show enp1s0\n",
    "```\n",
    "\n",
    "9. Modify an existing static connection address to `192.168.122.125`:\n",
    "```bash\n",
    "nmcli con modify eth_static ipv4.addresses 192.168.122.125/24\n",
    "nmcli con up eth_static\n",
    "ip addr show enp1s0\n",
    "```\n",
    "\n",
    "### Alternative Configuration File Method\n",
    "`nmcli` generates configuration profiles under `/etc/NetworkManager/system-connections/`. You can manually edit these files and apply changes:\n",
    "```bash\n",
    "systemctl restart NetworkManager\n",
    "nmcli con up eth_static\n",
    "```\n",
    "\n",
    "### Reverting to Dynamic DHCP\n",
    "To restore automatic network addressing, reactivate the DHCP profile:\n",
    "```bash\n",
    "nmcli conn up \"Wired connection 1\"\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## Internet Control Message Protocol (ICMP)\n",
    "\n",
    "ICMP is deployed across end-hosts and routers to perform:\n",
    "- Datagram processing status reporting.\n",
    "- Routing diagnostics and connectivity verification.\n",
    "- Diagnostic error reporting.\n",
    "\n",
    "### Command Diagnostic 1: `ping`\n",
    "\n",
    "```text\n",
    "[ Source Host ]                                   [ Target Host ]\n",
    "       |                                                 |\n",
    "       | --- ICMP Echo Request (Type 8) ---------------> |\n",
    "       |                                                 |\n",
    "       | <-- ICMP Echo Reply (Type 0) ------------------ |\n",
    "       v                                                 v\n",
    "```\n",
    "\n",
    "![](img/ping.jpg)\n",
    "\n",
    "**Accessible Image Description:**\n",
    "- **Overview:** Flow diagram of ICMP Echo Request and Reply mechanism used by the `ping` utility.\n",
    "- **Setup:** Shows an origin host transmitting an echo request message to a remote host and receiving an echo reply message.\n",
    "- **Key Takeaways:** The `ping` command verifies network reachability and measures round-trip time by issuing ICMP echo requests and awaiting replies.\n",
    "\n",
    "#### Verification Exercise:\n",
    "```bash\n",
    "ping -c 5 capone.rutgers.edu\n",
    "ping -c 5 engsoft.rutgers.edu\n",
    "ping -c 5 google.com\n",
    "```\n",
    "\n",
    "### Command Diagnostic 2: `traceroute`\n",
    "\n",
    "`traceroute` tracks network pathways by leveraging the Time to Live (TTL) packet field. Intermediate routers decrement TTL by 1; when TTL reaches 0, the router returns an ICMP \"Time Exceeded\" message.\n",
    "\n",
    "```text\n",
    "[ Source Host ] ---> [ Router 1 (TTL=29) ] ===> (ICMP Time Exceeded returned)\n",
    "[ Source Host ] ---> [ Router 2 (TTL=28) ] ===> (ICMP Time Exceeded returned)\n",
    "[ Source Host ] ---> [ Target Host (TTL=30-N)] ===> (Destination ICMP Response)\n",
    "```\n",
    "\n",
    "![](img/traceroute.jpg)\n",
    "\n",
    "**Accessible Image Description:**\n",
    "- **Overview:** Diagram explaining the operating mechanism of the `traceroute` network diagnostic tool.\n",
    "- **Setup:** Shows decremental Time To Live (TTL) values assigned to outgoing IP packets by the routers. ICMP Time Exceeded messages are triggered at TTL=0.\n",
    "- **Key Takeaways:** `traceroute` identifies each router hop along the path to a destination host by systematically analyzing ICMP expiration responses.\n",
    "\n",
    "#### Verification Exercise:\n",
    "```bash\n",
    "sudo apt install traceroute\n",
    "sudo /usr/sbin/traceroute -I capone.rutgers.edu\n",
    "sudo /usr/sbin/traceroute -I engsoft.rutgers.edu\n",
    "sudo /usr/sbin/traceroute -I google.com\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## Subnets and Routing Fundamentals\n",
    "\n",
    "```text\n",
    "[ Host B (Subnet 1) ]\n",
    "         |\n",
    "         X <-- Missing Default Gateway Route\n",
    "         |\n",
    "    [ Router ]\n",
    "     /      \\\n",
    "    /        \\\n",
    "[ Host A ]  [ Host C (Subnet 2) ]\n",
    "```\n",
    "\n",
    "![](img/hosts-router.png)\n",
    "\n",
    "**Accessible Image Description:**\n",
    "- **Overview:** Network topology diagram illustrating a routing failure due to missing gateway configurations.\n",
    "- **Setup:** Host A, Host B, and Host C are connected via a central router across subnets, with Host B lacking a default gateway definition.\n",
    "- **Key Takeaways:** Communication across separate subnets requires correctly configured default gateways on participating host machines.\n",
    "\n",
    "### Network Calculation Principles\n",
    "- Devices attached to the same switch share a local subnet and communicate directly via MAC layer delivery.\n",
    "- Traffic destined for external subnets requires routing through a default gateway.\n",
    "- Subnet masks differentiate the network prefix from host identifiers.\n",
    "\n",
    "$$\\text{Network Address} = \\text{Host Address} \\text{ (bitwise AND) } \\text{Subnet Mask}$$\n",
    "\n",
    "| Parameter | Decimal Notation | Binary Form |\n",
    "| :--- | :---: | :--- |\n",
    "| **IP Address** | `192.168.5.10` | `11000000.10010100.00000101.00001010` |\n",
    "| **Subnet Mask** | `255.255.255.0` | `11111111.11111111.11111111.00000000` |\n",
    "| **Network Address** | `192.168.5.0` | `11000000.10010100.00000101.00000000` |\n",
    "\n",
    "- **Network Address:** Lowest address within subnet bounds (`192.168.5.0`).\n",
    "- **Broadcast Address:** Highest address within subnet bounds (`192.168.5.255`).\n",
    "- **Usable Host Capacity:** $2^8 - 2 = 254$ usable host IPs (`192.168.5.1` to `192.168.5.254`).\n",
    "\n",
    "---\n",
    "\n",
    "## IP Calculator Exercises (`ipcalc`)\n",
    "\n",
    "1. Install `ipcalc`:\n",
    "```bash\n",
    "sudo apt install ipcalc\n",
    "```\n",
    "\n",
    "2. Calculate network boundaries using CIDR or dotted decimal notation:\n",
    "```bash\n",
    "ipcalc 192.168.5.0/255.255.255.0\n",
    "ipcalc 192.168.5.0/24\n",
    "```\n",
    "\n",
    "3. Analyze subnetting for split network blocks:\n",
    "```bash\n",
    "ipcalc 192.168.5.0/25\n",
    "ipcalc 192.168.5.128/25\n",
    "```\n",
    "*(Notice that `192.168.5.5` resides within the lower `/25` subnet, whereas `192.168.5.250` falls into the upper `/25` subnet).*\n",
    "\n",
    "---\n",
    "\n",
    "## Network Applications: `ssh`, `sftp`, `scp`, `rsync`\n",
    "\n",
    "1. Determine the target IP address of `netplan`:\n",
    "```bash\n",
    "virsh domifaddr netplan\n",
    "```\n",
    "\n",
    "2. Append entry to `/etc/hosts`:\n",
    "```text\n",
    "192.168.122.222   netplan\n",
    "```\n",
    "\n",
    "3. Generate an SSH keypair on your desktop:\n",
    "```bash\n",
    "ssh-keygen -t rsa\n",
    "```\n",
    "\n",
    "4. Transfer public keys using interactive `sftp`:\n",
    "```bash\n",
    "sftp netplan\n",
    "sftp> cd .ssh\n",
    "sftp> lcd .ssh\n",
    "sftp> put id_rsa.pub authorized_keys\n",
    "sftp> quit\n",
    "```\n",
    "\n",
    "5. Automate key passphrase management using `ssh-agent`:\n",
    "```bash\n",
    "exec ssh-agent $SHELL\n",
    "ssh-add\n",
    "```\n",
    "\n",
    "6. Execute remote commands directly over SSH:\n",
    "```bash\n",
    "ssh netplan \"uname -a\"\n",
    "```\n",
    "\n",
    "7. Copy files remotely using `scp` and `rsync`:\n",
    "```bash\n",
    "# Copy file from remote to local\n",
    "scp netplan:/etc/hosts .\n",
    "\n",
    "# Copy local file to remote home directory\n",
    "touch somef.txt\n",
    "scp somef.txt netplan:/home/$USER\n",
    "\n",
    "# Copy directory recursively\n",
    "scp -r /etc netplan:\n",
    "\n",
    "# Synchronize directories via rsync\n",
    "mkdir -p dir1/dir2/dir3\n",
    "rsync -avz dir1 netplan:/home/$USER\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## Data Transfer via Netcat (`netcat` / `ncat`)\n",
    "\n",
    "| Step | Ubuntu Server (`netplan`) | Red Hat / Rocky Linux (`rocky10-1`) |\n",
    "| :- | :--- | :--- |\n",
    "| **1. Tool Installation** | `apt install netcat-openbsd` | `dnf install nmap-ncat` |\n",
    "| **2. UDP Stream Test** | `netcat -l -u 8080` | `ncat 192.168.122.222 -u 8080` |\n",
    "| **3. File Transfer** | `netcat -l 8080 > outputfile.txt` | `ncat 192.168.122.222 8080 < /etc/hosts` |\n",
    "| **4. Archive Stream** | `netcat -l 8080 \\| tar xf -` | `tar cf - /etc \\| nc -w 20 192.168.122.222 8080` |\n",
    "\n",
    "---\n",
    "\n",
    "## Network Traffic Sniffing with `tcpdump`\n",
    "\n",
    "1. Install `tcpdump` on `netplan`:\n",
    "```bash\n",
    "sudo apt install tcpdump\n",
    "```\n",
    "\n",
    "2. Capture traffic on interface `enp1s0`:\n",
    "```bash\n",
    "sudo tcpdump -v -n -i enp1s0\n",
    "```\n",
    "\n",
    "3. Filter capture for specific protocol and IP address:\n",
    "```bash\n",
    "sudo tcpdump -v -n -i enp1s0 host 192.168.122.126 and proto \\icmp\n",
    "```\n",
    "\n",
    "4. Capture payload data for specific incoming destination ports:\n",
    "```bash\n",
    "sudo tcpdump -i enp1s0 src host 192.168.122.126 and dst port 8080 -A\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## Web Server Deployment (`lighttpd`)\n",
    "\n",
    "1. Install `lighttpd` on `netplan`:\n",
    "```bash\n",
    "sudo apt install lighttpd\n",
    "```\n",
    "\n",
    "2. Enable directory browsing by editing `/etc/lighttpd/lighttpd.conf`:\n",
    "```text\n",
    "dir-listing.activate = \"enable\"\n",
    "```\n",
    "\n",
    "3. Restart web service:\n",
    "```bash\n",
    "sudo systemctl restart lighttpd\n",
    "```\n",
    "\n",
    "4. Populate document root directory (`/var/www/html`):\n",
    "```bash\n",
    "cd /var/www/html\n",
    "sudo apt download lua5.4\n",
    "sudo dpkg -X lua5.4_5.4.8-1build1_amd64.deb LUA\n",
    "sudo rm -f index.lighttpd.html\n",
    "```\n",
    "\n",
    "5. Retrieve data recursively from client host:\n",
    "```bash\n",
    "mkdir DOWNLOADS && cd DOWNLOADS\n",
    "wget http://192.168.122.222/lua5.4_5.4.8-1build1_amd64.deb\n",
    "wget --recursive -nH http://192.168.122.222/\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## SSH Local Port Tunneling\n",
    "\n",
    "Forward local client port `8080` to web port `80` on the target VM across an encrypted SSH connection:\n",
    "\n",
    "```bash\n",
    "ssh -L 8080:192.168.122.222:80 user@desktop-host\n",
    "```\n",
    "\n",
    "Access the forwarded remote service in your local Web Browser:\n",
    "```text\n",
    "http://localhost:8080/\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
