{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "remove-cell",
     "remove_cell"
    ]
   },
   "outputs": [],
   "source": [
    "my_notebook_text = \"\"\"\n",
    "# Start/stop services, environment file, job scheduling.\n",
    "\n",
    "- Bootup sequence\n",
    "- Systemd\n",
    "- Service start and stop\n",
    "- Unit files\n",
    "- Environment files\n",
    "- Scheduling jobs with <tt>at</tt> and <tt>cron</tt>\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Linux Bootup and Login Sequence\n",
    "\n",
    "1. BIOS boot-ups; checks peripherals; finds bootable device.\n",
    "\n",
    "2. First sector loaded into RAM and executed-- GRUB prompt appears.\n",
    "\n",
    "3. Kernel loaded from sector list, /boot/vmlinuz-7.0.0-15-generic; places the appropriate initial RAM disk image, initrd, into RAM.\n",
    "\n",
    "4. Kernel executed; unpacks.\n",
    "\n",
    "5. Kernel initializes hardware.\n",
    "\n",
    "6. Kernel mounts root, /, file system, say /dev/sda1.\n",
    "\n",
    "7. Kernel executes init (Systemd) as PID 1.\n",
    "\n",
    "8. Systemd initiates the units in the default target. The units start the services.\n",
    "\n",
    "9. Among the started services, there are agetty programs on each terminal and the login process.\n",
    "\n",
    "10. login requests user name and password and validates them against /etc/passwd, or NIS maps, or LDAP (depending on PAM settings).\n",
    "\n",
    "11. login starts a shell.\n",
    "\n",
    "12. The shell executes the appropriate startup files (.profile for sh, ksh; .bash_profile for BASH; .cshrc and .login for csh and tcsh).\n",
    "\n",
    "13. The shell prints a prompt and waits for input.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## VM setup for startup scripts. (Exercise)\n",
    " \n",
    "\n",
    "Clone kvm1 to a new NFS server VM host, run01, by executing the command below:\n",
    "```bash\n",
    "virsh shutdown kvm1\n",
    "virt-clone  -o kvm1 -n run01 -f /home/hostadm/KVM/run01.qcow2\n",
    "```\n",
    "Check if the new VM is in the list, start it, then login to its console:\n",
    "```bash\n",
    "virsh list --all\n",
    "virsh start run01\n",
    "virsh console run01\n",
    "```\n",
    "Fix the host name.\n",
    " Replace kvm1 with run01 in /etc/hostname file.\n",
    " Reset the machine ID by running the following commands on kvm3:\n",
    "```bash\n",
    "./vm_id_reset.sh\n",
    "```\n",
    "\n",
    "Reboot the VM:\n",
    "```bash\n",
    "reboot\n",
    "```\n",
    "After reboot, the new VM should come up with the correct host name.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Systemd is the parent of all processes (Exercise)\n",
    "\n",
    "Systemd is the parent of all processes on the system, it is executed by the kernel and is responsible for starting all other processes; it is the parent of all processes whose natural parents have died and it is responsible for reaping those when they die.\n",
    "\n",
    "```bash\n",
    "virsh console run01\n",
    "```\n",
    "Install package psmisc and run command pstree\n",
    "```bash\n",
    "sudo -s\n",
    "apt install psmisc\n",
    "pstree\n",
    "```\n",
    "```{admonition} Output:\n",
    "    systemd─┬─ModemManager───3*[{ModemManager}]\n",
    "            ├─accounts-daemon───3*[{accounts-daemon}]\n",
    "            ├─agetty\n",
    "            ├─atd\n",
    "            ├─cron\n",
    "            ├─dbus-daemon\n",
    "            ├─login───bash───pstree\n",
    "            ├─multipathd───6*[{multipathd}]\n",
    "            ├─networkd-dispat\n",
    "            ├─polkitd───3*[{polkitd}]\n",
    "            ├─rsyslogd───3*[{rsyslogd}]\n",
    "            ├─sshd\n",
    "            ├─systemd───(sd-pam)\n",
    "            ├─systemd-hostnam\n",
    "            ├─systemd-journal\n",
    "            ├─systemd-logind\n",
    "            ├─systemd-network\n",
    "            ├─systemd-resolve\n",
    "            ├─systemd-timesyn───{systemd-timesyn}\n",
    "            ├─systemd-udevd───10*[systemd-udevd]\n",
    "            ├─udisksd───5*[{udisksd}]\n",
    "            └─unattended-upgr───{unattended-upgr}\n",
    "```\n",
    "\n",
    "```bash\n",
    "pstree -p -l\n",
    "```\n",
    "shows the process tree with their PID in the long (non-truncated) format. <tt>/usr/bin/pstree</tt> comes with package psmisc.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Systemd in services startup.\n",
    "\n",
    "![](img/systemd_startup.jpeg)\n",
    "\n",
    "For technical details, read the man pages:\n",
    "```bash\n",
    "man init\n",
    "man systemd\n",
    "man systemctl\n",
    "```\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Administrative commands to start/stop services (Exercises)\n",
    "\n",
    "<b>Systemd</b> driven services are available in Ubuntu 18.04 and RedHat 7 and newer \n",
    "\n",
    "Command <tt>systemctl</tt> talks to <tt>systemd</tt>. To see all the services registered with <tt>systemd</tt>, run the following:\n",
    "```bash\n",
    "systemctl list-units --type service\n",
    "```\n",
    "To stop ```cron``` service, run\n",
    "```bash\n",
    "systemctl stop cron\n",
    "```\n",
    "To see the status of the service:\n",
    "```bash\n",
    "systemctl status cron\n",
    "```\n",
    "To start the service and see its status:\n",
    "```bash\n",
    "systemctl start cron\n",
    "systemctl status cron\n",
    "```\n",
    "\n",
    "You can check status of the services that are active\n",
    "```bash\n",
    "systemctl list-units --type service --state=active\n",
    "```\n",
    "running\n",
    "```bash\n",
    "systemctl list-units --type service --state=running\n",
    "```\n",
    "exited\n",
    "```bash\n",
    "systemctl list-units --type service --state=exited\n",
    "```\n",
    "failed\n",
    "```bash\n",
    "systemctl list-units --type service --state=failed\n",
    "```\n",
    "\n",
    "To see what services are enabled:\n",
    "```bash\n",
    "systemctl list-unit-files --type=service --state=enabled\n",
    "```\n",
    "***\n",
    "## Legacy System V services\n",
    "Used to be available on any Linux distro. Now they are provided within Systemd for backward compatibility.\n",
    "The System V (non-event driven) service startup scripts are located in directory <tt>/etc/init.d</tt>. For example to stop and then start cups printing service, use the following\n",
    "\n",
    "```bash\n",
    "apt install cups\n",
    "/etc/init.d/cups stop\n",
    "/etc/init.d/cups start\n",
    "/etc/init.d/cups status\n",
    "```\n",
    "Is the same as\n",
    "```bash\n",
    "service cups stop\n",
    "service cups start\n",
    "service cups status\n",
    "```\n",
    "\n",
    "<hr>\n",
    "\n",
    "## System Run Levels (Legacy System V)\n",
    "This is what used to be on older Linux distros.\n",
    "At bootup, init reads <tt>/etc/inittab</tt> file and executes all scripts for default run level. If the <tt>/etc/inittab</tt> is absent, the default run level is 2.\n",
    "On Unix, there are 7 conventional run levels (0,1,2,3,4,5,6). However, 10 levels are possible (0 -- 9).\n",
    "\n",
    "On Ubuntu and Debian,\n",
    "\n",
    "    Runlevel  0  - is halt.\n",
    "    Runlevel  1  - is single-user.\n",
    "    Runlevels 2-5  are multi-user.\n",
    "    Runlevel  6  - is reboot.\n",
    "\n",
    "On the other Linux systems, such as RedHat,\n",
    " \n",
    "    Runlevel  0 - halt\n",
    "    Runlevel  1 - Single user mode\n",
    "    Runlevel  2 - Multiuser, without NFS (The same as 3, if you do not have networking)\n",
    "    Runlevel  3 - Full multiuser mode\n",
    "    Runlevel  4 - unused\n",
    "    Runlevel  5 - X11\n",
    "    Runlevel  6 - reboot\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Comparison of SysV Runlevels with systemd Targets\n",
    "| Runlevel\t| Target Units | \tDescription \n",
    "| :- | :- | -: \n",
    "|0\t| runlevel0.target, poweroff.target\t| Shut down and power off the system.\n",
    "|1\t|runlevel1.target, rescue.target\t| Set up a rescue shell.\n",
    "|2\t|runlevel2.target, multi-user.target  | Set up a non-graphical multi-user system.\n",
    "|3\t|runlevel3.target, multi-user.target  | Set up a non-graphical multi-user system.\n",
    "|4\t|runlevel4.target, multi-user.target  |\tSet up a non-graphical multi-user system.\n",
    "|5\t|runlevel5.target, graphical.target\t| Set up a graphical multi-user system.\n",
    "|6\t|runlevel6.target, reboot.target\t| Shut down and reboot the system.\n",
    "\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Working with Systemd targets (Exercises)\n",
    "\n",
    "Viewing the Default Target:\n",
    "```bash\n",
    "systemctl get-default\n",
    "```\n",
    "Viewing the Current Target\n",
    "```bash\n",
    "systemctl list-units --type target\n",
    "```\n",
    "Changing the Default Target\n",
    "```bash\n",
    "systemctl set-default multi-user.target\n",
    "```\n",
    "Reboot. Run commands\n",
    "```bash\n",
    "systemctl get-default\n",
    "systemctl list-units --type target\n",
    "```\n",
    "Check the status of the targets:\n",
    "```bash\n",
    "systemctl status multi-user.target\n",
    "systemctl status graphical.target\n",
    "```\n",
    "\n",
    "\n",
    "Changing the Current Target\n",
    "```bash\n",
    "systemctl isolate graphical.target\n",
    "```\n",
    "Check the Target units\n",
    "```bash\n",
    "systemctl list-units --type target\n",
    "```\n",
    "Check the status of the targets:\n",
    "```bash\n",
    "systemctl status multi-user.target\n",
    "systemctl status graphical.target\n",
    "```\n",
    "\n",
    "You can see the target dependencies on the other targets and services with command ```systemctl list-dependencies <target name>```.\n",
    "For example, for the graphical target:\n",
    "```bash\n",
    "systemctl list-dependencies graphical.target\n",
    "```\n",
    "To see only the targets in the dependencies\n",
    "```bash\n",
    "systemctl list-dependencies graphical.target | grep target\n",
    "```\n",
    "\n",
    "Special target, Rescue, for fixing a system with minimum set of services running. Need to be root to login to these targets.\n",
    "Set password for root:\n",
    "```bash\n",
    "sudo -s\n",
    "passwd\n",
    "```\n",
    "\n",
    "Changing to Rescue Mode\n",
    "```bash\n",
    "systemctl rescue\n",
    "```\n",
    "Check the Target units\n",
    "```bash\n",
    "systemctl list-units --type target\n",
    "```\n",
    "Changing to Emergency Mode\n",
    "```bash\n",
    "systemctl emergency\n",
    "```\n",
    "Check the Target units\n",
    "```bash\n",
    "systemctl list-units --type target\n",
    "```\n",
    "Run command <tt>pstree</tt> to see the number of processes running.\n",
    "\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Adding service to systemd startup (Exercise)\n",
    "\n",
    "In directory <tt>/home/hostadm</tt>, create a shell script file, <tt>test.sh</tt>:\n",
    "```{admonition} File content:\n",
    "    #!/bin/bash\n",
    "    echo \"current date/time is $(/bin/date)\" >> /home/hostadm/timecheck.txt\n",
    "```\n",
    "\n",
    "Make the script executable\n",
    "```bash\n",
    "chmod 755  /home/hostadm/test.sh\n",
    "```\n",
    "\n",
    "In directory <tt>/lib/systemd/system</tt>, create unit service file, <tt>test.service</tt>:\n",
    "```{admonition} File content:\n",
    "    [Unit]\n",
    "    Description=test startup\n",
    "\n",
    "    [Service]\n",
    "    Type=simple\n",
    "    ExecStart=/home/hostadm/test.sh\n",
    "\n",
    "    [Install]\n",
    "    WantedBy=default.target\n",
    "```\n",
    "\n",
    "Enable service test in systemd:\n",
    "```bash\n",
    "systemctl daemon-reload\n",
    "systemctl enable test\n",
    "```\n",
    "Run commands\n",
    "```bash\n",
    "systemctl start test\n",
    "systemctl status test\n",
    "```\n",
    "Check if file <tt>/home/hostadm/timecheck.txt</tt> has been updated.\n",
    "Reboot the VM. Run command\n",
    "```bash\n",
    "systemctl status test\n",
    "```\n",
    "Check if file <tt>/home/hostadm/timecheck.txt</tt> has been updated.\n",
    "\n",
    "<hr>\n",
    "\n",
    "\n",
    "## Adding stop script to the unit (Exercise)\n",
    "\n",
    "In directory <tt>/home/hostadm</tt>, create another shell script file, <tt>stop.sh</tt>:\n",
    "\n",
    "```{admonition} script file:\n",
    "    #!/bin/bash\n",
    "    echo STOP >> /home/hostadm/timecheck.txt\n",
    "```\n",
    "Make the script executable\n",
    "```bash\n",
    "chmod 755  /home/hostadm/stop.sh\n",
    "```\n",
    "In directory <tt>/lib/systemd/system</tt>, modify the unit service file, <tt>test.service</tt> by adding two lines in the [Service] block:\n",
    "```{admonition} test.service:\n",
    "    [Unit]\n",
    "    Description=test startup\n",
    "\n",
    "    [Service]\n",
    "    Type=simple\n",
    "    ExecStart=/home/hostadm/test.sh\n",
    "    RemainAfterExit=true\n",
    "    ExecStop=/home/hostadm/stop.sh \n",
    "\n",
    "    [Install]\n",
    "    WantedBy=default.target\n",
    "```\n",
    "Reload systemd:\n",
    "```bash\n",
    "systemctl daemon-reload\n",
    "```\n",
    "\n",
    "Run commands\n",
    "```bash\n",
    "systemctl stop test\n",
    "systemctl status test\n",
    "```\n",
    "Check if file <tt>/home/hostadm/timecheck.txt</tt> has been updated with entry \"STOP\", generated by the <tt>stop.sh</tt> script.\n",
    "<hr>\n",
    "\n",
    "\n",
    "## User settings at login.\n",
    "\n",
    "When a user logins, getty gives him SHELL.\n",
    "\n",
    "In case of /bin/bash, the following initialization/configuration scripts are executed:\n",
    "\n",
    "| /etc/profile\t| System wide initialization file. Runs at log in.\n",
    "| :--            |  -:    |\n",
    "|~/.bash_profile  |\tExecuted automatically after /etc/profile\n",
    "|~/.bash_login\t| If ~/.bash_profile is absent, the script is executed after /etc/profile, otherwise, ignored.\n",
    "|~/.profile\t| If both ~/.bash_profile and ~/.bash_login are absent, the script is executed after /etc/profile, otherwise, ignored.\n",
    "|~/.bashrc\t | It is called from ~/.bash_profile at startup. An interactive shell that is not a login shell also reads ~/.bashrc .\n",
    "|~/.bash_logout\t| Executed automatically during log out\n",
    "\n",
    "\n",
    "When you need to specify environment variables, for example PATH, you can place them in /etc/profile for all users on the system, or in ~/.bashrc for the specific user.\n",
    "\n",
    "When the user logs out, control returns to init, which wakes up and spawns a new getty on the terminal port.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Login scripts  (Exercise).\n",
    "\n",
    "Complete the exercises below on run01 VM\n",
    "\n",
    "In a user home directory, there should be scripts .profile and .bashrc.\n",
    "Create a new user account, for example jerry:\n",
    "```bash\n",
    "useradd -m -s /bin/bash jerry\n",
    "passwd jerry\n",
    "```\n",
    "In file .profile, put entry\n",
    "```{admonition} file content\n",
    "echo PROFILE\n",
    "```\n",
    "\n",
    "In file .bashrc, put entry\n",
    "```{admonition} file content\n",
    "echo BASHRC\n",
    "```\n",
    "\n",
    "Logout and login as user jerry.\n",
    "You can also switch to user jerry from the root:\n",
    "```bash\n",
    "su - jerry\n",
    "```\n",
    "\n",
    "Notice PROFILE in the login prompt. \n",
    "\n",
    "Start a new shell,\n",
    "```bash\n",
    "/bin/bash\n",
    "```\n",
    "notice BASHRC in the new shell prompt.\n",
    "\n",
    "Logout from user jerry:\n",
    "```bash\n",
    "exit\n",
    "```\n",
    "\n",
    "<hr>\n",
    "\n",
    "\n",
    "## One time scheduled jobs: at\n",
    "To schedule a job to run once at a particular time, use command at time_to_run, for example, to run at 8:40 am:\n",
    "\n",
    "```bash\n",
    "at 0840\n",
    "```\n",
    "Then type-in commands and finish with ^D\n",
    "\n",
    "Time format examples:\n",
    "```bash\n",
    "at 1825 Mar 4\n",
    "at now + 1 minute\n",
    "at now + 1 hour\n",
    "at now + 1 day\n",
    "```\n",
    "\n",
    "To run an executable file or a script:\n",
    "```bash\n",
    "at -f filename time_to_run\n",
    "```\n",
    "For example,\n",
    "```bash\n",
    "at -f /home/jerry/script.sh 2030\n",
    "at -f /home/jerry/script.sh 11am Mar 30\n",
    "```\n",
    "Command at puts a scheduled job in a queue in <tt>/var/spool/at/</tt>; daemon <tt>/usr/sbin/atd</tt> checks for queued jobs in that directory and runs them.\n",
    "\n",
    "To check scheduled job status, run\n",
    "```bash\n",
    "atq\n",
    "```\n",
    "To delete scheduled job, run\n",
    "```bash\n",
    "atrm\n",
    "```\n",
    "\n",
    "Permssions on using at:\n",
    "\n",
    "If <tt>/etc/at.allow</tt> does not exist, <tt>/etc/at.deny</tt> is checked, every user name not mentioned in it is then allowed to use at.\n",
    "If neither exists, only the superuser is allowed use of at.\n",
    "An empty <tt>/etc/at.deny</tt> means that every user is allowed use these commands.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Scheduled periodical jobs: cron.\n",
    "\n",
    "To schedule jobs to execute repetitively at regular intervals, use cron. Daemon crond runs every minute and searches /var/spool/cron for crontab files and checks /etc/crontab and /etc/cron.d.\n",
    "/etc/crontab:\n",
    "\n",
    "```{admonition} cron file content\n",
    "    SHELL=/bin/sh\n",
    "    PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin\n",
    "    # m h dom mon dow user  command\n",
    "    17 *    * * *   root    run-parts --report /etc/cron.hourly\n",
    "    25 6    * * *   root    test -x /usr/sbin/anacron || run-parts --report /etc/cron.daily\n",
    "    47 6    * * 0   root    test -x /usr/sbin/anacron || run-parts --report /etc/cron.weekly\n",
    "    52 6    1 * *   root    test -x /usr/sbin/anacron || run-parts --report /etc/cron.monthly\n",
    "```\n",
    "\n",
    "Format: (time) (user) (executable)\n",
    "time: minute (0-59), hour(0-23), day of the month(1-31), month(1-12), week day (0-6) starting with Sun.\n",
    "Example:\n",
    "\n",
    "```{admonition} cron file content:\n",
    "    25 10 * * * root  /root/scheduled.sh   # runs /root/scheduled.sh every day at 10:25\n",
    "    25 10 * * 6 root  /root/scheduled.sh   # runs every Sat. at 10:25\n",
    "    */5 * * * 6 root  /root/scheduled.sh   # runs every 5 min on Sat.\n",
    "```\n",
    "\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Scheduling at and cron. (Exercises).\n",
    "Complete the exercises below on run01 VM\n",
    "\n",
    "Install package at:\n",
    "```bash\n",
    "apt install at\n",
    "```\n",
    "As user hostadm, Create script, <tt>mem.sh</tt>, to check memory status:\n",
    "```{admonition} file content\n",
    "    #!/bin/bash\n",
    "    # Checks memory status through cron\n",
    "\n",
    "    of=/home/hostadm/mem.out\n",
    "    dt=$(date)\n",
    "\n",
    "    echo \"Memory status (in MB) on $dt:\" >> $of\n",
    "    free -m >> $of\n",
    "    echo \"------------------\" >> $of\n",
    "\n",
    "```\n",
    "Make it executable\n",
    "```bash\n",
    "chmod 755 mem.sh\n",
    "```\n",
    "and run through at:\n",
    "```bash\n",
    "at -f mem.sh now + 1 minute\n",
    "```\n",
    "Run <tt>atq</tt> to make sure the job is scheduled. In a minute, check content of a new file, <tt>mem.out</tt>; run <tt>atq</tt> again.\n",
    "\n",
    "Remove file <tt>mem.out</tt>. Add the following entry in <tt>/etc/crontab</tt> below the other scheduled tasks:\n",
    "```{admonition} file content\n",
    "    */2 * * * *  root  /home/hostadm/mem.sh\n",
    "```\n",
    "Check if the content of <tt>mem.out</tt> is updated every 2 minutes.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## References on systemd and cron\n",
    "\n",
    "[Managing Services with systemd (RedHat documentation)](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system_administrators_guide/chap-managing_services_with_systemd)\n",
    "\n",
    "[Cron (Wikipedia)](https://en.wikipedia.org/wiki/Cron)\n",
    "\n",
    "[Crontab calculator](https://crontab.guru/)\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "remove-cell",
     "remove_cell"
    ]
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Both GOOGLE_API_KEY and GEMINI_API_KEY are set. Using GOOGLE_API_KEY.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# Service Management, Environment Files, and Job Scheduling\n",
      "\n",
      "- Bootup sequence\n",
      "- Systemd initialization\n",
      "- Starting and stopping services\n",
      "- Unit files\n",
      "- Environment files\n",
      "- Scheduling jobs with `at` and `cron`\n",
      "\n",
      "<hr>\n",
      "\n",
      "## Linux Bootup and Login Sequence\n",
      "\n",
      "1. The BIOS initiates, checks system peripherals, and identifies a bootable storage device.\n",
      "2. The system loads the first sector into RAM and executes it, displaying the GRUB bootloader prompt.\n",
      "3. The bootloader loads the Linux kernel from the sector list (`/boot/vmlinuz-7.0.0-15-generic`) and places the initial RAM disk image (`initrd`) into RAM.\n",
      "4. The kernel executes and unpacks its compressed image.\n",
      "5. The kernel initializes underlying hardware components.\n",
      "6. The kernel mounts the root (`/`) file system (for example, `/dev/sda1`).\n",
      "7. The kernel executes `init` (Systemd) as Process ID 1 (PID 1).\n",
      "8. Systemd initializes the units specified within the default target, which in turn starts core system services.\n",
      "9. Among these initialized services, `agetty` programs launch on each virtual terminal, initializing the login process.\n",
      "10. The `login` program requests a username and password, validating credentials against `/etc/passwd`, Network Information Service (NIS) maps, or Lightweight Directory Access Protocol (LDAP), depending on Pluggable Authentication Modules (PAM) settings.\n",
      "11. Upon successful authentication, `login` spawns an interactive user shell.\n",
      "12. The shell executes appropriate startup scripts (`.profile` for `sh` and `ksh`; `.bash_profile` for Bash; `.cshrc` and `.login` for `csh` and `tcsh`).\n",
      "13. The shell outputs an interactive command prompt and awaits user input.\n",
      "\n",
      "<hr>\n",
      "\n",
      "## Virtual Machine Setup for Startup Scripts (Exercise)\n",
      "\n",
      "Clone the template virtual machine `kvm1` to a new Network File System (NFS) server VM host named `run01` by executing the commands below:\n",
      "\n",
      "```bash\n",
      "virsh shutdown kvm1\n",
      "virt-clone -o kvm1 -n run01 -f /home/hostadm/KVM/run01.qcow2\n",
      "```\n",
      "\n",
      "Verify that the new VM appears in the guest list, boot it up, and connect to its console interface:\n",
      "\n",
      "```bash\n",
      "virsh list --all\n",
      "virsh start run01\n",
      "virsh console run01\n",
      "```\n",
      "\n",
      "Update the system hostname by replacing `kvm1` with `run01` inside the `/etc/hostname` file. \n",
      "\n",
      "Reset the unique machine identifier by running the helper script located on `kvm3`:\n",
      "\n",
      "```bash\n",
      "./vm_id_reset.sh\n",
      "```\n",
      "\n",
      "Reboot the virtual machine to apply changes:\n",
      "\n",
      "```bash\n",
      "reboot\n",
      "```\n",
      "\n",
      "Following the reboot cycle, the virtual machine should load successfully with the updated hostname.\n",
      "\n",
      "<hr>\n",
      "\n",
      "## Systemd as the Parent of All Processes (Exercise)\n",
      "\n",
      "Systemd serves as the primordial parent process for all user-space tasks on modern Linux systems. Executed directly by the kernel, it is responsible for launching background services and adopting orphaned processes whose original parent tasks have terminated, ensuring proper process resource reaping.\n",
      "\n",
      "Access the virtual machine console:\n",
      "\n",
      "```bash\n",
      "virsh console run01\n",
      "```\n",
      "\n",
      "Install the `psmisc` package and execute the `pstree` command to visualize the active process hierarchy:\n",
      "\n",
      "```bash\n",
      "sudo -s\n",
      "apt install psmisc\n",
      "pstree\n",
      "```\n",
      "\n",
      "```{admonition} Output:\n",
      "    systemd─┬─ModemManager───3*[{ModemManager}]\n",
      "            ├─accounts-daemon───3*[{accounts-daemon}]\n",
      "            ├─agetty\n",
      "            ├─atd\n",
      "            ├─cron\n",
      "            ├─dbus-daemon\n",
      "            ├─login───bash───pstree\n",
      "            ├─multipathd───6*[{multipathd}]\n",
      "            ├─networkd-dispat\n",
      "            ├─polkitd───3*[{polkitd}]\n",
      "            ├─rsyslogd───3*[{rsyslogd}]\n",
      "            ├─sshd\n",
      "            ├─systemd───(sd-pam)\n",
      "            ├─systemd-hostnam\n",
      "            ├─systemd-journal\n",
      "            ├─systemd-logind\n",
      "            ├─systemd-network\n",
      "            ├─systemd-resolve\n",
      "            ├─systemd-timesyn───{systemd-timesyn}\n",
      "            ├─systemd-udevd───10*[systemd-udevd]\n",
      "            ├─udisksd───5*[{udisksd}]\n",
      "            └─unattended-upgr───{unattended-upgr}\n",
      "```\n",
      "\n",
      "To display the process tree with corresponding Process IDs (PIDs) in an untruncated format, execute:\n",
      "\n",
      "```bash\n",
      "pstree -p -l\n",
      "```\n",
      "\n",
      "The `/usr/bin/pstree` utility is provided by the `psmisc` package.\n",
      "\n",
      "<hr>\n",
      "\n",
      "## Systemd in Service Startup\n",
      "\n",
      "```text\n",
      "+-------------------------------------------------------------+\n",
      "|                     Systemd Initialization                  |\n",
      "|          (Reads Unit Files & Resolves Dependencies)         |\n",
      "+------------------------------+------------------------------+\n",
      "                               |\n",
      "                               v\n",
      "+-------------------------------------------------------------+\n",
      "|                    Target Units Activation                  |\n",
      "|                 (Multi-User / Graphical Target)             |\n",
      "+------------------------------+------------------------------+\n",
      "                               |\n",
      "                               v\n",
      "+-------------------------------------------------------------+\n",
      "|                  Service Daemons & Daemons                  |\n",
      "|                (Network, SSH, Cron, Agetty)                 |\n",
      "+-------------------------------------------------------------+\n",
      "```\n",
      "\n",
      "![](img/systemd_startup.jpeg)\n",
      "\n",
      "**Accessible Image Description:**\n",
      "- **Overview:** This diagram illustrates the structural hierarchy of the Systemd service management and startup architecture.\n",
      "- **Setup:** The flowchart maps out how Systemd reads configuration unit files, resolves inter-service dependencies, initializes target environments, and launches active daemons.\n",
      "- **Key Takeaways:** Systemd manages services in an event-driven, parallelized manner to accelerate boot times compared to legacy sequential initialization scripts.\n",
      "\n",
      "For technical reference details, consult the manual pages:\n",
      "\n",
      "```bash\n",
      "man init\n",
      "man systemd\n",
      "man systemctl\n",
      "```\n",
      "\n",
      "<hr>\n",
      "\n",
      "## Administrative Commands to Start and Stop Services (Exercises)\n",
      "\n",
      "Systemd-driven services are standard across Ubuntu 18.04, Red Hat Enterprise Linux 7, and newer distributions.\n",
      "\n",
      "The `systemctl` utility interfaces directly with the Systemd manager. To list all services currently registered with Systemd, execute:\n",
      "\n",
      "```bash\n",
      "systemctl list-units --type service\n",
      "```\n",
      "\n",
      "To stop the `cron` background service, run:\n",
      "\n",
      "```bash\n",
      "systemctl stop cron\n",
      "```\n",
      "\n",
      "To verify the current operational status of the service:\n",
      "\n",
      "```bash\n",
      "systemctl status cron\n",
      "```\n",
      "\n",
      "To start the service and immediately inspect its state:\n",
      "\n",
      "```bash\n",
      "systemctl start cron\n",
      "systemctl status cron\n",
      "```\n",
      "\n",
      "You can filter service states using targeted list commands:\n",
      "\n",
      "- Active services:\n",
      "  ```bash\n",
      "  systemctl list-units --type service --state=active\n",
      "  ```\n",
      "- Running services:\n",
      "  ```bash\n",
      "  systemctl list-units --type service --state=running\n",
      "  ```\n",
      "- Exited services:\n",
      "  ```bash\n",
      "  systemctl list-units --type service --state=exited\n",
      "  ```\n",
      "- Failed services:\n",
      "  ```bash\n",
      "  systemctl list-units --type service --state=failed\n",
      "  ```\n",
      "\n",
      "To view services configured to start automatically at boot (enabled units):\n",
      "\n",
      "```bash\n",
      "systemctl list-unit-files --type=service --state=enabled\n",
      "```\n",
      "\n",
      "***\n",
      "\n",
      "## Legacy System V Services\n",
      "\n",
      "Legacy System V initialization scripts were historically used across all Unix-like distributions. Modern distributions retain them within Systemd for backward compatibility. \n",
      "\n",
      "System V non-event-driven startup scripts reside in the `/etc/init.d` directory. For example, to stop, start, and query the status of the CUPS printing service, use:\n",
      "\n",
      "```bash\n",
      "apt install cups\n",
      "/etc/init.d/cups stop\n",
      "/etc/init.d/cups start\n",
      "/etc/init.d/cups status\n",
      "```\n",
      "\n",
      "This is functionally equivalent to using the wrapper utility:\n",
      "\n",
      "```bash\n",
      "service cups stop\n",
      "service cups start\n",
      "service cups status\n",
      "```\n",
      "\n",
      "<hr>\n",
      "\n",
      "## System Run Levels (Legacy System V)\n",
      "\n",
      "Traditional UNIX environments relied on numbered run levels managed by the `init` process. Upon boot, `init` reads the `/etc/inittab` configuration file and executes initialization scripts associated with the designated default run level. If `/etc/inittab` is absent, the default run level defaults to `2`.\n",
      "\n",
      "Unix environments define 7 standard run levels (0 through 6), though up to 10 levels (0–9) are technically configurable.\n",
      "\n",
      "### Ubuntu and Debian Run Level Conventions:\n",
      "- **Runlevel 0:** System halt and power off.\n",
      "- **Runlevel 1:** Single-user administrative maintenance mode.\n",
      "- **Runlevels 2–5:** Multi-user operation modes.\n",
      "- **Runlevel 6:** System reboot.\n",
      "\n",
      "### Red Hat Enterprise Linux Run Level Conventions:\n",
      "- **Runlevel 0:** Halt.\n",
      "- **Runlevel 1:** Single-user mode.\n",
      "- **Runlevel 2:** Multi-user mode without network file systems (identical to level 3 if networking is unconfigured).\n",
      "- **Runlevel 3:** Full multi-user text mode with networking.\n",
      "- **Runlevel 4:** Unused / user-definable.\n",
      "- **Runlevel 5:** Multi-user graphical environment (X11).\n",
      "- **Runlevel 6:** Reboot.\n",
      "\n",
      "<hr>\n",
      "\n",
      "## Comparison of SysV Run Levels with Systemd Targets\n",
      "\n",
      "| Runlevel | Target Units | Description |\n",
      "| :--- | :--- | :--- |\n",
      "| **0** | `runlevel0.target`, `poweroff.target` | Shut down and power off the system. |\n",
      "| **1** | `runlevel1.target`, `rescue.target` | Initialize a single-user rescue shell environment. |\n",
      "| **2** | `runlevel2.target`, `multi-user.target` | Set up a non-graphical multi-user system. |\n",
      "| **3** | `runlevel3.target`, `multi-user.target` | Set up a non-graphical multi-user system. |\n",
      "| **4** | `runlevel4.target`, `multi-user.target` | Set up a non-graphical multi-user system. |\n",
      "| **5** | `runlevel5.target`, `graphical.target` | Set up a graphical multi-user environment. |\n",
      "| **6** | `runlevel6.target`, `reboot.target` | Shut down and restart the system. |\n",
      "\n",
      "<hr>\n",
      "\n",
      "## Working with Systemd Targets (Exercises)\n",
      "\n",
      "Query the default boot target:\n",
      "```bash\n",
      "systemctl get-default\n",
      "```\n",
      "\n",
      "List all currently loaded target units:\n",
      "```bash\n",
      "systemctl list-units --type target\n",
      "```\n",
      "\n",
      "Modify the default startup target:\n",
      "```bash\n",
      "systemctl set-default multi-user.target\n",
      "```\n",
      "\n",
      "Reboot the system, then verify your changes:\n",
      "```bash\n",
      "systemctl get-default\n",
      "systemctl list-units --type target\n",
      "```\n",
      "\n",
      "Inspect the active status of specific targets:\n",
      "```bash\n",
      "systemctl status multi-user.target\n",
      "systemctl status graphical.target\n",
      "```\n",
      "\n",
      "Switch the active target dynamically at runtime:\n",
      "```bash\n",
      "systemctl isolate graphical.target\n",
      "```\n",
      "\n",
      "Verify target units and their operational status:\n",
      "```bash\n",
      "systemctl list-units --type target\n",
      "systemctl status multi-user.target\n",
      "systemctl status graphical.target\n",
      "```\n",
      "\n",
      "You can inspect target dependencies using the `systemctl list-dependencies` command. For instance, to examine dependencies for the graphical target:\n",
      "\n",
      "```bash\n",
      "systemctl list-dependencies graphical.target\n",
      "```\n",
      "\n",
      "To filter the output and display only subordinate target units:\n",
      "\n",
      "```bash\n",
      "systemctl list-dependencies graphical.target | grep target\n",
      "```\n",
      "\n",
      "The Rescue target provides a specialized environment for system recovery with a minimal set of running services. Administrative access requires root privileges.\n",
      "\n",
      "Set a secure password for the root account:\n",
      "```bash\n",
      "sudo -s\n",
      "passwd\n",
      "```\n",
      "\n",
      "Switch dynamically into Rescue Mode:\n",
      "```bash\n",
      "systemctl rescue\n",
      "```\n",
      "\n",
      "Verify active target units:\n",
      "```bash\n",
      "systemctl list-units --type target\n",
      "```\n",
      "\n",
      "To switch to Emergency Mode (minimal kernel mounting):\n",
      "```bash\n",
      "systemctl emergency\n",
      "```\n",
      "\n",
      "Check the active target units:\n",
      "```bash\n",
      "systemctl list-units --type target\n",
      "```\n",
      "\n",
      "Run `pstree` to review the reduced set of active processes.\n",
      "\n",
      "<hr>\n",
      "\n",
      "## Adding a Service to Systemd Startup (Exercise)\n",
      "\n",
      "Create a custom shell script named `test.sh` in the `/home/hostadm` directory:\n",
      "\n",
      "```{admonition} File content:\n",
      "    #!/bin/bash\n",
      "    echo \"current date/time is $(/bin/date)\" >> /home/hostadm/timecheck.txt\n",
      "```\n",
      "\n",
      "Grant executable permissions to the script:\n",
      "\n",
      "```bash\n",
      "chmod 755 /home/hostadm/test.sh\n",
      "```\n",
      "\n",
      "Create a corresponding systemd unit service file named `test.service` in the `/lib/systemd/system` directory:\n",
      "\n",
      "```{admonition} File content:\n",
      "    [Unit]\n",
      "    Description=test startup\n",
      "\n",
      "    [Service]\n",
      "    Type=simple\n",
      "    ExecStart=/home/hostadm/test.sh\n",
      "\n",
      "    [Install]\n",
      "    WantedBy=default.target\n",
      "```\n",
      "\n",
      "Register and enable the new service within Systemd:\n",
      "\n",
      "```bash\n",
      "systemctl daemon-reload\n",
      "systemctl enable test\n",
      "```\n",
      "\n",
      "Start the service and review its execution status:\n",
      "\n",
      "```bash\n",
      "systemctl start test\n",
      "systemctl status test\n",
      "```\n",
      "\n",
      "Verify that the target output file `/home/hostadm/timecheck.txt` has been updated. Reboot the virtual machine and verify persistence:\n",
      "\n",
      "```bash\n",
      "systemctl status test\n",
      "```\n",
      "\n",
      "Check the contents of `/home/hostadm/timecheck.txt` to confirm successful execution during boot.\n",
      "\n",
      "<hr>\n",
      "\n",
      "## Adding a Stop Script to a Unit (Exercise)\n",
      "\n",
      "In the `/home/hostadm` directory, create a companion shutdown script named `stop.sh`:\n",
      "\n",
      "```{admonition} Script file:\n",
      "    #!/bin/bash\n",
      "    echo STOP >> /home/hostadm/timecheck.txt\n",
      "```\n",
      "\n",
      "Make the shutdown script executable:\n",
      "\n",
      "```bash\n",
      "chmod 755 /home/hostadm/stop.sh\n",
      "```\n",
      "\n",
      "Modify the existing `test.service` unit file in `/lib/systemd/system` by adding two directives to the `[Service]` block:\n",
      "\n",
      "```{admonition} test.service:\n",
      "    [Unit]\n",
      "    Description=test startup\n",
      "\n",
      "    [Service]\n",
      "    Type=simple\n",
      "    ExecStart=/home/hostadm/test.sh\n",
      "    RemainAfterExit=true\n",
      "    ExecStop=/home/hostadm/stop.sh \n",
      "\n",
      "    [Install]\n",
      "    WantedBy=default.target\n",
      "```\n",
      "\n",
      "Reload the Systemd manager configuration:\n",
      "\n",
      "```bash\n",
      "systemctl daemon-reload\n",
      "```\n",
      "\n",
      "Stop the service and query its operational status:\n",
      "\n",
      "```bash\n",
      "systemctl stop test\n",
      "systemctl status test\n",
      "```\n",
      "\n",
      "Verify that `/home/hostadm/timecheck.txt` now includes the \"STOP\" entry generated by `stop.sh`.\n",
      "\n",
      "<hr>\n",
      "\n",
      "## User Settings at Login\n",
      "\n",
      "When a user logs in, the `getty` process spawns an interactive shell environment. \n",
      "\n",
      "For the standard Bourne-again shell (`/bin/bash`), the system evaluates initialization scripts in the following sequence:\n",
      "\n",
      "| Configuration File | Description |\n",
      "| :--- | :--- |\n",
      "| `/etc/profile` | System-wide initialization file executed upon login. |\n",
      "| `~/.bash_profile` | Executed automatically after `/etc/profile`. |\n",
      "| `~/.bash_login` | Evaluated if `~/.bash_profile` is absent; otherwise, ignored. |\n",
      "| `~/.profile` | Executed if both `~/.bash_profile` and `~/.bash_login` are absent; otherwise, ignored. |\n",
      "| `~/.bashrc` | Sourced from `~/.bash_profile` at startup. Also executed for non-login interactive shells. |\n",
      "| `~/.bash_logout` | Executed automatically upon session logout. |\n",
      "\n",
      "Environment variables such as `PATH` can be defined globally in `/etc/profile` for all users, or locally in `~/.bashrc` for individual user accounts.\n",
      "\n",
      "When a user logs out, control returns to the `init` process, which respawns a fresh `getty` instance on the terminal port.\n",
      "\n",
      "<hr>\n",
      "\n",
      "## Login Scripts (Exercise)\n",
      "\n",
      "Complete the following exercises on the `run01` virtual machine.\n",
      "\n",
      "User home directories should include `.profile` and `.bashrc` configuration scripts. Create a new user account named `jerry`:\n",
      "\n",
      "```bash\n",
      "useradd -m -s /bin/bash jerry\n",
      "passwd jerry\n",
      "```\n",
      "\n",
      "Add an informational entry to user jerry's `.profile` file:\n",
      "\n",
      "```{admonition} File content\n",
      "echo PROFILE\n",
      "```\n",
      "\n",
      "Add an informational entry to user jerry's `.bashrc` file:\n",
      "\n",
      "```{admonition} File content\n",
      "echo BASHRC\n",
      "```\n",
      "\n",
      "Log out and log back in as user `jerry`. Alternatively, switch user contexts directly from the root account:\n",
      "\n",
      "```bash\n",
      "su - jerry\n",
      "```\n",
      "\n",
      "Observe that `PROFILE` appears in the login shell prompt.\n",
      "\n",
      "Start a subshell session:\n",
      "\n",
      "```bash\n",
      "/bin/bash\n",
      "```\n",
      "\n",
      "Notice that `BASHRC` outputs to the prompt for the newly spawned interactive subshell.\n",
      "\n",
      "Terminate the user session and return to the previous user context:\n",
      "\n",
      "```bash\n",
      "exit\n",
      "```\n",
      "\n",
      "<hr>\n",
      "\n",
      "## One-Time Scheduled Jobs: `at`\n",
      "\n",
      "To schedule a command or script to execute once at a specific time, use the `at` utility followed by the target execution time. For example, to schedule a task for 8:40 AM:\n",
      "\n",
      "```bash\n",
      "at 0840\n",
      "```\n",
      "\n",
      "Input your desired shell commands and conclude the stream with `Ctrl+D` (`^D`).\n",
      "\n",
      "Time specification examples:\n",
      "```bash\n",
      "at 1825 Mar 4\n",
      "at now + 1 minute\n",
      "at now + 1 hour\n",
      "at now + 1 day\n",
      "```\n",
      "\n",
      "To schedule the execution of an existing script file:\n",
      "\n",
      "```bash\n",
      "at -f filename time_to_run\n",
      "```\n",
      "\n",
      "For example:\n",
      "```bash\n",
      "at -f /home/jerry/script.sh 2030\n",
      "at -f /home/jerry/script.sh 11am Mar 30\n",
      "```\n",
      "\n",
      "The `at` command places queued tasks into the spool directory at `/var/spool/at/`. The background daemon `/usr/sbin/atd` monitors this directory and executes queued jobs at their designated times.\n",
      "\n",
      "To inspect the queue of pending scheduled jobs:\n",
      "\n",
      "```bash\n",
      "atq\n",
      "```\n",
      "\n",
      "To remove a scheduled job from the queue:\n",
      "\n",
      "```bash\n",
      "atrm <job_number>\n",
      "```\n",
      "\n",
      "### Access Control for `at`:\n",
      "- If `/etc/at.allow` exists, only users listed within it are permitted to use `at`.\n",
      "- If `/etc/at.allow` is absent, `/etc/at.deny` is checked; any user not listed within it is permitted access.\n",
      "- If neither file exists, execution rights are restricted exclusively to the superuser (`root`).\n",
      "- An empty `/etc/at.deny` file permits all system users to use scheduling commands.\n",
      "\n",
      "<hr>\n",
      "\n",
      "## Periodic Scheduled Jobs: `cron`\n",
      "\n",
      "To execute tasks repeatedly at defined intervals, use the `cron` daemon. The `crond` daemon evaluates system crontab files in `/var/spool/cron`, `/etc/crontab`, and directory `/etc/cron.d/` every minute.\n",
      "\n",
      "Example configuration from `/etc/crontab`:\n",
      "\n",
      "```{admonition} Cron file content\n",
      "    SHELL=/bin/sh\n",
      "    PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin\n",
      "    # m h dom mon dow user  command\n",
      "    17 *    * * *   root    run-parts --report /etc/cron.hourly\n",
      "    25 6    * * *   root    test -x /usr/sbin/anacron || run-parts --report /etc/cron.daily\n",
      "    47 6    * * 0   root    test -x /usr/sbin/anacron || run-parts --report /etc/cron.weekly\n",
      "    52 6    1 * *   root    test -x /usr/sbin/anacron || run-parts --report /etc/cron.monthly\n",
      "```\n",
      "\n",
      "### Syntax Format:\n",
      "`[Minute] [Hour] [Day_of_Month] [Month] [Day_of_Week] [User] [Command]`\n",
      "\n",
      "- **Minute:** `0` through `59`\n",
      "- **Hour:** `0` through `23`\n",
      "- **Day of Month:** `1` through `31`\n",
      "- **Month:** `1` through `12`\n",
      "- **Day of Week:** `0` through `6` (starting with Sunday)\n",
      "\n",
      "### Examples:\n",
      "```{admonition} Cron file content:\n",
      "    25 10 * * * root  /root/scheduled.sh   # Executes /root/scheduled.sh daily at 10:25 AM\n",
      "    25 10 * * 6 root  /root/scheduled.sh   # Executes every Saturday at 10:25 AM\n",
      "    */5 * * * 6 root  /root/scheduled.sh   # Executes every 5 minutes on Saturdays\n",
      "```\n",
      "\n",
      "<hr>\n",
      "\n",
      "## Scheduling with `at` and `cron` (Exercises)\n",
      "\n",
      "Complete the exercises below on the `run01` virtual machine.\n",
      "\n",
      "Install the `at` utility package:\n",
      "\n",
      "```bash\n",
      "apt install at\n",
      "```\n",
      "\n",
      "As the `hostadm` user, create a script named `mem.sh` to check system memory utilization:\n",
      "\n",
      "```{admonition} File content\n",
      "    #!/bin/bash\n",
      "    # Checks memory status through cron\n",
      "\n",
      "    of=/home/hostadm/mem.out\n",
      "    dt=$(date)\n",
      "\n",
      "    echo \"Memory status (in MB) on $dt:\" >> $of\n",
      "    free -m >> $of\n",
      "    echo \"------------------\" >> $of\n",
      "```\n",
      "\n",
      "Make the script executable:\n",
      "\n",
      "```bash\n",
      "chmod 755 mem.sh\n",
      "```\n",
      "\n",
      "Queue the script for execution using `at`:\n",
      "\n",
      "```bash\n",
      "at -f mem.sh now + 1 minute\n",
      "```\n",
      "\n",
      "Run `atq` to verify that the job is registered in the queue. After one minute, inspect the newly generated `mem.out` file and run `atq` again to confirm completion.\n",
      "\n",
      "Delete the `mem.out` output file. Next, append a periodic execution rule to the bottom of `/etc/crontab`:\n",
      "\n",
      "```{admonition} File content\n",
      "    */2 * * * *  root  /home/hostadm/mem.sh\n",
      "```\n",
      "\n",
      "Verify that the contents of `mem.out` update automatically every two minutes.\n",
      "\n",
      "<hr>\n",
      "\n",
      "## References on Systemd and Cron\n",
      "\n",
      "- [Managing Services with systemd (Red Hat Documentation)](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system_administrators_guide/chap-managing_services_with_systemd)\n",
      "- [Cron (Wikipedia)](https://en.wikipedia.org/wiki/Cron)\n",
      "- [Crontab Guru (Cron Schedule Expression Editor)](https://crontab.guru/)\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import time\n",
    "import base64 \n",
    "from google import genai\n",
    "from google.genai.errors import APIError\n",
    "\n",
    "os.environ[\"GEMINI_API_KEY\"] = \"AQ.Ab8RN6K9G5S-h6IWgpUgfmHbIk7b8iZlWTuYptMFqOT4Ph6FnQ\"\n",
    "client = genai.Client()\n",
    "\n",
    "# Use triple quotes \"\" \" so you can paste long paragraphs easily!\n",
    "\n",
    "prompt_instructions_simple = f\"Rewrite this text for a university course. Make it clear and write WCAG 2.1 AA compliant alt text descriptions for any charts mentioned: {my_notebook_text}\"\n",
    "        \n",
    "\n",
    "\n",
    "\n",
    "prompt_instructions_detailed5 = (\n",
    "    \"You are processing raw course text meant for Jupyter Notebook markdown cells.\\n\\n\"\n",
    "    \n",
    "    \"CRITICAL RULE: Do NOT delete, alter, or remove any original image links, image paths, \"\n",
    "    \"or image tags (e.g., lines starting with `![...](...)` or containing `<img ...>`). \"\n",
    "    \"Leave them exactly where they are in the original text sequence.\\n\\n\"\n",
    "    \n",
    "    \"INSTRUCTION:\\n\"\n",
    "    \"Rewrite the text to make it clear and university-appropriate.Make it clear and write WCAG 2.1 AA compliant. Use the following strict sequence \"\n",
    "    \"when dealing with image placements:\\n\\n\"\n",
    "    \n",
    "    \"1. First, insert the NEW Text-Based Structured Alternative ABOVE the image:\\n\"\n",
    "    \"   - For numerical data charts or grid plots: Provide a clean Markdown Data Table.\\n\"\n",
    "    \"   - For hardware architecture, network layouts, or OS Process lifecycles (like fork(), process states, execution threads): \"\n",
    "    \"Provide a high-level text flowchart inside a preformatted ```text code block.\\n\"\n",
    "    \"   - For Terminal Commands or Pipelines (like stdin/stdout piping, shell interactions): \"\n",
    "    \"Provide a vertical box-and-arrow sequence map illustrating data flow from one utility to the next.\\n\"\n",
    "    \"   - Keep diagram characters perfectly aligned and compatible with static HTML page exports.\\n\\n\"\n",
    "    \"   - CRITICAL RULE FOR VIRTUAL DESKTOP LOGIN CONTEXT:\\n\"\n",
    "    \"     If the text or diagram is about logging into the Virtual Desktop or using Ravada, \"\n",
    "    \"the text diagram sequence MUST start exactly with this workflow path:\\n\"\n",
    "    \"     [Web access to Ravada portal with authentication] ===> [Start VM] ===> [Access VM via SSH with provided port]\\n\\n\"\n",
    "    \n",
    "    \"2. Second, place the ORIGINAL untouched image link exactly as it was.\\n\\n\"\n",
    "    \n",
    "    \"3. Third, immediately BELOW the original image link, add the Comprehensive Detailed Description:\\n\"\n",
    "    \"Inside this caption block, you MUST start with the exact text '**Accessible Image Description:**'. Place it always under the image.\"\n",
    "    \"   - Overview: One sentence describing what the visual shows at a high level.\\n\"\n",
    "    \"   - Setup: Explain the primary components, labels, axes, or code pathways without hyper-granular specs.\\n\"\n",
    "    \"   - Key Takeaways: Core conceptual trends or execution results shown.\\n\\n\"\n",
    "\n",
    "    f\"Course Text:\\n{my_notebook_text}\"\n",
    ")\n",
    "prompt_instructions = prompt_instructions_detailed5\n",
    "\n",
    "# Trying the 2.5-flash model which often has lower server congestion\n",
    "try:\n",
    "    response = client.models.generate_content(\n",
    "        model='models/gemini-3.5-flash-lite',\n",
    "        contents= prompt_instructions,\n",
    "        )\n",
    "\n",
    "    #print(response.text)\n",
    "\n",
    "    output_text = response.text\n",
    "    print(output_text)\n",
    "    \n",
    "    # --- THE MAGIC TRICK ---\n",
    "    # This forces Jupyter to create a new cell below and paste the AI response inside it!\n",
    "    #get_ipython().set_next_input(response.text)\n",
    "    \n",
    "except APIError as e:\n",
    "    if e.code == 503:\n",
    "        print(\"Servers are still busy. Waiting 5 seconds before a quick automatic retry...\")\n",
    "        time.sleep(5)\n",
    "        # Automatic second attempt\n",
    "        response = client.models.generate_content(\n",
    "            model='models/gemini-3.5-flash-lite',\n",
    "            contents= prompt_instructions,\n",
    "            )\n",
    "        print(response.text)\n",
    "        # --- THE MAGIC TRICK ---\n",
    "        # This forces Jupyter to create a new cell below and paste the AI response inside it!\n",
    "        #get_ipython().set_next_input(response.text)\n",
    "    else:\n",
    "        raise e"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "# Service Management, Environment Files, and Job Scheduling\n",
    "\n",
    "- Bootup sequence\n",
    "- Systemd initialization\n",
    "- Starting and stopping services\n",
    "- Unit files\n",
    "- Environment files\n",
    "- Scheduling jobs with `at` and `cron`\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Linux Bootup and Login Sequence\n",
    "\n",
    "1. The BIOS initiates, checks system peripherals, and identifies a bootable storage device.\n",
    "2. The system loads the first sector into RAM and executes it, displaying the GRUB bootloader prompt.\n",
    "3. The bootloader loads the Linux kernel from the sector list (`/boot/vmlinuz-7.0.0-15-generic`) and places the initial RAM disk image (`initrd`) into RAM.\n",
    "4. The kernel executes and unpacks its compressed image.\n",
    "5. The kernel initializes underlying hardware components.\n",
    "6. The kernel mounts the root (`/`) file system (for example, `/dev/sda1`).\n",
    "7. The kernel executes `init` (Systemd) as Process ID 1 (PID 1).\n",
    "8. Systemd initializes the units specified within the default target, which in turn starts core system services.\n",
    "9. Among these initialized services, `agetty` programs launch on each virtual terminal, initializing the login process.\n",
    "10. The `login` program requests a username and password, validating credentials against `/etc/passwd`, Network Information Service (NIS) maps, or Lightweight Directory Access Protocol (LDAP), depending on Pluggable Authentication Modules (PAM) settings.\n",
    "11. Upon successful authentication, `login` spawns an interactive user shell.\n",
    "12. The shell executes appropriate startup scripts (`.profile` for `sh` and `ksh`; `.bash_profile` for Bash; `.cshrc` and `.login` for `csh` and `tcsh`).\n",
    "13. The shell outputs an interactive command prompt and awaits user input.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Virtual Machine Setup for Startup Scripts (Exercise)\n",
    "\n",
    "Clone the template virtual machine `kvm1` to a new Network File System (NFS) server VM host named `run01` by executing the commands below:\n",
    "\n",
    "```bash\n",
    "virsh shutdown kvm1\n",
    "virt-clone -o kvm1 -n run01 -f /home/hostadm/KVM/run01.qcow2\n",
    "```\n",
    "\n",
    "Verify that the new VM appears in the guest list, boot it up, and connect to its console interface:\n",
    "\n",
    "```bash\n",
    "virsh list --all\n",
    "virsh start run01\n",
    "virsh console run01\n",
    "```\n",
    "\n",
    "Update the system hostname by replacing `kvm1` with `run01` inside the `/etc/hostname` file. \n",
    "\n",
    "Reset the unique machine identifier by running the helper script located on `kvm3`:\n",
    "\n",
    "```bash\n",
    "./vm_id_reset.sh\n",
    "```\n",
    "\n",
    "Reboot the virtual machine to apply changes:\n",
    "\n",
    "```bash\n",
    "reboot\n",
    "```\n",
    "\n",
    "Following the reboot cycle, the virtual machine should load successfully with the updated hostname.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Systemd as the Parent of All Processes (Exercise)\n",
    "\n",
    "Systemd serves as the primordial parent process for all user-space tasks on modern Linux systems. Executed directly by the kernel, it is responsible for launching background services and adopting orphaned processes whose original parent tasks have terminated, ensuring proper process resource reaping.\n",
    "\n",
    "Access the virtual machine console:\n",
    "\n",
    "```bash\n",
    "virsh console run01\n",
    "```\n",
    "\n",
    "Install the `psmisc` package and execute the `pstree` command to visualize the active process hierarchy:\n",
    "\n",
    "```bash\n",
    "sudo -s\n",
    "apt install psmisc\n",
    "pstree\n",
    "```\n",
    "\n",
    "```{admonition} Output:\n",
    "    systemd─┬─ModemManager───3*[{ModemManager}]\n",
    "            ├─accounts-daemon───3*[{accounts-daemon}]\n",
    "            ├─agetty\n",
    "            ├─atd\n",
    "            ├─cron\n",
    "            ├─dbus-daemon\n",
    "            ├─login───bash───pstree\n",
    "            ├─multipathd───6*[{multipathd}]\n",
    "            ├─networkd-dispat\n",
    "            ├─polkitd───3*[{polkitd}]\n",
    "            ├─rsyslogd───3*[{rsyslogd}]\n",
    "            ├─sshd\n",
    "            ├─systemd───(sd-pam)\n",
    "            ├─systemd-hostnam\n",
    "            ├─systemd-journal\n",
    "            ├─systemd-logind\n",
    "            ├─systemd-network\n",
    "            ├─systemd-resolve\n",
    "            ├─systemd-timesyn───{systemd-timesyn}\n",
    "            ├─systemd-udevd───10*[systemd-udevd]\n",
    "            ├─udisksd───5*[{udisksd}]\n",
    "            └─unattended-upgr───{unattended-upgr}\n",
    "```\n",
    "\n",
    "To display the process tree with corresponding Process IDs (PIDs) in an untruncated format, execute:\n",
    "\n",
    "```bash\n",
    "pstree -p -l\n",
    "```\n",
    "\n",
    "The `/usr/bin/pstree` utility is provided by the `psmisc` package.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Systemd in Service Startup\n",
    "\n",
    "```text\n",
    "+-------------------------------------------------------------+\n",
    "|                     Systemd Initialization                  |\n",
    "|          (Reads Unit Files & Resolves Dependencies)         |\n",
    "+------------------------------+------------------------------+\n",
    "                               |\n",
    "                               v\n",
    "+-------------------------------------------------------------+\n",
    "|                    Target Units Activation                  |\n",
    "|                 (Multi-User / Graphical Target)             |\n",
    "+------------------------------+------------------------------+\n",
    "                               |\n",
    "                               v\n",
    "+-------------------------------------------------------------+\n",
    "|                  Service Daemons & Daemons                  |\n",
    "|                (Network, SSH, Cron, Agetty)                 |\n",
    "+-------------------------------------------------------------+\n",
    "```\n",
    "\n",
    "![](img/systemd_startup.jpeg)\n",
    "\n",
    "**Accessible Image Description:**\n",
    "- **Overview:** This diagram illustrates the structural hierarchy of the Systemd service management and startup architecture.\n",
    "- **Setup:** The flowchart maps out how Systemd reads configuration unit files, resolves inter-service dependencies, initializes target environments, and launches active daemons.\n",
    "- **Key Takeaways:** Systemd manages services in an event-driven, parallelized manner to accelerate boot times compared to legacy sequential initialization scripts.\n",
    "\n",
    "For technical reference details, consult the manual pages:\n",
    "\n",
    "```bash\n",
    "man init\n",
    "man systemd\n",
    "man systemctl\n",
    "```\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Administrative Commands to Start and Stop Services (Exercises)\n",
    "\n",
    "Systemd-driven services are standard across Ubuntu 18.04, Red Hat Enterprise Linux 7, and newer distributions.\n",
    "\n",
    "The `systemctl` utility interfaces directly with the Systemd manager. To list all services currently registered with Systemd, execute:\n",
    "\n",
    "```bash\n",
    "systemctl list-units --type service\n",
    "```\n",
    "\n",
    "To stop the `cron` background service, run:\n",
    "\n",
    "```bash\n",
    "systemctl stop cron\n",
    "```\n",
    "\n",
    "To verify the current operational status of the service:\n",
    "\n",
    "```bash\n",
    "systemctl status cron\n",
    "```\n",
    "\n",
    "To start the service and immediately inspect its state:\n",
    "\n",
    "```bash\n",
    "systemctl start cron\n",
    "systemctl status cron\n",
    "```\n",
    "\n",
    "You can filter service states using targeted list commands:\n",
    "\n",
    "- Active services:\n",
    "  ```bash\n",
    "  systemctl list-units --type service --state=active\n",
    "  ```\n",
    "- Running services:\n",
    "  ```bash\n",
    "  systemctl list-units --type service --state=running\n",
    "  ```\n",
    "- Exited services:\n",
    "  ```bash\n",
    "  systemctl list-units --type service --state=exited\n",
    "  ```\n",
    "- Failed services:\n",
    "  ```bash\n",
    "  systemctl list-units --type service --state=failed\n",
    "  ```\n",
    "\n",
    "To view services configured to start automatically at boot (enabled units):\n",
    "\n",
    "```bash\n",
    "systemctl list-unit-files --type=service --state=enabled\n",
    "```\n",
    "\n",
    "***\n",
    "\n",
    "## Legacy System V Services\n",
    "\n",
    "Legacy System V initialization scripts were historically used across all Unix-like distributions. Modern distributions retain them within Systemd for backward compatibility. \n",
    "\n",
    "System V non-event-driven startup scripts reside in the `/etc/init.d` directory. For example, to stop, start, and query the status of the CUPS printing service, use:\n",
    "\n",
    "```bash\n",
    "apt install cups\n",
    "/etc/init.d/cups stop\n",
    "/etc/init.d/cups start\n",
    "/etc/init.d/cups status\n",
    "```\n",
    "\n",
    "This is functionally equivalent to using the wrapper utility:\n",
    "\n",
    "```bash\n",
    "service cups stop\n",
    "service cups start\n",
    "service cups status\n",
    "```\n",
    "\n",
    "<hr>\n",
    "\n",
    "## System Run Levels (Legacy System V)\n",
    "\n",
    "Traditional UNIX environments relied on numbered run levels managed by the `init` process. Upon boot, `init` reads the `/etc/inittab` configuration file and executes initialization scripts associated with the designated default run level. If `/etc/inittab` is absent, the default run level defaults to `2`.\n",
    "\n",
    "Unix environments define 7 standard run levels (0 through 6), though up to 10 levels (0–9) are technically configurable.\n",
    "\n",
    "### Ubuntu and Debian Run Level Conventions:\n",
    "- **Runlevel 0:** System halt and power off.\n",
    "- **Runlevel 1:** Single-user administrative maintenance mode.\n",
    "- **Runlevels 2–5:** Multi-user operation modes.\n",
    "- **Runlevel 6:** System reboot.\n",
    "\n",
    "### Red Hat Enterprise Linux Run Level Conventions:\n",
    "- **Runlevel 0:** Halt.\n",
    "- **Runlevel 1:** Single-user mode.\n",
    "- **Runlevel 2:** Multi-user mode without network file systems (identical to level 3 if networking is unconfigured).\n",
    "- **Runlevel 3:** Full multi-user text mode with networking.\n",
    "- **Runlevel 4:** Unused / user-definable.\n",
    "- **Runlevel 5:** Multi-user graphical environment (X11).\n",
    "- **Runlevel 6:** Reboot.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Comparison of SysV Run Levels with Systemd Targets\n",
    "\n",
    "| Runlevel | Target Units | Description |\n",
    "| :--- | :--- | :--- |\n",
    "| **0** | `runlevel0.target`, `poweroff.target` | Shut down and power off the system. |\n",
    "| **1** | `runlevel1.target`, `rescue.target` | Initialize a single-user rescue shell environment. |\n",
    "| **2** | `runlevel2.target`, `multi-user.target` | Set up a non-graphical multi-user system. |\n",
    "| **3** | `runlevel3.target`, `multi-user.target` | Set up a non-graphical multi-user system. |\n",
    "| **4** | `runlevel4.target`, `multi-user.target` | Set up a non-graphical multi-user system. |\n",
    "| **5** | `runlevel5.target`, `graphical.target` | Set up a graphical multi-user environment. |\n",
    "| **6** | `runlevel6.target`, `reboot.target` | Shut down and restart the system. |\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Working with Systemd Targets (Exercises)\n",
    "\n",
    "Query the default boot target:\n",
    "```bash\n",
    "systemctl get-default\n",
    "```\n",
    "\n",
    "List all currently loaded target units:\n",
    "```bash\n",
    "systemctl list-units --type target\n",
    "```\n",
    "\n",
    "Modify the default startup target:\n",
    "```bash\n",
    "systemctl set-default multi-user.target\n",
    "```\n",
    "\n",
    "Reboot the system, then verify your changes:\n",
    "```bash\n",
    "systemctl get-default\n",
    "systemctl list-units --type target\n",
    "```\n",
    "\n",
    "Inspect the active status of specific targets:\n",
    "```bash\n",
    "systemctl status multi-user.target\n",
    "systemctl status graphical.target\n",
    "```\n",
    "\n",
    "Switch the active target dynamically at runtime:\n",
    "```bash\n",
    "systemctl isolate graphical.target\n",
    "```\n",
    "\n",
    "Verify target units and their operational status:\n",
    "```bash\n",
    "systemctl list-units --type target\n",
    "systemctl status multi-user.target\n",
    "systemctl status graphical.target\n",
    "```\n",
    "\n",
    "You can inspect target dependencies using the `systemctl list-dependencies` command. For instance, to examine dependencies for the graphical target:\n",
    "\n",
    "```bash\n",
    "systemctl list-dependencies graphical.target\n",
    "```\n",
    "\n",
    "To filter the output and display only subordinate target units:\n",
    "\n",
    "```bash\n",
    "systemctl list-dependencies graphical.target | grep target\n",
    "```\n",
    "\n",
    "The Rescue target provides a specialized environment for system recovery with a minimal set of running services. Administrative access requires root privileges.\n",
    "\n",
    "Set a secure password for the root account:\n",
    "```bash\n",
    "sudo -s\n",
    "passwd\n",
    "```\n",
    "\n",
    "Switch dynamically into Rescue Mode:\n",
    "```bash\n",
    "systemctl rescue\n",
    "```\n",
    "\n",
    "Verify active target units:\n",
    "```bash\n",
    "systemctl list-units --type target\n",
    "```\n",
    "\n",
    "To switch to Emergency Mode (minimal kernel mounting):\n",
    "```bash\n",
    "systemctl emergency\n",
    "```\n",
    "\n",
    "Check the active target units:\n",
    "```bash\n",
    "systemctl list-units --type target\n",
    "```\n",
    "\n",
    "Run `pstree` to review the reduced set of active processes.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Adding a Service to Systemd Startup (Exercise)\n",
    "\n",
    "Create a custom shell script named `test.sh` in the `/home/hostadm` directory:\n",
    "\n",
    "```{admonition} File content:\n",
    "    #!/bin/bash\n",
    "    echo \"current date/time is $(/bin/date)\" >> /home/hostadm/timecheck.txt\n",
    "```\n",
    "\n",
    "Grant executable permissions to the script:\n",
    "\n",
    "```bash\n",
    "chmod 755 /home/hostadm/test.sh\n",
    "```\n",
    "\n",
    "Create a corresponding systemd unit service file named `test.service` in the `/lib/systemd/system` directory:\n",
    "\n",
    "```{admonition} File content:\n",
    "    [Unit]\n",
    "    Description=test startup\n",
    "\n",
    "    [Service]\n",
    "    Type=simple\n",
    "    ExecStart=/home/hostadm/test.sh\n",
    "\n",
    "    [Install]\n",
    "    WantedBy=default.target\n",
    "```\n",
    "\n",
    "Register and enable the new service within Systemd:\n",
    "\n",
    "```bash\n",
    "systemctl daemon-reload\n",
    "systemctl enable test\n",
    "```\n",
    "\n",
    "Start the service and review its execution status:\n",
    "\n",
    "```bash\n",
    "systemctl start test\n",
    "systemctl status test\n",
    "```\n",
    "\n",
    "Verify that the target output file `/home/hostadm/timecheck.txt` has been updated. Reboot the virtual machine and verify persistence:\n",
    "\n",
    "```bash\n",
    "systemctl status test\n",
    "```\n",
    "\n",
    "Check the contents of `/home/hostadm/timecheck.txt` to confirm successful execution during boot.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Adding a Stop Script to a Unit (Exercise)\n",
    "\n",
    "In the `/home/hostadm` directory, create a companion shutdown script named `stop.sh`:\n",
    "\n",
    "```{admonition} Script file:\n",
    "    #!/bin/bash\n",
    "    echo STOP >> /home/hostadm/timecheck.txt\n",
    "```\n",
    "\n",
    "Make the shutdown script executable:\n",
    "\n",
    "```bash\n",
    "chmod 755 /home/hostadm/stop.sh\n",
    "```\n",
    "\n",
    "Modify the existing `test.service` unit file in `/lib/systemd/system` by adding two directives to the `[Service]` block:\n",
    "\n",
    "```{admonition} test.service:\n",
    "    [Unit]\n",
    "    Description=test startup\n",
    "\n",
    "    [Service]\n",
    "    Type=simple\n",
    "    ExecStart=/home/hostadm/test.sh\n",
    "    RemainAfterExit=true\n",
    "    ExecStop=/home/hostadm/stop.sh \n",
    "\n",
    "    [Install]\n",
    "    WantedBy=default.target\n",
    "```\n",
    "\n",
    "Reload the Systemd manager configuration:\n",
    "\n",
    "```bash\n",
    "systemctl daemon-reload\n",
    "```\n",
    "\n",
    "Stop the service and query its operational status:\n",
    "\n",
    "```bash\n",
    "systemctl stop test\n",
    "systemctl status test\n",
    "```\n",
    "\n",
    "Verify that `/home/hostadm/timecheck.txt` now includes the \"STOP\" entry generated by `stop.sh`.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## User Settings at Login\n",
    "\n",
    "When a user logs in, the `getty` process spawns an interactive shell environment. \n",
    "\n",
    "For the standard Bourne-again shell (`/bin/bash`), the system evaluates initialization scripts in the following sequence:\n",
    "\n",
    "| Configuration File | Description |\n",
    "| :--- | :--- |\n",
    "| `/etc/profile` | System-wide initialization file executed upon login. |\n",
    "| `~/.bash_profile` | Executed automatically after `/etc/profile`. |\n",
    "| `~/.bash_login` | Evaluated if `~/.bash_profile` is absent; otherwise, ignored. |\n",
    "| `~/.profile` | Executed if both `~/.bash_profile` and `~/.bash_login` are absent; otherwise, ignored. |\n",
    "| `~/.bashrc` | Sourced from `~/.bash_profile` at startup. Also executed for non-login interactive shells. |\n",
    "| `~/.bash_logout` | Executed automatically upon session logout. |\n",
    "\n",
    "Environment variables such as `PATH` can be defined globally in `/etc/profile` for all users, or locally in `~/.bashrc` for individual user accounts.\n",
    "\n",
    "When a user logs out, control returns to the `init` process, which respawns a fresh `getty` instance on the terminal port.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Login Scripts (Exercise)\n",
    "\n",
    "Complete the following exercises on the `run01` virtual machine.\n",
    "\n",
    "User home directories should include `.profile` and `.bashrc` configuration scripts. Create a new user account named `jerry`:\n",
    "\n",
    "```bash\n",
    "useradd -m -s /bin/bash jerry\n",
    "passwd jerry\n",
    "```\n",
    "\n",
    "Add an informational entry to user jerry's `.profile` file:\n",
    "\n",
    "```{admonition} File content\n",
    "echo PROFILE\n",
    "```\n",
    "\n",
    "Add an informational entry to user jerry's `.bashrc` file:\n",
    "\n",
    "```{admonition} File content\n",
    "echo BASHRC\n",
    "```\n",
    "\n",
    "Log out and log back in as user `jerry`. Alternatively, switch user contexts directly from the root account:\n",
    "\n",
    "```bash\n",
    "su - jerry\n",
    "```\n",
    "\n",
    "Observe that `PROFILE` appears in the login shell prompt.\n",
    "\n",
    "Start a subshell session:\n",
    "\n",
    "```bash\n",
    "/bin/bash\n",
    "```\n",
    "\n",
    "Notice that `BASHRC` outputs to the prompt for the newly spawned interactive subshell.\n",
    "\n",
    "Terminate the user session and return to the previous user context:\n",
    "\n",
    "```bash\n",
    "exit\n",
    "```\n",
    "\n",
    "<hr>\n",
    "\n",
    "## One-Time Scheduled Jobs: `at`\n",
    "\n",
    "To schedule a command or script to execute once at a specific time, use the `at` utility followed by the target execution time. For example, to schedule a task for 8:40 AM:\n",
    "\n",
    "```bash\n",
    "at 0840\n",
    "```\n",
    "\n",
    "Input your desired shell commands and conclude the stream with `Ctrl+D` (`^D`).\n",
    "\n",
    "Time specification examples:\n",
    "```bash\n",
    "at 1825 Mar 4\n",
    "at now + 1 minute\n",
    "at now + 1 hour\n",
    "at now + 1 day\n",
    "```\n",
    "\n",
    "To schedule the execution of an existing script file:\n",
    "\n",
    "```bash\n",
    "at -f filename time_to_run\n",
    "```\n",
    "\n",
    "For example:\n",
    "```bash\n",
    "at -f /home/jerry/script.sh 2030\n",
    "at -f /home/jerry/script.sh 11am Mar 30\n",
    "```\n",
    "\n",
    "The `at` command places queued tasks into the spool directory at `/var/spool/at/`. The background daemon `/usr/sbin/atd` monitors this directory and executes queued jobs at their designated times.\n",
    "\n",
    "To inspect the queue of pending scheduled jobs:\n",
    "\n",
    "```bash\n",
    "atq\n",
    "```\n",
    "\n",
    "To remove a scheduled job from the queue:\n",
    "\n",
    "```bash\n",
    "atrm <job_number>\n",
    "```\n",
    "\n",
    "### Access Control for `at`:\n",
    "- If `/etc/at.allow` exists, only users listed within it are permitted to use `at`.\n",
    "- If `/etc/at.allow` is absent, `/etc/at.deny` is checked; any user not listed within it is permitted access.\n",
    "- If neither file exists, execution rights are restricted exclusively to the superuser (`root`).\n",
    "- An empty `/etc/at.deny` file permits all system users to use scheduling commands.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Periodic Scheduled Jobs: `cron`\n",
    "\n",
    "To execute tasks repeatedly at defined intervals, use the `cron` daemon. The `crond` daemon evaluates system crontab files in `/var/spool/cron`, `/etc/crontab`, and directory `/etc/cron.d/` every minute.\n",
    "\n",
    "Example configuration from `/etc/crontab`:\n",
    "\n",
    "```{admonition} Cron file content\n",
    "    SHELL=/bin/sh\n",
    "    PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin\n",
    "    # m h dom mon dow user  command\n",
    "    17 *    * * *   root    run-parts --report /etc/cron.hourly\n",
    "    25 6    * * *   root    test -x /usr/sbin/anacron || run-parts --report /etc/cron.daily\n",
    "    47 6    * * 0   root    test -x /usr/sbin/anacron || run-parts --report /etc/cron.weekly\n",
    "    52 6    1 * *   root    test -x /usr/sbin/anacron || run-parts --report /etc/cron.monthly\n",
    "```\n",
    "\n",
    "### Syntax Format:\n",
    "`[Minute] [Hour] [Day_of_Month] [Month] [Day_of_Week] [User] [Command]`\n",
    "\n",
    "- **Minute:** `0` through `59`\n",
    "- **Hour:** `0` through `23`\n",
    "- **Day of Month:** `1` through `31`\n",
    "- **Month:** `1` through `12`\n",
    "- **Day of Week:** `0` through `6` (starting with Sunday)\n",
    "\n",
    "### Examples:\n",
    "```{admonition} Cron file content:\n",
    "    25 10 * * * root  /root/scheduled.sh   # Executes /root/scheduled.sh daily at 10:25 AM\n",
    "    25 10 * * 6 root  /root/scheduled.sh   # Executes every Saturday at 10:25 AM\n",
    "    */5 * * * 6 root  /root/scheduled.sh   # Executes every 5 minutes on Saturdays\n",
    "```\n",
    "\n",
    "<hr>\n",
    "\n",
    "## Scheduling with `at` and `cron` (Exercises)\n",
    "\n",
    "Complete the exercises below on the `run01` virtual machine.\n",
    "\n",
    "Install the `at` utility package:\n",
    "\n",
    "```bash\n",
    "apt install at\n",
    "```\n",
    "\n",
    "As the `hostadm` user, create a script named `mem.sh` to check system memory utilization:\n",
    "\n",
    "```{admonition} File content\n",
    "    #!/bin/bash\n",
    "    # Checks memory status through cron\n",
    "\n",
    "    of=/home/hostadm/mem.out\n",
    "    dt=$(date)\n",
    "\n",
    "    echo \"Memory status (in MB) on $dt:\" >> $of\n",
    "    free -m >> $of\n",
    "    echo \"------------------\" >> $of\n",
    "```\n",
    "\n",
    "Make the script executable:\n",
    "\n",
    "```bash\n",
    "chmod 755 mem.sh\n",
    "```\n",
    "\n",
    "Queue the script for execution using `at`:\n",
    "\n",
    "```bash\n",
    "at -f mem.sh now + 1 minute\n",
    "```\n",
    "\n",
    "Run `atq` to verify that the job is registered in the queue. After one minute, inspect the newly generated `mem.out` file and run `atq` again to confirm completion.\n",
    "\n",
    "Delete the `mem.out` output file. Next, append a periodic execution rule to the bottom of `/etc/crontab`:\n",
    "\n",
    "```{admonition} File content\n",
    "    */2 * * * *  root  /home/hostadm/mem.sh\n",
    "```\n",
    "\n",
    "Verify that the contents of `mem.out` update automatically every two minutes.\n",
    "\n",
    "<hr>\n",
    "\n",
    "## References on Systemd and Cron\n",
    "\n",
    "- [Managing Services with systemd (Red Hat Documentation)](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system_administrators_guide/chap-managing_services_with_systemd)\n",
    "- [Cron (Wikipedia)](https://en.wikipedia.org/wiki/Cron)\n",
    "- [Crontab Guru (Cron Schedule Expression Editor)](https://crontab.guru/)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
