{ "cells": [ { "cell_type": "markdown", "id": "21df53b0", "metadata": {}, "source": [ "## More Python modules\n", "\n", "- ```gzip``` - compresses/decompresses binary data, reads/writes data into *.gzip file.\n", "- ```zipfile``` - creates, adds, reads, writes to zip archives.\n", "- ```getpass``` - reads password from keyboard.\n", "- ```glob``` - finds files with matched regexp patterns.\n", "- ```pathlib``` - object-oriented approach to handling filesystem paths.\n", "- ```json``` - encodes/decodes python object into json, reads/writes jason files.\n", "- ```argparse``` - passes parameters to a python script.\n", "- Exception handling." ] }, { "cell_type": "markdown", "id": "1cb7d85e", "metadata": {}, "source": [ "***\n", "### Module gzip (Exercise)\n", "\n", "\n", "The module is utilized to\n", "\n", "- compress/decompress binary data\n", "- write data into compressed *.gz file\n", "- read data from compressed *.gz file" ] }, { "cell_type": "code", "execution_count": 66, "id": "83d38686", "metadata": {}, "outputs": [], "source": [ "## See all the methods in the module" ] }, { "cell_type": "code", "execution_count": 67, "id": "45b7b54c", "metadata": {}, "outputs": [], "source": [ "import gzip" ] }, { "cell_type": "code", "execution_count": 68, "id": "8ab6d14c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " Create some data. \n", "In this particular case, \n", "the data is just two sentences stored in a string\n", " \n", " Line 4 \n", "\n", " Line 5 \n", "\n", " Line 6 \n", " \n", " Line 7\n", " \n" ] } ], "source": [ "data = '''\n", " Create some data. \\nIn this particular case, \\nthe data is just two sentences stored in a string\\n \n", " Line 4 \\n\n", " Line 5 \\n\n", " Line 6 \\n \n", " Line 7\n", " '''\n", "\n", "print(data)" ] }, { "cell_type": "code", "execution_count": 69, "id": "f1a9be74", "metadata": {}, "outputs": [], "source": [ "## Store data into gzip file, data.txt.gz. The procedure is very similar to writing data into a regular file. \n", "## Notice the write option 'wt' is for writing text:\n", "\n", "with gzip.open(\"data.txt.gz\",\"wt\") as f:\n", " f.write(data)" ] }, { "cell_type": "code", "execution_count": 77, "id": "f5619a0e", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[]" ] }, "execution_count": 77, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Check if file data.txt.gz shows up in the directory:\n", "\n", "import os\n", "os.listdir()" ] }, { "cell_type": "code", "execution_count": 78, "id": "760f3f09", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[0m\u001b[01;31mdata.txt.gz\u001b[0m\n" ] } ], "source": [ "%ls data.txt.gz\n" ] }, { "cell_type": "code", "execution_count": 79, "id": "9c94d27d", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " Create some data. \n", "In this particular case, \n", "the data is just two sentences stored in a string\n", " \n", " Line 4 \n", "\n", " Line 5 \n", "\n", " Line 6 \n", " \n", " Line 7\n", " \n" ] } ], "source": [ "## Let's check if we can read data back from the file. Notice the read option 'rt' is for reading text:\n", "\n", "with gzip.open(\"data.txt.gz\",\"rt\") as f:\n", " data_read = f.read()\n", " \n", "print(data_read)" ] }, { "cell_type": "code", "execution_count": 80, "id": "d5371d94", "metadata": {}, "outputs": [], "source": [ "## We cal also convert the data into a binary form, then write it into gzip file with 'wb' option" ] }, { "cell_type": "code", "execution_count": 81, "id": "1802c8c4", "metadata": {}, "outputs": [], "source": [ "#data_byte = bytes(data, 'utf-8')\n", "data_byte = data.encode('utf-8')\n" ] }, { "cell_type": "code", "execution_count": 82, "id": "96ef849f", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "bytes" ] }, "execution_count": 82, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(data_byte)" ] }, { "cell_type": "code", "execution_count": 83, "id": "0340e9a2", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "b'\\n Create some data. \\nIn this particular case, \\nthe data is just two sentences stored in a string\\n \\n Line 4 \\n\\n Line 5 \\n\\n Line 6 \\n \\n Line 7\\n '\n" ] } ], "source": [ "print(data_byte)" ] }, { "cell_type": "code", "execution_count": 84, "id": "281a73c6", "metadata": {}, "outputs": [], "source": [ "## Store the binary data into gzip file" ] }, { "cell_type": "code", "execution_count": 85, "id": "e1e2842c", "metadata": {}, "outputs": [], "source": [ "with gzip.open(\"data.txt.gz\",\"wb\") as f:\n", " f.write(data_byte)" ] }, { "cell_type": "code", "execution_count": null, "id": "c9ad51bb", "metadata": { "collapsed": true, "jupyter": { "outputs_hidden": true } }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": 90, "id": "37555e80", "metadata": {}, "outputs": [], "source": [ "## This also can be done with GzipFile method:\n", "with gzip.GzipFile(\"data.txt.gz\",\"wb\") as f:\n", " f.write(data_byte)" ] }, { "cell_type": "code", "execution_count": null, "id": "2891b9f5", "metadata": { "collapsed": true, "jupyter": { "outputs_hidden": true } }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": 91, "id": "34ed3ed5", "metadata": {}, "outputs": [], "source": [ "## We can also read the gzip file into binary data type by using \"rb\" option:\n", "\n", "with gzip.open(\"data.txt.gz\",\"rb\") as f:\n", " out_byte = f.read()" ] }, { "cell_type": "code", "execution_count": 92, "id": "f512b94d", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "b'\\n Create some data. \\nIn this particular case, \\nthe data is just two sentences stored in a string\\n \\n Line 4 \\n\\n Line 5 \\n\\n Line 6 \\n \\n Line 7\\n '\n" ] } ], "source": [ "print(out_byte)" ] }, { "cell_type": "code", "execution_count": 93, "id": "a9af5126", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "180 > 172 > 151\n" ] } ], "source": [ "## Compressing/decompressing data. It can be applied to binary data only.\n", "## Check the sizes of text data, byte data, and compressed byte data:\n", "\n", "import sys\n", "\n", "size_txt = sys.getsizeof(data)\n", "\n", "out_byte = data.encode('utf-8')\n", "size_bin = sys.getsizeof(out_byte)\n", "\n", "out_compressed = gzip.compress(out_byte)\n", "size_compressed = sys.getsizeof(out_compressed)\n", "\n", "print(size_txt, '>', size_bin, '>', size_compressed)" ] }, { "cell_type": "markdown", "id": "e0a9ee00", "metadata": {}, "source": [ "***\n", "### Module zipfile (Exercise)\n", "\n", "Works with zip archives:\n", "\n", "- Reads list of files and directories from zip archive\n", "- Extracts files from an archive\n", "- Reads specific file from an archive\n", "- Creates a new archive\n", "- Adds/appends file to the archive" ] }, { "cell_type": "markdown", "id": "3ea19215", "metadata": {}, "source": [ "#### Download a zip file:\n", "\n" ] }, { "cell_type": "code", "execution_count": 94, "id": "82ddc823", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "--2025-11-18 22:13:41-- https://linuxcourse.rutgers.edu/2024/html/files/test.zip\n" ] }, { "data": { "text/plain": [ "0" ] }, "execution_count": 94, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import os\n", "os.system('wget --no-check-certificate https://linuxcourse.rutgers.edu/2024/html/files/test.zip')" ] }, { "cell_type": "code", "execution_count": 95, "id": "0007a2b8", "metadata": {}, "outputs": [], "source": [ "import zipfile" ] }, { "cell_type": "code", "execution_count": 96, "id": "619c6b05", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "File Name Modified Size\n", "NEW_DIR1/ 2021-05-25 14:34:44 0\n", "py_script.py 2024-01-10 17:14:22 198\n", "list.txt 2024-01-10 17:14:00 105\n", "ip.txt 2024-01-11 11:55:14 12\n", "None\n" ] } ], "source": [ "with zipfile.ZipFile('test.zip') as file: \n", " # printing all the information of archive file contents using 'printdir' method\n", " print(file.printdir())" ] }, { "cell_type": "code", "execution_count": 97, "id": "24f231d9", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "['NEW_DIR1/', 'py_script.py', 'list.txt', 'ip.txt']\n" ] } ], "source": [ "with zipfile.ZipFile('test.zip') as file: \n", " # ZipFile.namelist() returns a list containing all the members with names of an archive file\n", " print(file.namelist())" ] }, { "cell_type": "code", "execution_count": 150, "id": "4c1077eb", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "#!/home/akotelnikov/miniconda3/bin/python3\n", "\n", "import sys, os\n", "script_name = sys.argv[0]\n", "par1 = sys.argv[1]\n", "par2 = sys.argv[2]\n", "\n", "print('name=',script_name, 'par1=', par1, 'par2=',par2)\n", "\n", "\n", "os.system(par1)\n", "\n" ] } ], "source": [ "with zipfile.ZipFile('test.zip') as file:\n", " # Open and read the content of a file from the list:\n", " with file.open(name = file.namelist()[1], mode = 'r') as text_file:\n", " print(text_file.read().decode('utf-8')) " ] }, { "cell_type": "code", "execution_count": 99, "id": "d68cd841", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[, , , ]\n" ] } ], "source": [ " with zipfile.ZipFile('test.zip') as file:\n", "\n", " # ZipFile.infolist() returns a list containing all the members of an archive file\n", " print(file.infolist())" ] }, { "cell_type": "code", "execution_count": 100, "id": "dc5e8b54", "metadata": {}, "outputs": [], "source": [ "## Extract the content of zip file into a new directory, EXTRACT_DIR\n", "\n", "\n", "with zipfile.ZipFile('test.zip') as f:\n", " f.extractall('EXTRACT_DIR')" ] }, { "cell_type": "code", "execution_count": 101, "id": "9a439065", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['list.txt', 'ip.txt', 'NEW_DIR1', 'py_script.py']" ] }, "execution_count": 101, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import os\n", "os.listdir('EXTRACT_DIR')" ] }, { "cell_type": "code", "execution_count": 102, "id": "74a1e78d", "metadata": {}, "outputs": [], "source": [ "## Create an empty zipfile:\n", "\n", "archive_name = 'new_test.zip'\n", "f = zipfile.ZipFile(archive_name, 'w')\n", "f.close()" ] }, { "cell_type": "code", "execution_count": 103, "id": "dd8203b9", "metadata": {}, "outputs": [], "source": [ "## Add a new file, /etc/hosts, to the existing archive. Ovewrite existing files there.\n", "with zipfile.ZipFile('new_test.zip', 'w') as f:\n", " f.write('/etc/hosts')" ] }, { "cell_type": "code", "execution_count": 104, "id": "32637ec7", "metadata": {}, "outputs": [], "source": [ "## Append a new file, /etc/passwd. This doesn't overwrite the existing files.\n", "with zipfile.ZipFile('new_test.zip', 'a') as f:\n", " f.write('/etc/passwd')" ] }, { "cell_type": "code", "execution_count": 105, "id": "fe56c934", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "File Name Modified Size\n", "etc/hosts 2025-05-04 02:17:52 179\n", "etc/passwd 2024-07-27 16:12:20 1188\n", "None\n" ] } ], "source": [ "## Check the content of archive new_test.zip\n", "with zipfile.ZipFile('new_test.zip','r') as f:\n", " print(f.printdir())" ] }, { "cell_type": "markdown", "id": "1683608f", "metadata": {}, "source": [ "***\n", "### Module getpass (Exercise)\n", "\n", "- Promps for password and reads it from stdin\n", "- Figures out the user name from the environment" ] }, { "cell_type": "code", "execution_count": 106, "id": "866e7ecb", "metadata": {}, "outputs": [], "source": [ "import getpass" ] }, { "cell_type": "code", "execution_count": 107, "id": "f2ed6b23", "metadata": {}, "outputs": [], "source": [ "## Prompt for password and read user password:" ] }, { "cell_type": "code", "execution_count": 108, "id": "a2c95695", "metadata": {}, "outputs": [ { "name": "stdin", "output_type": "stream", "text": [ "Enter the password: ········\n" ] } ], "source": [ "p = getpass.getpass(prompt='Enter the password: ')" ] }, { "cell_type": "code", "execution_count": 109, "id": "db53ffc9", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "password incorrect\n" ] } ], "source": [ "if p == 'this':\n", " print('correct password')\n", "else:\n", " print('password incorrect')" ] }, { "cell_type": "code", "execution_count": 110, "id": "370b26b2", "metadata": {}, "outputs": [], "source": [ "## identify the user and prompt in the loop until success. Maximum 10 attempts" ] }, { "cell_type": "code", "execution_count": 111, "id": "67225af7", "metadata": {}, "outputs": [ { "name": "stdin", "output_type": "stream", "text": [ "User Name : hostadm ········\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "The password you entered is incorrect.\n" ] }, { "name": "stdin", "output_type": "stream", "text": [ "User Name : hostadm ········\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "The password you entered is incorrect.\n" ] }, { "name": "stdin", "output_type": "stream", "text": [ "User Name : hostadm ········\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "The password you entered is incorrect.\n" ] }, { "name": "stdin", "output_type": "stream", "text": [ "User Name : hostadm ········\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "login successful\n" ] } ], "source": [ "user = getpass.getuser()\n", "\n", "count = 0\n", "max_count = 10\n", "\n", "#while True: \n", "while count <= max_count:\n", " \n", " pwd = getpass.getpass(\"User Name : %s\" % user)\n", " \n", " count += 1\n", "\n", " if pwd == 'password1':\n", " print(\"login successful\")\n", " break\n", " else:\n", " print(\"The password you entered is incorrect.\")" ] }, { "cell_type": "markdown", "id": "d5eb69d8", "metadata": {}, "source": [ "***\n", "### Module glob (Exercise)\n", "\n", "- Finds files matching a specified pattern and\n", " returns them in a list [ ]\n", "- `*` matches zero or more characters\n", "- `?` matches one character\n", "- [0-9] matches any number\n", "- [a-z] matches any low case character\n", "- [A-Z] matches any upper case character" ] }, { "cell_type": "code", "execution_count": 112, "id": "e5c9cc38", "metadata": {}, "outputs": [], "source": [ "import glob" ] }, { "cell_type": "code", "execution_count": 113, "id": "ed59ddfc", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['new_test.zip', 'test.zip']" ] }, "execution_count": 113, "metadata": {}, "output_type": "execute_result" } ], "source": [ "glob.glob('*.zip')" ] }, { "cell_type": "code", "execution_count": 114, "id": "2752d8d6", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[]" ] }, "execution_count": 114, "metadata": {}, "output_type": "execute_result" } ], "source": [ "glob.glob('*[0-9].zip')" ] }, { "cell_type": "code", "execution_count": 115, "id": "8c318926", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['new_test.zip', 'test.zip']" ] }, "execution_count": 115, "metadata": {}, "output_type": "execute_result" } ], "source": [ "glob.glob('*[a-z].zip')" ] }, { "cell_type": "code", "execution_count": 116, "id": "3fcf4405", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[]" ] }, "execution_count": 116, "metadata": {}, "output_type": "execute_result" } ], "source": [ "glob.glob('[A-Z]*[0-9].zip')" ] }, { "cell_type": "code", "execution_count": 117, "id": "c1d8fc3b", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['new_test.zip', 'networking.ipynb']" ] }, "execution_count": 117, "metadata": {}, "output_type": "execute_result" } ], "source": [ "glob.glob('[n,I]e*')" ] }, { "cell_type": "code", "execution_count": 118, "id": "b7faa403", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['/etc/nftables.conf',\n", " '/etc/deluser.conf',\n", " '/etc/ld.so.conf',\n", " '/etc/reportbug.conf',\n", " '/etc/sysctl.conf',\n", " '/etc/xattr.conf',\n", " '/etc/libaudit.conf',\n", " '/etc/nsswitch.conf',\n", " '/etc/sudo_logsrvd.conf',\n", " '/etc/sensors3.conf',\n", " '/etc/mke2fs.conf',\n", " '/etc/ca-certificates.conf',\n", " '/etc/e2scrub.conf',\n", " '/etc/logrotate.conf',\n", " '/etc/gai.conf',\n", " '/etc/ucf.conf',\n", " '/etc/resolv.conf',\n", " '/etc/host.conf',\n", " '/etc/sudo.conf',\n", " '/etc/debconf.conf',\n", " '/etc/pam.conf',\n", " '/etc/adduser.conf']" ] }, "execution_count": 118, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Search for files specifically in /etc\n", "glob.glob('/etc/*.conf')" ] }, { "cell_type": "code", "execution_count": 119, "id": "789bab20", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['/etc/ld.so.conf.d/x86_64-linux-gnu.conf',\n", " '/etc/ld.so.conf.d/libc.conf',\n", " '/etc/fonts/fonts.conf',\n", " '/etc/apt/listchanges.conf',\n", " '/etc/gtk-3.0/im-multipress.conf',\n", " '/etc/security/limits.conf',\n", " '/etc/security/access.conf',\n", " '/etc/security/pam_env.conf',\n", " '/etc/security/namespace.conf',\n", " '/etc/security/time.conf',\n", " '/etc/security/faillock.conf',\n", " '/etc/security/group.conf',\n", " '/etc/security/sepermit.conf',\n", " '/etc/selinux/semanage.conf',\n", " '/etc/environment.d/90qt-a11y.conf',\n", " '/etc/udev/udev.conf',\n", " '/etc/dhcp/dhclient.conf',\n", " '/etc/xdg/user-dirs.conf',\n", " '/etc/modules-load.d/modules.conf',\n", " '/etc/rsyslog.d/postfix.conf',\n", " '/etc/sysctl.d/99-sysctl.conf',\n", " '/etc/gtk-2.0/im-multipress.conf',\n", " '/etc/systemd/system.conf',\n", " '/etc/systemd/networkd.conf',\n", " '/etc/systemd/pstore.conf',\n", " '/etc/systemd/timesyncd.conf',\n", " '/etc/systemd/user.conf',\n", " '/etc/systemd/logind.conf',\n", " '/etc/systemd/sleep.conf',\n", " '/etc/systemd/journald.conf']" ] }, "execution_count": 119, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Search for files in directory and subdirectories /etc\n", "glob.glob('/etc/*/*.conf')" ] }, { "cell_type": "code", "execution_count": 120, "id": "627e5f17", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['/etc/nftables.conf',\n", " '/etc/deluser.conf',\n", " '/etc/ld.so.conf',\n", " '/etc/reportbug.conf',\n", " '/etc/sysctl.conf',\n", " '/etc/xattr.conf',\n", " '/etc/libaudit.conf',\n", " '/etc/nsswitch.conf',\n", " '/etc/sudo_logsrvd.conf',\n", " '/etc/sensors3.conf',\n", " '/etc/mke2fs.conf',\n", " '/etc/ca-certificates.conf',\n", " '/etc/e2scrub.conf',\n", " '/etc/logrotate.conf',\n", " '/etc/gai.conf',\n", " '/etc/ucf.conf',\n", " '/etc/resolv.conf',\n", " '/etc/host.conf',\n", " '/etc/sudo.conf',\n", " '/etc/debconf.conf',\n", " '/etc/pam.conf',\n", " '/etc/adduser.conf',\n", " '/etc/ld.so.conf.d/x86_64-linux-gnu.conf',\n", " '/etc/ld.so.conf.d/libc.conf',\n", " '/etc/fonts/fonts.conf',\n", " '/etc/fonts/conf.avail/20-unhint-small-dejavu-sans.conf',\n", " '/etc/fonts/conf.avail/64-arphic-uming.conf',\n", " '/etc/fonts/conf.avail/65-fonts-lmodern.conf',\n", " '/etc/fonts/conf.avail/20-unhint-small-dejavu-lgc-sans.conf',\n", " '/etc/fonts/conf.avail/30-droid-noto-mono.conf',\n", " '/etc/fonts/conf.avail/30-droid-noto.conf',\n", " '/etc/fonts/conf.avail/58-dejavu-lgc-serif.conf',\n", " '/etc/fonts/conf.avail/20-unhint-small-dejavu-lgc-serif.conf',\n", " '/etc/fonts/conf.avail/65-fonts-texgyre-math.conf',\n", " '/etc/fonts/conf.avail/57-dejavu-sans-mono.conf',\n", " '/etc/fonts/conf.avail/20-unhint-small-dejavu-lgc-sans-mono.conf',\n", " '/etc/fonts/conf.avail/65-fonts-texgyre.conf',\n", " '/etc/fonts/conf.avail/58-dejavu-lgc-sans-mono.conf',\n", " '/etc/fonts/conf.avail/25-arphic-uming-render.conf',\n", " '/etc/fonts/conf.avail/65-droid-sans-fallback.conf',\n", " '/etc/fonts/conf.avail/25-arphic-uming-bitmaps.conf',\n", " '/etc/fonts/conf.avail/57-dejavu-sans.conf',\n", " '/etc/fonts/conf.avail/58-dejavu-lgc-sans.conf',\n", " '/etc/fonts/conf.avail/35-arphic-uming-aliases.conf',\n", " '/etc/fonts/conf.avail/20-unhint-small-dejavu-serif.conf',\n", " '/etc/fonts/conf.avail/41-arphic-uming.conf',\n", " '/etc/fonts/conf.avail/20-unhint-small-dejavu-sans-mono.conf',\n", " '/etc/fonts/conf.avail/57-dejavu-serif.conf',\n", " '/etc/fonts/conf.avail/90-arphic-uming-embolden.conf',\n", " '/etc/fonts/conf.d/60-latin.conf',\n", " '/etc/fonts/conf.d/20-unhint-small-dejavu-sans.conf',\n", " '/etc/fonts/conf.d/61-urw-nimbus-sans.conf',\n", " '/etc/fonts/conf.d/61-urw-fallback-backwards.conf',\n", " '/etc/fonts/conf.d/30-metric-aliases.conf',\n", " '/etc/fonts/conf.d/64-arphic-uming.conf',\n", " '/etc/fonts/conf.d/61-urw-z003.conf',\n", " '/etc/fonts/conf.d/61-urw-nimbus-mono-ps.conf',\n", " '/etc/fonts/conf.d/65-fonts-lmodern.conf',\n", " '/etc/fonts/conf.d/61-urw-standard-symbols-ps.conf',\n", " '/etc/fonts/conf.d/90-fonts-unfonts-extra.conf',\n", " '/etc/fonts/conf.d/64-11-tlwg-waree.conf',\n", " '/etc/fonts/conf.d/20-unhint-small-dejavu-lgc-sans.conf',\n", " '/etc/fonts/conf.d/61-urw-nimbus-roman.conf',\n", " '/etc/fonts/conf.d/10-yes-antialias.conf',\n", " '/etc/fonts/conf.d/61-urw-p052.conf',\n", " '/etc/fonts/conf.d/61-urw-d050000l.conf',\n", " '/etc/fonts/conf.d/40-nonlatin.conf',\n", " '/etc/fonts/conf.d/89-tlwg-umpush-synthetic.conf',\n", " '/etc/fonts/conf.d/80-delicious.conf',\n", " '/etc/fonts/conf.d/69-unifont.conf',\n", " '/etc/fonts/conf.d/48-spacing.conf',\n", " '/etc/fonts/conf.d/64-21-tlwg-typo.conf',\n", " '/etc/fonts/conf.d/50-user.conf',\n", " '/etc/fonts/conf.d/51-local.conf',\n", " '/etc/fonts/conf.d/45-generic.conf',\n", " '/etc/fonts/conf.d/90-fonts-linux-libertine.conf',\n", " '/etc/fonts/conf.d/58-dejavu-lgc-serif.conf',\n", " '/etc/fonts/conf.d/64-22-tlwg-typist.conf',\n", " '/etc/fonts/conf.d/89-tlwg-garuda-synthetic.conf',\n", " '/etc/fonts/conf.d/90-synthetic.conf',\n", " '/etc/fonts/conf.d/20-unhint-small-dejavu-lgc-serif.conf',\n", " '/etc/fonts/conf.d/65-nonlatin.conf',\n", " '/etc/fonts/conf.d/65-fonts-texgyre-math.conf',\n", " '/etc/fonts/conf.d/64-15-laksaman.conf',\n", " '/etc/fonts/conf.d/57-dejavu-sans-mono.conf',\n", " '/etc/fonts/conf.d/65-fonts-persian.conf',\n", " '/etc/fonts/conf.d/20-unhint-small-vera.conf',\n", " '/etc/fonts/conf.d/10-hinting-slight.conf',\n", " '/etc/fonts/conf.d/61-urw-fallback-generics.conf',\n", " '/etc/fonts/conf.d/64-02-tlwg-norasi.conf',\n", " '/etc/fonts/conf.d/20-unhint-small-dejavu-lgc-sans-mono.conf',\n", " '/etc/fonts/conf.d/61-urw-c059.conf',\n", " '/etc/fonts/conf.d/64-23-tlwg-mono.conf',\n", " '/etc/fonts/conf.d/65-fonts-texgyre.conf',\n", " '/etc/fonts/conf.d/58-dejavu-lgc-sans-mono.conf',\n", " '/etc/fonts/conf.d/70-no-bitmaps.conf',\n", " '/etc/fonts/conf.d/64-13-tlwg-garuda.conf',\n", " '/etc/fonts/conf.d/25-arphic-uming-render.conf',\n", " '/etc/fonts/conf.d/89-tlwg-laksaman-synthetic.conf',\n", " '/etc/fonts/conf.d/30-intel-clear.conf',\n", " '/etc/fonts/conf.d/57-dejavu-sans.conf',\n", " '/etc/fonts/conf.d/10-scale-bitmap-fonts.conf',\n", " '/etc/fonts/conf.d/61-urw-bookman.conf',\n", " '/etc/fonts/conf.d/58-dejavu-lgc-sans.conf',\n", " '/etc/fonts/conf.d/35-arphic-uming-aliases.conf',\n", " '/etc/fonts/conf.d/90-fonts-baekmuk.conf',\n", " '/etc/fonts/conf.d/20-unhint-small-dejavu-serif.conf',\n", " '/etc/fonts/conf.d/30-comic-sans.conf',\n", " '/etc/fonts/conf.d/41-arphic-uming.conf',\n", " '/etc/fonts/conf.d/60-generic.conf',\n", " '/etc/fonts/conf.d/11-lcdfilter-default.conf',\n", " '/etc/fonts/conf.d/64-01-tlwg-kinnari.conf',\n", " '/etc/fonts/conf.d/20-unhint-small-dejavu-sans-mono.conf',\n", " '/etc/fonts/conf.d/45-latin.conf',\n", " '/etc/fonts/conf.d/64-14-tlwg-umpush.conf',\n", " '/etc/fonts/conf.d/49-sansserif.conf',\n", " '/etc/fonts/conf.d/89-tlwg-kinnari-synthetic.conf',\n", " '/etc/fonts/conf.d/61-urw-gothic.conf',\n", " '/etc/fonts/conf.d/57-dejavu-serif.conf',\n", " '/etc/fonts/conf.d/90-arphic-uming-embolden.conf',\n", " '/etc/fonts/conf.d/90-fonts-unfonts-core.conf',\n", " '/etc/fonts/conf.d/64-10-tlwg-loma.conf',\n", " '/etc/ghostscript/cidfmap.d/90gs-cjk-resource-korea1.conf',\n", " '/etc/ghostscript/cidfmap.d/90gs-cjk-resource-gb1.conf',\n", " '/etc/ghostscript/cidfmap.d/90gs-cjk-resource-japan2.conf',\n", " '/etc/ghostscript/cidfmap.d/90gs-cjk-resource-cns1.conf',\n", " '/etc/ghostscript/cidfmap.d/90gs-cjk-resource-japan1.conf',\n", " '/etc/ghostscript/fontmap.d/10fonts-urw-base35.conf',\n", " '/etc/apt/listchanges.conf',\n", " '/etc/gtk-3.0/im-multipress.conf',\n", " '/etc/security/limits.conf',\n", " '/etc/security/access.conf',\n", " '/etc/security/pam_env.conf',\n", " '/etc/security/namespace.conf',\n", " '/etc/security/time.conf',\n", " '/etc/security/faillock.conf',\n", " '/etc/security/group.conf',\n", " '/etc/security/sepermit.conf',\n", " '/etc/selinux/semanage.conf',\n", " '/etc/lighttpd/conf-available/90-javascript-alias.conf',\n", " '/etc/lighttpd/conf-enabled/90-javascript-alias.conf',\n", " '/etc/environment.d/90qt-a11y.conf',\n", " '/etc/udev/udev.conf',\n", " '/etc/dhcp/dhclient.conf',\n", " '/etc/xdg/user-dirs.conf',\n", " '/etc/modules-load.d/modules.conf',\n", " '/etc/rsyslog.d/postfix.conf',\n", " '/etc/sysctl.d/99-sysctl.conf',\n", " '/etc/apache2/conf-available/javascript-common.conf',\n", " '/etc/gtk-2.0/im-multipress.conf',\n", " '/etc/systemd/system.conf',\n", " '/etc/systemd/networkd.conf',\n", " '/etc/systemd/pstore.conf',\n", " '/etc/systemd/timesyncd.conf',\n", " '/etc/systemd/user.conf',\n", " '/etc/systemd/logind.conf',\n", " '/etc/systemd/sleep.conf',\n", " '/etc/systemd/journald.conf']" ] }, "execution_count": 120, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Recursive search in a directory tree\n", "glob.glob('/etc/**/*.conf', recursive=True)" ] }, { "cell_type": "code", "execution_count": 121, "id": "758a975d", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/etc/nftables.conf\n", "/etc/deluser.conf\n", "/etc/ld.so.conf\n", "/etc/reportbug.conf\n", "/etc/sysctl.conf\n", "/etc/xattr.conf\n", "/etc/libaudit.conf\n", "/etc/nsswitch.conf\n", "/etc/sudo_logsrvd.conf\n", "/etc/sensors3.conf\n", "/etc/mke2fs.conf\n", "/etc/ca-certificates.conf\n", "/etc/e2scrub.conf\n", "/etc/logrotate.conf\n", "/etc/gai.conf\n", "/etc/ucf.conf\n", "/etc/resolv.conf\n", "/etc/host.conf\n", "/etc/sudo.conf\n", "/etc/debconf.conf\n", "/etc/pam.conf\n", "/etc/adduser.conf\n", "/etc/ld.so.conf.d/x86_64-linux-gnu.conf\n", "/etc/ld.so.conf.d/libc.conf\n", "/etc/fonts/fonts.conf\n", "/etc/fonts/conf.avail/20-unhint-small-dejavu-sans.conf\n", "/etc/fonts/conf.avail/64-arphic-uming.conf\n", "/etc/fonts/conf.avail/65-fonts-lmodern.conf\n", "/etc/fonts/conf.avail/20-unhint-small-dejavu-lgc-sans.conf\n", "/etc/fonts/conf.avail/30-droid-noto-mono.conf\n", "/etc/fonts/conf.avail/30-droid-noto.conf\n", "/etc/fonts/conf.avail/58-dejavu-lgc-serif.conf\n", "/etc/fonts/conf.avail/20-unhint-small-dejavu-lgc-serif.conf\n", "/etc/fonts/conf.avail/65-fonts-texgyre-math.conf\n", "/etc/fonts/conf.avail/57-dejavu-sans-mono.conf\n", "/etc/fonts/conf.avail/20-unhint-small-dejavu-lgc-sans-mono.conf\n", "/etc/fonts/conf.avail/65-fonts-texgyre.conf\n", "/etc/fonts/conf.avail/58-dejavu-lgc-sans-mono.conf\n", "/etc/fonts/conf.avail/25-arphic-uming-render.conf\n", "/etc/fonts/conf.avail/65-droid-sans-fallback.conf\n", "/etc/fonts/conf.avail/25-arphic-uming-bitmaps.conf\n", "/etc/fonts/conf.avail/57-dejavu-sans.conf\n", "/etc/fonts/conf.avail/58-dejavu-lgc-sans.conf\n", "/etc/fonts/conf.avail/35-arphic-uming-aliases.conf\n", "/etc/fonts/conf.avail/20-unhint-small-dejavu-serif.conf\n", "/etc/fonts/conf.avail/41-arphic-uming.conf\n", "/etc/fonts/conf.avail/20-unhint-small-dejavu-sans-mono.conf\n", "/etc/fonts/conf.avail/57-dejavu-serif.conf\n", "/etc/fonts/conf.avail/90-arphic-uming-embolden.conf\n", "/etc/fonts/conf.d/60-latin.conf\n", "/etc/fonts/conf.d/20-unhint-small-dejavu-sans.conf\n", "/etc/fonts/conf.d/61-urw-nimbus-sans.conf\n", "/etc/fonts/conf.d/61-urw-fallback-backwards.conf\n", "/etc/fonts/conf.d/30-metric-aliases.conf\n", "/etc/fonts/conf.d/64-arphic-uming.conf\n", "/etc/fonts/conf.d/61-urw-z003.conf\n", "/etc/fonts/conf.d/61-urw-nimbus-mono-ps.conf\n", "/etc/fonts/conf.d/65-fonts-lmodern.conf\n", "/etc/fonts/conf.d/61-urw-standard-symbols-ps.conf\n", "/etc/fonts/conf.d/90-fonts-unfonts-extra.conf\n", "/etc/fonts/conf.d/64-11-tlwg-waree.conf\n", "/etc/fonts/conf.d/20-unhint-small-dejavu-lgc-sans.conf\n", "/etc/fonts/conf.d/61-urw-nimbus-roman.conf\n", "/etc/fonts/conf.d/10-yes-antialias.conf\n", "/etc/fonts/conf.d/61-urw-p052.conf\n", "/etc/fonts/conf.d/61-urw-d050000l.conf\n", "/etc/fonts/conf.d/40-nonlatin.conf\n", "/etc/fonts/conf.d/89-tlwg-umpush-synthetic.conf\n", "/etc/fonts/conf.d/80-delicious.conf\n", "/etc/fonts/conf.d/69-unifont.conf\n", "/etc/fonts/conf.d/48-spacing.conf\n", "/etc/fonts/conf.d/64-21-tlwg-typo.conf\n", "/etc/fonts/conf.d/50-user.conf\n", "/etc/fonts/conf.d/51-local.conf\n", "/etc/fonts/conf.d/45-generic.conf\n", "/etc/fonts/conf.d/90-fonts-linux-libertine.conf\n", "/etc/fonts/conf.d/58-dejavu-lgc-serif.conf\n", "/etc/fonts/conf.d/64-22-tlwg-typist.conf\n", "/etc/fonts/conf.d/89-tlwg-garuda-synthetic.conf\n", "/etc/fonts/conf.d/90-synthetic.conf\n", "/etc/fonts/conf.d/20-unhint-small-dejavu-lgc-serif.conf\n", "/etc/fonts/conf.d/65-nonlatin.conf\n", "/etc/fonts/conf.d/65-fonts-texgyre-math.conf\n", "/etc/fonts/conf.d/64-15-laksaman.conf\n", "/etc/fonts/conf.d/57-dejavu-sans-mono.conf\n", "/etc/fonts/conf.d/65-fonts-persian.conf\n", "/etc/fonts/conf.d/20-unhint-small-vera.conf\n", "/etc/fonts/conf.d/10-hinting-slight.conf\n", "/etc/fonts/conf.d/61-urw-fallback-generics.conf\n", "/etc/fonts/conf.d/64-02-tlwg-norasi.conf\n", "/etc/fonts/conf.d/20-unhint-small-dejavu-lgc-sans-mono.conf\n", "/etc/fonts/conf.d/61-urw-c059.conf\n", "/etc/fonts/conf.d/64-23-tlwg-mono.conf\n", "/etc/fonts/conf.d/65-fonts-texgyre.conf\n", "/etc/fonts/conf.d/58-dejavu-lgc-sans-mono.conf\n", "/etc/fonts/conf.d/70-no-bitmaps.conf\n", "/etc/fonts/conf.d/64-13-tlwg-garuda.conf\n", "/etc/fonts/conf.d/25-arphic-uming-render.conf\n", "/etc/fonts/conf.d/89-tlwg-laksaman-synthetic.conf\n", "/etc/fonts/conf.d/30-intel-clear.conf\n", "/etc/fonts/conf.d/57-dejavu-sans.conf\n", "/etc/fonts/conf.d/10-scale-bitmap-fonts.conf\n", "/etc/fonts/conf.d/61-urw-bookman.conf\n", "/etc/fonts/conf.d/58-dejavu-lgc-sans.conf\n", "/etc/fonts/conf.d/35-arphic-uming-aliases.conf\n", "/etc/fonts/conf.d/90-fonts-baekmuk.conf\n", "/etc/fonts/conf.d/20-unhint-small-dejavu-serif.conf\n", "/etc/fonts/conf.d/30-comic-sans.conf\n", "/etc/fonts/conf.d/41-arphic-uming.conf\n", "/etc/fonts/conf.d/60-generic.conf\n", "/etc/fonts/conf.d/11-lcdfilter-default.conf\n", "/etc/fonts/conf.d/64-01-tlwg-kinnari.conf\n", "/etc/fonts/conf.d/20-unhint-small-dejavu-sans-mono.conf\n", "/etc/fonts/conf.d/45-latin.conf\n", "/etc/fonts/conf.d/64-14-tlwg-umpush.conf\n", "/etc/fonts/conf.d/49-sansserif.conf\n", "/etc/fonts/conf.d/89-tlwg-kinnari-synthetic.conf\n", "/etc/fonts/conf.d/61-urw-gothic.conf\n", "/etc/fonts/conf.d/57-dejavu-serif.conf\n", "/etc/fonts/conf.d/90-arphic-uming-embolden.conf\n", "/etc/fonts/conf.d/90-fonts-unfonts-core.conf\n", "/etc/fonts/conf.d/64-10-tlwg-loma.conf\n", "/etc/ghostscript/cidfmap.d/90gs-cjk-resource-korea1.conf\n", "/etc/ghostscript/cidfmap.d/90gs-cjk-resource-gb1.conf\n", "/etc/ghostscript/cidfmap.d/90gs-cjk-resource-japan2.conf\n", "/etc/ghostscript/cidfmap.d/90gs-cjk-resource-cns1.conf\n", "/etc/ghostscript/cidfmap.d/90gs-cjk-resource-japan1.conf\n", "/etc/ghostscript/fontmap.d/10fonts-urw-base35.conf\n", "/etc/apt/listchanges.conf\n", "/etc/gtk-3.0/im-multipress.conf\n", "/etc/security/limits.conf\n", "/etc/security/access.conf\n", "/etc/security/pam_env.conf\n", "/etc/security/namespace.conf\n", "/etc/security/time.conf\n", "/etc/security/faillock.conf\n", "/etc/security/group.conf\n", "/etc/security/sepermit.conf\n", "/etc/selinux/semanage.conf\n", "/etc/lighttpd/conf-available/90-javascript-alias.conf\n", "/etc/lighttpd/conf-enabled/90-javascript-alias.conf\n", "/etc/environment.d/90qt-a11y.conf\n", "/etc/udev/udev.conf\n", "/etc/dhcp/dhclient.conf\n", "/etc/xdg/user-dirs.conf\n", "/etc/modules-load.d/modules.conf\n", "/etc/rsyslog.d/postfix.conf\n", "/etc/sysctl.d/99-sysctl.conf\n", "/etc/apache2/conf-available/javascript-common.conf\n", "/etc/gtk-2.0/im-multipress.conf\n", "/etc/systemd/system.conf\n", "/etc/systemd/networkd.conf\n", "/etc/systemd/pstore.conf\n", "/etc/systemd/timesyncd.conf\n", "/etc/systemd/user.conf\n", "/etc/systemd/logind.conf\n", "/etc/systemd/sleep.conf\n", "/etc/systemd/journald.conf\n" ] } ], "source": [ "## For recursive search, it is better using iterative method `iglob` to save space in RAM. \n", "## The iglob runs iteratively so doesn't buffer the output unlike glob.\n", "\n", "for filename in glob.iglob('/etc/**/*.conf', recursive=True):\n", " print(filename)" ] }, { "cell_type": "markdown", "id": "70d68fea-c26e-494f-b7f8-2fbdb82aad8e", "metadata": {}, "source": [ "***\n", "### Module pathlib (Exercise) \n", "\n", "The module provides an object-oriented approach to handling filesystem paths. \n" ] }, { "cell_type": "code", "execution_count": 122, "id": "915f3047-2ca0-4d31-9fda-0217b6445744", "metadata": {}, "outputs": [], "source": [ "from pathlib import Path" ] }, { "cell_type": "code", "execution_count": 123, "id": "eb2f5830-e1e7-43d4-8acd-9cfcec209ca9", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "PosixPath('/home/hostadm')" ] }, "execution_count": 123, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Path.home()" ] }, { "cell_type": "markdown", "id": "740a6f6a-7159-4921-b5cc-3d1d75f5cd01", "metadata": {}, "source": [ "Create a new directory, ```/home/hostadm/Python2```" ] }, { "cell_type": "code", "execution_count": 124, "id": "edd8d45b-a9d4-43b3-a9fb-7e3010249407", "metadata": {}, "outputs": [], "source": [ "p = Path.home() / 'Python2'\n", "p.mkdir()" ] }, { "cell_type": "markdown", "id": "62ade510-648d-4de0-b80a-04e0838ab169", "metadata": {}, "source": [ "Create a new file in the directory:" ] }, { "cell_type": "code", "execution_count": 125, "id": "88624ccd-55b1-47ab-bc60-3c0adce1dd58", "metadata": {}, "outputs": [], "source": [ "nf = p / 'new_file.txt'\n", "nf\n", "nf.touch()" ] }, { "cell_type": "markdown", "id": "a94c192e-bea4-43ae-8adb-91b172fe7710", "metadata": {}, "source": [ "Check if the directory and the file exist:" ] }, { "cell_type": "code", "execution_count": 126, "id": "7928da26-6c13-4162-8661-2fc8b803f750", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 126, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nf.is_file()" ] }, { "cell_type": "code", "execution_count": 127, "id": "cf35ebb1-444f-4d9d-9343-a637b555f142", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 127, "metadata": {}, "output_type": "execute_result" } ], "source": [ "p.is_dir()" ] }, { "cell_type": "markdown", "id": "46cf4804-9330-4ba1-b131-738bf7da841a", "metadata": {}, "source": [ "Rename the file:" ] }, { "cell_type": "code", "execution_count": 128, "id": "67d0307d-637d-4db0-93a2-37bff28a73a8", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "PosixPath('/home/hostadm/Python2/new_file2.txt')" ] }, "execution_count": 128, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nfn = p / 'new_file2.txt'\n", "\n", "nf.rename(nfn)" ] }, { "cell_type": "markdown", "id": "402949ea-43b7-406c-8f05-42366e957394", "metadata": {}, "source": [ "Write data into file:" ] }, { "cell_type": "code", "execution_count": 129, "id": "e418c50c-ba71-4fe9-9af0-f2b023cecc2d", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "139" ] }, "execution_count": 129, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nf.write_text(data)" ] }, { "cell_type": "markdown", "id": "8911b265-9ea2-4b92-b71d-fbcbc4ed4d1a", "metadata": {}, "source": [ "Read text from data:" ] }, { "cell_type": "code", "execution_count": 130, "id": "a6ad83c3-3d3a-4cb6-b56e-d3a450b21a8d", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " Create some data. \n", "In this particular case, \n", "the data is just two sentences stored in a string\n", " \n", " Line 4 \n", "\n", " Line 5 \n", "\n", " Line 6 \n", " \n", " Line 7\n", " \n" ] } ], "source": [ "s = Path('/home/hostadm/Python2/new_file.txt').read_text()\n", "print(s)" ] }, { "cell_type": "markdown", "id": "67b7e150-7cea-4e6c-9638-148b651fd0b9", "metadata": {}, "source": [ "List files in the directory:" ] }, { "cell_type": "code", "execution_count": 131, "id": "0d0f9818-d92c-4475-8642-19825d43dada", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "['new_file2.txt', 'new_file.txt']\n" ] } ], "source": [ "s = [ s.name for s in list(p.glob('*'))]\n", "print(s)" ] }, { "cell_type": "markdown", "id": "96708569", "metadata": {}, "source": [ "***\n", "### Exception handling in Python (Exercise)" ] }, { "cell_type": "code", "execution_count": null, "id": "9e6f34ed-05c9-4857-8c31-15c9ccca746c", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": 132, "id": "beac1c4a", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/etc/sysctl.conf\n", "/etc/nsswitch.conf\n", "/etc/sudo.conf\n", "/etc/security/access.conf\n", "/etc/sysctl.d/99-sysctl.conf\n", "/etc/systemd/networkd.conf\n" ] } ], "source": [ "## Example: find all conf files that contain keyword `network`.\n", "## This will throw an error since some files in /etc are not readable by the user.\n", "\n", "for filename in glob.iglob('/etc/**/*.conf', recursive=True):\n", " with open(filename) as f:\n", " line = f.read()\n", " if 'network' in line:\n", " print(filename)" ] }, { "cell_type": "markdown", "id": "3908e45e", "metadata": {}, "source": [ "To allow the script to run and prompt us about errors, we run it within block\n", "```python\n", "try:\n", "...\n", "except ...\n", "```" ] }, { "cell_type": "code", "execution_count": 133, "id": "473502d3", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/etc/sysctl.conf\n", "/etc/nsswitch.conf\n", "/etc/sudo.conf\n", "/etc/security/access.conf\n", "/etc/sysctl.d/99-sysctl.conf\n", "/etc/systemd/networkd.conf\n" ] } ], "source": [ "\n", "## Lets handle the exception above\n", "for filename in glob.iglob('/etc/**/*.conf', recursive=True):\n", " try:\n", " with open(filename) as f:\n", " line = f.read()\n", " if 'network' in line:\n", " print(filename)\n", " \n", " except PermissionError as e:\n", " print(f\"can't read {filename}: {e}\")" ] }, { "cell_type": "markdown", "id": "d836f232", "metadata": {}, "source": [ "There is another, ```FileNotFoundError```, error due to a symbolic link to a missing file.\n", "\n", "We can handle multipple exceptions:" ] }, { "cell_type": "code", "execution_count": 134, "id": "dbdad043", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/etc/sysctl.conf\n", "/etc/nsswitch.conf\n", "/etc/sudo.conf\n", "/etc/security/access.conf\n", "/etc/sysctl.d/99-sysctl.conf\n", "/etc/systemd/networkd.conf\n" ] } ], "source": [ "for filename in glob.iglob('/etc/**/*.conf', recursive=True):\n", " try:\n", " with open(filename) as f:\n", " line = f.read()\n", " if 'network' in line:\n", " print(filename)\n", " \n", " except (PermissionError, FileNotFoundError) as e:\n", " print(f\"can't read {filename}: {e}\")" ] }, { "cell_type": "markdown", "id": "530d8a90", "metadata": {}, "source": [ "The last two exceptions we can catch with a generic ```OSError```:\n", "```python\n", "except OSError as e:\n", "```" ] }, { "cell_type": "markdown", "id": "cdef3816", "metadata": {}, "source": [ "In numerical computations, you may need to deal with the division by zero. For example:" ] }, { "cell_type": "code", "execution_count": 135, "id": "9b01431e", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0.08333333333333334\n", "0.025\n", "100.0\n", "0.0022222222222222222\n" ] }, { "ename": "ZeroDivisionError", "evalue": "float division by zero", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mZeroDivisionError\u001b[0m Traceback (most recent call last)", "Cell \u001b[0;32mIn[135], line 6\u001b[0m\n\u001b[1;32m 3\u001b[0m numerator \u001b[38;5;241m=\u001b[39m \u001b[38;5;241m0.1\u001b[39m\n\u001b[1;32m 5\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m i \u001b[38;5;129;01min\u001b[39;00m denominator:\n\u001b[0;32m----> 6\u001b[0m s \u001b[38;5;241m=\u001b[39m \u001b[43mnumerator\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m/\u001b[39;49m\u001b[43m \u001b[49m\u001b[43mi\u001b[49m\n\u001b[1;32m 7\u001b[0m \u001b[38;5;28mprint\u001b[39m(s)\n", "\u001b[0;31mZeroDivisionError\u001b[0m: float division by zero" ] } ], "source": [ "denominator = [1.2, 4.0, 0.001, 45.0, 0.0]\n", "\n", "numerator = 0.1\n", "\n", "for i in denominator:\n", " s = numerator / i\n", " print(s)" ] }, { "cell_type": "markdown", "id": "c4141532", "metadata": {}, "source": [ "We need to put safe guards with \n", "```python\n", "except ZeroDivisionError as e:\n", "```" ] }, { "cell_type": "code", "execution_count": 136, "id": "95eca0cd", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0.08333333333333334\n", "0.025\n", "100.0\n", "0.0022222222222222222\n", "Division by zero, i = 0.0\n" ] } ], "source": [ "try:\n", " for i in denominator:\n", " s = numerator / i\n", " print(s)\n", "\n", "except ZeroDivisionError as e:\n", " print('Division by zero, i = ', i)" ] }, { "cell_type": "markdown", "id": "7b467631", "metadata": {}, "source": [ "***\n", "### JSON (Exercise)\n", "\n", "JSON is JavaScript Object Notation. It is used to store and transfer data. \n", "\n", "- jason.dump() writes data in json file\n", "- json.load() reads data from json file\n", "- json.dumps() encodes a python object (usually dictionary) into json\n", "- json.loads() decodes json into python object" ] }, { "cell_type": "markdown", "id": "eb475498", "metadata": {}, "source": [ "#### Dump data to json file" ] }, { "cell_type": "code", "execution_count": 137, "id": "a023cace", "metadata": {}, "outputs": [], "source": [ "import json\n", "\n", "# Define dictionary with services and their server IP addresses\n", "\n", "services = {\n", " # for all servers in NJ (building : re)\n", "'re':\n", "{'dns': ['192.168.3.250', '192.168.11.104'],\n", "'dhcp': '192.168.3.250',\n", "'ldapmaster': '192.168.3.100' ,\n", "'ldap': ['192.168.3.100', '192.168.11.104'],\n", "'ravada': '192.168.3.40'\n", "},\n", " # For all the servers in FL (building : fl)\n", "'fl':\n", "{'dns': ['192.168.11.104', '192.168.3.250'],\n", "'dhcp': '192.168.11.104',\n", "'ldapmaster': '192.168.3.100',\n", "'ldap': '[192.168.11.104, 192.168.3.100]',\n", "'ravada': '192.168.11.100'\n", "}\n", "}\n", "\n", "# the json file where the output must be stored \n", "with open(\"services.json\", \"w\") as out_file:\n", " json.dump(services, out_file, indent = 4) \n", " \n" ] }, { "cell_type": "code", "execution_count": 138, "id": "d02fadce", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"re\": {\n", " \"dns\": [\n", " \"192.168.3.250\",\n", " \"192.168.11.104\"\n", " ],\n", " \"dhcp\": \"192.168.3.250\",\n", " \"ldapmaster\": \"192.168.3.100\",\n", " \"ldap\": [\n", " \"192.168.3.100\",\n", " \"192.168.11.104\"\n", " ],\n", " \"ravada\": \"192.168.3.40\"\n", " },\n", " \"fl\": {\n", " \"dns\": [\n", " \"192.168.11.104\",\n", " \"192.168.3.250\"\n", " ],\n", " \"dhcp\": \"192.168.11.104\",\n", " \"ldapmaster\": \"192.168.3.100\",\n", " \"ldap\": \"[192.168.11.104, 192.168.3.100]\",\n", " \"ravada\": \"192.168.11.100\"\n", " }\n", "}" ] } ], "source": [ "%cat services.json" ] }, { "cell_type": "markdown", "id": "41ae6207", "metadata": {}, "source": [ "#### Read data from json file:" ] }, { "cell_type": "code", "execution_count": 139, "id": "890b2d29", "metadata": {}, "outputs": [], "source": [ "\n", "with open('services.json','r') as f:\n", " data= json.load(f)" ] }, { "cell_type": "code", "execution_count": 140, "id": "c950f5cf", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"re\": {\n", " \"dns\": [\n", " \"192.168.3.250\",\n", " \"192.168.11.104\"\n", " ],\n", " \"dhcp\": \"192.168.3.250\",\n", " \"ldapmaster\": \"192.168.3.100\",\n", " \"ldap\": [\n", " \"192.168.3.100\",\n", " \"192.168.11.104\"\n", " ],\n", " \"ravada\": \"192.168.3.40\"\n", " },\n", " \"fl\": {\n", " \"dns\": [\n", " \"192.168.11.104\",\n", " \"192.168.3.250\"\n", " ],\n", " \"dhcp\": \"192.168.11.104\",\n", " \"ldapmaster\": \"192.168.3.100\",\n", " \"ldap\": \"[192.168.11.104, 192.168.3.100]\",\n", " \"ravada\": \"192.168.11.100\"\n", " }\n", "}" ] } ], "source": [ "%cat services.json" ] }, { "cell_type": "code", "execution_count": 141, "id": "5cc69123", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'re': {'dns': ['192.168.3.250', '192.168.11.104'],\n", " 'dhcp': '192.168.3.250',\n", " 'ldapmaster': '192.168.3.100',\n", " 'ldap': ['192.168.3.100', '192.168.11.104'],\n", " 'ravada': '192.168.3.40'},\n", " 'fl': {'dns': ['192.168.11.104', '192.168.3.250'],\n", " 'dhcp': '192.168.11.104',\n", " 'ldapmaster': '192.168.3.100',\n", " 'ldap': '[192.168.11.104, 192.168.3.100]',\n", " 'ravada': '192.168.11.100'}}" ] }, "execution_count": 141, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data" ] }, { "cell_type": "code", "execution_count": 142, "id": "0cc56268", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "dict" ] }, "execution_count": 142, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(data)" ] }, { "cell_type": "code", "execution_count": 143, "id": "c52634a5", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "re : {'dns': ['192.168.3.250', '192.168.11.104'], 'dhcp': '192.168.3.250', 'ldapmaster': '192.168.3.100', 'ldap': ['192.168.3.100', '192.168.11.104'], 'ravada': '192.168.3.40'} \n", "\n", "fl : {'dns': ['192.168.11.104', '192.168.3.250'], 'dhcp': '192.168.11.104', 'ldapmaster': '192.168.3.100', 'ldap': '[192.168.11.104, 192.168.3.100]', 'ravada': '192.168.11.100'} \n", "\n" ] } ], "source": [ "for keys in data:\n", " print(keys,':',data[keys],'\\n')" ] }, { "cell_type": "markdown", "id": "52199b4c", "metadata": {}, "source": [ "#### Encode a dictionary into json string:\n", "\n" ] }, { "cell_type": "code", "execution_count": 144, "id": "d878d152", "metadata": {}, "outputs": [], "source": [ "json_string = json.dumps(services)" ] }, { "cell_type": "code", "execution_count": 145, "id": "a3c7e890", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\"re\": {\"dns\": [\"192.168.3.250\", \"192.168.11.104\"], \"dhcp\": \"192.168.3.250\", \"ldapmaster\": \"192.168.3.100\", \"ldap\": [\"192.168.3.100\", \"192.168.11.104\"], \"ravada\": \"192.168.3.40\"}, \"fl\": {\"dns\": [\"192.168.11.104\", \"192.168.3.250\"], \"dhcp\": \"192.168.11.104\", \"ldapmaster\": \"192.168.3.100\", \"ldap\": \"[192.168.11.104, 192.168.3.100]\", \"ravada\": \"192.168.11.100\"}}\n" ] } ], "source": [ "print(json_string)" ] }, { "cell_type": "code", "execution_count": 146, "id": "286c7e8f", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"re\": {\n", " \"dns\": [\n", " \"192.168.3.250\",\n", " \"192.168.11.104\"\n", " ],\n", " \"dhcp\": \"192.168.3.250\",\n", " \"ldapmaster\": \"192.168.3.100\",\n", " \"ldap\": [\n", " \"192.168.3.100\",\n", " \"192.168.11.104\"\n", " ],\n", " \"ravada\": \"192.168.3.40\"\n", " },\n", " \"fl\": {\n", " \"dns\": [\n", " \"192.168.11.104\",\n", " \"192.168.3.250\"\n", " ],\n", " \"dhcp\": \"192.168.11.104\",\n", " \"ldapmaster\": \"192.168.3.100\",\n", " \"ldap\": \"[192.168.11.104, 192.168.3.100]\",\n", " \"ravada\": \"192.168.11.100\"\n", " }\n", "}\n" ] } ], "source": [ "print(json.dumps(services, indent=4))" ] }, { "cell_type": "code", "execution_count": null, "id": "c16c9fcb", "metadata": { "collapsed": true, "jupyter": { "outputs_hidden": true } }, "outputs": [], "source": [] }, { "cell_type": "markdown", "id": "d84da0d6", "metadata": {}, "source": [ "#### Decode json string into a dictionary:" ] }, { "cell_type": "code", "execution_count": 147, "id": "8617fb91", "metadata": {}, "outputs": [], "source": [ "dict = json.loads(json_string)" ] }, { "cell_type": "code", "execution_count": 148, "id": "5ac38e77", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'re': {'dns': ['192.168.3.250', '192.168.11.104'], 'dhcp': '192.168.3.250', 'ldapmaster': '192.168.3.100', 'ldap': ['192.168.3.100', '192.168.11.104'], 'ravada': '192.168.3.40'}, 'fl': {'dns': ['192.168.11.104', '192.168.3.250'], 'dhcp': '192.168.11.104', 'ldapmaster': '192.168.3.100', 'ldap': '[192.168.11.104, 192.168.3.100]', 'ravada': '192.168.11.100'}}\n" ] } ], "source": [ "print(dict)" ] }, { "cell_type": "markdown", "id": "84f73121", "metadata": {}, "source": [ "#### Get the values by referencing the keys in the dictionaries:" ] }, { "cell_type": "code", "execution_count": 149, "id": "4b288bef", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "['192.168.3.250', '192.168.11.104']\n", "['192.168.11.104', '192.168.3.250']\n" ] } ], "source": [ "for keys in dict:\n", " dict2 = dict[keys]\n", " print(dict2['dns'])" ] }, { "cell_type": "code", "execution_count": null, "id": "08dd30b1", "metadata": { "collapsed": true, "jupyter": { "outputs_hidden": true } }, "outputs": [], "source": [] }, { "cell_type": "markdown", "id": "e7c23586", "metadata": {}, "source": [ "### Parsing arguments with argparse (Exercise)\n", "\n", "It allows to pass parameters to a python script.\n", "\n", "All the exercises below should run in the Linux command prompt.\n", "There are 3 python scripts in the directory: parser_1.py, parser_2.py, and parser_3.py\n", "\n", "\n", "Exersizes:" ] }, { "cell_type": "markdown", "id": "7c071176", "metadata": {}, "source": [ "```bash\n", "# Import the library\n", "import argparse\n", "# Create the parser\n", "parser = argparse.ArgumentParser()\n", "# Add an argument\n", "parser.add_argument('--alpha', type=int, required=True)\n", "# Parse the argument\n", "args = parser.parse_args()\n", "\n", "# use and print the argument\n", "value = args.alpha * 10\n", "print('alpha = ', args.alpha, f', value = {args.alpha} * 10 = ', value)\n", "```" ] }, { "cell_type": "markdown", "id": "d5921bb9", "metadata": {}, "source": [ "#### Run the script\n", "\n", "Save the above cell in a script, say parser_1.py, specify that it is a python script in the header:\n", "```bash\n", "#!/home/hostadm/miniconda3/bin/python3\n", "```\n", "\n", "Make it executable:\n", "```bash\n", "chmod a+x parser_1.py\n", "```\n", "\n", "Run the script with the parameter:\n", "```bash\n", "./parser_1.py --alpha 15\n", "```" ] }, { "cell_type": "markdown", "id": "06bd71d8", "metadata": {}, "source": [ "#### Multiple arguments with help options" ] }, { "cell_type": "markdown", "id": "34659ec3", "metadata": {}, "source": [ "```bash\n", "import argparse\n", "parser = argparse.ArgumentParser()\n", "\n", "parser.add_argument('--a1', type = str, required=True, help = 'First coefficient')\n", "parser.add_argument('--a2', type = str, required=True, help = 'Second coefficient')\n", "parser.add_argument('--a3', type = str, help = 'Third coefficient')\n", "\n", "args = parser.parse_args()\n", "\n", "s = f'{args.a1}*x^2 + {args.a2}*x + {args.a2}'\n", "\n", "print(s)\n" ] }, { "cell_type": "markdown", "id": "f81d008c", "metadata": {}, "source": [ "#### Invoking help\n", "\n", "Save the content of the cell above in file, say, parser_2.py, then run as follows:\n", "\n", "```bash\n", "./parser_2.py -h\n", "./parser_2.py --a1 34 --a2 55 --a3 56\n", "```" ] }, { "cell_type": "markdown", "id": "80cc7d46", "metadata": {}, "source": [ "#### Multiple arguments of the same type\n", "\n", "```bash\n", "import argparse\n", "parser = argparse.ArgumentParser()\n", "\n", "parser.add_argument('--values', type=int, nargs=3)\n", "\n", "args = parser.parse_args()\n", "sum = sum(args.values)\n", "print('Sum:', sum)\n", "```" ] }, { "cell_type": "markdown", "id": "227541f2", "metadata": {}, "source": [ "Run it with values:\n", "\n", "```bash\n", "./parser_3.py --values 4 8 9\n", "```" ] }, { "cell_type": "markdown", "id": "362d5c58", "metadata": {}, "source": [ "For any number of values, set ```nargs = '+'```" ] }, { "cell_type": "markdown", "id": "df4de3be", "metadata": {}, "source": [ "### Reference\n", "\n", "[Good tutorial on subparsers](https://towardsdatascience.com/a-simple-guide-to-command-line-arguments-with-argparse-6824c30ab1c3)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.2" } }, "nbformat": 4, "nbformat_minor": 5 }