Lmod NDSU lab

Lmod, A Modern Environment Module System

Alexei Kotelnikov, PhD
School of Engineering, Rutgers University

Outline of the presentation

  • User environments in the Unix/Linux shell.

  • Tcl/C module examples and motivation for the Lua based Lmod modules.

  • Setup Lmod for the lab exercises.

  • MODULEPATH and explanation how LMOD works

  • Practice with basic module commands and examples.

  • Module lua file structure.

  • Useful features of LMOD

  • LMOD usage tracking through syslog

Why to use environment modules

Reason: Environment variables affected:
Different Compilers: gcc 10.2.1, gcc 12.0.0, Intel 2020, etc PATH, CPATH, LIBRARY_PATH
Different apps: shared object libs (*.so), MPI, Java, Licenses, etc LD_LIBRARY_PATH, MPI_HOME, JAVA_HOME, LM_LICENSE_FILE

Environment modules allow switching seamlessly between the user environments without need for modifying /etc/profile or .bashrc files.


Classical TCL/C Environment Modules

  • To overcome the difficulty of setting and changing the Environment variables, the TCL/C Modules were introduced over 2 decades ago.

  • They were written in C and used TCL language for environment configuration.

  • Very robust and easy to use, for example, to load Intel 2018 compiler:

module load intel/2018
  • Check what the modules are loaded:

module list
  • See what the modules are available to load:

module avail
  • Unload the module:

module unload intel/2018

TCL/C modules drawback and motivation for migration to Lmod

  • The classical TCL/C modules allow conflicting environment modules to be loaded at the same time. For example:

module switch intel/2018 intel/2013
module list

shows: intel/2013 openmpi/2.1.2-intel2018

  • Users should watch what modules are loaded before performing compilation and running their codes.

  • The rule of thumb is, first, to unload all the modules with command

module purge
  • Then load specifically what is needed, for example:

module load intel/2018
module load openmpi/2.1.2-intel2018
  • Better approach is switching to Lmod.

  • The Lmod configuration can protect users from loading incompatible environment settings.

Lmod (“L” stands for Lua) vs TCL/C modules

  • Lmod is an environment modules implementation that provides all of the functionality of TCL/C Environment Modules plus more features.

  • Lmod can read modules written in TCL and in lua.

  • Support for hierarchical module file structure.

  • MODULEPATH is dynamically updated when modules are loaded.

  • Makes loaded modules inactive and active to provide sane environment.

  • User module collections.

  • Hidden modules.

  • Optional usage tracking.

  • Many useful functions in Lua module configuration files, such as family(“compiler”), property(“gpu”), etc

LMOD installation steps

  • Install Lua

  • Install LMOD

  • Setup the environment configuration in <module_name>.lua files.

  • Initialize MODULEPATH and MODULEPATH_ROOT entries in .bashrc

  • Details are here: Installing Lua and Lmod

The Lab cluster

LMOD setup on your lab cluster

SSH to to the Orbit firewall.

SSH to one of the nodes, lxc01, lxc02, etc.

Install ansible:

sudo apt install ansible

Download a tar ball with Ansible playbook for LMOD

wget https://linuxcourse.rutgers.edu/LCI_2023/Ansible-lmod-ndsu.tgz
tar -zxvf Ansible-lmod-ndsu.tgz
cd Ansible-lmod* 
ansible-playbook install_lmod.yml

In .bashrc file of user ccast, remove the settings for openmpi, and add the following block at the bottom:

if [ -z "$BASHRC_READ" ]
  export BASHRC_READ=1
  . /usr/share/lmod/lmod/init/profile
  export MODULEPATH_ROOT=/usr/share/lmod/lmod/modulefiles
  export LMOD_PACKAGE_PATH=/usr/share/lmod/lmod/libexec

Alternatively you can overwrite his .bashrc file as follows:

cp files/bashrc-lmod ~ccast/.bashrc
module avail

LMOD demo with loading and unloading the modules.

  • Check available modules:

module available
  • Shortcut:

ml av
  • See all modules via spider:

module spider
  • Shortcut:

ml spider
  • Load module gcc/10.2.1

module load gcc/10.2.1
  • Shortcut

ml gcc/10.2.1

  • List loaded modules:

module list
  • Shortcut:

  • See what modules need to be loaded for application lammps:

module spider lammps

  • Load the prereq modules:

module load openmpi/4.1.5
  • See what modules are available now:

ml av
  • See if there are hidden modules:

ml --show-hidden av
  • Load lammps:

ml lammps

  • Unloading the base module, gcc/10.2.1, will deactivate the others, openmpi/4.1.4 and lammps:

module unload gcc/10.2.1
  • The dependent modules will also get deactivated if the different compiler is loaded, for example:

module load intel/2018
  • To unload all the modules:

module purge

How the things work

  • File .bashrc references the Lua modules and initialize MODULEPATH_ROOT and MODULEPATH environment variables:

if [ -z "$BASHRC_READ" ]
  export BASHRC_READ=1
  . /usr/share/lmod/lmod/init/profile
  export MODULEPATH_ROOT=/usr/share/lmod/lmod/modulefiles


  • The initial MODULEPATH=modulefiles/Core

  • After command module load gcc/10.2.1, the MODULEPATH gets updated: MODULEPATH=modulefiles/Compiler/gcc/10.2.1:modulefiles/Core

  • After adding another module, module load openmpi/4.1.5, the MODULEPATH contains the directory with file lammps.lua

|-- Compiler
|   |-- gcc
|   |    \-- 10.2.1
|   |         \-- openmpi
|   |              \-- 4.1.5.lua <-- 2 updates: 
|    \-- intel      "MODULEPATH=modulefiles/MPI/gcc/10.2.1/4.1.5:$MODULEPATH"
|       |-- 2013
|       |    \-- openmpi
|       |       |-- 1.6.5.lua
|       |        \-- 1.8.8.lua
|        \-- 2018
|            -- openmpi
|               |-- 1.8.8.lua
|               |-- 2.1.1.lua
|               \-- 2.1.2.lua
|-- Core
|   |-- gcc
|   |    \-- 10.2.1.lua <--- 1 updates:  
|    \-- intel     "MODULEPATH=modulefiles/Compiler/gcc/10.2.1:modulefiles/Core"
|       |-- 2013.lua
|        \-- 2018.lua
 \-- MPI
    |-- gcc
    |    \-- 10.2.1
    |         \-- 4.1.5
    |             \-- lammps.lua  <-- 3 is in the updated MODULEPATH
     \-- intel
          \-- 2018
              \-- 2.1.2
                  \-- vasp.lua

Module file example: 10.2.1.lua

-- -*- lua -*-

help([[ This is a gcc 10.2.1 compiler, came with Red Hat 8.6 packages]])

-- Local variables
local version = "10.2.1"

-- Whatis description
whatis("Description: GCC compiler") 

-- Setup Modulepath for packages built by this compiler
local mroot = os.getenv("MODULEPATH_ROOT")
local mdir = pathJoin(mroot,"Compiler/gcc", version)
prepend_path("MODULEPATH", mdir)
-- Set family for this module

Helpful LMOD functionalities.

  • Module tags, for example:

  • Hide modules when software is updated or decommissioned. In the recent versions or lmod, version 8.x, make the module file hodden:

mv lammps.lua .lammps.lua

In the older versions of lmod, 6.x and 7.x, place file .modulerc.lua in the same directory with the module, for example, in directory modulefiles/MPI/gcc/10.2.1/4.1.5:

  • See what modules need to be loaded for application lammps:

module --show-hidden spider
  • User module collections can be saved and restored by a user, for example:

module load gcc/10.2.1
module load openmpi/4.1.5
module save gcc-openmpi

It saves the environment configuration into ~/.lmod.d/gcc-openmpi Lua file.

To restore the environment:

module restore gcc-openmpi

One can restore the module collections through .bashrc:

module -q restore gcc-openmpi
  • Lmod can track environment module usage by users.

Lmod configuration for module usage.

  • Add new path, /usr/share/lmod/lmod/libexec_log, to LMOD_PACKAGE_PATH into file .bashrc for ccast:

export LMOD_PACKAGE_PATH=/usr/share/lmod/lmod/libexec:/usr/share/lmod/lmod/libexec_log
  • Create directory /usr/share/lmod/lmod/libexec_log:

sudo mkdir /usr/share/lmod/lmod/libexec_log
  • Copy file SitePackage_LOGS.lua into SitePackage.lua in the new directory:

sudo cp /usr/share/lmod/lmod/libexec/SitePackage_LOGS.lua /usr/share/lmod/lmod/libexec_log/SitePackage.lua
  • Login to ccast.

sudo su - ccast
  • Make sure the updated LMOD_PACKAGE_PATH environment variable is in the user environment.

  • Start loading modules as ccast, for example:

module load gcc/10.2.1 openmpi/4.1.5
  • Check the log updates with ModuleUsageTracking tag in /var/log/messages

grep ModuleUsageTracking /var/log/messages

Questions and discussion: