email – Troubleshooting slurm e-mail settings

I am trying to setup a slurm installation and I have advanced towards the e-mail stage. So far I do not receive any mails. I have a working setup using msmtp-mta and msmtp. When I batch a script the slurmctld log shows

email msg to **@**: Slurm Job_id=73 Name=example_script.sh Began, Queued time 00:00:00

But I receive nothing. To troubleshoot the issue, I wrote a small script that does nothing but log calls:

#!/bin/bash

LOGFILE="${HOME}/mail.log"

echo "Called with args" >> ${LOGFILE}
echo $* >> ${LOGFILE}

# exit 1

I saved it as /usr/bin/slurm_mail (ls -l: -r-xr-xr-x 1 slurm slurm 109 Jan 27 14:00 /usr/bin/slurm_mail ), expecting some kind of output and set the configuration accordingly:

> scontrol show config | grep MailProg
MailProg                = /usr/bin/slurm_mail

But I still get absolutely nothing, so the script is apparently (?) never executed, despite being present on all hosts (actually one host, running slurmd and slurmctld). Is there any way to debug this issue (I set SlurmctldDebug to debug5 without getting any new information)?

Read more here: Source link