Submit a job for the serial version of the project

This section put everything that has been discussed previously together in order to submit a job running the serial version of the project.

Prepare your environment

Once we are logged in to NIC5, we will load the module that set up the environment for the project. The module name Info0939Tools and we load it with

module load Info0939Tools

Loading this module gives us access to the get_info0939_project command. We can use this command to get starting point source code and example inputs for the project.

 $ get_info0939_project
Project copied in the project_info0939 directory

Next, we use the cd command to move to the project_info0939 directory.

cd project_info0939

Compile your code

Now that we have the source code, we can compile it to produce an executable. Here, we will put the executable in a directory with name bin. For that, we create a directory with mkdir and then we use gcc to compile the code.

mkdir bin
gcc -O3 -Wall -o bin/fdtd fdtd.c -lm

In the table below, a summary of the options used and their meaning.

	Meaning
`-O3`	enable optimization at the `O3` level
`-Wall`	enable all compiler warnings
`-o bin/fdtd`	specify the name and location of the executable after compilation
`fdtd.c`	the source file
`-lm`	link with the math library (`libm`)

Submit a job

Now that we have an executable, we will create a directory from which we will submit a job.

mkdir -p workdir/serial
cd workdir/serial

Good practice

It's considered good practice to create a new directory for each job you submit.

To submit the job, we will create a text file with name fdtd_serial.sh

nano fdtd_serial.sh

and copy the content of the code block below.

Source code for this example

fdtd_serial.sh

#!/bin/bash
#SBATCH --job-name="fdtd serial"
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH --time=05:00
#SBATCH --output=fdtd_serial_%j.out

# define some variables
PROJECT_DIR="${HOME}/project_info0939"
EXEC_DIR="${PROJECT_DIR}/bin"
INPUT_DIR="${PROJECT_DIR}/example_inputs/simple3d"
PARAM_FILE="param_3d.txt"

# copy all files from the input directory to
# the directory from which we submitted the job
cp ${INPUT_DIR}/* ${SLURM_SUBMIT_DIR}

# move to the directory from which we submitted the job
cd ${SLURM_SUBMIT_DIR}

# run the simulation
${EXEC_DIR}/fdtd ${PARAM_FILE}

To save the changes to the file, press the Ctrl+X keys then press Y to confirm you want to save the change and finally, press Enter to confirm the name of the file you want to write.

Additional details about the job batch script

The following line for the Slurm output specification:
```
#SBATCH --output=fdtd_serial_%j.out
```
means that the job ID will be inserted in the name of the output file. %j will be automatically replaced by Slurm and set to the job ID.
In the script we use variables to define various paths. A variable is defined with
```
VARIABLE_NAME=VARIABLE_VALUE
```
Once a variable is defined it can be used using $VARIABLE_NAME or ${VARIABLE_NAME}.
In the script, we use the HOME environment variable. The value of the variable is the path to your home directory. If run the following command
```
 $ echo $HOME
/home/ulg/info0939/USERNAME
```
the output will be the path to your home directory.
The expression ${INPUT_DIR}/* use a wildcard. In the context of our script, the wildcard (*) means "match any files". If we use
```
${INPUT_DIR}/in_*
```
then, the expression will match any file starting with in_.

The next step is to submit the job to the queue

 $ sbatch fdtd_serial.sh
Submitted batch job 5980715

and check the status of the job using the squeue command

 $ squeue --me
  JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
5980715     batch fdtd ser  olouant  R       0:13      1 nic5-w058

Once the job is finished, the output should contain

 $ cat fdtd_serial_5980715.out

 Grid spacing: 0.01
  Grid size X: 100
  Grid size Y: 100
  Grid size Z: 100
    Time step: 5e-07
 Maximum time: 0.001

     Output rate: every 200 step(s)
 Output sampling: 10000 Hz

 Output files:

      pressure: complete dump to file out_p.dat
    velocity X: complete dump to file out_vx.dat
    velocity Y: complete dump to file out_vy.dat
    velocity Z: complete dump to file out_vz.dat

 Source infos:

          type: sine wave
     frequency: 3400 Hz
    position x: 0.5
    position y: 0.5
    position z: 0.5

step      200/2000 (ETA:  105.374 seconds)
step      400/2000 (ETA:   93.607 seconds)
step      600/2000 (ETA:   81.944 seconds)
step      800/2000 (ETA:   70.176 seconds)
step     1000/2000 (ETA:   58.412 seconds)
step     1200/2000 (ETA:   46.609 seconds)
step     1400/2000 (ETA:   34.924 seconds)
step     1600/2000 (ETA:   23.320 seconds)
step     1800/2000 (ETA:   11.649 seconds)
step     2000/2000

Elapsed 116.589941 seconds (17.163 Mupdates/s)