Job Submission¶

SLURM Overview¶

The cluster uses SLURM to manage and schedule jobs. You do not run simulations directly — instead, you submit a job to SLURM, which assigns it to available computing nodes.

How scheduling works¶

When you submit a job, it enters a queue.
Jobs are processed in first-come, first-served order.
If the requested nodes are busy, your job waits in the queue until resources become available.
Once resources free up, SLURM automatically starts the next job in line.
Each job has a walltime limit — if the job exceeds it, SLURM will kill it.

Partitions¶

Nodes are grouped into partitions. You must specify a partition when submitting a job. When running parallel jobs across multiple nodes, all nodes must be within the same partition.

Partition	Nodes	Max walltime
gen1	t01-t02	14 days
gen2	t03-t04	14 days
gen3	t05-t09	14 days
gen4	t10-t14	14 days

Submitting a CFX Job (runCFX)¶

runCFX is the command to submit an ANSYS CFX job to SLURM.

runCFX, pestat and query_license_ansys are shell aliases that your ~/.bashrc loads from /nopt/common/bash_aliases. If they are missing, copy /nopt/common/skel/.bashrc to your home directory and log in again.

Required options¶

Option	Description
`-f`	Path to the `.def` file (e.g. `-f mycase.def`)
`-c`	Number of CPU cores to use (e.g. `-c 128`)
`-p`	Partition name (e.g. `-p gen3`)

Optional options¶

Option	Description
`-w`	Specify nodes (e.g. `-w t05,t06` or `-w t[09-12]`)
`-t`	Walltime limit (format: `D-HH:MM:SS`, default: `3-00:00:00`)
`-o`	Pass additional options to cfx5solve (e.g. `-o "-double -ini-file file.res"`). See CFX Solver Manager User Guide Chapter 13 for all available options.

Examples¶

runCFX -c 32 -f mycase.def -p gen1
runCFX -c 256 -f mycase.def -p gen3 -w t[05-06] -t 7-00:00:00
runCFX -c 128 -f mycase.def -p gen4 -o "-double -ini-file restart.res"

Important rules¶

Run runCFX -h for full usage details.
When running across multiple nodes, use full node allocations only (e.g. 128, 256). Do not use partial allocations like 192 or 200.
Do not use more than 256 cores at once.

Monitoring Jobs¶

`squeue` — View job queue¶

Shows all currently queued and running jobs.

squeue

`pestat` — View node status¶

Shows the state of each node (idle, allocated, mixed, down).

pestat

Node states:

idle — No jobs running, available
alloc — Fully allocated
mix — Partially allocated
down — Offline
drain — Marked for maintenance, no new jobs accepted

`scancel` — Cancel a job¶

You can only cancel your own jobs.

scancel <job_id>

`query_license_ansys` — Check ANSYS license usage¶

Shows which licenses are currently in use and by whom.

query_license_ansys

Log Files¶

SLURM job logs are saved to ~/.slurm_log/.

.out file — standard output
.err file — error output

Filename format: <job_name>.<username>.<job_id>.out (job name defaults to CFX, e.g. CFX.shlee.4175.out)