Skip to content

Vector Engine

Introduction

The NEC Vector Engine (VE) compute node in ACES has 8 VE Type 20B-P cards. Each VE card has 8 VE cores and 48GB HBM2 memory.

Accessing Vector Engines

This VE compute node is available via the nec partition.

Interactive

To access the VE compute node interactively to develop your programs against the VE cards:

srun --partition nec --pty --time=4:00:00 --nodes=1 --gres=gpu:ve:8 /bin/bash

The requested number of VEs can be 1 to 8.

To setup your interactive job environment for the NEC compiler:

export PATH=/opt/nec/ve/bin/:$PATH
source /opt/nec/ve/mpi/3.4.0/bin64/necmpivars.sh

Turn on printing details about VE card usage:

export VE_PROGINF=DETAIL

Job Submission

To submit batch jobs against the VE node, include the following Slurm directives in your job scripts:

#SBATCH --partition=nec
#SBATCH --gres=gpu:ve:N

The requested number of VEs can be 1 to 8.

Information

MPI Usage

Example MPI runs:

mpirun –ve 0 –np 8 executable     # run on 1 VE card (ve card 0) with 8 VE cores
mpirun –ve 0-1 –np 16 executable  # run on 2 VE cards (VE card 0 and 1) with 8 VE cores per card
mpirun –ve 0-7 –np 64 executable  # run on 8 VE cards (VE cards 0-7) with 8 VE cores per card

References

Additional details and tutorials:

NEC Vector Engine: Vectorization for HPC applications
VASP on NEC VE

NEC Compiler user manuals:

C/C++ Compiler
Fortran Compiler