Skip to content

PyTorch

Description

PyTorch is deep learning framework that puts Python first.

Access

PyTorch is available on the Grace, FASTER, ACES, and Launch clusters. PyTorch is open to all HPRC users.

For a list of modules that have been optimized for HPRC clusters, run:

mla | grep PyTorch

The ones with 'cuda' in the name (e.g. PyTorch/2.1.2-CUDA-12.1.1) are optimized to utilize NVIDIA GPUs for acceleration using the CUDA toolkit.

You can learn more about the module system on our SW:Modules page.

Example PyTorch Script

As with any job on the system, PyTorch should be used via the submission of a job file. Scripts using PyTorch are written in Python, and thus PyTorch scripts should not be written directly inside a job file or entered in the shell line by line. Instead, a separate file for the Python/PyTorch script should be created, which can then be executed by the job file.

To create a new script file, simply open up the text editor of your choice.

Below is an example script (entered in the text editor of your choice) from http://pytorch.org/tutorials/beginner/pytorch_with_examples.html:

# -*- coding: utf-8 -*-

import torch
import math


dtype = torch.float
device = torch.device("cpu")
# device = torch.device("cuda:0") # Uncomment this to run on GPU

# Create random input and output data
x = torch.linspace(-math.pi, math.pi, 2000, device=device, dtype=dtype)
y = torch.sin(x)

# Randomly initialize weights
a = torch.randn((), device=device, dtype=dtype)
b = torch.randn((), device=device, dtype=dtype)
c = torch.randn((), device=device, dtype=dtype)
d = torch.randn((), device=device, dtype=dtype)

learning_rate = 1e-6
for t in range(2000):
    # Forward pass: compute predicted y
    y_pred = a + b * x + c * x ** 2 + d * x ** 3

    # Compute and print loss
    loss = (y_pred - y).pow(2).sum().item()
    if t % 100 == 99:
        print(t, loss)

    # Backprop to compute gradients of a, b, c, d with respect to loss
    grad_y_pred = 2.0 * (y_pred - y)
    grad_a = grad_y_pred.sum()
    grad_b = (grad_y_pred * x).sum()
    grad_c = (grad_y_pred * x ** 2).sum()
    grad_d = (grad_y_pred * x ** 3).sum()

    # Update weights using gradient descent
    a -= learning_rate * grad_a
    b -= learning_rate * grad_b
    c -= learning_rate * grad_c
    d -= learning_rate * grad_d


print(f'Result: y = {a.item()} + {b.item()} x + {c.item()} x^2 + {d.item()} x^3')

It is recommended to save this script with a .py file extension, but not necessary.

Once saved, the script can be tested on a login node by entering:

[NetID@cluster ~]$ python testscript.py

(Note: not all login nodes have a GPU. Please check before testing the GPU option.)