PyTorch
Description
PyTorch is deep learning framework that puts Python first.
- Homepage: https://PyTorch.org/
Access
PyTorch is available on the Grace, FASTER, ACES, and Launch clusters. PyTorch is open to all HPRC users.
PyTorch modules (recommended)
For a list of modules that have been optimized for HPRC clusters, run:
mla | grep PyTorch
The ones with 'cuda' in the name (e.g.
PyTorch/2.1.2-CUDA-12.1.1
) are optimized to utilize NVIDIA GPUs for acceleration using the CUDA toolkit.
You can learn more about the module system on our SW:Modules page.
Example PyTorch Script
As with any job on the system, PyTorch should be used via the submission of a job file. Scripts using PyTorch are written in Python, and thus PyTorch scripts should not be written directly inside a job file or entered in the shell line by line. Instead, a separate file for the Python/PyTorch script should be created, which can then be executed by the job file.
To create a new script file, simply open up the text editor of your choice.
Below is an example script (entered in the text editor of your choice) from http://pytorch.org/tutorials/beginner/pytorch_with_examples.html:
# -*- coding: utf-8 -*-
import torch
import math
dtype = torch.float
device = torch.device("cpu")
# device = torch.device("cuda:0") # Uncomment this to run on GPU
# Create random input and output data
x = torch.linspace(-math.pi, math.pi, 2000, device=device, dtype=dtype)
y = torch.sin(x)
# Randomly initialize weights
a = torch.randn((), device=device, dtype=dtype)
b = torch.randn((), device=device, dtype=dtype)
c = torch.randn((), device=device, dtype=dtype)
d = torch.randn((), device=device, dtype=dtype)
learning_rate = 1e-6
for t in range(2000):
# Forward pass: compute predicted y
y_pred = a + b * x + c * x ** 2 + d * x ** 3
# Compute and print loss
loss = (y_pred - y).pow(2).sum().item()
if t % 100 == 99:
print(t, loss)
# Backprop to compute gradients of a, b, c, d with respect to loss
grad_y_pred = 2.0 * (y_pred - y)
grad_a = grad_y_pred.sum()
grad_b = (grad_y_pred * x).sum()
grad_c = (grad_y_pred * x ** 2).sum()
grad_d = (grad_y_pred * x ** 3).sum()
# Update weights using gradient descent
a -= learning_rate * grad_a
b -= learning_rate * grad_b
c -= learning_rate * grad_c
d -= learning_rate * grad_d
print(f'Result: y = {a.item()} + {b.item()} x + {c.item()} x^2 + {d.item()} x^3')
It is recommended to save this script with a .py file extension, but not necessary.
Once saved, the script can be tested on a login node by entering:
[NetID@cluster ~]$ python testscript.py
(Note: not all login nodes have a GPU. Please check before testing the GPU option.)