API reference

PyTorch API reference

PyTorch deep learning

Identifiers: 126
With full guides: 107
Types: 3

Topics covered

Tensors
Autograd
nn.Module
Optimizers
DataLoader
Training loop
GPU / CUDA
Saving models

Class

Name	What it does
`torch.Tensor`	`torch.Tensor` is PyTorch’s multidimensional container for values sharing one data type. Create tensors from existing data with `torch.tensor()` or use factory operations to control shape, dtype, device, and related properties.
`nn.Module`	torch.nn.Module is the base class for PyTorch neural-network modules. Subclass it to define models or reusable components, compose child modules, and manage registered parameters and module-wide conversions.
`nn.Linear`	torch.nn.Linear is a trainable affine layer that maps each input vector from in_features to out_features. It preserves all leading dimensions and can optionally include a learnable bias.
`nn.Conv2d`	torch.nn.Conv2d applies a learnable 2D cross-correlation to channel-first image-like tensors. Stride, padding, dilation, and groups control spatial sampling and channel connectivity.
`DataLoader`	DataLoader wraps a dataset in an iterable that can customize sample order, batch samples, use single or multiple processes, and optionally pin fetched tensors in memory.
`Dataset`	torch.utils.data.Dataset is the abstract base for map-style datasets. Subclasses provide keyed sample retrieval with __getitem__ and may provide their size through __len__.
`nn.Conv1d`	nn.Conv1d applies learnable one-dimensional filters to multi-channel signals. It converts C_in input channels into C_out output channels while stride, padding, dilation, and groups control how the filters scan and connect.
`nn.MaxPool2d`	Class
`nn.AvgPool2d`	Applies 2D average pooling to spatial dimensions while preserving the number of channels. Its window, stride, padding, boundary rounding, padding inclusion, and divisor are configurable.
`nn.BatchNorm2d`	torch.nn.BatchNorm2d normalizes four-dimensional N×C×H×W inputs independently per channel. It normally tracks running statistics for evaluation and can apply learnable channel-wise scale and bias.
`nn.LayerNorm`	torch.nn.LayerNorm normalizes the final dimensions of each input using its mean and variance. It can apply optional learnable per-element scale and bias while preserving the input shape.
`nn.Dropout`	torch.nn.Dropout randomly zeros input elements during training with probability p. It samples independently on each forward call and preserves the input shape.
`nn.LSTM`	Applies a multi-layer long short-term memory recurrent network to an input sequence. It supports stacked, bidirectional, dropout, and projected configurations.
`nn.GRU`	torch.nn.GRU applies one or more gated recurrent-unit layers to an input sequence. It supports configurable hidden size, stacking, dropout between layers, batch-first layouts, and bidirectional processing.
`nn.RNN`	nn.RNN is a multi-layer Elman recurrent module for sequence data. It supports tanh or ReLU activation, optional inter-layer dropout, and bidirectional processing.
`nn.Embedding`	nn.Embedding is a trainable lookup table for a fixed dictionary of vectors. Integer indices retrieve the corresponding rows, commonly converting token IDs into dense representations.
`nn.MultiheadAttention`	torch.nn.MultiheadAttention implements Transformer multi-head attention, combining attention from multiple representation subspaces. It supports configurable head count, dropout, projection biases, key/value dimensions, layouts, devices, and dtypes.
`nn.Transformer`	nn.Transformer is a reference encoder-decoder implementation of the original Transformer architecture. It processes source and target feature sequences and returns decoder outputs.
`nn.TransformerEncoder`	TransformerEncoder is a stack of TransformerEncoderLayer modules that processes a source sequence in order. It supports optional normalization, attention masks, causal masking, and nested-tensor conversion for highly padded inputs.
`nn.Sequential`	torch.nn.Sequential is a registered container that applies modules in order, passing each output to the next. It is useful for compact feed-forward chains and can accept positional modules or an OrderedDict of named modules.
`nn.ModuleList`	ModuleList stores child modules in a list-like container while registering them with the parent module. It supports indexed access and iteration, but the enclosing module controls how the children are executed.
`nn.Parameter`	torch.nn.Parameter is a Tensor subclass that marks tensor data as a module parameter. When assigned to a Module attribute, it is automatically registered and exposed through the module's parameter iterator.
`nn.ReLU`	nn.ReLU applies the rectified linear unit element-wise, replacing negative values with zero and retaining nonnegative values. It supports inputs of any dimensionality and returns the same shape.
`nn.GELU`	nn.GELU is a Gaussian Error Linear Units activation module. It supports the default formulation or a tanh-based approximation and preserves the input shape.
`nn.Softmax`	nn.Softmax normalizes tensor slices along a selected dimension into values from 0 to 1 whose sum is 1. It preserves the input shape.
`nn.Sigmoid`	`torch.nn.Sigmoid` applies the sigmoid function independently to every input element. It accepts tensors of any dimensionality and returns a tensor with the same shape.
`nn.Tanh`	Class
`nn.Flatten`	torch.nn.Flatten is a module that combines a contiguous range of dimensions into one. By default, it preserves dimension 0 and flattens dimensions 1 through the last dimension.
`nn.CrossEntropyLoss`	CrossEntropyLoss computes classification cross entropy from unnormalized logits and either class-index or class-probability targets. It supports class weighting, ignored indices, configurable reduction, and label smoothing.
`nn.MSELoss`	torch.nn.MSELoss measures squared differences between corresponding input and target elements. By default, it returns their mean; it can also return unreduced values or their sum.
`nn.BCELoss`	BCELoss measures binary cross-entropy between target values and input probabilities. It supports optional weighting and configurable none, mean, or sum reduction.
`nn.BCEWithLogitsLoss`	BCEWithLogitsLoss combines sigmoid and binary cross-entropy in one numerically stable module. It accepts logits and targets, supports optional weighting, and can return unreduced, averaged, or summed losses.
`nn.NLLLoss`	Class
`nn.L1Loss`	torch.nn.L1Loss measures mean absolute error between corresponding input and target elements. Its reduction can be disabled, averaged, or summed; mean is the default.
`nn.SmoothL1Loss`	SmoothL1Loss computes a robust regression loss that is quadratic for small errors and L1 for larger errors. Its reduction defaults to the mean, and beta controls the transition threshold.
`optim.Adam`	torch.optim.Adam implements the Adam optimization algorithm for neural-network parameters. It supports configurable learning rate, moment coefficients, numerical stability, weight decay, AMSGrad, and multiple execution modes.
`optim.SGD`	Class
`optim.AdamW`	Class
`optim.RMSprop`	RMSprop is a PyTorch optimizer that scales parameter updates using a moving average of squared gradients. It also supports momentum, weight decay, centered normalization, graph capture, maximization, and differentiable optimization steps.
`optim.Adagrad`	Class
`lr_scheduler.StepLR`	Class
`lr_scheduler.CosineAnnealingLR`	CosineAnnealingLR applies cosine annealing to every optimizer parameter group. It gradually moves learning rates toward eta_min over T_max scheduler iterations.
`TensorDataset`	Class

Function

Name	What it does
`torch.tensor`	Creates a new tensor by copying array-like data. It can configure dtype, device, autograd tracking, and pinned CPU memory.
`torch.no_grad`	torch.no_grad disables gradient calculation for a scope or decorated function. It is useful for inference because ordinary results stop requiring gradients and autograd can use less memory.
`backward`	Function
`forward`	torch.nn.Module.forward defines the computation performed whenever a module is called. Subclasses override it to transform inputs into outputs, while calling the module instance preserves registered hook behavior.
`torch.save`	Serializes a PyTorch object to a file path or file-like destination. It uses Python pickling and defaults to a newer ZIP-based file format.
`F.relu`	Function
`torch.zeros`	Creates a tensor filled with zeros, using dimensions supplied as separate arguments or as a collection. Optional keywords select its dtype, layout, device, output buffer, and autograd behavior.
`torch.ones`	Creates a tensor filled with 1, using the supplied dimensions as its shape. Optional keywords control its output tensor, dtype, layout, device, and autograd behavior.
`torch.empty`	Allocates a tensor with a requested shape and optional dtype, layout, device, gradient, memory, and format settings. Its elements are uninitialized, so write valid values before reading them.
`torch.full`	Creates a tensor with a specified shape and fills every element with one scalar value. Its dtype is inferred from the fill value unless dtype is provided explicitly.
`torch.arange`	Creates a 1-D tensor containing values from start up to, but excluding, end, separated by step. start defaults to 0 and step to 1; dtype is inferred unless specified.
`torch.linspace`	Creates a one-dimensional tensor with a chosen number of evenly spaced values from start to end, including both endpoints. Optional arguments configure its dtype, layout, device, storage, and autograd behavior.
`torch.eye`	Creates a two-dimensional tensor with ones on its diagonal and zeros elsewhere. It can produce square or rectangular shapes and accepts options for dtype, layout, device, output reuse, and gradient tracking.
`torch.rand`	torch.rand creates a tensor of a requested shape with uniformly distributed random values in the half-open interval [0, 1). Optional arguments control the generator, output properties, device, gradient tracking, and memory allocation.
`torch.randn`	Creates a tensor of a requested shape with values sampled from the standard normal distribution. Shape dimensions may be separate arguments or a list/tuple, while keyword arguments control dtype, device, autograd, storage, and randomness.
`torch.randint`	Creates a tensor of uniformly distributed random integers in the half-open interval [low, high). The requested size determines its shape, and low defaults to zero.
`torch.from_numpy`	Creates a PyTorch Tensor from a NumPy ndarray while sharing the same memory. Changes made through either object are visible through the other, and the resulting tensor is not resizable.
`torch.as_tensor`	Converts array-like data to a tensor, sharing storage and preserving autograd history when possible. Matching tensors can be returned directly; dtype or device differences cause conversion.
`torch.zeros_like`	Creates a zero-filled tensor with the same size as an input tensor. By default, it also follows the input's dtype, layout, device, and preserved memory format.
`torch.cat`	Concatenates a sequence of tensors along a specified existing dimension. Inputs must match in every other dimension, with a documented exception for a 1-D empty tensor.
`torch.stack`	torch.stack combines equally sized tensors by inserting a new dimension. The dim argument chooses where that dimension goes and defaults to 0.
`torch.reshape`	Function
`torch.permute`	Reorders a tensor’s dimensions and returns a view of the original tensor. The dims argument specifies the desired axis order.
`torch.transpose`	Swaps two selected dimensions of a tensor and returns the transposed result. For strided tensors, the result shares underlying storage with the input.
`torch.squeeze`	torch.squeeze removes dimensions of size 1 from a tensor. Supplying dim limits removal to the selected dimensions.
`torch.unsqueeze`	Adds a size-one dimension to a tensor at a chosen position. The returned tensor shares the input's underlying data.
`torch.flatten`	Flattens a tensor by combining a selected inclusive range of dimensions into one. By default, all dimensions are flattened while element order is preserved.
`torch.split`	Splits a tensor into chunks along a selected dimension. Use an integer for repeated chunk sizes or a list for explicit sizes; returned chunks are views of the input.
`torch.chunk`	Splits a tensor along a chosen dimension into a requested number of chunks. Outputs are views and may be fewer than requested when the division is not possible.
`torch.gather`	torch.gather selects values from input along a chosen dimension using an index tensor. The result has the same shape as index.
`torch.where`	torch.where selects elements from input or other according to a condition, supporting broadcasting across the operands. With only a condition, it returns the indices of true elements.
`torch.clamp`	Clamps every element of a tensor to a chosen lower and/or upper bound. Either bound can be omitted, and the bounds may be scalar values or tensors.
`torch.add`	Adds other, scaled by alpha, to input elementwise. Tensor operands support broadcasting and the operation returns a tensor.
`torch.mul`	torch.mul performs elementwise multiplication between a tensor and another tensor or number. It supports broadcasting and returns a tensor containing the products.
`torch.matmul`	torch.matmul computes tensor products whose behavior depends on operand dimensions, covering dot, matrix, vector, and batched matrix multiplication. Higher-rank batch dimensions broadcast when compatible.
`torch.mm`	torch.mm multiplies two 2-D tensors as matrices. Compatible shapes (n, m) and (m, p) produce a (n, p) tensor, without broadcasting.
`torch.bmm`	Performs matrix multiplication independently on each corresponding pair of matrices in two 3-D tensor batches. The batches must have equal sizes, and the result has shape (b, n, p) for inputs shaped (b, n, m) and (b, m, p).
`torch.einsum`	Computes tensor contractions using Einstein-summation notation. It aligns dimensions by labels, multiplies corresponding elements, and sums labels omitted from the output.
`torch.sum`	Returns the sum of all elements in a tensor, or reduces selected dimension(s). Use keepdim to retain reduced axes and dtype to control the accumulation and result type.
`torch.mean`	Computes the mean of all tensor elements or reduces selected dimensions. It supports optional dtype conversion and dimension retention.
`torch.std`	Calculates standard deviation over selected tensor dimensions. By default it uses Bessel’s correction; choose correction=0 for population standard deviation and keepdim=True to retain reduced axes.
`torch.max`	Returns the maximum value of a tensor, either globally or along a chosen dimension. Dimension-wise reduction also returns the index of each maximum.
`torch.min`	Returns the minimum value of a tensor, globally or along a selected dimension. Dimension-wise reduction also returns the index of each minimum.
`torch.argmax`	Returns the indices of maximum values in a tensor, either globally or across a selected dimension. If maxima are tied, it returns the first maximal index.
`torch.softmax`	torch.softmax converts tensor values into normalized, nonnegative scores along a chosen dimension. Each slice along that dimension sums to 1, and an optional dtype controls the computation type.
`torch.sigmoid`	Function
`torch.relu`	Function
`torch.exp`	Computes the elementwise exponential of a tensor. It can optionally write the result to a supplied output tensor.
`torch.log`	Computes the element-wise natural logarithm of a tensor. It returns a new tensor and optionally accepts an output tensor.
`torch.sqrt`	torch.sqrt returns a new tensor containing the element-wise square roots of an input tensor. An optional out tensor can receive the result.
`torch.abs`	Computes the absolute value of every element in a tensor and returns the resulting tensor. An optional output tensor can receive the result.
`torch.norm`	torch.norm computes a matrix or vector norm of a tensor, with configurable order, reduction dimensions, shape retention, output destination, and computation dtype. It is deprecated; new code should use the corresponding torch.linalg norm function.
`F.softmax`	Applies softmax along a selected tensor dimension, converting each slice into values from 0 to 1 that sum to 1. An optional dtype controls casting before computation.
`F.cross_entropy`	Computes cross-entropy loss from unnormalized logits and class-index or probability targets. Supports class weighting, ignored indices, configurable reduction, and label smoothing.
`F.mse_loss`	Function
`F.dropout`	F.dropout randomly zeroes input elements during training with probability p. Use training=False to disable masking, such as during evaluation.
`F.linear`	Function
`F.conv2d`	F.conv2d applies a 2D convolution to image-like input using explicit filters and optional bias. Parameters control kernel movement, padding, dilation, and channel grouping.
`F.normalize`	Normalizes vectors in a tensor along a chosen dimension using an Lp norm. It divides by the norm or eps, whichever is larger, with defaults of p=2 and dim=1.
`F.pad`	F.pad pads selected trailing dimensions of an N-dimensional tensor. It supports constant, reflection, replication, and circular modes, with zero-filled constant padding by default.
`F.interpolate`	Resizes 3-D, 4-D, or 5-D tensors by upsampling or downsampling their spatial dimensions. You choose the output size or scale factor and an interpolation mode such as nearest, bilinear, bicubic, trilinear, area, or nearest-exact.
`torch.load`	Loads an object saved by torch.save, restoring tensor storages to their saved devices or a location selected with map_location. Optional mmap enables lazy storage loading, while weights_only provides restricted deserialization.
`torch.device`	torch.device identifies where a tensor is or will be allocated. It supports device strings, type-plus-index construction, accelerator-relative integer indices, and temporary default-device contexts.
`autograd.grad`	Computes and returns gradients of selected outputs with respect to selected inputs, without accumulating them in the inputs' .grad fields. It supports vector-Jacobian products, graph retention, higher-order derivatives, and batched gradient vectors.

Module

Name	What it does
`torch.optim`	torch.optim provides optimization algorithms for updating model parameters from computed gradients. Create an optimizer with parameters and algorithm-specific options, then use it in the training loop to apply updates.
`autograd`	torch.autograd provides PyTorch’s automatic differentiation machinery for scalar-valued functions. Mark tensors with requires_grad=True, execute the computation, and use backward or related APIs to obtain gradients.
`cuda`	`torch.cuda` provides PyTorch support for CUDA tensor types and GPU computation. Import it safely, then use `torch.cuda.is_available()` to determine whether CUDA can be used.
`tensor.backward`	Computes gradients of a tensor through its autograd graph and accumulates them in leaf tensors. Non-scalar outputs require a matching gradient tensor.
`tensor.item`	Converts a one-element tensor into its value as a standard Python number. The operation is not differentiable and is intended for scalar extraction.
`tensor.detach`	Returns a tensor disconnected from the current autograd graph. The result shares storage with the source, does not require gradients, and has no forward-mode AD gradients.
`tensor.to`	Converts a tensor to a requested dtype, device, or both. It can also match another tensor's dtype and device, and may avoid copying when no conversion is needed.
`tensor.cuda`	Module
`tensor.cpu`	Moves a tensor to CPU memory, returning a copy when needed. If it is already on CPU, no copy occurs and the original object is returned.
`tensor.numpy`	Converts a tensor to a NumPy ndarray. By default, conversion requires a CPU tensor with no gradients or unresolved conjugate bit and a NumPy-compatible dtype and layout; `force=True` permits preparation steps and possible copying.
`tensor.requires_grad_`	Changes a tensor’s `requires_grad` setting in place. By default, it enables autograd recording for subsequent operations involving that tensor.
`model.parameters`	Module.parameters() returns an iterator over a module's registered learnable parameters, including parameters in child modules by default. It is commonly passed to an optimizer for training.
`model.state_dict`	Returns a dictionary-like mapping of a module’s learnable parameters and registered buffers. Use it to save, inspect, modify, or restore model state.
`model.load_state_dict`	Module
`model.eval`	Module
`model.train`	Sets a module and its descendants to training mode. This affects layers with training-specific behavior, but it does not control gradient tracking.
`model.zero_grad`	Module
`optimizer.step`	optimizer.step() applies one optimizer update after gradients are computed. Most optimizers use it without arguments; closure-based optimizers can reevaluate the model and loss.
`optimizer.zero_grad`	Resets gradients for all tensors optimized by an optimizer. By default, it sets gradients to None instead of zero, usually reducing memory use and sometimes improving performance.

Search across every package and filter by type in your dashboard.

Start learning