Python PyTorch: How to Access the Metadata of a Tensor in PyTorch

When working with PyTorch tensors, understanding their metadata - such as shape, data type, device, and memory layout - is essential for debugging, ensuring compatibility between operations, and optimizing performance. Metadata tells you everything about a tensor's structure without looking at the actual data.

This guide covers all the key metadata properties and methods available on PyTorch tensors, with practical examples.

What Is Tensor Metadata?

Tensor metadata is information about the tensor's structure rather than its values. This includes:

• Shape / Size - the dimensions of the tensor
• Data type (dtype) - whether it holds floats, integers, etc.
• Device - whether it's on CPU or GPU
• Number of elements - total count of values
• Number of dimensions - how many axes the tensor has
• Memory layout - how the data is stored in memory

Shape and Size

The most commonly accessed metadata is a tensor's shape - its dimensions. PyTorch provides two equivalent ways to get this:

import torch

tensor = torch.tensor([
    [5, 1, 7],
    [7, 2, 9],
    [4, 7, 9],
    [8, 12, 14],
    [2, 4, 7]
])

# Both return the same result
print("Using .size():", tensor.size())
print("Using .shape: ", tensor.shape)

Output:

Using .size(): torch.Size([5, 3])
Using .shape:  torch.Size([5, 3])

• .size() is a method (can accept a dimension argument).
• .shape is a property (more concise, same as NumPy).

Getting the Size of a Specific Dimension

import torch

tensor = torch.randn(3, 4, 5)

print("Full shape:", tensor.shape)
print("Dimension 0:", tensor.size(0))  # 3
print("Dimension 1:", tensor.size(1))  # 4
print("Dimension 2:", tensor.size(2))  # 5

Output:

Full shape: torch.Size([3, 4, 5])
Dimension 0: 3
Dimension 1: 4
Dimension 2: 5

Number of Dimensions

The ndim property (or dim() method) returns how many axes the tensor has:

import torch

scalar = torch.tensor(42)
vector = torch.tensor([1, 2, 3])
matrix = torch.tensor([[1, 2], [3, 4]])
tensor_3d = torch.randn(2, 3, 4)

print(f"Scalar dimensions: {scalar.ndim}")
print(f"Vector dimensions: {vector.ndim}")
print(f"Matrix dimensions: {matrix.ndim}")
print(f"3D tensor dimensions: {tensor_3d.dim()}")

Output:

Scalar dimensions: 0
Vector dimensions: 1
Matrix dimensions: 2
3D tensor dimensions: 3

Total Number of Elements

Use torch.numel() or .numel() to count the total number of elements in a tensor:

import torch

tensor = torch.tensor([
    [5, 1, 7],
    [7, 2, 9],
    [4, 7, 9],
    [8, 12, 14],
    [2, 4, 7]
])

print("Shape:", tensor.shape)
print("Total elements:", torch.numel(tensor))
print("Total elements:", tensor.numel())  # Equivalent

Output:

Shape: torch.Size([5, 3])
Total elements: 15
Total elements: 15

The total element count is the product of all dimensions: 5 × 3 = 15.

Data Type (dtype)

Every tensor has a data type that determines what kind of values it stores:

import torch

int_tensor = torch.tensor([1, 2, 3])
float_tensor = torch.tensor([1.0, 2.0, 3.0])
bool_tensor = torch.tensor([True, False, True])
explicit_tensor = torch.zeros(3, dtype=torch.float16)

print("Integer tensor dtype:", int_tensor.dtype)
print("Float tensor dtype:", float_tensor.dtype)
print("Boolean tensor dtype:", bool_tensor.dtype)
print("Explicit dtype:", explicit_tensor.dtype)

Output:

Integer tensor dtype: torch.int64
Float tensor dtype: torch.float32
Boolean tensor dtype: torch.bool
Explicit dtype: torch.float16

Common PyTorch data types

dtype	Description	Common Use
torch.float32	32-bit float (default for floats)	Neural network weights
torch.float16	16-bit float	Mixed precision training
torch.int64	64-bit integer (default for ints)	Indices, labels
torch.bool	Boolean	Masks, conditions

Device (CPU vs GPU)

The .device property tells you where the tensor is stored - CPU or a specific GPU:

import torch

cpu_tensor = torch.tensor([1, 2, 3])
print("Device:", cpu_tensor.device)

# Move to GPU if available
if torch.cuda.is_available():
    gpu_tensor = cpu_tensor.to('cuda')
    print("GPU Device:", gpu_tensor.device)
else:
    print("CUDA not available, tensor stays on CPU")

Output:

Device: cpu
CUDA not available, tensor stays on CPU

Device mismatch errors

Operations between tensors on different devices will raise a RuntimeError. Always check devices before performing operations:

# This would fail if tensors are on different devices:
result = cpu_tensor + gpu_tensor  # RuntimeError!

# Fix: move to the same device
result = cpu_tensor.to('cuda') + gpu_tensor

Memory Layout and Strides

Strides describe how many elements to skip in memory to move to the next position along each dimension. This metadata is important for understanding memory efficiency and contiguity:

import torch

tensor = torch.tensor([
    [1, 2, 3],
    [4, 5, 6]
])

print("Shape:", tensor.shape)
print("Strides:", tensor.stride())
print("Is contiguous:", tensor.is_contiguous())

Output:

Shape: torch.Size([2, 3])
Strides: (3, 1)
Is contiguous: True

The stride (3, 1) means: moving to the next row requires skipping 3 elements, and moving to the next column requires skipping 1 element.

After Transposing

import torch

tensor = torch.tensor([[1, 2, 3], [4, 5, 6]])
transposed = tensor.T

print("Transposed shape:", transposed.shape)
print("Transposed strides:", transposed.stride())
print("Is contiguous:", transposed.is_contiguous())

Output:

Transposed shape: torch.Size([3, 2])
Transposed strides: (1, 3)
Is contiguous: False

After transposing, the tensor is no longer contiguous in memory. Some operations require contiguous tensors, which you can create with .contiguous().

Requires Grad (Gradient Tracking)

The requires_grad property indicates whether PyTorch tracks operations on this tensor for automatic differentiation (essential for training neural networks):

import torch

# By default, requires_grad is False
data = torch.tensor([1.0, 2.0, 3.0])
print("Requires grad:", data.requires_grad)

# Enable gradient tracking
weights = torch.tensor([1.0, 2.0, 3.0], requires_grad=True)
print("Requires grad:", weights.requires_grad)
print("Grad function:", weights.grad_fn)

Output:

Requires grad: False
Requires grad: True
Grad function: None

Complete Metadata Summary Function

Here's a utility function that prints all metadata for any tensor:

import torch

def tensor_info(tensor, name="Tensor"):
    """Print comprehensive metadata about a tensor."""
    print(f"--- {name} ---")
    print(f"  Shape:         {tensor.shape}")
    print(f"  Dimensions:    {tensor.ndim}")
    print(f"  Total elements:{tensor.numel()}")
    print(f"  Data type:     {tensor.dtype}")
    print(f"  Device:        {tensor.device}")
    print(f"  Strides:       {tensor.stride()}")
    print(f"  Contiguous:    {tensor.is_contiguous()}")
    print(f"  Requires grad: {tensor.requires_grad}")
    print()

# Test with different tensors
tensor_info(torch.randn(3, 4, 5), "3D Random Tensor")
tensor_info(torch.zeros(10, dtype=torch.int32), "Integer Zeros")
tensor_info(torch.ones(2, 2, requires_grad=True), "Trainable Weights")

Output:

--- 3D Random Tensor ---
  Shape:         torch.Size([3, 4, 5])
  Dimensions:    3
  Total elements:60
  Data type:     torch.float32
  Device:        cpu
  Strides:       (20, 5, 1)
  Contiguous:    True
  Requires grad: False

--- Integer Zeros ---
  Shape:         torch.Size([10])
  Dimensions:    1
  Total elements:10
  Data type:     torch.int32
  Device:        cpu
  Strides:       (1,)
  Contiguous:    True
  Requires grad: False

--- Trainable Weights ---
  Shape:         torch.Size([2, 2])
  Dimensions:    2
  Total elements:4
  Data type:     torch.float32
  Device:        cpu
  Strides:       (2, 1)
  Contiguous:    True
  Requires grad: True

Quick Reference

Property/Method	Returns	Description
.shape	torch.Size	Dimensions of the tensor
.size()	torch.Size	Same as .shape (can take dim argument)
.ndim / .dim()	int	Number of dimensions
.numel()	int	Total number of elements
.dtype	torch.dtype	Data type of elements
.device	torch.device	CPU or GPU location
.stride()	tuple	Memory strides per dimension
.is_contiguous()	bool	Whether memory layout is contiguous
.requires_grad	bool	Whether gradients are tracked
.grad_fn	Function or None	Gradient function (if part of computation graph)

Conclusion

Accessing tensor metadata in PyTorch is straightforward and essential for building reliable deep learning pipelines.

• Use .shape and .dtype for everyday checks,
• Use .device to ensure tensors are on the correct hardware
• Use .stride() and .is_contiguous() when optimizing memory layout.

Understanding these properties helps you debug shape mismatches, prevent device errors, and write more efficient PyTorch code.