Python PyTorch: How to Create and Initialize Tensors in Python with PyTorch

Tensors are the fundamental data structures in PyTorch, serving as the building blocks for all deep learning operations. Similar to NumPy arrays but with GPU acceleration and automatic differentiation support, tensors store everything from input data to model parameters. Mastering tensor creation and initialization is essential for building efficient neural networks and implementing custom deep learning solutions.

Creating Tensors from Existing Data

Convert Python lists or NumPy arrays into PyTorch tensors using torch.tensor():

import torch
import numpy as np

# From Python list - 1D tensor (vector)
vector = torch.tensor([1.0, 2.0, 3.0, 4.0])
print(f"Vector: {vector}")
print(f"Shape: {vector.shape}, Dtype: {vector.dtype}")

# From nested list - 2D tensor (matrix)
matrix = torch.tensor([[1, 2, 3], [4, 5, 6]])
print(f"\nMatrix:\n{matrix}")
print(f"Shape: {matrix.shape}")

# From NumPy array
np_array = np.array([[1.0, 2.0], [3.0, 4.0]])
from_numpy = torch.from_numpy(np_array)
print(f"\nFrom NumPy:\n{from_numpy}")

Output:

Vector: tensor([1., 2., 3., 4.])
Shape: torch.Size([4]), Dtype: torch.float32

Matrix:
tensor([[1, 2, 3],
        [4, 5, 6]])
Shape: torch.Size([2, 3])

From NumPy:
tensor([[1., 2.],
        [3., 4.]], dtype=torch.float64)

Shape Consistency Required

Tensors must have uniform dimensions. Jagged arrays cause errors:

# ❌ This fails - inconsistent row lengths
torch.tensor([[1, 2], [3, 4, 5]])  # ValueError

Initializing with Constant Values

Create tensors filled with specific values for initializing biases, masks, or placeholders:

import torch

# Tensor of zeros
zeros = torch.zeros(3, 4)
print(f"Zeros (3x4):\n{zeros}")

# Tensor of ones
ones = torch.ones(2, 3)
print(f"\nOnes (2x3):\n{ones}")

# Tensor filled with custom value
filled = torch.full((2, 2), fill_value=3.14)
print(f"\nFilled with 3.14:\n{filled}")

# Identity matrix
identity = torch.eye(3)
print(f"\nIdentity (3x3):\n{identity}")

Output:

Zeros (3x4):
tensor([[0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.]])

Ones (2x3):
tensor([[1., 1., 1.],
        [1., 1., 1.]])

Filled with 3.14:
tensor([[3.1400, 3.1400],
        [3.1400, 3.1400]])

Identity (3x3):
tensor([[1., 0., 0.],
        [0., 1., 0.],
        [0., 0., 1.]])

Common Initialization Patterns

• torch.zeros(): Bias initialization, padding masks
• torch.ones(): Attention masks, scaling factors
• torch.eye(): Linear algebra operations, initialization baselines

Random Tensor Initialization

Random initialization is critical for neural network weights. PyTorch provides various distributions:

import torch

# Uniform distribution [0, 1)
uniform = torch.rand(3, 3)
print(f"Uniform [0,1):\n{uniform}")

# Standard normal distribution (mean=0, std=1)
normal = torch.randn(3, 3)
print(f"\nNormal (μ=0, σ=1):\n{normal}")

# Random integers in range
integers = torch.randint(low=0, high=10, size=(2, 3))
print(f"\nRandom integers [0,10):\n{integers}")

# Custom normal distribution
custom_normal = torch.normal(mean=5.0, std=2.0, size=(2, 2))
print(f"\nCustom normal (μ=5, σ=2):\n{custom_normal}")

Output:

Uniform [0,1):
tensor([[0.8427, 0.2165, 0.8221],
        [0.6094, 0.9972, 0.6888],
        [0.4389, 0.0244, 0.7633]])

Normal (μ=0, σ=1):
tensor([[-1.1526, -1.1820,  0.5721],
        [ 0.3129,  0.2032,  0.9088],
        [-0.2895, -0.7636, -0.5447]])

Random integers [0,10):
tensor([[3, 8, 8],
        [2, 0, 9]])

Custom normal (μ=5, σ=2):
tensor([[2.6919, 2.0961],
        [4.1802, 1.8831]])

Reproducibility

Set a manual seed for reproducible results:

torch.manual_seed(42)
random_tensor = torch.randn(3, 3)  # Same values every run

Sequential and Range Tensors

Create tensors with sequential or evenly spaced values:

import torch

# Integer range (like Python range)
range_tensor = torch.arange(0, 10, 2)
print(f"Arange (0 to 10, step 2): {range_tensor}")

# Evenly spaced values (like NumPy linspace)
linspace = torch.linspace(0, 1, steps=5)
print(f"Linspace (0 to 1, 5 steps): {linspace}")

# Logarithmically spaced
logspace = torch.logspace(0, 2, steps=3)  # 10^0 to 10^2
print(f"Logspace: {logspace}")

Output:

Arange (0 to 10, step 2): tensor([0, 2, 4, 6, 8])
Linspace (0 to 1, 5 steps): tensor([0.0000, 0.2500, 0.5000, 0.7500, 1.0000])
Logspace: tensor([  1.,  10., 100.])

Specifying Data Types and Device

Control tensor precision and hardware placement:

import torch

# Specify data type
float32_tensor = torch.zeros(2, 2, dtype=torch.float32)
float64_tensor = torch.zeros(2, 2, dtype=torch.float64)
int_tensor = torch.zeros(2, 2, dtype=torch.int64)

print(f"Float32: {float32_tensor.dtype}")
print(f"Float64: {float64_tensor.dtype}")
print(f"Int64: {int_tensor.dtype}")

# Create on GPU directly (if available)
device = "cuda" if torch.cuda.is_available() else "cpu"
gpu_tensor = torch.randn(3, 3, device=device)
print(f"\nDevice: {gpu_tensor.device}")

# Move existing tensor to GPU
cpu_tensor = torch.randn(2, 2)
gpu_tensor = cpu_tensor.to(device)

Output:

Float32: torch.float32
Float64: torch.float64
Int64: torch.int64

Device: cpu

GPU Memory

Tensors created on GPU consume graphics memory. Monitor usage with torch.cuda.memory_allocated() and free memory by deleting tensors and calling torch.cuda.empty_cache().

Creating Tensors Like Existing Ones

Create new tensors matching the shape, dtype, or device of existing tensors:

import torch

original = torch.randn(3, 4, dtype=torch.float32, device="cpu")

# Same shape, filled with zeros
zeros_like = torch.zeros_like(original)

# Same shape, filled with ones
ones_like = torch.ones_like(original)

# Same shape, random values
rand_like = torch.rand_like(original)

# Same properties, different values
new_tensor = torch.empty_like(original).fill_(5.0)

print(f"Original shape: {original.shape}")
print(f"All match dtype ({zeros_like.dtype}) and device ({zeros_like.device})")

Output:

Original shape: torch.Size([3, 4])
All match dtype (torch.float32) and device (cpu)

Quick Reference

Method	Description	Typical Use Case
torch.tensor()	From existing data	Loading datasets
torch.zeros()	All zeros	Bias initialization
torch.ones()	All ones	Masks, scaling
torch.randn()	Normal distribution	Weight initialization
torch.rand()	Uniform [0, 1)	Dropout masks
torch.arange()	Sequential integers	Positional encoding
torch.eye()	Identity matrix	Linear algebra
torch.empty()	Uninitialized	Performance-critical allocation

Mastering these tensor creation methods provides the foundation for implementing efficient PyTorch models, from simple linear layers to complex transformer architectures.