Python Pandas: How to Add a New Calculated Column to a Pandas DataFrame
Creating calculated columns from existing data is one of the most fundamental operations in data transformation. Whether you are computing totals from prices and quantities, deriving new metrics from raw measurements, or building features for a machine learning model, Pandas provides vectorized operations that process entire columns at once. This makes calculations fast, concise, and easy to read.
In this guide, you will learn how to add calculated columns using direct assignment and the .assign() method, understand why vectorized operations vastly outperform loops, and explore common calculation patterns you will encounter in everyday data work.
Direct Assignment: In-Place Column Creation
The simplest way to add a calculated column is to assign an expression directly to a new column name. Pandas evaluates the expression across all rows simultaneously:
import pandas as pd
df = pd.DataFrame({
'Price': [10, 20, 15],
'Qty': [5, 3, 4]
})
# Add a calculated column
df['Total'] = df['Price'] * df['Qty']
print(df)
Output:
Price Qty Total
0 10 5 50
1 20 3 60
2 15 4 60
This approach modifies the DataFrame in place, which is efficient and intuitive for straightforward scripts.
Building Multiple Calculated Columns
You can chain multiple assignments, with each new column referencing previously created ones:
import pandas as pd
df = pd.DataFrame({
'Price': [100, 200, 150],
'Qty': [2, 3, 4]
})
df['Subtotal'] = df['Price'] * df['Qty']
df['Tax'] = df['Subtotal'] * 0.08
df['Total'] = df['Subtotal'] + df['Tax']
print(df)
Output:
Price Qty Subtotal Tax Total
0 100 2 200 16.0 216.0
1 200 3 600 48.0 648.0
2 150 4 600 48.0 648.0
Each column builds on the one before it, creating a clear and readable data pipeline.
Using .assign(): Returning a New DataFrame
The .assign() method creates and returns a new DataFrame with the added columns, leaving the original untouched. This makes it ideal for method chaining and functional-style workflows:
import pandas as pd
df = pd.DataFrame({
'Price': [10, 20],
'Qty': [5, 3]
})
df_new = df.assign(
Total=lambda x: x['Price'] * x['Qty'],
Tax=lambda x: x['Price'] * 0.1
)
print(df_new)
print()
# Original is unchanged
print(df)
Output:
Price Qty Total Tax
0 10 5 50 1.0
1 20 3 60 2.0
Price Qty
0 10 5
1 20 3
Lambda functions inside .assign() receive the DataFrame as it exists at that point in the chain. This means you can reference columns created in a previous .assign() call when you chain multiple calls together.
Chaining Operations with .assign()
The real strength of .assign() is building readable transformation pipelines:
import pandas as pd
df = pd.DataFrame({
'Price': [10, 20, 15],
'Qty': [5, 3, 4]
})
result = (df
.assign(Subtotal=lambda x: x['Price'] * x['Qty'])
.assign(Tax=lambda x: x['Subtotal'] * 0.08)
.assign(Total=lambda x: x['Subtotal'] + x['Tax'])
.query('Total > 50')
)
print(result)
Output:
Price Qty Subtotal Tax Total
0 10 5 50 4.0 54.0
1 20 3 60 4.8 64.8
2 15 4 60 4.8 64.8
Each step in the chain is self-contained and easy to understand, and the original DataFrame remains unmodified throughout.
Why You Should Avoid Loops
A common mistake, especially for those coming from other programming languages, is using row-by-row iteration to compute new columns. This approach is dramatically slower than vectorized operations because it bypasses Pandas' optimized C-based internals.
import pandas as pd
df = pd.DataFrame({'Price': [10, 20], 'Qty': [5, 3]})
# Slow: Row-by-row iteration
for i, row in df.iterrows():
df.loc[i, 'Total'] = row['Price'] * row['Qty']
print(df)
Output:
Price Qty Total
0 10 5 50.0
1 20 3 60.0
The vectorized equivalent produces the same result but runs orders of magnitude faster:
import pandas as pd
df = pd.DataFrame({'Price': [10, 20], 'Qty': [5, 3]})
# Fast: Vectorized operation
df['Total'] = df['Price'] * df['Qty']
print(df)
Output:
Price Qty Total
0 10 5 50
1 20 3 60
Performance Difference
To illustrate the scale of the difference, here is a timing comparison on a DataFrame with 10,000 rows:
import pandas as pd
import time
df = pd.DataFrame({'A': range(10000), 'B': range(10000)})
# Vectorized approach
start = time.time()
df['C'] = df['A'] * df['B']
vectorized_time = time.time() - start
# Loop approach
start = time.time()
for i, row in df.iterrows():
df.loc[i, 'C'] = row['A'] * row['B']
loop_time = time.time() - start
print(f"Vectorized: {vectorized_time:.4f}s")
print(f"Loop: {loop_time:.4f}s")
print(f"Loop is ~{loop_time / vectorized_time:.0f}x slower")
Output:
Vectorized: 0.0017s
Loop: 1.5670s
Loop is ~932x slower
The performance gap widens as your dataset grows. On DataFrames with millions of rows, loops can take minutes where vectorized operations finish in milliseconds. Always prefer vectorized operations unless your logic truly cannot be expressed without iteration.
Common Calculation Patterns
Arithmetic Operations
import pandas as pd
df = pd.DataFrame({
'A': [10, 20, 30],
'B': [3, 4, 5]
})
df['Sum'] = df['A'] + df['B']
df['Diff'] = df['A'] - df['B']
df['Product'] = df['A'] * df['B']
df['Ratio'] = df['A'] / df['B']
print(df)
Output:
A B Sum Diff Product Ratio
0 10 3 13 7 30 3.333333
1 20 4 24 16 80 5.000000
2 30 5 35 25 150 6.000000
Mathematical Functions with NumPy
import pandas as pd
import numpy as np
df = pd.DataFrame({
'A': [10, 20, 30]
})
df['Log_A'] = np.log(df['A'])
df['Sqrt_A'] = np.sqrt(df['A'])
print(df)
Output:
A Log_A Sqrt_A
0 10 2.302585 3.162278
1 20 2.995732 4.472136
2 30 3.401197 5.477226
String Concatenation
import pandas as pd
df = pd.DataFrame({
'First': ['John', 'Jane'],
'Last': ['Doe', 'Smith']
})
df['Full_Name'] = df['First'] + ' ' + df['Last']
print(df)
Output:
First Last Full_Name
0 John Doe John Doe
1 Jane Smith Jane Smith
Conditional Columns
Use np.where() for simple conditions or .apply() for more complex logic:
import pandas as pd
import numpy as np
df = pd.DataFrame({'Score': [85, 45, 72, 90, 55]})
# Simple binary condition
df['Pass'] = np.where(df['Score'] >= 60, 'Yes', 'No')
# Multi-category logic
df['Grade'] = df['Score'].apply(
lambda x: 'A' if x >= 90 else ('B' if x >= 80 else ('C' if x >= 60 else 'F'))
)
print(df)
Output:
Score Pass Grade
0 85 Yes B
1 45 No F
2 72 Yes C
3 90 Yes A
4 55 No F
For multiple conditions, np.select() is often cleaner than nested lambda expressions:
import numpy as np
import pandas as pd
df = pd.DataFrame({'Score': [85, 45, 72, 90, 55]})
conditions = [
df['Score'] >= 90,
df['Score'] >= 80,
df['Score'] >= 60
]
choices = ['A', 'B', 'C']
df['Grade'] = np.select(conditions, choices, default='F')
print(df)
Output:
Score Grade
0 85 B
1 45 F
2 72 C
3 90 A
4 55 F
Quick Reference
| Method | Modifies Original | Best For |
|---|---|---|
df['col'] = expr | Yes | Simple scripts, quick modifications |
df.assign(col=expr) | No | Method chaining, preserving the original |
| Operation Type | Example |
|---|---|
| Arithmetic | df['C'] = df['A'] + df['B'] |
| Math function | df['Log'] = np.log(df['A']) |
| Conditional | df['Flag'] = np.where(df['A'] > 0, 1, 0) |
| String | df['Full'] = df['First'] + ' ' + df['Last'] |
- Use direct assignment (
df['col'] = expression) for straightforward, in-place calculations. - Use
.assign()when you need method chaining or want to keep the original DataFrame unchanged.
In all cases, prefer vectorized operations over loops for performance gains that can reach several hundred times faster on real-world datasets.