Python Pandas: How to Compare Elements of Two Pandas Series in Python
When analyzing data with pandas, you'll often need to compare two Series - whether to find matching records, identify differences, detect outliers, or validate data transformations. Pandas provides intuitive ways to compare Series elements, from simple relational operators that compare element by element to methods that check overall equality.
In this guide, you'll learn how to compare two pandas Series using relational operators, built-in methods, and practical techniques for real-world data analysis.
Element-Wise Comparison with Relational Operators
Pandas Series support all standard relational operators (==, !=, >, <, >=, <=). These operators compare corresponding elements at each index position and return a new Series of boolean values.
Checking Equality and Inequality
import pandas as pd
ps1 = pd.Series([2.5, 4, 6, 8, 10, 1.75, 40])
ps2 = pd.Series([1.5, 3, 5, 7, 10, 1.75, 20])
print("Series 1:", ps1.tolist())
print("Series 2:", ps2.tolist())
print("\nEqual (==):")
print(ps1 == ps2)
print("\nNot Equal (!=):")
print(ps1 != ps2)
Output:
Series 1: [2.5, 4.0, 6.0, 8.0, 10.0, 1.75, 40.0]
Series 2: [1.5, 3.0, 5.0, 7.0, 10.0, 1.75, 20.0]
Equal (==):
0 False
1 False
2 False
3 False
4 True
5 True
6 False
dtype: bool
Not Equal (!=):
0 True
1 True
2 True
3 True
4 False
5 False
6 True
dtype: bool
Elements at indices 4 and 5 are equal in both Series (10 == 10 and 1.75 == 1.75), so they return True for == and False for !=.
Greater Than and Less Than
import pandas as pd
ps1 = pd.Series([2.5, 4, 6, 8, 10, 1.75, 40])
ps2 = pd.Series([1.5, 3, 5, 7, 10, 1.75, 20])
print("Greater than (ps1 > ps2):")
print(ps1 > ps2)
print("\nLess than (ps1 < ps2):")
print(ps1 < ps2)
Output:
Greater than (ps1 > ps2):
0 True
1 True
2 True
3 True
4 False
5 False
6 True
dtype: bool
Less than (ps1 < ps2):
0 False
1 False
2 False
3 False
4 False
5 False
6 False
dtype: bool
Greater Than or Equal / Less Than or Equal
import pandas as pd
ps1 = pd.Series([2.5, 4, 6, 8, 10, 1.75, 40])
ps2 = pd.Series([1.5, 3, 5, 7, 10, 1.75, 20])
print("Greater than or equal (ps1 >= ps2):")
print(ps1 >= ps2)
Output:
Greater than or equal (ps1 >= ps2):
0 True
1 True
2 True
3 True
4 True
5 True
6 True
dtype: bool
Every element in ps1 is greater than or equal to its counterpart in ps2.
Checking Overall Equality with Series.equals()
While relational operators compare element by element, the equals() method checks whether two Series contain the same elements overall. It returns a single True or False value.
import pandas as pd
ps1 = pd.Series([80, 25, 3, 25, 24, 6])
ps2 = pd.Series([80, 25, 3, 25, 24, 6])
ps3 = pd.Series([80, 25, 3, 25, 24, 7])
print(f"ps1 equals ps2: {ps1.equals(ps2)}")
print(f"ps1 equals ps3: {ps1.equals(ps3)}")
Output:
ps1 equals ps2: True
ps1 equals ps3: False
== vs equals()==returns a Series of booleans - one value per element.equals()returns a single boolean -Trueonly if all elements match.
import pandas as pd
ps1 = pd.Series([1, 2, 3])
ps2 = pd.Series([1, 2, 3])
print(type(ps1 == ps2)) # <class 'pandas.core.series.Series'>
print(type(ps1.equals(ps2))) # <class 'bool'>
Use equals() when you need an overall yes/no answer. Use == when you need to know which specific elements match.
How equals() Handles NaN Values
A major advantage of equals() over == is its handling of NaN values. In standard comparison, NaN != NaN, but equals() treats two NaN values in the same position as equal:
import pandas as pd
import numpy as np
ps1 = pd.Series([1, np.nan, 3])
ps2 = pd.Series([1, np.nan, 3])
print("Element-wise == :")
print(ps1 == ps2)
print(f"\nequals(): {ps1.equals(ps2)}")
Output:
Element-wise == :
0 True
1 False
2 True
dtype: bool
equals(): True
With ==, the NaN comparison at index 1 returns False (since NaN != NaN by IEEE 754 rules). The equals() method correctly identifies that both Series have NaN at the same position and returns True.
Using compare() for Detailed Differences
Starting with pandas 1.1.0, the compare() method shows exactly where two Series differ, displaying only the non-matching elements:
import pandas as pd
ps1 = pd.Series([10, 20, 30, 40, 50], index=['a', 'b', 'c', 'd', 'e'])
ps2 = pd.Series([10, 25, 30, 45, 50], index=['a', 'b', 'c', 'd', 'e'])
diff = ps1.compare(ps2)
print("Differences:")
print(diff)
Output:
Differences:
self other
b 20.0 25.0
d 40.0 45.0
Only the differing elements (at indices b and d) are displayed, with self showing the value from ps1 and other from ps2.
Combining Comparisons with Logical Operators
You can combine multiple comparisons using & (and), | (or), and ~ (not) to create complex filtering conditions:
import pandas as pd
ps1 = pd.Series([10, 20, 30, 40, 50])
ps2 = pd.Series([15, 20, 25, 40, 55])
# Elements where ps1 equals ps2 AND value is greater than 15
mask = (ps1 == ps2) & (ps1 > 15)
print("Equal and greater than 15:")
print(ps1[mask])
Output:
Equal and greater than 15:
1 20
3 40
dtype: int64
Due to Python's operator precedence, you must wrap each comparison in parentheses when using &, |, or ~:
# ❌ This raises an error
mask = ps1 == ps2 & ps1 > 15
# ✅ This works correctly
mask = (ps1 == ps2) & (ps1 > 15)
Practical Example: Comparing Before and After Data
A common real-world use case is comparing data before and after a transformation to identify what changed:
import pandas as pd
before = pd.Series([100, 200, 300, 400, 500],
index=['Product A', 'Product B', 'Product C', 'Product D', 'Product E'])
after = pd.Series([100, 220, 300, 380, 550],
index=['Product A', 'Product B', 'Product C', 'Product D', 'Product E'])
# Find changed items
changed = before != after
changes = pd.DataFrame({
'Before': before[changed],
'After': after[changed],
'Change': after[changed] - before[changed],
'Change %': ((after[changed] - before[changed]) / before[changed] * 100).round(1)
})
print("Price changes:")
print(changes)
Output:
Price changes:
Before After Change Change %
Product B 200 220 20 10.0
Product D 400 380 -20 -5.0
Product E 500 550 50 10.0
Comparing Series of Different Lengths
When comparing Series with different lengths or mismatched indices, pandas does NOT align them by index and does NOT fill unmatched positions with NaN:
import pandas as pd
ps1 = pd.Series([10, 20, 30], index=['a', 'b', 'c'])
ps2 = pd.Series([10, 25, 35, 40], index=['a', 'b', 'c', 'd'])
print(ps1 == ps2) # ValueError: Can only compare identically-labeled Series objects
You need to manually align the two series before doing the comparison:
import pandas as pd
ps1 = pd.Series([10, 20, 30], index=['a', 'b', 'c'])
ps2 = pd.Series([10, 25, 35, 40], index=['a', 'b', 'c', 'd'])
ps1_aligned, ps2_aligned = ps1.align(ps2)
print(ps1_aligned == ps2_aligned)
Output
a True
b False
c False
d False
dtype: bool
Quick Comparison of Methods
| Method | Returns | Handles NaN | Best For |
|---|---|---|---|
==, !=, >, <, >=, <= | Series of booleans | ❌ (NaN != NaN) | Element-wise comparison |
Series.equals() | Single boolean | ✅ (NaN == NaN) | Overall equality check |
Series.compare() | DataFrame of differences | ✅ | Identifying specific differences |
Logical operators (&, |, ~) | Series of booleans | Depends on base comparison | Complex filtering conditions |
Conclusion
Pandas offers flexible and powerful tools for comparing Series:
- Use relational operators (
==,>,<, etc.) for element-wise comparisons that return a boolean Series showing which positions match your condition. - Use
Series.equals()for a quick overall check that returns a singleTrueorFalse- with the added benefit of treatingNaNvalues at the same position as equal. - Use
Series.compare()to get a clear, concise view of exactly which elements differ between two Series. - Combine comparisons with logical operators (
&,|,~) to build complex conditions for data filtering and analysis.
Choose the method that matches your goal - whether you need a full element-by-element breakdown, a simple yes/no answer, or a detailed difference report.