Upgrade to pandas 3 Without Changing Your Results

A pandas 3 upgrade can change your results without raising a single error. Run the same script on your old and new versions, compare the outputs row by row, and fix the five patterns that most often change silently: chained assignment, chained inplace calls, object dtype checks, datetime-to-integer conversions, and category columns read from CSV.

pandas 3.0.0 was released on January 21, 2026, and pandas 3.1.0 followed on October 22, 2026. Many analysts are now moving notebooks and scheduled scripts off 2.x. This guide uses a four-row file to reproduce each trap. Every snippet was run on macOS with Python 3.13.12 and NumPy 2.5.4, under pandas 2.3.3 and pandas 3.1.1. The pandas 3.1.1 results were the same with and without pyarrow installed.

Why a clean run is not proof

Some upgrade problems are loud. resample("M") now raises ValueError because the month-end alias is "ME". You will notice that immediately.

The dangerous problems are quiet. Under pandas 3, the script below still runs to the end and still prints a tidy table, but the table is wrong. A chained-assignment warning appears in the console, which is easy to miss in a scheduled job or a long notebook.

The official pandas 3.0.0 release notes and pandas 3.1.0 release notes list many more changes. Use them as a checklist, but trust the output comparison below more than any list: it tests your code with your data.

Build a two-environment test bench

Do not upgrade the environment your reports run in. Create one environment pinned to your current version and one pinned to the target version, then run the same script in both. On macOS or Linux:

python3.13 -m venv .venv-pd2
.venv-pd2/bin/python -m pip install "pandas==2.3.3"

python3.13 -m venv .venv-pd3
.venv-pd3/bin/python -m pip install "pandas==3.1.1"

On Windows, the interpreter inside each environment is .venv-pd2\Scripts\python.exe. Replace 2.3.3 with the version your project actually uses; python -m pip show pandas tells you. pandas 3 requires Python 3.11 or newer, so an older interpreter needs to be upgraded first. If your project installs other packages, install the same versions of those in both environments so that pandas is the only difference.

Reproduce the problem with four rows

Save this as orders.csv. It contains the kinds of mess real exports have: a zip code with a leading zero, padded region names, and one missing region.

order_id,customer_zip,region,tier,amount,ordered_at
1,02134, East ,gold,100.0,2026-01-05 10:00
2,10001,East,silver,50.0,2026-01-05 11:30
3,02134,West ,gold,80.0,2026-01-06 09:15
4,94105,,bronze,20.0,2026-01-06 12:00

The business rules are simple. Trim every text field. Label a missing region Unknown. Gold orders are complimentary, so their amount becomes 0. Add an epoch-seconds column for a downstream system. Then total the amount by region.

Here is a version of that report written in a style that was common, and worked, under pandas 2.x. Save it as report_before.py:

import sys
import pandas as pd

df = pd.read_csv("orders.csv", dtype={"customer_zip": "category"},
                 parse_dates=["ordered_at"])

# Trim every text column
for col in df.columns:
    if df[col].dtype == object:
        df[col] = df[col].str.strip()

df["region"].fillna("Unknown", inplace=True)      # label missing regions
df["amount"][df["tier"] == "gold"] = 0            # gold orders are comped
df["epoch_s"] = df["ordered_at"].astype("int64") // 10**9

by_region = df.groupby("region", as_index=False)["amount"].sum()
print(by_region.to_string(index=False))
print("zips:", sorted(df["customer_zip"].astype(str).unique()))
print("first epoch:", df["epoch_s"].iloc[0])

Run it in both environments:

.venv-pd2/bin/python report_before.py
.venv-pd3/bin/python report_before.py

pandas 2.3.3 prints the intended answer, along with deprecation warnings:

 region  amount
   East    50.0
Unknown    20.0
   West     0.0
zips: ['02134', '10001', '94105']
first epoch: 1767607200

pandas 3.1.1 prints this, after two ChainedAssignmentError warnings:

region  amount
 East    100.0
  East    50.0
 West     80.0
zips: ['10001', '2134', '94105']
first epoch: 1767607

The total moved from 70 to 230, East split into two groups, the Unknown row disappeared, a zip code lost its leading zero, and the timestamp is off by a factor of 1,000. Nothing crashed.

What changed, line by line

1. Category columns read from CSV now infer numbers

In pandas 3.1, read_csv(dtype="category") infers numeric and boolean categories for every parser engine. Before 3.1, only the pyarrow engine did that. 02134 is a valid integer, so it becomes 2134. Zip codes, account numbers, product codes, and any other zero-padded identifier are at risk. Read identifiers as text, and convert to category afterward if you need the memory savings:

df = pd.read_csv("orders.csv", dtype={"customer_zip": "string"})
df["customer_zip"] = df["customer_zip"].astype("category")  # optional

2. Text columns are no longer object

pandas 3 stores text in the new default str dtype. A check such as df[col].dtype == object is now False for text columns, so the trimming loop silently does nothing. Untrimmed labels then form separate groups. Name the text columns you mean instead of guessing them from the dtype:

text_cols = ["customer_zip", "region", "tier"]
df[text_cols] = df[text_cols].apply(lambda s: s.str.strip())

select_dtypes(include="object") still returns str columns in pandas 3.1.1, but it emits a Pandas4Warning that this backward-compatibility behavior is deprecated. Explicit column lists avoid both problems.

3. Chained inplace calls no longer reach the DataFrame

Under Copy-on-Write, which is always on in pandas 3, df["region"] behaves as a new object. Calling fillna(..., inplace=True) on it changes that temporary object, not df. The missing region stays missing, and groupby drops missing keys by default, so the row vanishes from the totals. Assign the result back:

df["region"] = df["region"].fillna("Unknown")

4. Chained assignment no longer writes

df["amount"][mask] = 0 is two operations: select a column, then assign into that selection. Under Copy-on-Write, the selection is never a view of df, so the assignment is lost. Use one .loc call with both the row mask and the column:

df.loc[df["tier"] == "gold", "amount"] = 0

5. Datetime resolution is no longer always nanoseconds

pandas 2.x parsed datetimes as datetime64[ns], so dividing the int64 value by 10**9 produced epoch seconds. pandas 3 infers the resolution; this file parses as datetime64[us], so the same arithmetic produces a value 1,000 times too small. pandas 3.1 also defaults to_datetime(..., unit="s") on numbers to microseconds. Set the unit explicitly before converting to integers:

df["epoch_s"] = df["ordered_at"].dt.as_unit("s").astype("int64")

as_unit("s") floors fractional seconds. In both versions, 1969-12-31 23:59:59.500 became -1. If your downstream system expects rounding instead, round first.

Write the version-independent version

Save this as report_after.py. It applies all five fixes and uses only APIs that behave the same in 2.3.3 and 3.1.1:

import pandas as pd

df = pd.read_csv(
    "orders.csv",
    dtype={"customer_zip": "string", "region": "string", "tier": "string"},
    parse_dates=["ordered_at"],
)

text_cols = ["customer_zip", "region", "tier"]          # named, not dtype-guessed
df[text_cols] = df[text_cols].apply(lambda s: s.str.strip())

df["region"] = df["region"].fillna("Unknown")
df.loc[df["tier"] == "gold", "amount"] = 0
df["epoch_s"] = df["ordered_at"].dt.as_unit("s").astype("int64")

by_region = df.groupby("region", as_index=False)["amount"].sum()
print(by_region.to_string(index=False))
print("zips:", sorted(df["customer_zip"].unique()))
print("first epoch:", df["epoch_s"].iloc[0])

Run it with every warning turned into an error, so a missed deprecation stops the run instead of scrolling past:

.venv-pd2/bin/python -W error report_after.py
.venv-pd3/bin/python -W error report_after.py

Both runs completed without warnings and printed the same intended result:

 region  amount
   East    50.0
Unknown    20.0
   West     0.0
zips: ['02134', '10001', '94105']
first epoch: 1767607200

Compare outputs instead of reading them

Looking at printed tables does not scale to a real report with hundreds of rows. Make each run write its key output to a file, then compare the files by key. Change the report's print line so it writes the regional totals to a path you pass in:

import sys
by_region.to_csv(sys.argv[1], index=False)

Save this checker as compare_outputs.py. It reads both files as plain text, so a '02134' that became '2134', or a 100.0 that became 100, counts as a difference. It reports missing keys, extra keys, and changed values, and exits with status 1 when anything differs:

import sys
import pandas as pd

old_path, new_path, *keys = sys.argv[1:]
old = pd.read_csv(old_path, dtype=str, keep_default_na=False)
new = pd.read_csv(new_path, dtype=str, keep_default_na=False)

if list(old.columns) != list(new.columns):
    sys.exit(f"Column mismatch: {list(old.columns)} vs {list(new.columns)}")

both = old.merge(new, on=keys, how="outer", suffixes=("_old", "_new"), indicator=True)
problems = []
for _, row in both.iterrows():
    key = {k: row[k] for k in keys}
    if row["_merge"] != "both":
        problems.append(f"{key}: only in {'old' if row['_merge'] == 'left_only' else 'new'}")
        continue
    for col in old.columns.difference(keys):
        if row[f"{col}_old"] != row[f"{col}_new"]:
            problems.append(f"{key}: {col} {row[f'{col}_old']!r} -> {row[f'{col}_new']!r}")

if problems:
    print(f"{len(problems)} difference(s):")
    print("\n".join(problems))
    sys.exit(1)
print(f"Identical: {len(old)} rows, {len(old.columns)} columns")

Run the same report once in each environment, then compare the two files on the region key:

.venv-pd2/bin/python report_before.py old.csv
.venv-pd3/bin/python report_before.py new.csv
.venv-pd3/bin/python compare_outputs.py old.csv new.csv region

For the original script, the checker failed with four differences:

4 difference(s):
{'region': ' East '}: only in new
{'region': 'Unknown'}: only in old
{'region': 'West'}: only in old
{'region': 'West '}: only in new

For the rewritten script, it printed Identical: 3 rows, 2 columns. Use the same pattern for every output that matters: the final table, a row count per key, and a few totals. If an output has no natural key, add one, such as the grouping columns or a row number from a stable sort. The expected-results testing guide explains the same both-directions comparison for SQL.

Preview pandas 3 behavior before you upgrade

pandas 2.3 can show you some of the problems while you are still on 2.x. Put these two options at the top of a test run, before any data is loaded:

import pandas as pd
pd.options.mode.copy_on_write = "warn"
pd.options.future.infer_string = True

With these options on pandas 2.3.3, report_before.py emitted FutureWarning messages pointing at the chained fillna line and the chained assignment line. Its output also showed the untrimmed regions, because infer_string switches text to the new string dtype. Setting copy_on_write = True instead of "warn" reproduces the pandas 3 result, which is useful for a dry run.

The preview did not reproduce the zip-code change or the epoch change. The zip-code change arrived in 3.1, and datetime resolution inference is a 3.0 change with no 2.x switch. That is why the side-by-side comparison is the real test; the preview options only help you find problems sooner.

Upgrade checklist for real projects

1. Pin the current state. Record your current versions with python -m pip freeze > requirements-before.txt so you can rebuild the old environment.
2. Search for the risky patterns. This command lists likely candidates in scripts and notebooks. It finds suspects, not proof; read each match.

grep -rnE 'inplace=True|\]\[.*\] *=|dtype *== *.?object|select_dtypes|astype\(.int64.\)|category|freq=.(M|Q|Y|BM).\b|resample\(.(M|Q|Y).\)' --include='*.py' --include='*.ipynb' .

3. Fix the patterns on 2.3 first. The rewrites in this guide work on both versions, so you can deploy them before the upgrade and keep the upgrade itself small.
4. Compare real outputs. Run each report on a frozen copy of real input in both environments and compare the results by key. Freeze the input so a data refresh does not look like an upgrade bug.
5. Run with warnings as errors. Use python -W error on the new version until the run is clean. pandas 3.1 also deprecates inplace=True on methods such as sort_values, reset_index, and drop; df = df.sort_values(...) works on both versions.
6. Pin the new version. Add the exact pandas version to your requirements file so the next environment build does not quietly move you to another release.

If the new environment cannot even install or import pandas, fix that first: why Jupyter cannot import a package you installed covers interpreter mismatches, and what pip does covers isolated installs. For datetime parsing beyond resolution, see the pandas datetime and time zone guide.

FAQ

Should I skip pandas 3.0 and go straight to 3.1?

You can install 3.1 directly; there is no requirement to stop at 3.0. The pandas team recommends upgrading to 2.3 first and clearing its warnings, because 2.3 warns about many 3.0 changes. Test against the exact version you will deploy, since 3.1 added changes of its own, such as numeric category inference in read_csv.

Can I turn Copy-on-Write off in pandas 3?

No. In pandas 3, Copy-on-Write is the only behavior, and the mode.copy_on_write option has no effect. Code that relied on chained assignment has to be rewritten with .loc or by assigning results back.

My comparison shows tiny floating-point differences. Is that a bug?

Not necessarily. The pandas 3.1 release notes say that grouped float sums and means on sorted keys may differ in the last digits. Decide a tolerance from the business meaning, for example cents for currency, and round both outputs to that precision before comparing. Do not apply a tolerance to identifiers, counts, or text keys.