Pandas fillna: different values for different columns

Select a timestamp to watch that moment on YouTube.

1. 00:00

   Pandas fillna replaces values that pandas recognizes as missing, such as None, NaN, and pd.NA. A single scalar applies the same replacement across the selected DataFrame, which is rarely appropriate when columns represent different kinds of information. Filling a city label with 0 and a numeric score with 'Unknown' produces confusing data and can change a useful numeric dtype into an object dtype. Today we will test the claim with a tiny example, inspect a misleading first attempt, and finish with a verification case. Fill missing city and score values with separate defaults while leaving other

2. 00:40

   columns and the input DataFrame unchanged. Pass a dictionary to DataFrame.fillna for column-specific defaults: frame.fillna({'city': 'Unknown', 'score': 0}). Keys are column labels and values are replacements. Columns omitted from the mapping remain untouched. The call returns a new DataFrame by default, so the original is available for comparison and audit. The first run is intentionally incomplete or incorrect. The city column gets a numeric zero instead of a meaningful label. Read the actual output before editing the code; a familiar-looking command can still answer the wrong question.

3. 01:21

   Begin by naming what each column represents. In the sample, city is text, score is numeric, and note is optional free text. Replacing every missing cell with 0 makes the numeric column appear complete, but it also makes city equal a number. A table can still print without an error while its schema becomes less useful. Pandas recognizes None and NaN as missing here. An empty string is not treated as missing by fillna, so a blank city would require a separate normalization decision. Do not merge these two policies casually: an empty string might be intentionally different from unknown in some data sources.

4. 02:02

   We use a second run to make the cause visible. The printed values are small enough to check by hand. If they disagree with your prediction, pause and identify the assumption that changed. The direct answer to pandas fillna different values for different columns is a dictionary passed to DataFrame.fillna. Its keys are column names. The city key maps to 'Unknown'; the score key maps to 0. Each replacement applies only where that named column is missing. Existing values, including the Rome label and numeric four, remain as they were. The call returns a new DataFrame by default, so a caller can inspect both versions.

5. 02:43

   This is useful for data cleaning because the original record is evidence of what arrived. Avoid inplace=True unless there is a clear reason to discard that original state. For larger pipelines, make the fill mapping a named policy rather than scattered literals. Now compare the corrected run with the earlier one. The important change is the rule encoded in the operation, not merely the final display. Run the code and inspect the returned values as well as their interpretation. A missing value is different from an empty string or a string containing spaces. fillna does not rewrite blank text; normalize text separately if blank text is

6. 03:23

   invalid in your source. It also does not decide whether zero is an honest replacement for an unknown score. Zero may change averages or totals, so choose a default only when the data contract justifies it; otherwise keep missing values or use an imputation rule and document it. A mapping need not cover every column. Here note is intentionally absent from the fill mapping, so a missing note remains missing. That is often better than inventing a note that no one wrote. Verify with pd.isna instead of comparing a missing value with equality; NaN does not behave like an ordinary string or number in comparisons.

7. 04:08

   Also inspect the input after computing fixed. By default fillna did not mutate it, so its city is still missing. This makes it possible to compare raw and cleaned records or test the transformation. Filling unknown scores with zero may bias averages, so a production analysis must justify that rule; preserving missing scores can be the more honest choice when zero has a real measurement meaning. The final run checks a different condition so a special-case answer cannot pass unnoticed. Try the linked browser task and use the guide when you need a smaller step. The source links on the lesson page give the complete API behavior.

8. 04:49

   The browser task uses in-memory records with city, score, and note fields. In local Python install pandas with python -m pip install pandas. The task fills only city and score, leaves note missing, and verifies the source remains unchanged. It checks a second DataFrame so the answer cannot be a hard-coded output list. Before adopting a fill rule, count how many values are missing in each column with isna().sum(). A default can hide a source-quality problem if a large fraction of records is incomplete. Keep the original DataFrame or a copy so you can measure what changed. If a numeric column uses pandas nullable integer dtype, choose a replacement that

9. 05:32

   preserves its intended type and verify the dtype after filling. The examples use floating scores because missing values in a regular numeric column commonly produce floats. The important habit is to choose a meaning first, then apply the fill operation.