python 23 lines · 1 tab

Cleaning missing values and normalizing messy CSV exports

Dr. Elena Vasquez Apr 2026

1 tab

import pandas as pd

df = pd.read_csv('customers.csv')

df.columns = df.columns.str.strip().str.lower().str.replace(' ', '_', regex=False)

df['email'] = df['email'].str.strip().str.lower()
df['country'] = df['country'].fillna('unknown').str.strip().str.upper()
df['signup_date'] = pd.to_datetime(df['signup_date'], errors='coerce')
df['lifetime_value'] = pd.to_numeric(df['lifetime_value'], errors='coerce')

df = df.drop_duplicates(subset=['email'], keep='last')
df = df[df['email'].notna()]
df['lifetime_value'] = df['lifetime_value'].fillna(df['lifetime_value'].median())

missing_summary = (
    df.isna()
    .mean()
    .sort_values(ascending=False)
    .rename('missing_ratio')
)

assert missing_summary.loc['email'] == 0, 'email should be fully populated'

1 file · python Explain with highlit

Real data arrives dirty. I usually start with missing-value audits, duplicate removal, explicit type conversion, and canonical text cleanup. The trick is to make each cleanup rule reproducible rather than burying it in notebook state. I prefer small, composable transformations and assertions that fail loudly when source feeds drift.

Related snips

python

import cv2

image = cv2.imread('receipt.jpg')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (5, 5), 0)
thresholded = cv2.adaptiveThreshold(

OpenCV image preprocessing for OCR and vision pipelines

opencv image-processing computer-vision

by Dr. Elena Vasquez 1 tab

python

import pandas as pd

df = pd.read_csv('traffic.csv', parse_dates=['timestamp'])
df['timestamp'] = pd.to_datetime(df['timestamp'], utc=True)
df = df.set_index('timestamp').sort_index()

Time series resampling and rolling windows in pandas

pandas time-series resampling

by Dr. Elena Vasquez 1 tab

python

import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

Encoding categorical variables without creating leakage

categorical-encoding preprocessing scikit-learn

by Dr. Elena Vasquez 1 tab

python

import pandas as pd

df = pd.read_parquet('events.parquet')
df['event_date'] = pd.to_datetime(df['event_date'])
df['month'] = df['event_date'].dt.to_period('M').astype(str)

GroupBy aggregations and pivot tables for business reporting

pandas groupby pivot-table

by Dr. Elena Vasquez 1 tab

python

from sklearn.compose import ColumnTransformer
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.ensemble import RandomForestClassifier

ColumnTransformer pipelines that keep preprocessing honest

scikit-learn pipelines columntransformer

by Dr. Elena Vasquez 1 tab

python

import pandas as pd

df = pd.read_csv(
    'orders.csv',
    parse_dates=['created_at'],
    dtype={

pandas DataFrame essentials: loading, indexing, and selection

pandas python dataframe

by Dr. Elena Vasquez 1 tab

Share this code

Here's the card — post it anywhere.

Cleaning missing values and normalizing messy CSV exports — share card

Link copied