bulk

Streaming CSV import with batched inserts

CSV imports are where memory usage quietly explodes if you read everything at once. I stream rows with encoding/csv, validate each record, and batch inserts to keep DB overhead low. The important detail is backpressure: if the database is slow, the im

Bulk Upsert with insert_all + Unique Index

I stopped doing row-by-row imports once they started hammering the DB. In Migration (unique index), I added a real uniqueness guarantee on provider + uid, because bulk writes only stay correct if the database can enforce constraints. Then, in Bulk ups