codesnips

etl

sql 
-- Import CSV with COPY (fastest method)
COPY users (username, email, age, created_at)
FROM '/path/to/users.csv'
WITH (
  FORMAT csv,
  HEADER true,

Efficient data import and export strategies

database import export
by Maria Garcia 2 tabs
typescript 
import { Writable } from "node:stream";
import type { Pool } from "pg";

interface Row {
  email: string;
  name: string;

Streaming CSV import (Node streams)

streams postgres nodejs
by codesnips 3 tabs
java 
package com.example.demo.batch;

import com.example.demo.model.User;
import com.example.demo.model.UserDTO;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;

Batch processing with Spring Batch

java spring-batch batch-processing
by David Kumar 2 tabs
sql 
CREATE TABLE events (
    id          bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    source      text        NOT NULL,
    external_id text        NOT NULL,
    payload     jsonb       NOT NULL,
    occurred_at timestamptz NOT NULL,

Batched writes with COPY (conceptual)

postgres performance sql
by codesnips 3 tabs
javascript 
const fs = require('fs');
const readline = require('readline');

async function* streamCsvRows(filePath) {
  const stream = fs.createReadStream(filePath, { encoding: 'utf8' });
  const rl = readline.createInterface({ input: stream, crlfDelay: Infinity });

Stream and Import a Large CSV File Line-by-Line with Node.js readline

nodejs streams readline
by codesnips 3 tabs