python 23 lines · 1 tab

Custom Datasets and DataLoaders for robust training input pipelines

Dr. Elena Vasquez Apr 2026

1 tab

import pandas as pd
import torch
from PIL import Image
from torch.utils.data import Dataset, DataLoader

class ProductImageDataset(Dataset):
    def __init__(self, metadata_path, image_root, transform=None):
        self.df = pd.read_csv(metadata_path)
        self.image_root = image_root
        self.transform = transform

    def __len__(self):
        return len(self.df)

    def __getitem__(self, index):
        row = self.df.iloc[index]
        image = Image.open(f"{self.image_root}/{row['image_name']}").convert('RGB')
        if self.transform:
            image = self.transform(image)
        label = torch.tensor(row['label'], dtype=torch.long)
        return image, label

train_loader = DataLoader(dataset=train_dataset, batch_size=64, shuffle=True, num_workers=4, pin_memory=True)

1 file · python Explain with highlit

Input pipelines are part of the model system, not an afterthought. I keep dataset classes deterministic, move expensive transforms into explicit stages, and use DataLoader settings that match hardware limits. Good batching and collation logic can remove a surprising amount of GPU idle time.

Related snips

python

import torch.nn as nn

class SmallCNN(nn.Module):
    def __init__(self, num_classes: int) -> None:
        super().__init__()
        self.features = nn.Sequential(

Convolutional neural networks for image classification in PyTorch

pytorch cnn computer-vision

by Dr. Elena Vasquez 1 tab

typescript

import DataLoader from "dataloader";
import { Pool } from "pg";

export interface User { id: number; name: string; }
export interface Post { id: number; user_id: number; title: string; }

N+1 avoidance with DataLoader (GraphQL)

graphql performance dataloader

by codesnips 3 tabs

ruby

# Gemfile
gem 'graphql'

# Installation
# rails generate graphql:install

GraphQL APIs with graphql-ruby gem

ruby rails graphql

by Sarah Mitchell 3 tabs

typescript

import { PrismaClient } from '@prisma/client';

const prisma = new PrismaClient();

export async function getFeedNaive(limit = 20) {
  const posts = await prisma.post.findMany({

Prisma: avoid N+1 with include/select

prisma performance typescript

by codesnips 3 tabs

python

import torch

best_val_loss = float('inf')

for epoch in range(1, num_epochs + 1):
    model.train()

A clean PyTorch training loop with validation and checkpoints

pytorch training-loop checkpoints

by Dr. Elena Vasquez 1 tab

python

import torch

device = 'cuda' if torch.cuda.is_available() else 'cpu'
features = torch.tensor([[1.0, 2.0], [3.0, 4.0]], requires_grad=True, device=device)
weights = torch.tensor([[0.2], [0.8]], requires_grad=True, device=device)

PyTorch tensor basics and automatic differentiation

pytorch tensors autograd

by Dr. Elena Vasquez 1 tab

Share this code

Here's the card — post it anywhere.

Custom Datasets and DataLoaders for robust training input pipelines — share card

Link copied