retries

ruby
class CreateDeadJobs < ActiveRecord::Migration[7.1]
  def change
    create_table :dead_jobs do |t|
      t.string  :jid, null: false
      t.string  :queue, null: false
      t.string  :klass, null: false

Background Job Dead Letter Queue (DLQ) Table

rails reliability background-jobs
by codesnips 4 tabs
ruby
require "faraday"
require "faraday/retry"

module Http
  class RetryableError < StandardError; end
  class CircuitOpenError < StandardError; end

HTTP Timeouts + Retries Wrapper (Faraday)

rails http reliability
by codesnips 3 tabs
typescript
import { Queue } from "bullmq";
import IORedis from "ioredis";

export const connection = new IORedis(process.env.REDIS_URL ?? "redis://localhost:6379", {
  maxRetriesPerRequest: null,
});

BullMQ worker with retries + dead-letter

node redis background-jobs
by codesnips 3 tabs
typescript
export type CircuitState = "closed" | "open" | "half-open";

export interface CircuitBreakerOptions {
  failureThreshold: number;
  resetTimeout: number; // ms to wait in "open" before probing
  onStateChange?: (from: CircuitState, to: CircuitState) => void;

Circuit breaker wrapper for flaky third-party APIs

circuit-breaker resilience http
by codesnips 3 tabs
ruby
module Retryable
  module_function

  def with_retries(tries: 3, base: 0.3, cap: 5.0, on: [StandardError])
    attempt = 0
    begin

Retry a Flaky HTTP Call with Exponential Backoff Using a Rails Service Object

rails http retries
by codesnips 3 tabs
typescript
import { Module } from '@nestjs/common';
import { BullModule } from '@nestjs/bull';
import { ScheduleModule } from '@nestjs/schedule';
import { DigestProducer } from './digest.producer';
import { DigestProcessor } from './digest.processor';

Scheduling and Processing Email Digests with a Bull Queue in NestJS

nestjs bull redis
by codesnips 3 tabs
javascript
const express = require('express');
const { enqueueEmail } = require('./emailQueue');

const router = express.Router();

router.post('/users/:id/welcome-email', async (req, res, next) => {

Reliable Background Email Jobs With BullMQ, Redis, and a Worker Process

bullmq redis background-jobs
by codesnips 3 tabs
sql
CREATE TABLE idempotency_keys (
    request_key         text        NOT NULL,
    endpoint            text        NOT NULL,
    request_fingerprint text        NOT NULL,
    status              text        NOT NULL DEFAULT 'in_progress'
                                    CHECK (status IN ('in_progress', 'completed')),

Idempotency keys for “create” endpoints

reliability postgres idempotency
by codesnips 3 tabs
ruby
class SyncContactJob < ApplicationJob
  queue_as :external

  BACKOFF = ->(executions) do
    (2**executions) + rand(0.0..1.0) # exponential + jitter, in seconds
  end

Exponential Backoff with Jitter for Flaky External API Calls in ActiveJob

rails activejob background-jobs
by codesnips 3 tabs
typescript
import { Prisma, PrismaClient } from "@prisma/client";

const prisma = new PrismaClient();

type TxClient = Prisma.TransactionClient;

Prisma transaction with retries for serialization errors

prisma postgres concurrency
by codesnips 3 tabs
typescript
import { EventEmitter } from "events";

export interface Job<T> {
  id: string;
  payload: T;
  attempts: number;

Typed In-Memory Job Queue With a Concurrency-Limited Worker Pool

typescript job-queue concurrency
by codesnips 3 tabs
python
import queue
import time
from dataclasses import dataclass, field, replace
from typing import Any, Dict, Tuple

In-Process Threaded Background Job Queue for Sending Emails Without Redis

background-jobs threading queue
by codesnips 4 tabs