// latest_posts.filter(by=<tag>)
Patterns for resilient Laravel queue workers — retries, backoff, idempotency, and graceful shutdown under load.
php artisan queue:work \
--tries=3 --backoff=5 \
--max-time=3600Practical goroutine patterns — worker pools, fan-out/fan-in, and context cancellation — for production Go services.
ctx, cancel := context.WithTimeout(
ctx, 2*time.Second)
defer cancel()Reproducible local environments for polyglot backends — Postgres, Redis, and your app in one compose file.
services:
app:
build: .
db:
image: postgres:16How to detect, diagnose, and eliminate N+1 query problems in Laravel Eloquent using eager loading, constrained loads, and preventLazyLoading().
Post::with(['author', 'tags'])
->withCount('comments')
->get();A practical guide to Laravel Sanctum — personal access tokens with scopes, SPA cookie auth, middleware setup, revocation, and when Passport is the right tool instead.
$token = $user->createToken(
'mobile-app',
['orders:read', 'orders:write']
)->plainTextToken;A practical look at the PHP 8.1–8.2 features that actually change how you write production code — backed enums, readonly properties, constructor promotion, first-class callables, and a clear-eyed take on Fibers.
enum Status: string {
case Active = 'active';
case Archived = 'archived';
}How to diagnose and eliminate Django ORM performance problems — from N+1 queries with select_related and prefetch_related to deferred fields, annotations, and streaming large querysets.
posts = Post.objects.select_related('author')\
.prefetch_related('tags')\
.only('title', 'published_at', 'author__username')A ground-level tour of asyncio for backend engineers — event loop mechanics, gather, Semaphore-bounded concurrency, timeouts, and avoiding the blocking-call trap with run_in_executor.
async with asyncio.Semaphore(20):\
results = await asyncio.gather(\
*[fetch(url) for url in urls],\
return_exceptions=True)Sentinel errors, wrapping with fmt.Errorf, errors.Is/As, and custom error types — the patterns that keep Go error handling honest and usable.
if errors.Is(err, ErrNotFound) {
return http.StatusNotFound
}
var ve *ValidationError
if errors.As(err, &ve) {
log.Println(ve.Field)
}B-tree, GIN, partial, and composite indexes explained with real EXPLAIN ANALYZE output — plus covering indexes and how to spot bloat before it kills your queries.
CREATE INDEX CONCURRENTLY idx_orders_user
ON orders (user_id, created_at DESC)
INCLUDE (status, total);Cache-aside, write-through, stampede protection, and cache key design — practical Redis patterns with the failure modes that make or break them in production.
SET user:42 '{"id":42,"email":"[email protected]"}'
EX 300 NX
SET lock:user:42 1 EX 5 NXWordPress ships a capable REST API — here's how to extend it with custom routes, ACF fields, and proper authentication so a decoupled frontend can treat WP as a real backend.
add_action('rest_api_init', function() {
register_rest_route('myapp/v1', '/products',
['callback' => 'myapp_get_products']);
});Exchanges, bindings, consumer acks, dead-letter queues, and idempotent consumers — the pieces you need to build reliable async messaging with RabbitMQ.
channel.basic_qos(prefetch_count=10)
channel.basic_consume(
queue='orders',
on_message_callback=handle,
auto_ack=False)Sessions and JWTs solve different problems — here's the honest tradeoff analysis so you pick the right one instead of cargo-culting whichever you've seen most recently.
// Short-lived access + refresh token pair
{
"access_token": "eyJ...",
"expires_in": 900,
"refresh_token": "def50200..."
}Zero-downtime schema changes require more than running migrations before a deploy — here's the expand/contract pattern, lock-safe Postgres operations, and rollback strategy that actually works.
-- Add column without locking the table
ALTER TABLE orders
ADD COLUMN status_v2 varchar(32);
CREATE INDEX CONCURRENTLY idx_orders_status_v2
ON orders (status_v2);Default PHP-FPM and Nginx configs leave most of your server's capacity on the table — here's how to size worker pools correctly, tune FastCGI buffers, enable OPcache properly, and stop bleeding requests under load.
pm = dynamic
pm.max_children = 40
pm.start_servers = 10
pm.min_spare_servers = 5
pm.max_spare_servers = 20How Elasticsearch analyzers, mappings, and query DSL work together to deliver relevant full-text search — and when Postgres FTS is a better starting point.
GET /posts/_search
{
"query": {
"match": { "title": "django performance" }
}
}How to layer a PHP application into domain, application, and infrastructure tiers — keeping business logic framework-free, testable, and honest about its dependencies.
interface OrderRepository {
public function findById(OrderId $id): Order;
public function save(Order $order): void;
}A complete GitHub Actions pipeline for a PHP/Laravel project — dependency caching, matrix test runs, Docker image builds, and a deploy step that only fires on main.
jobs:
test:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- uses: shivammathur/setup-php@v2Fixed window, sliding window, and token bucket algorithms for API rate limiting — implemented with Redis atomics and paired with correct 429 response headers.
local count = redis.call('INCR', KEYS[1])
if count == 1 then
redis.call('EXPIRE', KEYS[1], ARGV[1])
end
return countAnything slow or failure-prone — email, image processing, third-party API calls —
belongs off the request path. Laravel’s queue system makes that easy, but a naive
queue:work invocation will bite you in production.
Configure retry behavior per job, not just on the CLI:
class ProcessPayment implements ShouldQueue
{
public int $tries = 3;
public array $backoff = [5, 15, 60];
public function handle(): void
{
// ...
}
public function failed(\Throwable $e): void
{
Log::error('Payment job failed', ['error' => $e->getMessage()]);
}
}Workers can crash mid-job and re-run. Make handlers safe to repeat:
public function handle(): void
{
if (Payment::where('idempotency_key', $this->key)->exists()) {
return; // already processed
}
Payment::create([...]);
}Run with --max-time and a process supervisor (Horizon or systemd) so workers
restart cleanly and never leak memory:
php artisan queue:work redis \
--tries=3 --backoff=5 --max-time=3600 --sleep=1Reliable queues are about assuming failure: retry with backoff, stay idempotent, and let a supervisor recycle workers. Do that and background work just works.
Spawning a goroutine per request works until it doesn’t. Unbounded concurrency exhausts connections and memory. Bound it with a worker pool.
func pool(jobs <-chan Job, workers int) <-chan Result {
out := make(chan Result)
var wg sync.WaitGroup
for i := 0; i < workers; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for j := range jobs {
out <- process(j)
}
}()
}
go func() { wg.Wait(); close(out) }()
return out
}Always thread a context.Context so slow work stops when the client disconnects:
func fetch(ctx context.Context, url string) ([]byte, error) {
req, _ := http.NewRequestWithContext(ctx, "GET", url, nil)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
return io.ReadAll(resp.Body)
}Distribute work across workers, then merge results back into one channel. The pool
above is fan-out; a select loop or sync.WaitGroup collector is fan-in.
Bound your concurrency, propagate context everywhere, and close channels from the sender. These three habits prevent most production Go concurrency bugs.
“Works on my machine” dies the day your environment is code. Compose defines the database, cache, and app together so every developer runs the same stack.
services:
app:
build: .
ports:
- "8000:8000"
environment:
DATABASE_URL: postgres://dev:dev@db:5432/app
REDIS_URL: redis://cache:6379
depends_on:
- db
- cache
db:
image: postgres:16-alpine
environment:
POSTGRES_USER: dev
POSTGRES_PASSWORD: dev
POSTGRES_DB: app
volumes:
- pgdata:/var/lib/postgresql/data
cache:
image: redis:7-alpine
volumes:
pgdata:Don’t sleep 5 waiting for Postgres. Use a healthcheck so depends_on waits for
readiness:
db:
healthcheck:
test: ["CMD-SHELL", "pg_isready -U dev"]
interval: 2s
retries: 10One docker compose up gives every developer an identical backend. No version
drift, no missing services, no onboarding day lost to setup.
N+1 queries are one of those bugs that feel fine locally — three posts, three extra queries, who cares — and then detonate in production when you’re paging through ten thousand rows. The pattern is always the same: you load a collection, then inside a loop you touch a relationship that hasn’t been loaded, triggering a new query per row. Eloquent’s lazy loading is the enabler; understanding it is the cure.
Before you can fix it you need to see it. In a AppServiceProvider boot method, DB::listen gives you raw visibility with zero dependencies:
DB::listen(function ($query) {
if (config('app.debug')) {
Log::debug($query->sql, [
'bindings' => $query->bindings,
'time' => $query->time,
]);
}
});For a browser-facing app, Laravel Debugbar is more ergonomic — it groups queries per request, highlights duplicates, and shows backtraces. Install it dev-only:
composer require barryvdh/laravel-debugbar --devThe “Queries” tab will immediately show you 1 query turning into 51.
The fix for most N+1s is a single with() call:
// Bad — 1 + N queries
$posts = Post::all();
foreach ($posts as $post) {
echo $post->author->name; // lazy loads author per post
}
// Good — 2 queries total
$posts = Post::with('author')->get();
foreach ($posts as $post) {
echo $post->author->name; // already in memory
}Nest dot-notation for deeper relationships:
Post::with(['author', 'author.profile', 'comments.user'])->get();Eloquent resolves these in the minimum number of queries — one per relationship level, not one per model.
with() accepts a closure when you need to scope what gets loaded. This avoids pulling the entire comments table just to show recent activity:
Post::with([
'comments' => function ($query) {
$query->where('approved', true)
->latest()
->limit(5);
},
'tags:id,name', // select-constrained shorthand
])->get();The 'tags:id,name' syntax is shorthand for selecting only specific columns on the eager-loaded relation — useful for keeping payloads small.
Sometimes you don’t know upfront what you’ll need, or you receive a collection from a method you don’t own. load() works on an already-retrieved collection:
$posts = $this->postRepository->getPublished(); // returns Collection
if ($showAuthors) {
$posts->load('author');
}load() fires the same two-query pattern as with() — it does not trigger one query per model. Use it for conditional loading rather than as an escape hatch for bad architecture.
Aggregates are a common source of invisible N+1s. Counting comments per post in a loop means N COUNT(*) queries:
// Bad
foreach (Post::all() as $post) {
echo $post->comments()->count();
}
// Good — adds comments_count column in the original query
Post::withCount('comments')->get();
// Access as $post->comments_countwithSum, withAvg, withMin, withMax follow the same pattern when you need aggregates beyond counts:
Post::withSum('orderItems', 'quantity')->get();
// $post->order_items_sum_quantityThe most reliable guardrail is telling Eloquent to throw an exception whenever lazy loading happens in development. Add this to AppServiceProvider::boot():
use Illuminate\Database\Eloquent\Model;
Model::preventLazyLoading(! app()->isProduction());Now any unloaded relationship access throws a LazyLoadingViolationException before it hits production. This converts a silent performance problem into a loud, test-blocking failure — exactly what you want in a CI pipeline. Your production behavior is unchanged; the check is disabled when APP_ENV=production.
N+1 queries are entirely preventable. Use DB::listen or Debugbar to spot them early, reach for with() as your default when fetching collections with relationships, constrain loads when you only need a subset, and make preventLazyLoading() a non-negotiable part of your dev setup. An application that never lazy-loads in dev won’t lazy-load in production.
Sanctum covers two distinct use cases and it’s worth being clear about which one you’re solving before writing a line of code. The first is personal access tokens for first-party mobile apps, CLI tools, or third-party integrations — stateless, Bearer-token auth where the token lives on the client. The second is SPA cookie auth for a JavaScript front-end on the same domain as the API — stateful, session-cookie auth that re-uses Laravel’s existing session infrastructure. They’re both Sanctum, but they work completely differently and should not be mixed up.
Install and migrate:
composer require laravel/sanctum
php artisan vendor:publish --provider="Laravel\Sanctum\SanctumServiceProvider"
php artisan migrateAdd the HasApiTokens trait to your User model:
use Laravel\Sanctum\HasApiTokens;
class User extends Authenticatable
{
use HasApiTokens, HasFactory, Notifiable;
}Issue a token on login. The plain-text value is only available once — hash the stored copy, never log it:
public function login(Request $request): JsonResponse
{
$request->validate([
'email' => 'required|email',
'password' => 'required',
]);
$user = User::where('email', $request->email)->firstOrFail();
if (! Hash::check($request->password, $user->password)) {
throw ValidationException::withMessages([
'email' => ['Invalid credentials.'],
]);
}
$token = $user->createToken(
$request->input('device_name', 'api'),
['orders:read', 'orders:write']
)->plainTextToken;
return response()->json(['token' => $token]);
}The client sends Authorization: Bearer <token> on every subsequent request.
The second argument to createToken() is an array of ability strings — Sanctum’s equivalent of OAuth scopes. Check them in controllers or middleware:
public function destroy(Order $order): JsonResponse
{
if (! $request->user()->tokenCan('orders:write')) {
abort(403, 'Token lacks orders:write ability.');
}
$order->delete();
return response()->json(['deleted' => true]);
}Define abilities as constants somewhere authoritative (an Ability enum or a service class) rather than scattering raw strings across controllers. Sanctum does no ability registry of its own, so the discipline is yours.
Wire the middleware in routes/api.php. Sanctum’s guard handles both token and cookie auth transparently:
Route::middleware('auth:sanctum')->group(function () {
Route::get('/user', [UserController::class, 'show']);
Route::apiResource('orders', OrderController::class);
});For the public login and registration endpoints, keep them outside that group. Don’t reach for auth:api (Passport’s guard) — that’s a different package.
For a same-domain SPA (e.g., Vue/React served from app.example.com calling api.example.com), skip tokens entirely. Sanctum re-uses the session cookie:
api.example.com to SANCTUM_STATEFUL_DOMAINS in .env.GET /sanctum/csrf-cookie to prime the CSRF cookie.POST /login using Axios (which sends cookies automatically).This approach is simpler, inherits Laravel’s session security, and avoids token storage problems in the browser. The catch: it only works when the SPA and API are on the same top-level domain.
Tokens are rows in personal_access_tokens. Deleting them revokes access immediately:
// Revoke the current token (logout)
$request->user()->currentAccessToken()->delete();
// Revoke all tokens for the user (force logout everywhere)
$request->user()->tokens()->delete();
// Revoke a specific token by ID
$request->user()->tokens()->where('id', $tokenId)->delete();Implement a GET /tokens endpoint so users can see active sessions and revoke them individually — this is table-stakes UX for any API that issues long-lived tokens.
Sanctum is the right default for the vast majority of Laravel APIs. Reach for Passport when:
Passport is a proper OAuth server backed by league/oauth2-server. Sanctum is a thin, session-aware token layer. If your use case doesn’t require delegated authorization from external parties, Sanctum is simpler, faster to set up, and easier to reason about.
Sanctum earns its place as the default Laravel auth package for APIs and SPAs. Personal access tokens with abilities cover mobile apps and third-party integrations cleanly; cookie auth covers same-domain SPAs without token-in-localStorage risks. Keep abilities well-defined, build revocation UI from day one, and only graduate to Passport when you genuinely need an OAuth authorization server.
The jump from PHP 7 to PHP 8.x isn’t just JIT and nullable types. The language has acquired genuinely expressive constructs that eliminate whole categories of boilerplate and make intent clearer. If you’re still writing const STATUS_ACTIVE = 'active' or ten-line constructors with manual property assignments, you’re leaving readability on the table.
Before 8.1, PHP had no native enum type. Developers faked it with class constants, final class workarounds, or packages like myclabs/php-enum. PHP 8.1 native enums are better in every way — they’re first-class types that the type system understands.
Backed enums bind each case to a primitive value (string or int), enabling serialization and database round-trips:
enum Status: string
{
case Active = 'active';
case Archived = 'archived';
case Banned = 'banned';
public function label(): string
{
return match($this) {
Status::Active => 'Active',
Status::Archived => 'Archived',
Status::Banned => 'Banned',
};
}
}
// Retrieve from a DB string value
$status = Status::from('active'); // Status::Active
$status = Status::tryFrom('unknown'); // null (safe)
// Strict type checking in function signatures
function activateUser(User $user, Status $status): void { ... }Enums implement interfaces, hold methods, and can be used in match expressions. The cases() static method returns all cases as an array — useful for populating dropdowns.
PHP 8.1 introduced readonly on properties; 8.2 added readonly classes. A readonly property can be written exactly once — during initialization — and never mutated afterward. This is the closest PHP gets to immutable value objects without a library:
class Money
{
public function __construct(
public readonly int $amount,
public readonly string $currency,
) {}
}
$price = new Money(1999, 'USD');
$price->amount = 2000; // Error: Cannot modify readonly propertyUse this on DTOs, value objects, and domain entities where mutation is semantically wrong. It’s self-documenting: the type declaration is the rule.
Constructor promotion (PHP 8.0) eliminates the repetitive declare-then-assign pattern. Combine it with readonly and you get concise, explicit value objects with zero boilerplate:
// Old way — four mentions of $name
class CreateUserDTO
{
public string $name;
public string $email;
public function __construct(string $name, string $email)
{
$this->name = $name;
$this->email = $email;
}
}
// New way
class CreateUserDTO
{
public function __construct(
public readonly string $name,
public readonly string $email,
) {}
}Named arguments (8.0) let you skip positional ordering and be explicit at the call site. They’re most valuable when calling built-in functions with many optional parameters:
// Before — had to remember argument order
$result = array_slice($items, 0, 5, true);
// After — intent is clear
$result = array_slice(array: $items, offset: 0, length: 5, preserve_keys: true);They also make complex constructor calls readable and let you skip optional arguments in the middle without passing null.
PHP 8.1 lets you create closures from any callable using the ... syntax, without wrapping in an anonymous function:
// Before
$fn = fn($x) => strlen($x);
usort($strings, fn($a, $b) => strcmp($a, $b));
// After
$fn = strlen(...);
usort($strings, strcmp(...));
// Works on methods too
$getter = $user->getName(...);
$handler = [OrderController::class, 'store'](...);This integrates cleanly with array_map, array_filter, and any higher-order pipeline code. It also plays well with static analysis because the tool knows the exact signature of the returned closure.
Fibers (PHP 8.1) are stackful coroutines: you can suspend execution mid-function and resume later. They’re the foundation for async primitives in frameworks like ReactPHP and Revolt.
$fiber = new Fiber(function (): void {
$value = Fiber::suspend('first suspension');
echo "Resumed with: $value\n";
});
$yielded = $fiber->start(); // 'first suspension'
$fiber->resume('hello'); // prints: Resumed with: helloFibers are not goroutines and they’re not magic async. PHP is still single-threaded. A Fiber suspends the current stack frame so the event loop can do other work — but only if you’re running inside an event loop (Revolt, Amp v3, ReactPHP). In a standard Laravel/PHP-FPM setup, Fibers add complexity with zero benefit. They matter for long-running processes handling concurrent I/O — not for the typical request-response cycle.
Backed enums, readonly properties, and constructor promotion are production-ready tools that belong in every PHP 8.1+ codebase now. Named arguments and first-class callables are minor quality-of-life wins that accumulate. Fibers are genuinely useful — in the right context, which is event-loop-driven applications, not typical framework request handling. Adopt the features that solve real problems; don’t adopt them because they’re new.
The Django ORM is expressive enough to build complex queries with almost no SQL knowledge. That’s the problem. It’s equally easy to accidentally fire 1,000 queries per request, and the framework will not stop you. Production load exposes this quickly; a page that benchmarks at 40 ms under local testing can degrade to 400 ms with real data the moment each rendered row triggers its own database round-trip.
The tools to fix this are built into Django. The key is knowing which tool applies to which relationship type — and using django-debug-toolbar to confirm the query count actually dropped.
The classic N+1 happens on forward foreign-key and one-to-one relationships. Iterating over posts and touching post.author inside the loop triggers a separate SELECT per row:
# Bad — one query per post
for post in Post.objects.all():
print(post.author.username) # hits DB each iterationselect_related rewrites the queryset to use a SQL JOIN, loading related objects in the same query:
# One query with JOIN
posts = Post.objects.select_related('author').all()
for post in posts:
print(post.author.username) # already in memoryYou can chain across relationships: select_related('author__profile') will JOIN through two tables. Use it when the relation depth is bounded and the join won’t explode row count (generally foreign-key and one-to-one, not many-to-many).
select_related uses a JOIN, which does not compose well with M2M or reverse FK relationships because it multiplies rows. prefetch_related runs a separate optimized query per relation and stitches results together in Python:
from django.db.models import Prefetch
# Two queries total, regardless of post count
posts = Post.objects.prefetch_related('tags', 'comments').all()
# Custom Prefetch for filtering or annotation inside the prefetch
recent_comments = Comment.objects.filter(approved=True).order_by('-created_at')
posts = Post.objects.prefetch_related(
Prefetch('comments', queryset=recent_comments, to_attr='approved_comments')
)
for post in posts:
for comment in post.approved_comments:
print(comment.body)The to_attr argument avoids touching the default manager cache and gives you a plain list, which is faster to iterate.
select_related and prefetch_related handle row counts. only() and defer() handle column width. If a table has a body TEXT column that is large and only needed on the detail view, exclude it from list queries:
# Load only what the list view needs
posts = (
Post.objects
.select_related('author')
.only('title', 'published_at', 'slug', 'author__username')
)defer() is the inverse — specify what to skip:
posts = Post.objects.defer('body', 'search_vector')Accessing a deferred field triggers a lazy query per object, so treat it as a bug if you accidentally do so inside a loop. django-debug-toolbar’s query panel makes deferred-field lazy-load queries obvious because they appear as single-column SELECT statements with a WHERE id = ? clause.
Computing aggregates in Python means loading rows you don’t need. Use annotate() to attach computed values at the database level:
from django.db.models import Count, Avg, Q
# Annotate each author with post count and average word count
authors = (
Author.objects
.annotate(
post_count=Count('posts'),
avg_words=Avg('posts__word_count'),
published_count=Count('posts', filter=Q(posts__status='published')),
)
.filter(post_count__gte=3)
.order_by('-post_count')
)aggregate() reduces to a single dict when you need a scalar:
from django.db.models import Max, Min
stats = Post.objects.filter(status='published').aggregate(
newest=Max('published_at'),
oldest=Min('published_at'),
total=Count('id'),
)Both delegate entirely to SQL. The equivalent Python loops would require fetching every row.
Django caches every object in a queryset’s internal _result_cache. For bulk exports or data migrations this can exhaust memory. .iterator() streams rows without caching:
# Does not load all rows into memory at once
for post in Post.objects.filter(status='published').iterator(chunk_size=2000):
export_post(post)The trade-off: you cannot call len() on the queryset or access prefetch_related results (prefetch and iterator are mutually exclusive). For exports that need related data, issue a separate queryset per chunk rather than combining the two.
None of this matters without measurement. In development, add django-debug-toolbar to INSTALLED_APPS and middleware:
# settings.py (development only)
if DEBUG:
INSTALLED_APPS += ['debug_toolbar']
MIDDLEWARE.insert(0, 'debug_toolbar.middleware.DebugToolbarMiddleware')
INTERNAL_IPS = ['127.0.0.1']The SQL panel shows every query, its duration, and which line of Python triggered it. A view with 80 similar SELECT statements with incrementing WHERE id = ? clauses is an N+1 — add select_related or prefetch_related and watch the count collapse to 2-3 queries.
Django ORM performance problems have a predictable shape: N+1 on relationships, unnecessary column loads, aggregate logic running in Python, and memory exhaustion on large sets. select_related and prefetch_related solve the first class, only()/defer() the second, annotate()/aggregate() the third, and .iterator() the fourth. Install django-debug-toolbar and check the query count on every view — if it scales with the number of rows rendered, you have a problem worth fixing before it reaches production.
Python’s asyncio is a decade old and still widely misunderstood. The confusion usually comes from equating concurrency with parallelism. asyncio gives you cooperative multitasking on a single thread: while one coroutine waits on I/O, the event loop runs another. CPU-bound work blocks the loop entirely. Once you internalize that distinction, the API clicks into place and the patterns become obvious.
This is not a tutorial on async def. It’s the operational layer: how to run many coroutines efficiently, keep concurrency bounded, add timeouts that actually fire, and avoid stalling the loop with synchronous code you didn’t write.
The event loop is the scheduler. Coroutines are not threads — they yield control only at await expressions. Everything between two await calls runs synchronously and blocks the loop for its duration.
import asyncio
async def main():
# Runs coroutines sequentially — 2 seconds total
await asyncio.sleep(1)
await asyncio.sleep(1)
asyncio.run(main())asyncio.run() creates a fresh event loop, runs the coroutine to completion, and tears the loop down. For application code, this is the entry point you want. Never call loop.run_until_complete() manually in new code unless you’re managing loop lifecycle explicitly (rare).
asyncio.gather schedules multiple coroutines concurrently and collects their results in order:
import asyncio
import httpx
async def fetch(client: httpx.AsyncClient, url: str) -> dict:
response = await client.get(url, timeout=10.0)
response.raise_for_status()
return response.json()
async def main():
urls = [
"https://api.example.com/users/1",
"https://api.example.com/users/2",
"https://api.example.com/users/3",
]
async with httpx.AsyncClient() as client:
# All three requests fire simultaneously
results = await asyncio.gather(
*[fetch(client, url) for url in urls],
return_exceptions=True,
)
for result in results:
if isinstance(result, Exception):
print(f"Request failed: {result}")
else:
print(result["username"])
asyncio.run(main())return_exceptions=True prevents one failed coroutine from cancelling siblings — essential for fan-out calls where partial success is acceptable. Without it, the first exception propagates immediately and any still-running coroutines are cancelled.
Firing 500 coroutines at once is not free. Each holds an open connection, and the remote server may rate-limit or reject you. A Semaphore caps how many run simultaneously:
import asyncio
import httpx
async def fetch_bounded(
client: httpx.AsyncClient,
url: str,
sem: asyncio.Semaphore,
) -> dict:
async with sem:
response = await client.get(url, timeout=10.0)
response.raise_for_status()
return response.json()
async def main():
urls = [f"https://api.example.com/items/{i}" for i in range(200)]
sem = asyncio.Semaphore(20) # max 20 in-flight at a time
async with httpx.AsyncClient() as client:
results = await asyncio.gather(
*[fetch_bounded(client, url, sem) for url in urls],
return_exceptions=True,
)
return [r for r in results if not isinstance(r, Exception)]
asyncio.run(main())Tune the semaphore limit against your connection pool size (httpx.AsyncClient defaults to 100 connections per host) and the target service’s rate limits. Starting at 10–20 and measuring throughput is safer than guessing.
asyncio.gather itself has no timeout parameter. Wrap individual coroutines with asyncio.wait_for:
import asyncio
async def slow_operation() -> str:
await asyncio.sleep(5)
return "done"
async def main():
try:
result = await asyncio.wait_for(slow_operation(), timeout=2.0)
except asyncio.TimeoutError:
print("Operation timed out after 2 seconds")
result = None
return result
asyncio.run(main())When the timeout fires, wait_for cancels the wrapped coroutine and raises asyncio.TimeoutError. The cancellation propagates into any await inside slow_operation, so cleanup code in finally blocks runs correctly.
For per-item timeouts inside a gather, wrap each coroutine individually:
tasks = [asyncio.wait_for(fetch(client, url), timeout=5.0) for url in urls]
results = await asyncio.gather(*tasks, return_exceptions=True)Any synchronous call that takes real time — a requests.get, a bcrypt.hashpw, reading a large file with the stdlib open() — blocks the entire event loop for its duration. Other coroutines freeze until it returns.
The fix is loop.run_in_executor, which offloads the call to a thread pool:
import asyncio
import bcrypt
from concurrent.futures import ThreadPoolExecutor
executor = ThreadPoolExecutor(max_workers=4)
async def hash_password(plain: str) -> str:
loop = asyncio.get_running_loop()
hashed = await loop.run_in_executor(
executor,
bcrypt.hashpw,
plain.encode(),
bcrypt.gensalt(rounds=12),
)
return hashed.decode()
async def main():
# bcrypt is CPU-bound; run_in_executor keeps the loop responsive
result = await hash_password("hunter2")
print(result)
asyncio.run(main())Pass None as the executor to use the default ThreadPoolExecutor. For CPU-bound work that needs true parallelism, use ProcessPoolExecutor instead — threads share the GIL, processes don’t.
Pulling the patterns together: a bounded async client with per-request timeouts and structured error handling:
import asyncio
import httpx
from dataclasses import dataclass
@dataclass
class FetchResult:
url: str
data: dict | None
error: str | None
async def fetch_safe(
client: httpx.AsyncClient,
url: str,
sem: asyncio.Semaphore,
) -> FetchResult:
async with sem:
try:
resp = await asyncio.wait_for(
client.get(url),
timeout=8.0,
)
resp.raise_for_status()
return FetchResult(url=url, data=resp.json(), error=None)
except asyncio.TimeoutError:
return FetchResult(url=url, data=None, error="timeout")
except httpx.HTTPStatusError as e:
return FetchResult(url=url, data=None, error=f"HTTP {e.response.status_code}")
async def bulk_fetch(urls: list[str]) -> list[FetchResult]:
sem = asyncio.Semaphore(15)
async with httpx.AsyncClient(timeout=httpx.Timeout(10.0)) as client:
return await asyncio.gather(*[fetch_safe(client, url, sem) for url in urls])
if __name__ == "__main__":
urls = [f"https://jsonplaceholder.typicode.com/posts/{i}" for i in range(1, 51)]
results = asyncio.run(bulk_fetch(urls))
errors = [r for r in results if r.error]
print(f"Fetched {len(results) - len(errors)} ok, {len(errors)} errors")asyncio earns its complexity budget when your service is I/O-bound and you need real concurrency without thread-per-request overhead. The patterns here — gather for fan-out, Semaphore to stay sane, wait_for for timeout discipline, run_in_executor for blocking callsites you can’t rewrite — cover the majority of production use cases. The remaining failure mode is accidentally blocking the loop; asyncio.get_event_loop().is_running() combined with profiling tools like aiomonitor will surface it when it happens.
Go’s error handling gets a bad reputation from people who use it wrong. Returning
plain errors.New("something failed") from ten layers deep tells the caller
nothing useful. Wrapping every error with a novel message creates noise. The real
discipline is knowing what information each layer owns and what it should add —
nothing more.
The standard library’s errors package (augmented by fmt.Errorf in Go 1.13)
gives you a complete toolkit. Sentinel errors for identity checks, wrapped errors
for context, and custom types for structured data. Here is how to use each one
correctly.
A sentinel is a package-level error value callers can compare against. Define one when you expect callers to branch on the specific condition.
var (
ErrNotFound = errors.New("not found")
ErrConflict = errors.New("conflict")
ErrForbidden = errors.New("forbidden")
)
func (r *UserRepo) Get(ctx context.Context, id int64) (*User, error) {
var u User
err := r.db.QueryRowContext(ctx,
"SELECT id, email FROM users WHERE id = $1", id,
).Scan(&u.ID, &u.Email)
if errors.Is(err, sql.ErrNoRows) {
return nil, ErrNotFound
}
return &u, err
}Notice the translation: sql.ErrNoRows is a storage-layer detail. The repository
returns ErrNotFound, a domain-level sentinel the HTTP handler understands.
Never let sql.ErrNoRows leak past the repository boundary.
Use %w when you need to add context without losing the underlying error’s
identity. The wrapped error remains matchable with errors.Is and errors.As.
func (s *OrderService) Submit(ctx context.Context, o *Order) error {
if err := s.inventory.Reserve(ctx, o.Items); err != nil {
return fmt.Errorf("submit order %d: reserve inventory: %w", o.ID, err)
}
if err := s.payments.Charge(ctx, o.Total); err != nil {
return fmt.Errorf("submit order %d: charge payment: %w", o.ID, err)
}
return nil
}The resulting error string reads like a call stack:
submit order 42: charge payment: card declined. Each layer adds its own noun
phrase — no repeating “error” or “failed”. Do not wrap errors at every function
boundary mechanically; add context only when it changes the meaning for the
reader of a log line.
func (h *OrderHandler) Submit(w http.ResponseWriter, r *http.Request) {
err := h.svc.Submit(r.Context(), order)
if err == nil {
w.WriteHeader(http.StatusCreated)
return
}
switch {
case errors.Is(err, ErrNotFound):
http.Error(w, "not found", http.StatusNotFound)
case errors.Is(err, ErrConflict):
http.Error(w, "conflict", http.StatusConflict)
default:
slog.ErrorContext(r.Context(), "submit order", "err", err)
http.Error(w, "internal error", http.StatusInternalServerError)
}
}errors.Is unwraps the chain automatically, so it matches ErrNotFound even
when it is three %w layers deep. No string comparison, no type assertion on
raw errors.
When callers need more than an identity — field names, HTTP status codes, retry delays — define a type.
type ValidationError struct {
Field string
Message string
}
func (e *ValidationError) Error() string {
return fmt.Sprintf("validation: %s: %s", e.Field, e.Message)
}
func validateEmail(email string) error {
if !strings.Contains(email, "@") {
return &ValidationError{Field: "email", Message: "must contain @"}
}
return nil
}At the call site, extract the type with errors.As:
var ve *ValidationError
if errors.As(err, &ve) {
http.Error(w, ve.Message, http.StatusUnprocessableEntity)
return
}errors.As also traverses the wrap chain, so you can wrap a *ValidationError
with fmt.Errorf and still unwrap it successfully higher up.
Panic for programmer errors, not runtime conditions. An out-of-bounds slice index, a nil pointer that should never be nil, an impossible switch branch — these are bugs, not errors. Panicking surfaces them loudly at development time.
Never panic for network failures, database errors, or any condition that depends on external state. A production service should handle those gracefully, not crash.
The recover pattern belongs in framework-level middleware, not in application
code:
func recoveryMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
defer func() {
if v := recover(); v != nil {
slog.Error("panic recovered", "value", v,
"stack", debug.Stack())
http.Error(w, "internal error", http.StatusInternalServerError)
}
}()
next.ServeHTTP(w, r)
})
}Define sentinels for conditions callers branch on. Wrap with %w to add context
without losing identity. Use custom types when callers need structured data. Check
with errors.Is and errors.As, never string comparison. Panic only for
programmer mistakes. Each layer should add meaning, not noise.
Adding an index is easy. Adding the right index is not. Most teams drop a B-tree on every foreign key column, watch query times improve slightly, and call it done. Then six months later a query that filters on three columns and returns a handful of fields still does a full sequential scan because the index doesn’t cover what the planner needs.
Understanding how Postgres chooses indexes — and why it sometimes refuses to use
one you created — saves hours of guesswork. EXPLAIN ANALYZE is the primary
tool. The index types are a close second.
B-tree indexes work on equality and range predicates against sortable types:
integers, timestamps, UUIDs, text. They are the right default for primary keys,
foreign keys, and any column you filter with =, <, >, BETWEEN, or ORDER BY.
-- Supports: WHERE created_at > '2026-01-01' ORDER BY created_at DESC
CREATE INDEX idx_events_created ON events (created_at DESC);B-tree does not help with:
@>)@@)LIKE '%suffix%' (only 'prefix%' works with text_pattern_ops)For those cases, use the right index type instead of assuming the default will do.
GIN (Generalized Inverted Index) indexes every element of a composite value, making
it ideal for array containment, JSONB, and tsvector full-text columns.
-- JSONB key existence and path queries
CREATE INDEX idx_products_attrs ON products USING GIN (attributes);
-- Query benefits from the index
SELECT * FROM products
WHERE attributes @> '{"color": "red", "size": "L"}';
-- Full-text search
CREATE INDEX idx_articles_search ON articles USING GIN (to_tsvector('english', body));
SELECT * FROM articles
WHERE to_tsvector('english', body) @@ plainto_tsquery('english', 'database indexing');GIN builds are slower and the indexes are larger than B-tree. Write-heavy tables
with GIN indexes will see higher insert latency. If that is a problem, GIN with
fastupdate = on (the default) batches pending insertions; VACUUM flushes them.
A partial index carries a WHERE clause. It only indexes rows that satisfy the
condition, making it much smaller than a full-column index and faster on queries
that match the same condition.
-- Only index unprocessed jobs — the hot working set
CREATE INDEX idx_jobs_pending ON jobs (created_at)
WHERE status = 'pending';
-- This query uses the index; a query without the WHERE clause does not
SELECT * FROM jobs WHERE status = 'pending' ORDER BY created_at LIMIT 100;Partial indexes shine for status-column patterns where one value represents
99% of the table (e.g., deleted_at IS NULL) and another is the active
minority that queries actually care about.
-- Unique constraint only on active records
CREATE UNIQUE INDEX idx_users_email_active ON users (email)
WHERE deleted_at IS NULL;Column order in a composite index is not arbitrary. Postgres can use a composite
index when the leading columns are constrained. Given (a, b, c):
a alone: uses the index.a and b: uses the index.b alone: does not use the index (no leading column).a and c: uses the index partially (only a prefix).-- Correct order: equality column first, then range/sort column
CREATE INDEX idx_orders_user_date ON orders (user_id, created_at DESC);
-- This query is fast — leading equality on user_id, then range on created_at
SELECT * FROM orders
WHERE user_id = 42
AND created_at > NOW() - INTERVAL '30 days'
ORDER BY created_at DESC;
-- This query cannot use the index (no user_id predicate)
-- SELECT * FROM orders WHERE created_at > NOW() - INTERVAL '30 days';Put your highest-cardinality, equality-constrained columns first. Put range or sort columns after.
A covering index satisfies a query entirely from the index, without a heap fetch
(a “Index Only Scan”). Add non-key columns with INCLUDE to cover the SELECT
list without bloating the sort key.
CREATE INDEX idx_orders_user_covering ON orders (user_id, created_at DESC)
INCLUDE (status, total);
-- Index Only Scan — no heap access
SELECT user_id, created_at, status, total
FROM orders
WHERE user_id = 42
ORDER BY created_at DESC
LIMIT 20;Columns in INCLUDE are stored in the leaf pages but not used for ordering.
They increase index size, so only include columns the specific critical queries
actually need.
EXPLAIN (ANALYZE, BUFFERS, FORMAT TEXT)
SELECT * FROM orders
WHERE user_id = 42 AND created_at > '2026-01-01'
ORDER BY created_at DESC
LIMIT 20;Key lines to read:
Seq Scan on a large table: your filter columns lack a usable index.Index Scan vs Index Only Scan: the latter means no heap fetch — good.Rows Removed by Filter: 48000: the index was used but a large post-filter happened; the index is not selective enough, or column order is wrong.Buffers: shared hit=12 read=4380: high read count means the working set is not in shared_buffers — may be a cache-size issue, not an index issue.ANALYZE orders.Every UPDATE or DELETE leaves dead tuples in the index. VACUUM reclaims
table space, but index pages are only cleaned by VACUUM too — and if autovacuum
can’t keep up on a write-heavy table, indexes grow without bound.
-- Estimate bloat per index (requires pgstattuple extension)
SELECT schemaname, tablename, indexname,
pg_size_pretty(pg_relation_size(indexrelid)) AS index_size
FROM pg_stat_user_indexes
ORDER BY pg_relation_size(indexrelid) DESC
LIMIT 20;
-- Rebuild a bloated index without locking writes
REINDEX INDEX CONCURRENTLY idx_orders_user_date;Watch pg_stat_user_indexes.idx_scan as well. An index with zero scans in the
past week is dead weight — it costs write overhead and autovacuum work for no
query benefit. Drop it.
Match index type to query pattern: B-tree for equality and ranges, GIN for
composite values. Put equality columns first in composites. Use INCLUDE to
eliminate heap fetches on critical read paths. Read EXPLAIN ANALYZE before and
after every index change — never assume. And audit idx_scan counts regularly;
unused indexes are not free.
Redis is fast enough to paper over almost any problem — until it isn’t, and then the failure is spectacular. Cache stampedes take down databases. Missing TTL jitter creates synchronized expiry waves. Stale keys from a broken invalidation strategy serve wrong data for hours. The patterns are not complicated, but skipping the failure-mode analysis is where production incidents come from.
These are the patterns worth understanding before you write your first SET.
Cache-aside is the most common pattern: the application reads from the cache, and on a miss, reads from the database and populates the cache before returning.
import redis
import json
r = redis.Redis(host="localhost", port=6379, decode_responses=True)
def get_user(user_id: int) -> dict:
key = f"user:{user_id}"
cached = r.get(key)
if cached:
return json.loads(cached)
# Cache miss — load from DB
user = db.query("SELECT id, email, name FROM users WHERE id = %s", user_id)
if user is None:
return None
r.setex(key, 300, json.dumps(user)) # TTL: 5 minutes
return userThe upside: only data that is actually requested gets cached. The downside: the first request after expiry always hits the database. Under load, that first request becomes thousands of requests simultaneously — the stampede.
Write-through updates the cache synchronously when writing to the database. The cache never serves a cold miss for written data.
def update_user(user_id: int, data: dict) -> None:
# Update DB first — source of truth
db.execute(
"UPDATE users SET email = %s, name = %s WHERE id = %s",
data["email"], data["name"], user_id,
)
# Then update cache
key = f"user:{user_id}"
r.setex(key, 300, json.dumps({"id": user_id, **data}))Write-through works well when you control all write paths. It breaks when multiple services write to the same table without going through the same cache layer — which is common in microservice setups. In those cases, cache-aside with explicit invalidation is more reliable.
Setting all cached objects to the same TTL causes synchronized expiry. At 5:00 PM every key written at 4:55 PM expires simultaneously. Every service instance races to the database at once.
Fix it with jitter — a random offset added to the base TTL:
import random
def cache_ttl(base_seconds: int, jitter_pct: float = 0.2) -> int:
"""Add ±jitter_pct of base_seconds to the TTL."""
delta = int(base_seconds * jitter_pct)
return base_seconds + random.randint(-delta, delta)
# 5-minute TTL with ±20% jitter → 240–360 seconds
r.setex(key, cache_ttl(300), value)Choose TTLs that match data volatility. Session tokens: 15–30 minutes. User profiles: 5–10 minutes. Reference data that rarely changes (country list, plan names): hours or longer with explicit invalidation on write.
When a key expires under high traffic, use a short-lived lock so only one process rebuilds the cache while others wait or return stale data.
import time
def get_user_protected(user_id: int) -> dict:
key = f"user:{user_id}"
lock_key = f"lock:{key}"
cached = r.get(key)
if cached:
return json.loads(cached)
# Try to acquire the rebuild lock (NX = only set if not exists)
acquired = r.set(lock_key, "1", ex=5, nx=True)
if acquired:
try:
user = db.query("SELECT id, email, name FROM users WHERE id = %s", user_id)
r.setex(key, cache_ttl(300), json.dumps(user))
return user
finally:
r.delete(lock_key)
else:
# Another process is rebuilding — wait briefly and retry
time.sleep(0.05)
cached = r.get(key)
return json.loads(cached) if cached else NoneIn Go or Python, a cleaner alternative is singleflight (Go’s
golang.org/x/sync/singleflight or a Python equivalent): deduplicate concurrent
requests for the same key at the application layer so only one goroutine/coroutine
actually calls the database, and the rest share its result.
Keys should be namespaced, versioned where needed, and deterministic.
# Bad: no namespace, no version
SET 42 '{"id":42}'
# Good: type:id
SET user:42 '{"id":42,"email":"[email protected]"}' EX 300
# Good: include query parameters in the key
SET "orders:user:42:status:pending:page:1" '...' EX 60
# Version bump to invalidate all keys after a schema change
SET "v2:user:42" '{"id":42,"email":"[email protected]","plan":"pro"}' EX 300# Inspect key TTL
redis-cli TTL user:42
# Find all keys for a user (use SCAN, never KEYS in production)
redis-cli SCAN 0 MATCH "user:42:*" COUNT 100
# Flush a specific key after a write
redis-cli DEL user:42Never use KEYS * in production — it blocks the event loop. SCAN is the
correct cursor-based alternative.
Cache invalidation is famously hard. The simplest strategies are also the most reliable:
INCR.def invalidate_user(user_id: int) -> None:
# Delete the primary key and any known derived keys
r.delete(
f"user:{user_id}",
f"user:{user_id}:orders",
f"user:{user_id}:sessions",
)Cache-aside is the right default. Add write-through only when you control all write paths. Always add TTL jitter. Protect high-traffic cache rebuilds with a short lock or singleflight deduplication. Design key names with namespacing and a version prefix you can bump. And when in doubt, invalidate on write — a brief cache miss is far better than serving stale data for an hour.
WordPress powers roughly 43% of the web, but coupling it to its own theme layer wastes its content-management strengths. A headless setup — WP handles data, a separate frontend (Next.js, Astro, whatever) handles rendering — gives you a mature admin UI, ACF for structured content, and a battle-tested auth story, without the PHP templating. The REST API is production-ready. The pain is knowing which knobs to turn.
This post covers the practical decisions: custom routes, exposing ACF fields, authentication options, and caching so you’re not hammering the database on every page request.
The built-in /wp/v2/posts endpoint works for simple blogs, but real projects need shaped responses. Don’t filter the default endpoint — register your own namespace.
add_action('rest_api_init', function () {
register_rest_route('myapp/v1', '/products', [
'methods' => WP_REST_Server::READABLE,
'callback' => 'myapp_get_products',
'permission_callback' => '__return_true', // public endpoint
'args' => [
'category' => [
'sanitize_callback' => 'sanitize_text_field',
],
],
]);
register_rest_route('myapp/v1', '/products/(?P<id>\d+)', [
'methods' => WP_REST_Server::READABLE,
'callback' => 'myapp_get_product',
'permission_callback' => '__return_true',
]);
});
function myapp_get_products(WP_REST_Request $request): WP_REST_Response {
$args = [
'post_type' => 'product',
'posts_per_page' => 20,
'post_status' => 'publish',
];
if ($category = $request->get_param('category')) {
$args['tax_query'] = [[
'taxonomy' => 'product_cat',
'field' => 'slug',
'terms' => $category,
]];
}
$posts = get_posts($args);
$data = array_map('myapp_format_product', $posts);
return new WP_REST_Response($data, 200);
}
function myapp_format_product(WP_Post $post): array {
return [
'id' => $post->ID,
'slug' => $post->post_name,
'title' => $post->post_title,
'description' => $post->post_content,
'price' => get_field('price', $post->ID),
'image' => get_the_post_thumbnail_url($post->ID, 'large'),
];
}Own the shape. A WP_REST_Response with a flat, predictable structure beats whatever the default endpoint gives you.
ACF Pro ships with native REST API support. Enable it per field group in the UI (Field Group > Settings > Show in REST API), then fields appear under the acf key on the post object. For Fine-Grained control, register fields manually:
add_action('rest_api_init', function () {
register_rest_field('product', 'price', [
'get_callback' => fn($post) => (float) get_field('price', $post['id']),
'schema' => ['type' => 'number'],
]);
register_rest_field('product', 'gallery', [
'get_callback' => function ($post) {
$images = get_field('gallery', $post['id']) ?: [];
return array_map(fn($img) => [
'url' => $img['url'],
'alt' => $img['alt'],
], $images);
},
'schema' => ['type' => 'array'],
]);
});This keeps the transformation logic server-side and out of your frontend.
Application Passwords (built into WordPress since 5.6) are the right choice for server-to-server requests — a Next.js build fetching draft posts, a cron job pushing data. Generate one in the user profile, then use HTTP Basic:
curl https://your-site.com/wp-json/myapp/v1/products \
-H "Authorization: Basic $(echo -n 'username:app-password' | base64)"JWT makes sense for browser clients. The wp-api-jwt-auth plugin is the standard choice. Issue a token on login, send it as a Bearer header on subsequent requests:
# Obtain token
curl -X POST https://your-site.com/wp-json/jwt-auth/v1/token \
-d '{"username":"editor","password":"secret"}' \
-H "Content-Type: application/json"
# Use token
curl https://your-site.com/wp-json/myapp/v1/products \
-H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGci..."For public endpoints set permission_callback to '__return_true'. For write operations, check capabilities explicitly — never skip this:
'permission_callback' => function () {
return current_user_can('edit_posts');
},WP REST responses are not cached by default. A request to /wp-json/myapp/v1/products hits the database every time. Fix this with the transients API and invalidate on post save:
function myapp_get_products(WP_REST_Request $request): WP_REST_Response {
$cache_key = 'myapp_products_' . md5(serialize($request->get_params()));
$cached = get_transient($cache_key);
if ($cached !== false) {
return new WP_REST_Response($cached, 200);
}
// ... build $data ...
set_transient($cache_key, $data, HOUR_IN_SECONDS);
return new WP_REST_Response($data, 200);
}
// Bust the cache when any product is saved
add_action('save_post_product', function () {
global $wpdb;
$wpdb->query(
"DELETE FROM {$wpdb->options}
WHERE option_name LIKE '_transient_myapp_products_%'
OR option_name LIKE '_transient_timeout_myapp_products_%'"
);
});For higher traffic, front the REST API with Nginx’s fastcgi_cache or a CDN with Cache-Control headers set in the response:
return new WP_REST_Response($data, 200, [
'Cache-Control' => 'public, max-age=300, s-maxage=600',
]);Headless WP is a good fit when: the content team already knows WP and you can’t retrain them; the site needs ACF-powered structured content across multiple frontends; or you’re migrating a legacy WP site and want the frontend decoupled without rewriting the content model.
It’s the wrong call when you’re starting fresh with no content history — reach for a purpose-built headless CMS or a database with a custom API. WP’s REST layer adds overhead you don’t need if you’re not also using the admin.
WordPress as an API backend is a pragmatic choice, not a compromise. Register typed routes under your own namespace, expose ACF fields explicitly, use Application Passwords for server-to-server auth and JWT for browser clients, and cache aggressively at the transient layer. The content team keeps their familiar UI; your frontend gets a clean JSON API.
Dropping a message into a queue and hoping the other side picks it up is not a messaging strategy. Without acknowledgements, your messages vanish on consumer crash. Without dead-letter queues, one bad message blocks processing forever. Without idempotent consumers, duplicate deliveries corrupt data. RabbitMQ gives you the building blocks for reliable async messaging — but you have to assemble them correctly.
This is the mental model and the implementation detail that matter.
Publishers send messages to exchanges, not directly to queues. The exchange routes them to one or more queues based on a binding and a routing key. Consumers read from queues. This indirection is what makes RabbitMQ flexible.
The three exchange types you’ll use:
orders.#, orders.*.created).
Use when consumers care about subsets of a message stream.import pika
connection = pika.BlockingConnection(pika.URLParameters("amqp://guest:guest@localhost/"))
channel = connection.channel()
# Declare a durable topic exchange
channel.exchange_declare(
exchange="events",
exchange_type="topic",
durable=True,
)
# Declare a durable queue with a dead-letter exchange
channel.queue_declare(
queue="order.processor",
durable=True,
arguments={
"x-dead-letter-exchange": "events.dlx",
"x-dead-letter-routing-key": "dead.order.processor",
"x-message-ttl": 60000, # messages expire after 60s if unacked
},
)
# Bind: this queue receives messages matching orders.*
channel.queue_bind(
queue="order.processor",
exchange="events",
routing_key="orders.*",
)Declare exchanges and queues as durable=True so they survive a broker restart.
Publish messages with delivery_mode=2 (persistent) for the same reason.
def publish_order_event(channel, order_id: int, event_type: str) -> None:
channel.basic_publish(
exchange="events",
routing_key=f"orders.{event_type}",
body=json.dumps({"order_id": order_id, "event": event_type}),
properties=pika.BasicProperties(
delivery_mode=2, # persistent — survives broker restart
content_type="application/json",
message_id=str(uuid.uuid4()), # for deduplication on consumer side
),
)
publish_order_event(channel, order_id=42, event_type="created")Note the message_id. Your consumers need it for idempotency.
auto_ack=True deletes the message the moment it is delivered. If the consumer
crashes mid-processing, the message is gone. Always use manual acks.
Prefetch (basic_qos) limits how many unacked messages a single consumer holds
at once. Without it, RabbitMQ dumps the entire queue into the first consumer that
connects, leaving other consumers idle.
def handle(channel, method, properties, body):
try:
data = json.loads(body)
process_order(data["order_id"])
# Acknowledge only after successful processing
channel.basic_ack(delivery_tag=method.delivery_tag)
except Exception as e:
log.error("processing failed", exc_info=e)
# Reject and requeue=False → message goes to dead-letter exchange
channel.basic_nack(delivery_tag=method.delivery_tag, requeue=False)
channel.basic_qos(prefetch_count=10) # max 10 unacked messages per consumer
channel.basic_consume(
queue="order.processor",
on_message_callback=handle,
auto_ack=False,
)
channel.start_consuming()requeue=False on a nack sends the message to the dead-letter exchange defined
on the queue. requeue=True puts it back at the front of the queue — risky if
the message is fundamentally broken (poison message loop).
A dead-letter exchange (DLX) is a regular exchange. Messages arrive there when
they are nacked with requeue=False, or when their x-message-ttl expires.
Bind a queue to the DLX to capture them for inspection and replay.
# Declare the dead-letter exchange
channel.exchange_declare(
exchange="events.dlx",
exchange_type="direct",
durable=True,
)
# Declare the dead-letter queue
channel.queue_declare(queue="dead.order.processor", durable=True)
channel.queue_bind(
queue="dead.order.processor",
exchange="events.dlx",
routing_key="dead.order.processor",
)Operationally: monitor the dead-letter queue. A growing DLQ means either
consumers are failing consistently (a bug) or you are receiving messages you
cannot process (a contract violation). Do not let it grow indefinitely — set
x-max-length on the DLQ and alert when it is non-empty.
RabbitMQ guarantees at-least-once delivery. A message may be delivered more than once — after a consumer crashes before acking, or after a network partition. Your consumer must handle duplicates without corrupting state.
The standard approach: track processed message IDs in a store the consumer can check atomically with the processing step.
import redis
dedup = redis.Redis(host="localhost", decode_responses=True)
def handle(channel, method, properties, body):
msg_id = properties.message_id
key = f"processed:{msg_id}"
# NX = only set if not exists; EX = expire after 24h
is_new = dedup.set(key, "1", ex=86400, nx=True)
if not is_new:
# Already processed — ack and skip
channel.basic_ack(delivery_tag=method.delivery_tag)
return
try:
data = json.loads(body)
process_order(data["order_id"])
channel.basic_ack(delivery_tag=method.delivery_tag)
except Exception as e:
dedup.delete(key) # Remove dedup key so retry is possible
log.error("processing failed", exc_info=e)
channel.basic_nack(delivery_tag=method.delivery_tag, requeue=False)If your processing step is a database write, you can also use a unique constraint
on the message_id column and treat the constraint violation as a duplicate signal.
That ties the deduplication to the same transaction as the write — stronger guarantee,
no Redis dependency.
At-least-once means:
auto_ack.If you need exactly-once, you are outside what RabbitMQ can guarantee at the broker level. The practical path is idempotent consumers with a deduplication store, which achieves the same observable outcome without distributed transaction overhead.
Wire your exchanges and queues with durable flags, binding keys that match your routing needs, and a dead-letter exchange from day one. Set prefetch to a number that balances throughput with memory pressure. Always ack after the work is done, nack without requeueing on unrecoverable errors. Make every consumer idempotent — RabbitMQ will redeliver, and your database cannot tell the difference between a retry and a bug unless you track it explicitly.
The “JWT vs sessions” debate is mostly a false dilemma driven by people who treat JWT as a drop-in session replacement. They are not the same thing. Sessions are stateful server-side records. JWTs are self-contained signed assertions. Each has a place. The mistake is using JWT because it sounds modern, then discovering the revocation problem when you need to log someone out immediately.
Here’s the tradeoff analysis without the hype.
Sessions: On login, the server creates a session record (in Redis, a database, or memory), returns an opaque session ID in a cookie, and on every request looks up that ID to verify the user. State lives on the server. The session ID is worthless without the backing store.
JWT: On login, the server signs a payload containing claims (user ID, roles, expiry) with a secret or private key and returns the token. On every request the server verifies the signature — no database lookup. State lives in the token.
A typical JWT payload looks like this:
{
"sub": "user_42",
"email": "[email protected]",
"roles": ["editor"],
"iat": 1700000000,
"exp": 1700000900
}The signature prevents tampering. But once issued, a valid JWT is valid until it expires — the server has no say.
This is where the “JWT is stateless” selling point turns into a liability. If you need to:
…you have a problem with plain JWT. The only real solutions are:
Option 3 is the industry standard approach for browser-facing APIs. Here’s the flow in Laravel:
// Issue token pair on login
public function login(Request $request): JsonResponse
{
$credentials = $request->only('email', 'password');
if (! Auth::attempt($credentials)) {
return response()->json(['error' => 'Invalid credentials'], 401);
}
$user = Auth::user();
// Short-lived access token (15 min)
$accessToken = JWT::encode([
'sub' => $user->id,
'exp' => now()->addMinutes(15)->timestamp,
], config('jwt.secret'), 'HS256');
// Long-lived refresh token stored in DB for revocation
$refreshToken = Str::random(64);
$user->refreshTokens()->create([
'token' => hash('sha256', $refreshToken),
'expires_at' => now()->addDays(30),
]);
return response()->json([
'access_token' => $accessToken,
'expires_in' => 900,
'refresh_token' => $refreshToken,
]);
}
// Refresh endpoint
public function refresh(Request $request): JsonResponse
{
$raw = $request->input('refresh_token');
$token = RefreshToken::where('token', hash('sha256', $raw))
->where('expires_at', '>', now())
->whereNull('revoked_at')
->firstOrFail();
// Rotate: revoke old, issue new
$token->update(['revoked_at' => now()]);
return $this->issueTokenPair($token->user);
}On logout, mark the refresh token as revoked. The access token expires on its own within 15 minutes.
Browser clients: Store the access token in memory (a JavaScript variable, not localStorage). Store the refresh token in an HttpOnly; Secure; SameSite=Strict cookie — JavaScript can’t read it, which blocks XSS token theft.
# Set-Cookie header from your server
Set-Cookie: refresh_token=def502...; HttpOnly; Secure; SameSite=Strict; Path=/auth/refresh; Max-Age=2592000Never store a long-lived token in localStorage. It’s accessible to any script on the page.
Server-to-server / mobile: Bearer token in the Authorization header is fine. Mobile apps can use secure storage APIs (iOS Keychain, Android Keystore).
Sessions are the better choice when:
Sessions scale horizontally without trouble if you back them with Redis:
# Laravel: store sessions in Redis
SESSION_DRIVER=redis
SESSION_LIFETIME=120The “sessions don’t scale” argument comes from storing sessions in server memory. That argument died when Redis became ubiquitous.
Use sessions for monoliths and server-rendered apps. Use JWT with refresh token rotation for APIs consumed by SPAs or mobile clients. Never use long-lived JWTs without a revocation strategy — the convenience isn’t worth it when an account needs to be locked out immediately.
If you find yourself adding a Redis blocklist to your JWT implementation, ask whether you just re-implemented sessions with extra steps.
JWT and sessions are tools for different contexts. Match the tool to the requirement: instant revocation, scaling constraints, client type, and how much state you’re willing to manage. Short-lived access tokens with rotating refresh tokens stored server-side give you the best of both worlds for modern API authentication.
Running php artisan migrate on a live production database and hoping nothing breaks is not a deployment strategy. A long-running ALTER TABLE on a millions-row table acquires an exclusive lock. Every write waits. Your P99 spikes. Users notice. The migration finishes in 90 seconds and you spend the next two hours explaining to stakeholders why the site was “slow.”
The solution is a discipline called expand/contract — and understanding which DDL operations are safe in Postgres and which are not.
Expand/contract separates schema changes across multiple deploys so the old code and new code can coexist with the database in an intermediate state. It has three phases:
This turns a dangerous single migration into three safe, independent deployments. The trade-off is time and discipline. The payoff is zero downtime.
Not all DDL is equal. Postgres is more permissive than MySQL on several operations:
Safe (no exclusive lock, or lock is brief):
-- Adding a nullable column with no DEFAULT is nearly instant
ALTER TABLE orders ADD COLUMN shipped_at timestamptz;
-- Adding a column with a DEFAULT in Postgres 11+ is also safe
-- (stored as a table default, not rewritten)
ALTER TABLE orders ADD COLUMN source varchar(32) DEFAULT 'web' NOT NULL;
-- Building an index without blocking writes
CREATE INDEX CONCURRENTLY idx_orders_shipped_at ON orders (shipped_at);
-- Adding a NOT VALID constraint — checks future rows only
ALTER TABLE orders ADD CONSTRAINT chk_status
CHECK (status IN ('pending','shipped','cancelled')) NOT VALID;
-- Validate separately (ShareUpdateExclusiveLock, allows reads/writes)
ALTER TABLE orders VALIDATE CONSTRAINT chk_status;Dangerous (takes AccessExclusiveLock, blocks reads and writes):
-- Dropping a column — still acquires AccessExclusiveLock
ALTER TABLE orders DROP COLUMN legacy_field;
-- Adding a column with DEFAULT in Postgres < 11 rewrites the table
ALTER TABLE orders ADD COLUMN flag boolean DEFAULT false NOT NULL; -- rewrite in PG10
-- Changing a column type almost always requires a table rewrite
ALTER TABLE orders ALTER COLUMN amount TYPE numeric(12,4);When in doubt, check pg_locks during a test run on a staging replica with production-size data.
Renaming orders.status to orders.status_v2 safely across three deploys:
Deploy 1 — Expand: Add the new column and start writing to both.
ALTER TABLE orders ADD COLUMN status_v2 varchar(32);
CREATE INDEX CONCURRENTLY idx_orders_status_v2 ON orders (status_v2);// Migration file (Laravel)
public function up(): void
{
Schema::table('orders', function (Blueprint $table) {
$table->string('status_v2', 32)->nullable()->after('status');
});
// Application code now writes to both columns during this phase
}Deploy 2 — Backfill and migrate data: Run a batched backfill job, not a single UPDATE that locks the table.
// Artisan command or queued job
Order::whereNull('status_v2')
->chunkById(500, function ($orders) {
foreach ($orders as $order) {
$order->update(['status_v2' => $order->status]);
}
});-- Verify backfill is complete before proceeding
SELECT COUNT(*) FROM orders WHERE status_v2 IS NULL;Once backfill is complete and the application reads from status_v2 only, set NOT NULL:
-- Safe in Postgres 12+ if status_v2 has no NULLs
ALTER TABLE orders ALTER COLUMN status_v2 SET NOT NULL;Deploy 3 — Contract: Drop the old column after the old code path is gone.
ALTER TABLE orders DROP COLUMN status;
ALTER TABLE orders RENAME COLUMN status_v2 TO status;Always write a down() method. But be honest: a down() that drops a column containing 48 hours of backfilled data is not a rollback, it’s data loss. The real rollback strategy is:
DROP COLUMN status_v2 — safe, no data yet.Keep migrations small and reversible. A migration file that does five things simultaneously has five blast radii.
Lock timeouts prevent a migration from queuing up behind a slow query and holding a lock indefinitely:
# Set a 2-second lock wait timeout for this session only
psql "$DATABASE_URL" -c "SET lock_timeout = '2s';" \
&& php artisan migrate --forceIf the lock can’t be acquired within 2 seconds, the migration fails fast rather than blocking production traffic. Retry it during a low-traffic window.
For Laravel, wrap this in your deploy script:
php artisan migrate --force --pretend # dry-run, log the SQL
php artisan migrate --force # applySafe production migrations require treating schema changes as a multi-deploy process, not a one-shot command. Use CREATE INDEX CONCURRENTLY, NOT VALID constraints, and nullable column additions for the expand phase. Backfill in batches. Contract only after every consumer of the old schema is gone. Set lock_timeout in your deploy pipeline so a blocked migration fails fast. The discipline is more work upfront — it is infinitely cheaper than an emergency rollback at 2 AM.
A freshly provisioned server running Nginx and PHP-FPM with default configuration will hold up fine under light traffic and then fall apart under real load in ways that are hard to diagnose. Requests queue, workers block, timeouts cascade. The defaults are conservative placeholders — not a production configuration.
This is a systematic walkthrough of the settings that actually move the needle: PHP-FPM process manager sizing, FastCGI buffer tuning, keepalive, OPcache, and gzip. All values come with the reasoning so you can adapt them to your hardware.
The pm directive controls how PHP-FPM manages worker processes. The three modes:
static — fixed pool, predictable memory usage, no spin-up latency.dynamic — pool grows and shrinks within bounds. Wastes CPU on management under sustained load.ondemand — workers spawn on request and die when idle. Good for low-traffic sites with memory pressure. Not for high-throughput.For production APIs and web apps under sustained load, dynamic with correctly sized bounds is the right choice.
; /etc/php/8.3/fpm/pool.d/www.conf
pm = dynamic
; Formula: floor(available_RAM / avg_worker_memory)
; Check worker memory: ps --no-headers -o rss -C php-fpm8.3 | awk '{sum+=$1} END {print sum/NR/1024 " MB"}'
; On a 4 GB server with ~50 MB avg worker: floor(3500 / 50) = 70, but leave headroom
pm.max_children = 40
; Start with ~25% of max on boot
pm.start_servers = 10
; Keep enough idle to handle bursts without spawning
pm.min_spare_servers = 5
pm.max_spare_servers = 20
; Recycle workers that have served many requests (memory leak protection)
pm.max_requests = 500
; Log slow requests
slowlog = /var/log/php-fpm/www-slow.log
request_slowlog_timeout = 5sCalculate pm.max_children empirically. Measure your average worker’s RSS memory under real traffic, then divide your available RAM (total minus OS overhead and Nginx) by that number. Guessing leads to either OOM kills or under-utilization.
# Check current worker memory usage
ps --no-headers -o rss -C php-fpm8.3 | \
awk '{sum+=$1; count++} END {printf "avg: %.0f MB, total: %.0f MB\n", sum/count/1024, sum/1024}'
# Watch FPM pool status in real time
watch -n1 'curl -s http://127.0.0.1/fpm-status'Enable the status page to watch pool utilization — if you’re hitting max_children frequently, you need more workers or faster code.
# In your Nginx server block
location = /fpm-status {
access_log off;
allow 127.0.0.1;
deny all;
fastcgi_pass unix:/run/php/php8.3-fpm.sock;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
}Default FastCGI buffer sizes cause Nginx to write large PHP responses to disk instead of memory, adding latency. Set buffers to cover your typical response size in memory.
# /etc/nginx/conf.d/fastcgi.conf (include in http block or server block)
fastcgi_buffers 16 16k;
fastcgi_buffer_size 32k;
fastcgi_busy_buffers_size 64k;
# How long Nginx waits for PHP-FPM to start sending a response
fastcgi_read_timeout 60s;
# How long to wait for FPM to accept the connection
fastcgi_connect_timeout 10s;
fastcgi_send_timeout 30s;
# Cache FastCGI connections (requires upstream keepalive)
fastcgi_keep_conn on;For high-traffic APIs, add a FastCGI cache to serve identical responses without hitting PHP at all:
# In http block
fastcgi_cache_path /var/cache/nginx levels=1:2 keys_zone=php_cache:10m
max_size=1g inactive=60m use_temp_path=off;
# In server block
fastcgi_cache_key "$scheme$request_method$host$request_uri";
location ~ \.php$ {
fastcgi_cache php_cache;
fastcgi_cache_valid 200 60s;
fastcgi_cache_bypass $http_pragma;
fastcgi_no_cache $http_authorization; # Never cache authenticated requests
add_header X-Cache-Status $upstream_cache_status;
# ... rest of fastcgi_pass config
}Nginx’s default keepalive_timeout of 75 seconds keeps connections open far too long. Under load this exhausts worker connections. Reduce it and increase worker_connections to match your concurrency.
# /etc/nginx/nginx.conf
worker_processes auto; # One per CPU core
worker_rlimit_nofile 65535;
events {
worker_connections 4096; # max connections per worker process
use epoll;
multi_accept on;
}
http {
keepalive_timeout 30s;
keepalive_requests 1000;
# Upstream keepalive to PHP-FPM (requires php-fpm to use TCP, not socket, for this)
upstream php_fpm {
server 127.0.0.1:9000;
keepalive 32;
}
}OPcache eliminates bytecode compilation on every request. Its defaults — especially opcache.memory_consumption — are far too low for modern apps.
; /etc/php/8.3/fpm/conf.d/10-opcache.ini
opcache.enable = 1
opcache.enable_cli = 0
; Compiled bytecode cache — Laravel + vendor needs ~256 MB on large apps
opcache.memory_consumption = 256
; Number of cached scripts — count your PHP files and add 20%
opcache.max_accelerated_files = 20000
; Never check mtime in production — significant speed gain
; Set to 1 during deployment only (or restart php-fpm on deploy)
opcache.validate_timestamps = 0
; JIT — beneficial for CPU-bound code, neutral-to-negative for I/O-bound apps
opcache.jit = tracing
opcache.jit_buffer_size = 64mWhen validate_timestamps = 0, PHP never checks if files changed on disk. You must restart (or reload) PHP-FPM after every deploy to clear the cache. Wire this into your deploy script:
# Deploy script
git pull
composer install --no-dev --optimize-autoloader
php artisan optimize
sudo systemctl reload php8.3-fpmCompressing API responses and serving static files efficiently rounds out the configuration.
http {
gzip on;
gzip_comp_level 5; # 1-9; 5 is the sweet spot for CPU vs ratio
gzip_min_length 1024; # Don't compress tiny responses
gzip_proxied any;
gzip_types
text/plain
text/css
text/javascript
application/json
application/javascript
application/x-javascript
image/svg+xml;
# Efficient static file serving
sendfile on;
tcp_nopush on;
tcp_nodelay on;
# Cache static assets at the client
location ~* \.(css|js|png|jpg|ico|woff2)$ {
expires 1y;
add_header Cache-Control "public, immutable";
access_log off;
}
}Tuning Nginx and PHP-FPM is not a one-time exercise — it’s a feedback loop. Enable the FPM status page, watch max_children utilization, measure worker memory, and adjust. Enable OPcache with validate_timestamps = 0 and make it part of your deploy. Get FastCGI buffer sizes out of temp-file territory. These changes collectively move a default configuration from “survives light traffic” to “handles production load” without adding hardware.
Most applications don’t need Elasticsearch. Postgres full-text search handles millions of documents comfortably, supports phrase queries and ranking, and eliminates an entire distributed system from your stack. But when you need language-aware tokenization, per-field relevance tuning, faceted aggregations across tens of millions of documents, or sub-second latency at high query volume, Elasticsearch earns its complexity. The key is understanding what it’s actually doing so you build the index correctly the first time.
Before a document is stored, Elasticsearch runs each text field through an analyzer. The default standard analyzer lowercases input, strips punctuation, and splits on whitespace and Unicode boundaries. The same analyzer runs at query time, so the tokens in the index and the tokens in the query match.
# Inspect what the standard analyzer produces
curl -X POST "localhost:9200/_analyze?pretty" \
-H 'Content-Type: application/json' \
-d '{
"analyzer": "standard",
"text": "Django's ORM performance — select_related vs prefetch_related"
}'For language-aware stemming (queries for “running” matching “runs”), swap in a language analyzer:
{
"settings": {
"analysis": {
"analyzer": {
"english_text": {
"type": "custom",
"tokenizer": "standard",
"filter": ["lowercase", "english_stemmer", "english_stop"]
}
},
"filter": {
"english_stemmer": { "type": "stemmer", "language": "english" },
"english_stop": { "type": "stop", "stopwords": "_english_" }
}
}
}
}Analyzer choice is a mapping decision — changing it after indexing requires a full reindex.
The text vs keyword distinction is the most common mapping mistake. text fields are analyzed and support full-text search. keyword fields are stored as-is and support exact-match, aggregations, and sorting.
{
"mappings": {
"properties": {
"title": { "type": "text", "analyzer": "english_text" },
"slug": { "type": "keyword" },
"tags": { "type": "keyword" },
"body": { "type": "text", "analyzer": "english_text" },
"author": {
"type": "text",
"fields": {
"keyword": { "type": "keyword" }
}
},
"published_at": { "type": "date" }
}
}
}The author.keyword sub-field pattern is common: author handles full-text search, author.keyword handles aggregations and exact filters. Dynamic mapping will guess types, but it guesses wrong often enough that explicit mappings are worth defining upfront.
match runs the input through the same analyzer as the field, making it the right choice for user-facing search on text fields. term skips analysis entirely — use it for keyword fields, IDs, and status codes:
{
"query": {
"bool": {
"must": [
{
"match": {
"body": {
"query": "database indexing performance",
"operator": "and"
}
}
}
],
"filter": [
{ "term": { "tags": "backend" } },
{ "range": { "published_at": { "gte": "2025-01-01" } } }
]
}
}
}filter clauses inside bool don’t affect relevance scores and are cached by Elasticsearch — always use filter for date ranges, status fields, and exact tags, reserving must/should for the scored full-text portion.
bool is the workhorse query. Its four clauses compose cleanly:
must — must match, contributes to scoreshould — optional, boosts score when it matchesfilter — must match, does not affect score (cached)must_not — must not match, does not affect scoreUse boost to weight title matches higher than body matches:
{
"query": {
"bool": {
"should": [
{ "match": { "title": { "query": "elasticsearch", "boost": 3.0 } } },
{ "match": { "body": { "query": "elasticsearch", "boost": 1.0 } } }
],
"filter": [
{ "term": { "status": "published" } }
],
"minimum_should_match": 1
}
},
"highlight": {
"fields": {
"title": {},
"body": { "fragment_size": 150, "number_of_fragments": 2 }
}
}
}highlight returns matched fragments with <em> markers — useful for search result snippets without a second application-layer scan.
Elasticsearch introduces real operational complexity: cluster management, index lifecycle policies, mapping migrations, cross-node shard allocation, and snapshot backups. Before committing to it, benchmark Postgres:
-- Create a GIN index on a tsvector column
ALTER TABLE posts ADD COLUMN search_vector tsvector;
UPDATE posts
SET search_vector =
setweight(to_tsvector('english', coalesce(title, '')), 'A') ||
setweight(to_tsvector('english', coalesce(body, '')), 'B');
CREATE INDEX idx_posts_search ON posts USING GIN(search_vector);
-- Query with ranking
SELECT id, title, ts_rank(search_vector, query) AS rank
FROM posts, plainto_tsquery('english', 'django performance') AS query
WHERE search_vector @@ query
ORDER BY rank DESC
LIMIT 20;Postgres FTS with a GIN index handles sub-50ms queries on single-table datasets up to roughly 5–10 million rows on commodity hardware, supports phrase search and ranking via ts_rank, and costs nothing extra to operate. If your search needs stay within one table, one language, and don’t require facets or complex multi-field scoring, Postgres is the correct answer.
Move to Elasticsearch when you need: multi-language analyzers per document, aggregation-backed faceted navigation (categories × price × rating simultaneously), sub-100ms latency at thousands of queries per second, or near-real-time index updates across federated data sources.
Elasticsearch is powerful because it exposes the full text retrieval pipeline — analyzers, mappings, scored queries — as explicit configuration. That exposure is also the source of its learning curve and operational burden. Get your mappings right before ingesting data, use filter clauses for non-scored conditions, boost fields proportionally to their signal quality, and seriously benchmark Postgres FTS before deciding the complexity of a distributed search cluster is justified.
Most Laravel projects start by generating Eloquent models and controllers, then cramming business logic wherever it fits — often in fat models or even fatter controllers. This works until requirements change, the team grows, or someone tries to write a unit test without booting the entire framework. The coupling to Laravel isn’t inherently wrong; what’s wrong is letting it reach all the way into your domain logic.
Clean Architecture (and its relatives — Hexagonal, Ports & Adapters, Onion) share one central idea: the further inward you go, the less the code knows about infrastructure. Your Order entity shouldn’t care whether it’s persisted to Postgres or a test double. Your PlaceOrderAction shouldn’t know it was invoked via HTTP. When the framework sits at the edge, not at the center, the codebase stays navigable.
Structure the codebase into three tiers with a strict dependency rule: inner layers never import from outer layers.
src/
├── Domain/ # entities, value objects, domain events, repository interfaces
├── Application/ # use cases / actions, DTOs, application services
└── Infrastructure/ # Eloquent models, repository implementations, HTTP controllers, jobsDomain is pure PHP — no framework, no database driver, no HTTP. It holds your entities, value objects, and the interfaces for everything it needs from outside (repositories, event dispatchers).
Application orchestrates domain objects to fulfill a single use case. It also knows only about domain interfaces, not implementations.
Infrastructure implements those interfaces using concrete tools — Eloquent, Redis, Laravel’s event dispatcher, etc. It also contains controllers, console commands, and jobs, which translate external input into application calls.
An entity has identity; it changes over time. A value object is defined by its data; it’s immutable. Keep both in the domain layer, framework-free:
// Domain/ValueObject/Money.php
final class Money
{
public function __construct(
public readonly int $amount, // in cents
public readonly string $currency,
) {}
public function add(Money $other): self
{
if ($this->currency !== $other->currency) {
throw new \DomainException('Currency mismatch');
}
return new self($this->amount + $other->amount, $this->currency);
}
}
// Domain/Entity/Order.php
class Order
{
private OrderStatus $status;
private array $lines = [];
public function __construct(
public readonly OrderId $id,
public readonly CustomerId $customerId,
) {
$this->status = OrderStatus::Draft;
}
public function addLine(ProductId $productId, int $qty, Money $unitPrice): void
{
if ($this->status !== OrderStatus::Draft) {
throw new \DomainException('Cannot modify a placed order.');
}
$this->lines[] = new OrderLine($productId, $qty, $unitPrice);
}
public function place(): void
{
if (empty($this->lines)) {
throw new \DomainException('Cannot place an empty order.');
}
$this->status = OrderStatus::Placed;
}
}Order enforces its own invariants. No controller or Eloquent observer can put it into an impossible state.
The domain defines what it needs; infrastructure provides how. The interface lives in the domain layer:
// Domain/Repository/OrderRepository.php
interface OrderRepository
{
public function findById(OrderId $id): Order;
public function save(Order $order): void;
}The Eloquent implementation lives in infrastructure:
// Infrastructure/Persistence/EloquentOrderRepository.php
class EloquentOrderRepository implements OrderRepository
{
public function findById(OrderId $id): Order
{
$record = OrderRecord::findOrFail($id->value());
return $this->toDomain($record);
}
public function save(Order $order): void
{
OrderRecord::updateOrCreate(
['id' => $order->id->value()],
$this->toRecord($order),
);
}
private function toDomain(OrderRecord $record): Order { /* ... */ }
private function toRecord(Order $order): array { /* ... */ }
}Bind the interface to the implementation in a service provider — this is one of the few places Laravel’s container explicitly belongs:
$this->app->bind(OrderRepository::class, EloquentOrderRepository::class);Now your application code depends on the interface. Swap to a Doctrine implementation, a Redis cache layer, or an in-memory fake for tests — without touching a single use case.
Single-responsibility action classes replace fat service classes and controller logic. One action, one public method, one responsibility:
// Application/Action/PlaceOrderAction.php
class PlaceOrderAction
{
public function __construct(
private readonly OrderRepository $orders,
private readonly EventDispatcherPort $events,
) {}
public function execute(PlaceOrderDTO $dto): Order
{
$order = $this->orders->findById(new OrderId($dto->orderId));
$order->place();
$this->orders->save($order);
$this->events->dispatch(new OrderPlaced($order->id));
return $order;
}
}PlaceOrderAction knows nothing about HTTP or Eloquent. It can be invoked from a controller, a CLI command, or a test — identically. The controller’s only job is to translate the HTTP request into a PlaceOrderDTO and pass it in:
// Infrastructure/Http/OrderController.php
public function store(PlaceOrderRequest $request, PlaceOrderAction $action): JsonResponse
{
$dto = PlaceOrderDTO::fromRequest($request);
$order = $action->execute($dto);
return response()->json(OrderResource::make($order), 201);
}DTOs (Data Transfer Objects) are the input/output contract between layers. They’re plain, readonly objects — no behavior, no framework traits:
final class PlaceOrderDTO
{
public function __construct(
public readonly string $orderId,
public readonly string $userId,
) {}
public static function fromRequest(PlaceOrderRequest $request): self
{
return new self(
orderId: $request->validated('order_id'),
userId: $request->user()->id,
);
}
}The factory method fromRequest is the only infrastructure-aware thing, and it sits in the DTO itself, not in the action. The action receives a plain PlaceOrderDTO and never touches a Request object.
This architecture doesn’t mean ejecting Laravel. It means containing it. Controllers, form requests, Eloquent models, jobs, listeners — these all belong in infrastructure. The container wires everything together. Facades and helpers are fine in infrastructure; they’re inappropriate in domain or application code.
When you want to test PlaceOrderAction, you new it up with a fake repository and a fake dispatcher — no database, no HTTP, no framework boot. That’s the payoff: fast, focused tests that prove your business rules work, independent of any external dependency.
Clean architecture in PHP is not about following Uncle Bob’s diagrams to the letter — it’s about maintaining a clear dependency direction. Domain inward, framework outward. Repository interfaces in the domain, Eloquent in infrastructure, actions in application. When every layer knows only what it must, adding a new feature means writing new code, not untangling existing code. That’s the only metric that matters at scale.
GitHub Actions is powerful enough to replace dedicated CI servers for most projects, but the default examples in documentation stop at “run your tests.” A real pipeline caches dependencies so it doesn’t reinstall Composer on every push, runs tests in a matrix against multiple PHP versions, builds and pushes a tagged Docker image, and deploys — but only on merges to main, not on every feature branch push.
This walks through a complete, production-grade workflow file for a Laravel application. Adapt the service containers and deploy step to your stack.
A workflow file lives in .github/workflows/. Jobs run in parallel by default; use needs to sequence them. Each job gets a fresh VM.
# .github/workflows/ci.yml
name: CI/CD
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
env:
PHP_VERSION: "8.3"
IMAGE_NAME: ghcr.io/${{ github.repository }}The on block defines triggers. Pull requests to main run tests; pushes to main additionally build and deploy. This keeps the build loop fast for feature branches.
Reinstalling Composer packages from scratch on every run costs 30–90 seconds depending on lock file size. Cache the vendor directory keyed on composer.lock:
jobs:
test:
runs-on: ubuntu-24.04
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_USER: test
POSTGRES_PASSWORD: test
POSTGRES_DB: testdb
ports: ["5432:5432"]
options: >-
--health-cmd pg_isready
--health-interval 5s
--health-retries 5
steps:
- uses: actions/checkout@v4
- uses: shivammathur/setup-php@v2
with:
php-version: ${{ env.PHP_VERSION }}
extensions: pdo_pgsql, redis, pcov
coverage: pcov
- name: Cache Composer dependencies
uses: actions/cache@v4
with:
path: vendor
key: composer-${{ hashFiles('composer.lock') }}
restore-keys: composer-
- name: Install dependencies
run: composer install --no-interaction --prefer-dist --optimize-autoloader
- name: Copy environment file
run: cp .env.ci .env
- name: Run migrations
run: php artisan migrate --force
env:
DB_HOST: 127.0.0.1
DB_DATABASE: testdb
DB_USERNAME: test
DB_PASSWORD: test
- name: Run tests with coverage
run: php artisan test --parallel --coverage --min=80The services block spins up a real Postgres container the job can connect to — no SQLite, no mocking the database layer.
Testing across multiple PHP versions catches compatibility issues before they reach production. Matrix runs all combinations in parallel:
test-matrix:
runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
php: ["8.2", "8.3"]
dependency-version: [prefer-lowest, prefer-stable]
steps:
- uses: actions/checkout@v4
- uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php }}
- name: Install ${{ matrix.dependency-version }} dependencies
run: composer update --${{ matrix.dependency-version }} --no-interaction
- name: Run tests
run: php artisan testfail-fast: false lets all matrix combinations complete even if one fails, so you see the full picture rather than stopping at the first failure.
Run linters and static analysis as a separate parallel job — they don’t need a database.
lint:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- uses: shivammathur/setup-php@v2
with:
php-version: ${{ env.PHP_VERSION }}
- name: Cache Composer dependencies
uses: actions/cache@v4
with:
path: vendor
key: composer-${{ hashFiles('composer.lock') }}
- run: composer install --no-interaction --prefer-dist
- name: PHP_CodeSniffer
run: vendor/bin/phpcs --standard=PSR12 app/
- name: PHPStan
run: vendor/bin/phpstan analyse --level=8 app/Only build and push when tests pass and the branch is main. Use GHCR (GitHub Container Registry) — it’s free for public repos and integrated with your repository’s permissions.
build:
runs-on: ubuntu-24.04
needs: [test, lint]
if: github.ref == 'refs/heads/main'
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v4
- name: Log in to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata for Docker
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.IMAGE_NAME }}
tags: |
type=sha,prefix=,format=short
type=raw,value=latest
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=maxcache-from: type=gha uses GitHub’s layer cache to speed up Docker builds significantly — layer changes that haven’t changed since the last run are reused.
The deploy step runs after a successful image push, again gated to main. Use SSH and secrets for credentials — never hardcode them.
deploy:
runs-on: ubuntu-24.04
needs: build
if: github.ref == 'refs/heads/main'
environment: production
steps:
- name: Deploy to server
uses: appleboy/ssh-action@v1
with:
host: ${{ secrets.DEPLOY_HOST }}
username: ${{ secrets.DEPLOY_USER }}
key: ${{ secrets.DEPLOY_SSH_KEY }}
script: |
docker pull ghcr.io/${{ github.repository }}:latest
docker compose -f /opt/myapp/docker-compose.prod.yml up -d --no-deps app
docker system prune -f --filter "until=24h"Store DEPLOY_HOST, DEPLOY_USER, and DEPLOY_SSH_KEY under Settings > Secrets > Actions. Use the environment: production key to require manual approval for production deploys if your plan supports it.
A complete Actions pipeline is roughly 120 lines of YAML. Cache your dependencies, run tests against real service containers, use matrix builds to catch edge cases early, build Docker images only on green main, and deploy with SSH using repository secrets. The sequential needs chain — test → build → deploy — ensures nothing reaches production until every gate passes.
Rate limiting is one of those features teams bolt on after the first abuse incident. By then the API has no consistent key strategy, callers have no idea what limits exist, and the implementation is a patchwork of per-endpoint middleware. Building it correctly from the start costs almost nothing — Redis makes the hard part (atomic counters) trivial, and the HTTP spec gives you the response shape for free.
The algorithm choice matters more than the implementation details, so start there.
Fixed window counts requests per discrete time bucket: POST /api/items — user:42 — 2025-07-25T14:00. At 14:01, the counter resets. Simple and fast, but it allows bursting at window boundaries: a user can fire 100 requests in the last second of minute N and 100 in the first second of minute N+1, for 200 in two seconds against a nominal limit of 100/minute.
Sliding window eliminates the boundary burst by counting requests in the trailing window relative to the current timestamp. The exact implementation uses a sorted set in Redis (score = request timestamp), pruning entries older than the window and counting the remainder. More accurate but costs more Redis operations per request.
Token bucket is the most natural model for bursty-but-bounded traffic. A bucket starts full at capacity C. Each request consumes one token; tokens refill at rate R per second. Users can burst up to C requests instantly, then are capped at R/sec. Most commercial APIs (GitHub, Stripe, Twilio) use token bucket semantics.
For most backend APIs, fixed window with a short window (1 minute) and a generous enough limit is the right starting point. Its boundary burst is rarely exploitable and it’s the easiest to reason about under load.
The atomic INCR + EXPIRE pattern is the canonical fixed window implementation:
import redis
import time
from dataclasses import dataclass
r = redis.Redis(host="localhost", port=6379, decode_responses=True)
WINDOW_SECONDS = 60
MAX_REQUESTS = 100
@dataclass
class RateLimitResult:
allowed: bool
count: int
limit: int
reset_at: int # Unix timestamp when window resets
def check_rate_limit(key: str) -> RateLimitResult:
pipe = r.pipeline()
pipe.incr(key)
pipe.ttl(key)
count, ttl = pipe.execute()
if count == 1:
# First request in this window — set expiry
r.expire(key, WINDOW_SECONDS)
ttl = WINDOW_SECONDS
reset_at = int(time.time()) + (ttl if ttl > 0 else WINDOW_SECONDS)
allowed = count <= MAX_REQUESTS
return RateLimitResult(
allowed=allowed,
count=count,
limit=MAX_REQUESTS,
reset_at=reset_at,
)The pipeline (pipe.execute()) sends INCR and TTL in a single round-trip. The first INCR returns 1 (key didn’t exist), triggering EXPIRE. This approach is not perfectly atomic — there’s a race between INCR and EXPIRE — which matters only if the process dies between those two calls, leaving a key without an expiry. A Lua script eliminates the race entirely:
# Atomic INCR+EXPIRE via Lua — no race condition
redis-cli EVAL "
local count = redis.call('INCR', KEYS[1])
if count == 1 then
redis.call('EXPIRE', KEYS[1], ARGV[1])
end
return count
" 1 "ratelimit:user:42:$(date +%Y%m%d%H%M)" 60For the sliding window, use a sorted set:
def check_sliding_window(key: str, user_id: str) -> RateLimitResult:
now = time.time()
window_start = now - WINDOW_SECONDS
request_id = f"{now}:{user_id}"
pipe = r.pipeline()
pipe.zremrangebyscore(key, "-inf", window_start) # prune old entries
pipe.zadd(key, {request_id: now}) # add current request
pipe.zcard(key) # count in window
pipe.expire(key, WINDOW_SECONDS + 1)
_, _, count, _ = pipe.execute()
return RateLimitResult(
allowed=count <= MAX_REQUESTS,
count=count,
limit=MAX_REQUESTS,
reset_at=int(now + WINDOW_SECONDS),
)The key structure determines what you’re actually limiting. Common patterns:
# Per authenticated user — right for authenticated APIs
key = f"ratelimit:user:{user_id}:{window_bucket}"
# Per IP — right for unauthenticated endpoints (login, signup)
key = f"ratelimit:ip:{client_ip}:{window_bucket}"
# Per user + endpoint — right for expensive or sensitive endpoints
key = f"ratelimit:user:{user_id}:endpoint:{route_name}:{window_bucket}"
# Per API key (for developer APIs)
key = f"ratelimit:apikey:{api_key_hash}:{window_bucket}"The window bucket for a fixed 1-minute window is simply int(time.time() // 60). For a sliding window, omit the bucket — the sorted set handles windowing internally.
Per-IP limiting for authenticated endpoints is almost never right. Shared NAT, corporate proxies, and IPv6 /64 allocations mean one IP can legitimately represent hundreds of users. Restrict per-IP to pre-authentication flows.
RFC 6585 defines 429 Too Many Requests. The response must include Retry-After (seconds until the client can retry) and should include X-RateLimit-* headers that proactive clients can use to self-throttle before hitting the limit:
from flask import Flask, request, jsonify, g
from functools import wraps
app = Flask(__name__)
def rate_limited(f):
@wraps(f)
def decorated(*args, **kwargs):
user_id = getattr(g, 'user_id', None) or request.remote_addr
key = f"ratelimit:user:{user_id}:{int(time.time() // 60)}"
result = check_rate_limit(key)
headers = {
"X-RateLimit-Limit": str(result.limit),
"X-RateLimit-Remaining": str(max(0, result.limit - result.count)),
"X-RateLimit-Reset": str(result.reset_at),
}
if not result.allowed:
retry_after = result.reset_at - int(time.time())
return jsonify({"error": "rate limit exceeded"}), 429, {
**headers,
"Retry-After": str(retry_after),
}
response = f(*args, **kwargs)
# Attach headers to successful responses too
if hasattr(response, 'headers'):
for k, v in headers.items():
response.headers[k] = v
return response
return decoratedAlways return Retry-After on a 429. Clients that respect it back off cleanly; clients that don’t will hammered and stay blocked. X-RateLimit-Remaining on every response lets well-behaved clients pre-emptively pause before exhausting their quota.
Pick fixed window unless the boundary burst is a real threat, implement counters with Redis INCR+EXPIRE or Lua for atomicity, key by authenticated user identity rather than IP for authenticated routes, and always return Retry-After with your 429s. The algorithm and response contract matter far more than which middleware library wraps them.
// whoami && cat skills.json