AI Systems (34)

LLM as a Judge: How It Works and How to Know It Works An LLM judge scores another model's output against a written criterion. What a judge is, why teams use one, how to build one, and how to check its verdicts against human labels before trusting the numbers.

Evaluating Agents How to evaluate an AI agent on your own tasks: tool-call correctness, planning efficiency and outcome, measured separately and in that order, then repeated to check it succeeds the same way twice.

When a Graph Earns Its Place in RAG GraphRAG builds an entity graph over your documents and retrieves subgraphs in place of passages. It helps with multi-hop and whole-corpus questions, at two to three times the query latency and a large one-off indexing cost.

Instrumentation Is the Deliverable The records written to debug a generation pipeline describe the process the pipeline replaced. That description is the artefact most organisations lack, and it can be produced before any model is bought.

Data Methods for RAG Chunking is one of many ways to prepare documents for retrieval. A working list of the methods, from fixed-size splitting through knowledge graphs to retrieving directly from document images.

RAG Risks and Costs A RAG pipeline on a hosted API sends your data to a third party twice per query, at the embedding step and again at generation, and its bill grows with the size of the corpus. A third risk, retrieval that ignores document permissions, is rarely named.

What Problem Does RAG Actually Solve? RAG is one answer to a specific problem: a model that does not know your data. Fine-tuning and plain search are the other two. A framework for choosing between them before any vector database gets picked.

Building Agent Tasks Most of what looks like a single design decision, how to build an agent, is actually four separable ones: tool schema, planning strategy, orchestration pattern, and memory. A breakdown of each.

Agent Risks, Costs, and Governance An agent acts on what it reads, so it can send, delete or pay for the wrong thing, and its cost grows with the number of steps it takes. Cost ceilings, least-privilege tools and memory injection (MINJA), in turn.

Agent Frameworks and Harnesses Agent tools divide by who builds the loop. Frameworks leave it to you, harnesses ship it finished for one domain, and agent SDKs expose a harness as a library. LangGraph, CrewAI, the OpenAI Agents SDK, Google ADK, Microsoft Agent Framework and the Claude Agent SDK compared on that, on model choice and on crash recovery.

What Are Agents An agent differs from a pipeline in one respect: it decides the next step at runtime instead of following one fixed in advance. That decision is expensive, so the question worth asking first is when it earns its cost.

Evaluating RAG How to evaluate a RAG system with four measurements taken in pipeline order. A fault at any stage produces the same symptom (a fluent, wrong answer), so each stage is measured on its own.

How do Neural Networks Learn? Backpropagation Explained F learns the mapping from X to Y by adjusting its weights against the error between what it predicted and what the data says was correct. Backpropagation is the mechanism that turns that error into a weight update, one layer at a time.

What Is Machine Learning? F(X) = Y, Explained Machine learning reduces to one expression: F(X) = Y. What that means, and how the four types of machine learning differ in which part of the equation is known before training starts.

Open Datasets for Training Vision-Language Models The datasets behind open vision-language models, from 108,000 hand-annotated images to 12.8 billion web-scraped pairs. Observation counts, disk sizes, and what's actually downloadable versus URL-only.

Model Choice Determines VLM Inference Cost Vision-language models are biased toward a characteristic output length regardless of image content. Because cost tracks output length, model choice determines inference cost. Benchmark data from four open-weight VLMs across 4,004 requests, with per-request power and GPU telemetry.

Why Recommendation Algorithms Resurface Old Content Recommendation systems resurface older content on purpose, to test it against new groups of users. How that exploration works, and what it means for anyone distributing content.

PII Detection with Named Entity Recognition How named entity recognition detects and redacts PII in text, how to measure detection accuracy per entity type on your own documents, and the tools to use: Presidio, spaCy, BERT NER and OpenAI's open-weight Privacy Filter.

Memory in Agentic AI The types of memory available to an AI agent, how they work, when each is appropriate, and what breaks when the wrong type is used or the memory layer is absent. Includes a survey of named systems: A-MEM, Karpathy's LLM Wiki, Letta, Zep, Mem0, and Cognee.

Context is a Build Step Every method of building context for an agent, from basic chunking to knowledge compilation, is a build step run before the query. The quality of that step sets what the agent can do at runtime.

How to Evaluate an AI Pipeline How to score classification, RAG and generation pipelines, with public evaluation datasets listed by task type and domain.

Embedding-Based Relationship Discovery: Finding the Real Network in Your Data Most organisations hold a formal model of their customer relationships and a real one that differs from it. How embeddings and community detection find the real structure, and where the pattern applies.

Format Choice and LLM Performance: A Taxonomy The format used to pass data to a language model changes both its reliability and its token cost. A taxonomy of the main formats and the conditions under which each performs well.

Task Specification is the Primary Artefact Foundation models lowered the cost of building an AI system. The first question is still where the labels come from, because a labelled dataset specifies the task and measures the result.

Static vs Dynamic Planning in AI Pipelines Most production AI workloads are high-volume and fixed-task. Dynamic planning adds cost and reduces auditability without adding anything a static pipeline cannot do.

Agent Alloys or Persona Leakage? Context leaks between agents in multi-agent systems. The XBOW experiment suggests that for exploratory tasks the leak improves results.

Why Model Architecture Matters Every neural network architecture builds in assumptions about the structure of its problem, and those assumptions limit what the model can learn from its data.

Filtered Vector Search Vector search in production needs metadata filters, business rules, and combined ranking signals. How index structure determines what actually executes efficiently.

LLM Training Fundamentals: From Tokens to Human Preference LLMs are trained in stages with different objectives. How the shift from statistical training to human preference ranking produces behaviours that metrics alone cannot explain.

Sampling Strategies for Imbalanced Data Imbalanced datasets lead models to ignore the minority class. The sampling strategies for imbalanced data, and how they apply in fraud detection, medical diagnosis and quality control.

LLM Evaluation Metrics LLM outputs vary between runs and depend on context, so one score cannot describe them. The metrics for evaluating a language model in production, and what each measures.

A Complete Guide to Categorical Encoding for Machine Learning One-hot, ordinal, target and embedding encodings for categorical features, how each affects model performance, and which to use with which model.

Why Are Vector Databases Difficult? A Deep Dive Vector databases are harder to run well than they appear. How similarity search and indexing trade speed against accuracy, and why simple implementations degrade at scale.

The Components of RAG Retrieval-augmented generation (RAG) is four independent decisions: how documents are chunked, indexed and retrieved, and how the answer is generated. Each decision, with the tools that serve it.

Product & Strategy (20)

Engineering in the Exponential Age: MCP as a Case Study MCP moved from a genuine differentiator to an assumed interface faster than REST did. A worked example of what that shift actually requires an engineering team to build, and what it returns.

How to Ensure Action Conversation between an entrepreneur and a technical person is free and valuable. The moment something gets built, it needs a company and a contract behind it, or the relationship drifts somewhere nobody chose.

How to Structure an ML Project ML projects open with a sequential setup phase before iteration can begin. What that phase involves, why it does not compress, and how the four project phases fit together.

Engineering in the Exponential Age Capability is arriving faster than teams can absorb it, so engineering becomes the assembly of protocols and interfaces that already exist.

When Training is a Commodity Inference is already a commodity. Training is next. When it arrives, every organisation that has spent the intervening period curating its own data will have a model. The AI provider will be as unremarkable as the web host.

A Measurement Framework for Machine Learning Projects A practical framework for defining, building, and maintaining the measurement system that determines whether an ML project is working. Includes scoring formulae, risk weighting, and a readiness checklist.

Don't Build Models, Build Measurements Most AI projects fail before a model is built. A project is viable when the organisation can define what a correct output looks like, and that definition comes before the choice of model.

What Software Engineers Already Know About Machine Learning Eval datasets, scoring pipelines, and deployment gates for ML systems are not new concepts. Software engineers have been doing this for years under different names.

Open Weight Models and the Separation of Concerns in AI Vertically integrated AI companies collect data, train models, and sell inference. This is a business process error. The organisation that understands the task should own the data that defines it.

The Austrian Case for AI Investment The Chicago school asks what AI allows you to stop paying for. The Austrian school asks what latent structure already exists in your accumulated data, waiting to be read.

Friction Builders and Friction Removers Technical people locate power in the ability to build things. In most organisations, the ability to deliver and the ability to block have considerably more impact.

Capital, Labour, and the Productivity Question Productivity growth has slowed across OECD economies. One explanation: the statistics measure the wrong competition. Capital and labour are not on the same curve.

The Indirection Problem Indirection (reaching something through a reference) is a C programming idea. It also explains a number of everyday frustrations with modern systems.

Metrics for Business Goals A model with 95% accuracy can destroy business value. How to choose evaluation metrics that reflect what the business actually needs.

Machine Learning Engineer vs Software Engineer: Four Structural Differences Machine learning engineer vs software engineer: four differences that change how ML work is scoped, managed and judged, from statistical success criteria to what happens after deployment.

5 Mistakes to Avoid When Building Your First AI Product The traps that derail early AI products are predictable. Five failure modes from years of helping startups build and ship.

How to Brief a Developer on a Feature Without Being Technical A practical guide for business and product owners who need to communicate clearly with AI engineers without speaking code.

Alignment: Semantics in the AI/ML Era Language models added a new problem to software: the model's reading of an instruction can differ from what the developer meant. What alignment means in practice for teams building with LLMs.

Machine Learning Design Patterns Choosing the machine learning design pattern decides more of a project's outcome than choosing the model. The main patterns, with examples from marketing and audience intelligence.

The Cost of Machine Learning: A Practical Breakdown How much a machine learning project costs, broken down by data, model and labels, with the running costs of hosted APIs, rented GPUs and owned hardware, and what monitoring, lineage and retraining add.

Data Strategy (6)

What Is Data Engineering? Two Jobs, One Title Data engineering describes two distinct disciplines that require different skills and produce different failures when confused. Understanding the split is the first step to hiring and structuring a data team that actually works.

What Your Data Already Knows Siloed data is a list. Connected data encodes structure -- communities, gaps, relationships -- that accumulated through ordinary operation and has never been made visible.

SQL Schema Documentation for RAG Pipelines Structured databases contain context that LLMs need to generate accurate queries. How to expose schema information in a way that preserves relationships and business logic.

How to Isolate Bad Training Data Using Cross-Validation Cross-validation is usually framed as a model evaluation technique. It is also the most reliable way to find which training examples are hurting your model.

What Is Data Maturity? Data maturity is a spectrum from ad hoc storage to strategic data infrastructure. What it is, how to assess it, and why it determines what AI is actually feasible.

What Data Do I Need? A Guide for Business and Product Owners A checklist for integrating machine learning into a product without overcommitting data infrastructure before you know what the problem is.

Architecture & Deployment (13)

Serverless on AWS vs Azure: The Real Differences Serverless migrations between AWS and Azure look symmetrical on paper. Lambda's one-function-one-deployment model and Azure's Function App grouping are not the same shape, and that gap costs more time than any other part of the move.

What AWS Bedrock Actually Costs You Bedrock's one-API-many-models pitch does not remove the ordinary difficulty of LLM integration or AWS's own configuration surface. It adds one on top of the other, with no library standing between a team and either.

AI Inference Infrastructure: A Decision Framework The best AI inference infrastructure depends on four decisions: who hosts it, which GPU (VRAM is the constraint), how it scales, and which serving software handles concurrent users. How to make each one for production.

Open Weights Mean the Tier Is a Deployment Decision, Not a Lock-In With open-weight models, a laptop, a private server and a cloud deployment run the same application at different scales, so the hosting tier can change later without rewriting code.

The Private Inference Stack: A Field Guide How to self-host an LLM, layer by layer: llama.cpp and Ollama for development, vLLM for concurrent production traffic, and when a managed private endpoint (Hugging Face, Baseten, SageMaker) is the better choice.

Multi-Tenant Architecture: A Practical Spectrum Multi-tenant architecture runs from shared tables to fully separate infrastructure per tenant. Where to sit on that range depends on what varies between tenants, how much operational complexity you can carry, and what a failure looks like.

Bronze, Silver, Gold: A Layered Data Architecture in PostgreSQL The medallion architecture (bronze for raw data, silver for validated records, gold for aggregates) built in PostgreSQL, with one concern per layer and the engineering in the transitions between them.

Event-Driven Embeddings in PostgreSQL: LISTEN/NOTIFY and the Async Pattern PostgreSQL's LISTEN/NOTIFY mechanism lets you trigger embedding generation the moment a row is inserted, without polling, without a separate scheduler, and without coupling your embedding service to your database schema.

What Is Private AI Inference? Data egress stops many AI projects in regulated sectors. Open-weight models deployed inside your own VPC keep prompts and documents within your network boundary, which answers the first question a data protection review asks.

Scaling Databases: Lessons from Facebook's MySQL Journey Facebook's progression from a single database to a sharded architecture contains practical lessons for any system that needs to grow. The constraints that drove each decision.

Platforms vs Pipelines: Engineering for Compounding Returns AWS, Borg and chaos engineering are examples of organisations building platforms where others tune individual pipelines. The choice decides whether engineering effort compounds.

AI Tooling and the Separation Between Coding and Programming AI-assisted development accelerates a long-running separation between writing code and building systems. Why that distinction matters and what it means for teams.

SaaS Architecture 101 for Non-CTOs LAMP, JAMstack, microservices and serverless explained for founders without a technical co-founder, and how to make architecture decisions with them.

Infrastructure (8)

Unified Memory Machines for Local AI: DGX Spark, Strix Halo, Mac Studio and Jetson Thor (Late 2026) Unified memory machines hold 64GB to 512GB of model weights in one box, and memory bandwidth decides how fast they generate. DGX Spark, AMD Strix Halo and Gorgon Halo, Mac Studio M5, Jetson Thor and RTX Spark compared on capacity, bandwidth, price and what it takes to run containers on each.

Private Inference: Costs What self-hosted LLM inference costs, measured: 14 open-weight models, 1,204 requests and 226 watt-hours on one RTX 3060, or 5.6p of electricity. With two faults the telemetry caught that an estimate would have missed.

Falling Token Prices and the Cost of Inference You Already Own Token prices are falling across the industry while usage grows faster than prices fall. What that means for organisations that already own inference hardware, illustrated by a 14-model benchmark on one GPU.

GPU Options for Local AI Inference: VRAM, Software Stacks, and What Actually Works (Mid-2026) Alternatives to standard CUDA-based inference stacks: AMD ROCm, Apple Silicon (Mac Studio, M3 Ultra), Chinese GPU vendors, and which frameworks actually run on non-NVIDIA, non-x86 hardware.

Why Private Inference Is Not Fully Private (Yet) Self-hosted AI keeps data inside your network, and the model still computes on it in plain text. Fully homomorphic encryption would close that gap, which is why private inference today is an architectural measure without a cryptographic guarantee.

Why Private Inference The specific regulatory (NHS, FCA, GDPR) and exposure risks that make private inference necessary for regulated data, distinct from the general architectural case for self-hosting AI models.

Deploying Models to AWS Lambda Open-source models on serverless infrastructure cost less than incumbent AI services. A practical guide to deploying on AWS Lambda.

What is Docker? A Deep Dive into Containers How Docker works: the Linux primitives behind containers (chroot, cgroups and mount points), what is inside an image file and its layers, and what changes when a container is deployed to the cloud.

Developer Tools (9)

Durable Execution vs Workflow Orchestration: Two Questions That Separate the Tools What is the difference between Temporal, DBOS, Airflow, Dagster, Prefect, Step Functions, LangGraph and runfox? Two questions sort them: when the shape of the work is fixed, and where progress is recorded. The answers decide recovery, repair and fit.

What Is DBOS? Durable Python Workflows Stored in Postgres What a durable workflow is, and how DBOS implements one by storing program state as ordinary rows in Postgres. How to use DBOS in Python, how an AI agent with a human approval step fits the model, and what it costs.

Marigold: A Self-Hosted Inference API for Open-Weight Models An OpenAI-compatible API for open-weight models, running entirely on your own hardware. Open source, installed with pip, airgapped once your models are cached.

Reheat: Intent Clustering for Google Search Console Reheat pulls your Search Console queries, embeds them as vectors, clusters by semantic intent, and surfaces ranked content gaps and opportunities as a report.

How to Build an AI Pipeline That Runs Anywhere: the runfox Backend Model Most workflow libraries couple the pipeline definition to the execution environment. runfox separates them: the same YAML definition runs in-process, against SQLite, or distributed across SQS and DynamoDB, with no changes to the workflow code.

Why JSON Logic Needs JSONPath: Adding the vars Operator Standard JSON Logic resolves single values via dot notation. When rules need to match against lists -- model outputs, tag arrays, multi-value fields -- dot notation is the wrong tool. json-logic-path adds a vars operator backed by JSONPath.

Defining Your AWS API Gateway in Python API Gateway configuration and Python route definitions live in separate files and drift apart. fastapi-aws generates both the AWS integration spec and the public API documentation from a single Python source.

DynamoDB Access Patterns: The Part the Documentation Skips DynamoDB rewards developers who define access patterns before writing schema. boto3 enforces none of that discipline. dynawrap moves the pattern onto the class and lets you swap the backend for PostgreSQL when AWS is not available.

Developing with APIs Without Rate Limits: Introducing APICache Rate limits slow down early-stage development. APICache is an open-source library that caches API responses locally so you can iterate without hitting limits.

Security & Resilience (2)

Are You a Data Controller or a Processor? Two regimes govern analytics data and they test different things. PECR asks whether you stored anything on the device. GDPR asks whether you can identify anyone. A 2025 CJEU judgment changed the answer to the second, and the first is still where most systems fail.

Security Risks in AI Systems with Public Access The security risks specific to AI systems open to the public, and the controls for each.

Perspectives (12)

Generation Speed and the Collapse of Review Delegation Generative video removes the production stage that gave one role authority over taste. Review then belongs to nobody in particular, and higher volume widens the gap.

Content Negotiation Was a Good Idea HTTP content negotiation was designed to decouple content from form. It stopped at format selection. Generative inference completes the original intention.

The Embedding is the Message AI communication inherited natural language from human communication. That constraint is not a technical necessity.

Adtech Ran This Experiment First The measure-optimise-adjust loop now running through AI first ran through advertising technology. The pathologies are the same. Adtech just got there a decade earlier.

The Fat Protocol Thesis Applied to AI Model weights are the fat layer -- general, beneath everything, available at marginal cost. Workflows are thin clients. Most AI infrastructure is built the wrong way round.

Software Engineering Solved Governance Single source of truth, idempotency, rollback and least privilege: software engineering formalised governance problems that other institutions still handle by convention.

Content is Dead: The Curation Inversion Generative AI removed the cost constraint that forced editorial discipline. When creation is free and unlimited, the signal collapses. Curation is the new scarcity.

Digital Archaeology: Bitcoin's 21 Million The 21 million bitcoin limit is not in the whitepaper. It derives from a 32-bit integer ceiling, a type migration, and a patch written under pressure after a 184 billion BTC exploit.

Similarities in the Bitcoin and AI Economies Bitcoin and AI both convert electrical energy into discrete computational units via open protocols. The economic structure is closer than it first appears.

PMFOps: Operationalising Product-Market Fit DevOps operationalised code and MLOps operationalised models. PMFOps applies the same discipline to product-market fit, treating the audience as an artefact to test and version.

The Problem with Faster Horses A foundation model can be treated as a non-deterministic data store. That framing raises more useful questions than the labour-displacement debate.

Funnels vs Bandits Content discovery online is run by recommendation engines solving a different problem entirely: a multi-armed bandit deciding what to show, to whom, out of a pool that keeps growing.

Technical Articles

AI Systems (34)

Product & Strategy (20)

Data Strategy (6)

Architecture & Deployment (13)

Infrastructure (8)

Developer Tools (9)

Security & Resilience (2)

Perspectives (12)