--- author: "Umesh Malik" canonical: "https://umesh-malik.com/topics/llm-engineering" description: "Technical guides on LLM engineering: RAG vs fine-tuning tradeoffs, running local LLMs for coding, and deploying GPT-5 and DeepSeek models in production." title: "LLM Engineering — RAG, Fine-Tuning & Production LLMs | Umesh Malik" tokens: 4148 generator: "scripts/generate-page-markdown.mjs" --- Topic Hub # LLM Engineering LLM Engineering is the discipline of shipping production systems built on large language models — covering RAG architecture, fine-tuning strategies, model evaluation, and the practical tradeoffs that determine what gets deployed versus what stays in a notebook. These articles cover the technical decisions that matter when you move from prototype to production. RAG Fine-Tuning LLM Architecture OpenAI Production AI ## Articles 27 [![Cover showing the inference engine attack surface with token stream flowing from model through vulnerable parser to arbitrary code execution, and the defense architecture separating GPU host from token parsing](https://umesh-malik.com/blog/secure-llm-inference-vllm-cve-2025-9141-cover.png) AI Security • Aug 25, 2026 ### How to Harden vLLM Inference: CVE-2025-9141 Defense Guide How to harden vLLM inference against token exploits. CVE-2025-9141 let models run code via eval(). Separate GPU hosts from parsers. 9 min read Read more →](https://umesh-malik.com/blog/secure-llm-inference-vllm-cve-2025-9141) [![Chart showing ChatGPT Search site-scoped query share jumping from 0.3% to 17% on August 8, 2026](https://umesh-malik.com/blog/chatgpt-search-site-scoping-geo-cover.png) LLM Engineering • Aug 24, 2026 ### ChatGPT Search Optimization After the Site-Scoping Shift ChatGPT search optimization changed when 17% of queries started scoping to specific sites. What the GPT-5.6 shift means and how to get cited. 7 min read Read more →](https://umesh-malik.com/blog/chatgpt-search-site-scoping-geo) [![How server-side compaction replaces a long agent transcript with a single summary block once input tokens cross the trigger, and what is kept versus permanently dropped](https://umesh-malik.com/blog/agent-context-compaction-what-survives-cover.png) AI Engineering • Aug 12, 2026 ### Agent context compaction: keep what the 150K cutoff drops Agent context compaction drops every block before the summary at 150K tokens. What survives, what instructions silently replaces, and the usage field that lies. 9 min read Read more →](https://umesh-malik.com/blog/agent-context-compaction-what-survives) [![Timeline comparing the knowledge-cutoff date vendors claim for Claude Opus 5 against the earlier cutoff its answers actually reveal](https://umesh-malik.com/blog/testing-llm-knowledge-cutoffs-opus-5-cover.png) LLM Engineering • Aug 11, 2026 ### How to Test an LLM's Knowledge Cutoff: Opus 5's May Claim Falls Short Here's how to test an LLM's knowledge cutoff with three reproducible probes — the method showing Opus 5 claims May 2026 but answers like January 2026. 7 min read Read more →](https://umesh-malik.com/blog/testing-llm-knowledge-cutoffs-opus-5) [![Cover showing the vLLM VRAM budget split into model weights, runtime overhead and KV cache, with the KV cache block math that converts free VRAM into concurrent sequences](https://umesh-malik.com/blog/vllm-throughput-tuning-flags-cover.png) LLM Engineering • Aug 8, 2026 ### vLLM throughput tuning: configure these four flags, not a bigger GPU vLLM throughput tuning starts with KV cache blocks, not a bigger GPU. The four flags that decide your tokens/sec, and the one that quietly backfires. 10 min read Read more →](https://umesh-malik.com/blog/vllm-throughput-tuning-flags) [![Diagram contrasting a sequential agent loop of ten model round trips with a two-level dependency graph running nine tool calls concurrently](https://umesh-malik.com/blog/parallel-agent-tool-calls-dag-harness-cover.png) AI Engineering • Aug 6, 2026 ### Run agent tool calls in parallel: 10 turns become 2 DAG levels Run agent tool calls in parallel by swapping the loop for a DAG planner: ten round trips become two levels, plus the cap, budget and critic on top. 10 min read Read more →](https://umesh-malik.com/blog/parallel-agent-tool-calls-dag-harness) [![Reinforcement fine-tuning: a 4B open model matching a frontier LLM on retrieval at a fraction of the cost](https://umesh-malik.com/blog/reinforcement-fine-tuning-small-models-retrieval-cover.png) LLM Engineering • Aug 6, 2026 ### Reinforcement Fine-Tuning: When a 4B Model Beats GPT-5.6 Reinforcement fine-tuning let a 4B open model match GPT-5.6 Sol on retrieval at 100x lower cost. How RFT works, and when it beats prompting a frontier LLM. 10 min read Read more →](https://umesh-malik.com/blog/reinforcement-fine-tuning-small-models-retrieval) [![Agent harness design diagram: a model's reasoning discarded each turn versus reasoning retained and context compacted across turns](https://umesh-malik.com/blog/agent-harness-design-arc-agi-3-cover.png) AI Engineering • Aug 5, 2026 ### Agent Harness Design: Why an ARC-AGI-3 Score Tripled Agent harness design decided a benchmark: OpenAI's ARC-AGI-3 score went 13.3% → 38.3% with zero model changes. What that means for your agent loop. 11 min read Read more →](https://umesh-malik.com/blog/agent-harness-design-arc-agi-3) [![LLM abuse detection diagram contrasting per-message content classification with account-level behavioural correlation across a session corpus](https://umesh-malik.com/blog/llm-abuse-detection-openai-scam-network-cover.png) AI Security • Aug 5, 2026 ### LLM Abuse Detection: What OpenAI's Scam Ban Reveals LLM abuse detection failed at the message level and worked at the account level. OpenAI's Cambodia scam ban shows which signal actually catches misuse. 11 min read Read more →](https://umesh-malik.com/blog/llm-abuse-detection-openai-scam-network) [![Diagram of an LLM eval framework pipeline: tasks and configs produce runs, graders apply checks to produce grades](https://umesh-malik.com/blog/llm-eval-framework-smevals-cover.png) LLM Engineering • Aug 4, 2026 ### LLM Eval Framework: Grade Prompts, Models and Harnesses An LLM eval framework turns vibes into scores. How smevals structures tasks, configs, runners and graders — and how to ship your first eval today. 10 min read Read more →](https://umesh-malik.com/blog/llm-eval-framework-smevals) [![Layer-by-layer LLM inference streaming one transformer layer at a time from disk onto a 4GB GPU](https://umesh-malik.com/blog/run-70b-llm-on-4gb-gpu-airllm-cover.png) LLM Engineering • Aug 4, 2026 ### Run 70B LLM on 4GB GPU: AirLLM's Real Tradeoff Run 70B LLM on 4GB GPU hardware with AirLLM's layer-by-layer inference. The VRAM math is real — you just pay for it in disk bandwidth. The honest tradeoff. 11 min read Read more →](https://umesh-malik.com/blog/run-70b-llm-on-4gb-gpu-airllm) [![A RAG chatbot in Next.js: embed the query, search pgvector, augment the prompt, stream a cited answer](https://umesh-malik.com/blog/rag-chatbot-nextjs-guide-cover.png) AI Engineering • Jul 21, 2026 ### Build a RAG Chatbot in Next.js: Retrieval, Streaming & Citations (2026) Build a RAG chatbot in Next.js with the AI SDK: embed the query, search pgvector, stream a grounded answer with citations, and stop hallucinations. 8 min read Read more →](https://umesh-malik.com/blog/rag-chatbot-nextjs-guide) [![The production layer of a Vercel AI SDK app: streaming, tool-calling, abort, rate limiting, and cost control](https://umesh-malik.com/blog/vercel-ai-sdk-production-guide-cover.png) AI Engineering • Jul 21, 2026 ### Vercel AI SDK in Production: Streaming, Tool-Calling & the Gotchas Nobody Tells You (2026) Vercel AI SDK in production: streaming, tool-calling, aborting generations, error retry UX, rate limiting, and cost control — the layer every tutorial skips. 9 min read Read more →](https://umesh-malik.com/blog/vercel-ai-sdk-production-guide) [![Editorial cover: the demo-to-production gap for autonomous AI agents in 2026](https://umesh-malik.com/blog/autonomous-ai-agents-production-gap-2026-cover.png) AI Engineering • Jun 14, 2026 ### Why 77% of Autonomous AI Agents Never Reach Production (2026) Only 23% of autonomous AI agents reach production in 2026. The demo-to-production gap, why agents fail, and the playbook the winners actually use. 9 min read Read more →](https://umesh-malik.com/blog/autonomous-ai-agents-production-gap-2026) [![The stages of a production retrieval-augmented generation pipeline](https://umesh-malik.com/blog/build-rag-pipeline-from-scratch-cover.png) AI Engineering • Jun 8, 2026 ### Build a RAG Pipeline From Scratch: Production Patterns That Matter Build a RAG pipeline from scratch: chunking, embeddings, retrieval, reranking, grounded generation, and the production patterns that decide whether it works. 7 min read Read more →](https://umesh-malik.com/blog/build-rag-pipeline-from-scratch) [![Architecture of a production Model Context Protocol server on Cloudflare Workers](https://umesh-malik.com/blog/how-to-build-mcp-server-cover.png) AI Engineering • Jun 8, 2026 ### How to Build an MCP Server: A Step-by-Step Guide (2026) How to build an MCP server, step by step: JSON-RPC 2.0, the Streamable HTTP transport, typed tools, and agent discovery — from a real one I shipped. 9 min read Read more →](https://umesh-malik.com/blog/how-to-build-mcp-server) [![RAG vs fine-tuning architecture comparison for LLMs](https://umesh-malik.com/blog/rag-vs-fine-tuning-llms-2026-cover.png) AI Engineering • Feb 28, 2026 ### RAG vs Fine-Tuning for LLMs in 2026: A Production Decision Framework With Real Tradeoffs RAG vs fine-tuning for LLMs in 2026: a practical decision framework covering architecture tradeoffs, cost, latency, and when to use each in production. 6 min read Read more →](https://umesh-malik.com/blog/rag-vs-fine-tuning-llms-2026) [![Ads for AI agents: TIME serves markdown with sponsored FAQ blocks to AI crawlers and HTML to humans](https://umesh-malik.com/blog/ads-for-ai-agents-time-markdown-crawlers-cover.png) AI Engineering • Aug 6, 2026 ### Ads for AI Agents: TIME Serves Crawlers a Different Site Ads for AI agents are live. TIME serves crawlers a 42 KB markdown site with sponsored blocks; humans and Googlebot get 1.2 MB of HTML. I measured all of it. 10 min read Read more →](https://umesh-malik.com/blog/ads-for-ai-agents-time-markdown-crawlers) [![Editorial cover: build enterprise-grade AI agents for free with open-source MaxKB, $0 API cost, self-hosted](https://umesh-malik.com/blog/build-enterprise-ai-agents-free-cover.png) AI Engineering • Jul 8, 2026 ### How to Build Enterprise-Grade AI Agents for Free (MaxKB, 2026) How to build enterprise-grade AI agents for free in 2026: a hands-on MaxKB + local LLM guide to RAG precision, security, and $0 API cost. 10 min read Read more →](https://umesh-malik.com/blog/build-enterprise-ai-agents-free) [![DeepSeek V4 Flash 0731 agent benchmark results compared against the preview build and V4-Pro Preview](https://umesh-malik.com/blog/deepseek-v4-flash-0731-benchmarks-cover.png) LLM Engineering • Aug 1, 2026 ### DeepSeek V4 Flash 0731 Benchmarks: 13B Active Beats 1.6T DeepSeek V4 Flash 0731 benchmarks: same 284B/13B architecture as the preview, re-post-trained only — and it beats the 1.6T V4-Pro Preview on nine agent tests. 9 min read Read more →](https://umesh-malik.com/blog/deepseek-v4-flash-0731-benchmarks) [![Kimi K3 vs Claude Fable 5 head-to-head showing benchmarks, pricing, and where the open 2.8T model wins](https://umesh-malik.com/blog/kimi-k3-vs-fable-5-cover.png) LLM Engineering • Jul 19, 2026 ### Kimi K3 vs Claude Fable 5: When the Open Model Is Worth the Switch Kimi K3 beats Claude Fable 5 on cost by a wide margin and loses on agentic tasks. The benchmarks that decide it, and how to run K3 where it actually wins. 6 min read Read more →](https://umesh-malik.com/blog/kimi-k3-vs-claude-fable-5) [![GPT-5.6 Sol vs Terra vs Luna comparison showing price, coding strength, and cost per task](https://umesh-malik.com/blog/gpt-5-6-sol-terra-luna-cover.png) LLM Engineering • Jul 11, 2026 ### GPT-5.6 Sol vs Terra vs Luna: The Routing Strategy That Cuts Cost GPT-5.6 Sol vs Terra vs Luna compared on price, coding, latency, and cost per task — plus a routing strategy that cuts your bill without wrecking quality. 5 min read Read more →](https://umesh-malik.com/blog/gpt-5-6-sol-vs-terra-vs-luna) [![OpenAI GPT-5.6 family showing Sol, Terra, and Luna tiers with benchmarks, pricing, and 1.05M context](https://umesh-malik.com/blog/gpt-5-6-cover.png) LLM Engineering • Jul 11, 2026 ### GPT-5.6 API: Pricing, Thinking Modes, and the Shared Context Trap GPT-5.6 API pricing ($1-$30/1M), the Ultra and Max thinking modes, and a 1.05M context window that is shared — with the fine print that breaks agent loops. 10 min read Read more →](https://umesh-malik.com/blog/openai-gpt-5-6-sol-terra-luna-guide) [![OpenAI GPT-5.4 overview showing professional work, coding, computer use, and 1M context](https://umesh-malik.com/blog/gpt-5-4-cover.png) LLM Engineering • Mar 6, 2026 ### GPT-5.4 for Agents: Computer Use, MCP Tool Calls, and Real Pricing GPT-5.4's native computer use and MCP tool calls are the real upgrade for agents. What holds up in a loop, what the 1M context costs, and how Pro compares. 12 min read Read more →](https://umesh-malik.com/blog/openai-gpt-5-4-complete-guide) [![OpenAI GPT-5.3 Instant overview showing three key improvements: fewer refusals, better web answers, and smoother conversational tone](https://umesh-malik.com/blog/gpt-5-3-instant-cover.png) LLM Engineering • Mar 4, 2026 ### OpenAI GPT-5.3 Instant: 26.8% Fewer Hallucinations, Reduced Refusals, and Better Web Answers GPT-5.3 Instant brings 26.8% fewer hallucinations, fewer needless refusals, and better web-sourced answers — what changed and why it matters for devs. 10 min read Read more →](https://umesh-malik.com/blog/openai-gpt-5-3-instant-fewer-refusals-better-answers) [![DeepSeek V4 launch preview showing AI model race between China-first chips and U.S. rivals](https://umesh-malik.com/blog/deepseek-v4-release-cover.png) LLM Engineering • Mar 1, 2026 ### DeepSeek V4 vs US AI Models: Benchmarks, Architecture, and What It Means for the Industry DeepSeek V4 is expected in early March 2026. Here is what is confirmed, what remains unverified, and how it challenges U.S. AI rivals. 10 min read Read more →](https://umesh-malik.com/blog/deepseek-v4-release-challenge-us-ai-rivals) [![A glowing AI brain being extracted through a network of fraudulent connections representing the massive distillation attack on Claude](https://umesh-malik.com/blog/distillation-attacks-cover.png) AI Security • Feb 24, 2026 ### AI Model Distillation: Inside the $100M Claude Heist Anthropic exposes an AI model distillation attack by DeepSeek, Moonshot, and MiniMax: 16 million exchanges, 24,000 fake accounts. The forensic breakdown. 33 min read Read more →](https://umesh-malik.com/blog/anthropic-detecting-preventing-distillation-attacks) ## Frequently Asked Questions ### What is LLM Engineering? LLM Engineering is the practice of building production systems with large language models. It covers model selection, prompt design, RAG architecture, fine-tuning strategies, evaluation pipelines, and inference optimization — the full technical stack between a raw model and a working AI product. ### What is the difference between RAG and fine-tuning for LLMs? RAG (Retrieval-Augmented Generation) fetches relevant context at inference time from an external knowledge base, making it ideal for dynamic or frequently-updated information. Fine-tuning adjusts model weights for specific tasks or communication styles and is better for consistent behavior and lower-latency responses. Most production systems combine both: fine-tuning for style and RAG for knowledge. ### How have GPT-5 models changed production LLM engineering? GPT-5.3 introduced significantly fewer refusals and better instruction following, reducing prompt engineering overhead. GPT-5.4 added expanded context windows and improved agentic tool use, making it easier to build reliable multi-step pipelines without complex fallback logic. ### When should I consider running LLMs locally? Local LLMs make sense for private codebases, high-volume batch tasks where API costs add up, offline workflows, and experimentation without usage limits. Models like Qwen3-Coder are viable for coding assistance on modern hardware. The tradeoff is quality: frontier models still outperform local alternatives on complex reasoning tasks. ## Related Topics [AI Coding Agents →](https://umesh-malik.com/topics/ai-coding-agents)[Claude Code →](https://umesh-malik.com/topics/claude-code) [All Articles →](https://umesh-malik.com/blog)