How Plan Caching Reduces LLM Agent Costs
Plan caching reuses planning templates across similar agent tasks, cutting cost and latency without throwing away accuracy.
Tag: llm
6 matching entries.
Stop filling your agent's context window just because you can
Bigger context windows do not remove failure modes. They create new ones when we stop being intentional about what goes into an agent's context.
I benchmarked 5 embedding models across 4 datasets
I benchmarked five embedding models across four NanoBEIR datasets and found that bigger embeddings did not always produce better retrieval.
Why reranking matters with cross-encoders
Bi-encoders make retrieval fast, but cross-encoders expose why reranking matters when meaning depends on the query.
A beginner-friendly guide to the GGUF model format
GGUF made local LLM inference feel practical by packaging model weights, vocabulary, hyperparameters, and architecture metadata into one runnable format.
Semantic Caching in Production
Repeated user intents can quietly inflate LLM cost and latency. Semantic caching helps, but production use comes with trade-offs.