About Me

By

 


Professional Snapshot

  • Platform architect with 13+ years of experience building large-scale software and production-ready AI systems

  • Led the design of enterprise AI platforms, agent orchestration frameworks, and high-performance RAG solutions

  • Strong focus on Kubernetes, cloud architecture, and taking AI products from early prototype to stable production

Fun Facts

  • Makes AI systems behave even when everything else refuses to

  • Loves Kubernetes but complains about it daily

  • Turns half-broken POCs into clean, scalable platforms

  • Writes blogs to simplify complex tech for real engineers

  • Protects his early-morning deep-work window like it’s gold

  • Constantly tinkering with agents, infra, and OSS tools

Connect with me on X: https://x.com/nithin_k_anil

Comments

Post a Comment

Popular posts from this blog

Deploying AI Agents in Production Using Open Source Architecture

By
Image
AI agents are becoming core parts of modern applications, but deploying them in production at scale is challenging. The diversity of agent frameworks, unpredictable latency, and the need for streaming responses make it hard to use simple REST-only patterns. Most proof-of-concept deployments break down when real workloads, traffic, and observability requirements hit. This blog post explains how to design a production-ready, open-source architecture for AI agents using FastAPI, Celery, Redis, Kubernetes, KEDA, Prometheus, Grafana, LangFuse, and LangGraph . Why Deploying AI Agents Is Hard Agent frameworks differ widely in required infrastructure. Latency varies from milliseconds to minutes depending on workflow complexity. Real-time streaming is needed for modern AI UX. REST-only patterns can’t handle long execution, retries, or async scheduling. Scaling compute-heavy agents is fundamentally different from scaling API servers. Why the Traditional REST Architecture Fail...
Read more

Welcome to Pods and Prompts

By
Image
Hi everyone ðŸ‘‹, Welcome to my blog,  Pods and Prompts ! This is the space where I’ll be sharing my thoughts and experiences on Generative AI, Large Language Models (LLMs), and the infrastructure that powers AI applications . Everything I write here reflects my personal views  and is in no way related to  my past or present employers. A little about me: I’m Nithin Anil , an AI/ML engineer with over 13 years in the software industry . Over the years, I’ve worked with AWS, Azure, and GCP , built and managed large-scale distributed big data  systems , and designed microservice-based architectures . For the past couple of years, my focus has been on Generative AI applications,  and one of my strengths is managing infrastructure for highly available, low-latency AI systems . I’m also deeply experienced in Kubernetes and Infrastructure as Code . System design for millisecond or even microsecond latency is one of my favourite challenges. Beyond work, I’m pass...
Read more

Semantic caching for LLM Applications and AI Agents

By
Image
Caching is one of the easiest ways to speed up applications and control cost. But LLM based systems don’t work well with traditional caching because users phrase the same idea in many different ways. This means most queries turn into cache misses. Still, caching is important. LLM agents can take time to run, and inference is expensive. A smarter caching method is needed. Why Traditional Caching Fails Works on exact matching. Natural language rarely matches exactly. Same intent written differently becomes a cache miss. Result: almost no benefit for LLM workloads. What Semantic caching Does Focuses on meaning instead of exact text. Convert each query into an embedding. Store embeddings in a vector database like Redis, Qdrant, or Milvus. Add TTL to control freshness. For each new query: Convert to embedding. Run similarity search. If similar enough, return cached output. Otherwise, run the agent. Redis Code Sample for Semantic caching Belo...
Read more