Welcome to Pods and Prompts

By -




Hi everyone ðŸ‘‹,

Welcome to my blog, Pods and Prompts! This is the space where I’ll be sharing my thoughts and experiences on Generative AI, Large Language Models (LLMs), and the infrastructure that powers AI applications. Everything I write here reflects my personal views and is in no way related to my past or present employers.

A little about me: I’m Nithin Anil, an AI/ML engineer with over 13 years in the software industry. Over the years, I’ve worked with AWS, Azure, and GCP, built and managed large-scale distributed big data systems, and designed microservice-based architectures.

For the past couple of years, my focus has been on Generative AI applications, and one of my strengths is managing infrastructure for highly available, low-latency AI systems. I’m also deeply experienced in Kubernetes and Infrastructure as Code. System design for millisecond or even microsecond latency is one of my favourite challenges.

Beyond work, I’m passionate about teaching and mentoring. I enjoy giving talks and sessions on emerging technologies for students and software engineers. Fun fact: Early in my career, I was an educator at Infosys Global Education Centre in Mysore. Someday, I’d love to pivot fully into teaching. If you’d like me to deliver a technical session on any of my areas of expertise, feel free to reach out!

On the personal side, I’m a big fan of TV shows and movies, especially the feel-good kind. The Big Bang Theory and The Office are all-time favourites, and recently, I finished watching Foundation on Apple TV. Lately, I’ve also gotten into vampire stories, thanks to Lokah: Chapter 1 - Chandra. Right now, I’m reading The Vampire Lestat by Anne Rice. If you have recommendations in this space (or just good feel-good shows), I’d love to hear them!

I also keep a close eye on technology and AI news, so expect me to sprinkle some of that here, too.

Thanks for stopping by, and I hope you’ll enjoy reading my posts as much as I’ll enjoy writing them. 

Location: Bengaluru, Karnataka, India

Comments

Post a Comment

Popular posts from this blog

Deploying AI Agents in Production Using Open Source Architecture

By -
Image
AI agents are becoming core parts of modern applications, but deploying them in production at scale is challenging. The diversity of agent frameworks, unpredictable latency, and the need for streaming responses make it hard to use simple REST-only patterns. Most proof-of-concept deployments break down when real workloads, traffic, and observability requirements hit. This blog post explains how to design a production-ready, open-source architecture for AI agents using FastAPI, Celery, Redis, Kubernetes, KEDA, Prometheus, Grafana, LangFuse, and LangGraph . Why Deploying AI Agents Is Hard Agent frameworks differ widely in required infrastructure. Latency varies from milliseconds to minutes depending on workflow complexity. Real-time streaming is needed for modern AI UX. REST-only patterns can’t handle long execution, retries, or async scheduling. Scaling compute-heavy agents is fundamentally different from scaling API servers. Why the Traditional REST Architecture Fail...
Read more

Semantic caching for LLM Applications and AI Agents

By -
Image
Caching is one of the easiest ways to speed up applications and control cost. But LLM based systems don’t work well with traditional caching because users phrase the same idea in many different ways. This means most queries turn into cache misses. Still, caching is important. LLM agents can take time to run, and inference is expensive. A smarter caching method is needed. Why Traditional Caching Fails Works on exact matching. Natural language rarely matches exactly. Same intent written differently becomes a cache miss. Result: almost no benefit for LLM workloads. What Semantic caching Does Focuses on meaning instead of exact text. Convert each query into an embedding. Store embeddings in a vector database like Redis, Qdrant, or Milvus. Add TTL to control freshness. For each new query: Convert to embedding. Run similarity search. If similar enough, return cached output. Otherwise, run the agent. Redis Code Sample for Semantic caching Belo...
Read more