a concept first tour of how transformers work – Transformer Architecture I’ve always been curious about what’s actually happening inside a transformer, not the API call, not the chat interface, but the actual mechanics underneath. How does attention “know” which words matter? What does a weight matrix encode meaning at all? This post is me … Read more →
Most engineers including myself may have met large language models through an API such as OpenAI’s GPT-4, Anthropic’s Claude, or Google’s Gemini. These are systems that process input tokens and return output tokens without exposing their internal mechanisms. In open-source LLMs, developers interact directly with tensors instead of relying on endpoint APIs. This post walks … Read more →
Retrieval-Augmented Generation (RAG) combines a language model (LLM) with a retrieval system to improve response accuracy by incorporating relevant external knowledge. Deploying RAG on mobile devices presents unique challenges, such as memory constraints, limited processing power, and the need for efficiency. In this blog, I want to build a lightweight RAG system optimized for mobile … Read more →
In the AI space, vector databases are emerging as essential tools for handling unstructured data, such as images, audio or even text. These databases leverage vector representations of data to facilitate efficient searching, retrieval, and analysis. This blog delves into how vector databases operate, the process of generating vector embeddings, the models used, and the … Read more →