Design patterns in agentic AI serve the same purpose they do in software engineering, they are reusable solutions to recurring problems. An agent system that works well for a simple Q&A task will break under a complex multi-step research task. An orchestrator pattern that works for sequential workflows is wrong for parallel ones. Choosing the … Read more →
In distributed systems, the same operation can be triggered more than once due to network retries, client timeouts, or message redelivery. Idempotency is the guarantee that running an operation multiple times produces the same result as running it once. This post breaks down the core concepts, the key design decisions every engineer faces, and applies … Read more →
Architectural Patterns for Modern Data Platforms Architectural Patterns for Modern Data Platforms Introduction In today’s data-driven world, choosing the right data architecture is crucial for organizational success. Data architecture defines how data is collected, stored, processed, and accessed across an organization. The evolution of data architectures has been driven by increasing data volumes, velocity, variety, … Read more →
Understanding Memory Management in Rust: Stack, Heap, and Object Creation Understanding Memory Management in Rust Stack, Heap, and Object Creation Table of Contents Introduction Stack vs Heap String Analysis Allocation Strategy Creating Objects Box Allocation Automatic Cleanup Ownership System Borrowing & References Lifetimes Concurrency Common Patterns Practical Example Real-World Example Memory Reference Key Principles Introduction … Read more →
Rust for Java Developers – Complete Guide Rust for Java Developers A comprehensive guide with side-by-side comparisons for Java engineers transitioning to Rust Table of Contents Introduction Hello World Variables & Mutability Data Types String vs &str Stack vs Heap Functions Ownership Structs vs Classes Enums & Pattern Matching Traits vs Interfaces Error Handling Collections … Read more →
Most engineers including myself may have met large language models through an API such as OpenAI’s GPT-4, Anthropic’s Claude, or Google’s Gemini. These are systems that process input tokens and return output tokens without exposing their internal mechanisms. In open-source LLMs, developers interact directly with tensors instead of relying on endpoint APIs. This post walks … Read more →
Model may perform with excellent accuracy at launch, but six months later its predictions are unreliable. What happened? Key principle: A model is only as good as the data it sees today, not the data it saw six months ago. The likely culprit is data drift, a change in the statistical properties of the input … Read more →
When working with Large Language Models(LLMs) prompts are everything. The way requests are structured directly affects accuracy, consistency and cost implication of the model. Recently, JSON-based prompting has been gaining traction as a structured alternative to free-text prompts, especially when building production-grade AI applications. Alongside that, token minimization is becoming an important engineering discipline, This … Read more →
Data is the new gold in today’s world. With businesses relying more and more on data-driven decisions, understanding where data comes from and how it flows is essential. Data is constantly moving from raw ingestion, to transformation, to analytics, and finally to end consumers like dashboards or machine learning models. While data navigates and transforms … Read more →
In recent years, data has grown exponentially, driving the need for efficient storage, processing, and analysis. This has led to the emergence of diverse architectural paradigms such as data lakes, data lakehouses, data warehouses, data marts, as well as metrics layers and semantic layers, each serving a distinct purpose within the modern data pipeline. Complementing … Read more →
Fraud detection is a critical aspect of any financial institution or e-commerce platform. Traditional fraud detection systems rely on batch processing and historical data, which can lead to delayed detection and increased false positives and false negatives. In 2017, Equifax, one of the largest credit reporting agencies in the United States, suffered a massive data … Read more →
Retrieval-Augmented Generation (RAG) combines a language model (LLM) with a retrieval system to improve response accuracy by incorporating relevant external knowledge. Deploying RAG on mobile devices presents unique challenges, such as memory constraints, limited processing power, and the need for efficiency. In this blog, I want to build a lightweight RAG system optimized for mobile … Read more →
Images tell stories, capture moments, and help us make sense of the world. But how do computers make sense of images? Unlike us, they don’t “see” things the way we do, they break images down into numbers and patterns. One of the most important steps in image processing is edge detection, which helps identify the … Read more →
Computing is evolving beyond traditional bits and bytes. Quantum computing, based on quantum mechanics, has the potential to solve problems much faster than today’s computers. But how does it work, and what does it mean for data processing and speed? I will explore qubits and how they differ from classical bits, highlighting their unique properties. … Read more →
The Growing Need for Federated Query Engines Have you ever encountered a use case where you need to gather data from multiple databases to arrive at a solution? For instance, analyzing sales across different categories or calculating revenue flowing through multiple channels —such cases often involve data may be residing in different data storage solutions. … Read more →
In the AI space, vector databases are emerging as essential tools for handling unstructured data, such as images, audio or even text. These databases leverage vector representations of data to facilitate efficient searching, retrieval, and analysis. This blog delves into how vector databases operate, the process of generating vector embeddings, the models used, and the … Read more →
Understanding the inner workings of Artificial Neural Network (NN) is crucial for anyone delving into the field of deep learning. This post will explore forward propagation, backward propagation, the critical role of weights and biases, and positive vs negative gradients. Using Python libraries like PyTorch or TensorFlow makes building a neural network architecture seem straightforward—until … Read more →
This blog post is an extension of my previous article on Event-Driven Architecture (EDA) with Event Sourcing. In this post, I will make an attempt to delve deeper into the CQRS (Command Query Responsibility Segregation) pattern, exploring its components and flow, and various considerations in terms of schema design, command and query model synchronization and … Read more →
Event-driven Architecture (EDA) needs no introduction, as it’s increasingly being adopted across various domains. However, there are key enhancements that can further bolster the resilience of EDA systems. In this blog, I will explore how the reliability and auditability of an Event-Driven Architecture (EDA) system can be enhanced by leveraging the Event Sourcing architectural pattern. … Read more →
Modern applications are increasingly built using microservices, where hundreds of small, independent services work together to handle user requests. While this architecture brings scalability and flexibility, it also introduces significant complexity. When a service slows down, fails, or when pipelines take an excessive amount of time, pinpointing the root cause across a complex web of … Read more →
In today’s interconnected world, software systems are expected to deliver uninterrupted services despite inevitable failures. With increasing demands for availability, reliability, and scalability, designing systems that can gracefully handle failures is paramount. In this blog post, I will explore how fault tolerance can be implemented at various levels of a microservices-based system architecture. Understanding Fault … Read more →
We see AI models excelling at complex tasks and delivering impressive predictions. While this is undoubtedly remarkable, it raises an important question: how can we ensure the predictions are unbiased and trustworthy? This is where Explainable AI comes into play What is Explainable AI ? Explainable AI(XAI) is a critical field in artificial intelligence which … Read more →
Latent factors are hidden building blocks that shape user preferences and item characteristics. They are not directly observable but are inferred from the data through a process called dimensionality reduction. These underlying characteristics or hidden dimensions explain the patterns in the interactions between users and items.Furthermore , they also influence the user interactions (like ratings, … Read more →
For the most part the primary reason why we want to use Spark is to accomplish optimal runtime performance from big data workloads. Given that is accepted motivation, how do we actually verify if performance is what is expected to be or see what is going on at basic level of task ? We can do that … Read more →