<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>must —   christova  </title>
    <link>https://christova.writeas.com/tag:must</link>
    <description>&lt;b&gt;&lt;h3&gt;Tech Articles&lt;/h3&gt;&lt;/b&gt;&lt;br/&gt;&lt;b&gt;Collated from various sources. Full copyright remains with original authors.&lt;/b&gt;</description>
    <pubDate>Mon, 16 Nov 2026 16:29:10 +0000</pubDate>
    <item>
      <title>System Design Concepts</title>
      <link>https://christova.writeas.com/system-design-concepts?pk_campaign=rss-feed</link>
      <description>#SystemDesign #systemdesignconcepts #must-know https://blog.algomaster.io/p/30-system-design-concepts 1. Client-Server Architecture Almos...</description>
      <content:encoded><![CDATA[<p><img src="https://i.snap.as/qKImsBHH.jpg" alt=""/>
<a href="https://christova.writeas.com/tag:SystemDesign" class="hashtag" rel="nofollow ugc"><span>#</span><span class="p-category">SystemDesign</span></a> <a href="https://christova.writeas.com/tag:systemdesignconcepts" class="hashtag" rel="nofollow ugc"><span>#</span><span class="p-category">systemdesignconcepts</span></a> <a href="https://christova.writeas.com/tag:must" class="hashtag" rel="nofollow ugc"><span>#</span><span class="p-category">must</span></a>-know</p>

<p><a href="https://blog.algomaster.io/p/30-system-design-concepts" rel="nofollow ugc">https://blog.algomaster.io/p/30-system-design-concepts</a></p>

<p><strong>1. Client-Server Architecture</strong></p>

<p>Almost every web application that you use is built on this simple yet powerful concept called <strong>client-server architecture</strong>.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!GgVl!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F05ba06b8-7730-4d3e-a6d9-1f8c81770b55_1364x446.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!GgVl!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F05ba06b8-7730-4d3e-a6d9-1f8c81770b55_1364x446.png" alt=""/></a></p>

<p>On one side, you have a <strong>client</strong>—this could be a web browser, a mobile app, or any other frontend application.</p>

<p>and on the other side, you have a <strong>server</strong>—a machine that runs continuously, waiting to handle incoming requests.</p>

<p>The client sends a request to <strong>store, retrieve, or modify data</strong>.</p>

<p>The server receives the request, processes it, performs the necessary operations, and sends back a response.</p>

<p>This sounds simple, but there’s a big question: <em><strong>How does the client even know where to find the server?</strong></em></p>

<hr/>

<h2 id="2-ip-address"><strong>2. IP Address</strong></h2>

<p>A client doesn’t magically know where a server is, it needs an <strong>address</strong> to locate and communicate with it.</p>

<p>On the internet, computers identify each other using <strong>IP addresses</strong>, which work like phone numbers for servers.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!7vrc!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbefd954a-3bf4-4000-9749-8bbe491b7881_1304x848.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!7vrc!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbefd954a-3bf4-4000-9749-8bbe491b7881_1304x848.png" alt=""/></a></p>

<p>Every publicly deployed server has a <strong>unique IP address</strong>. When a client wants to interact with a service, it must send requests to the correct IP address.</p>

<p>But there’s a problem:</p>
<ul><li>When we visit a website, we don’t type its IP address—we just enter the website name.</li>
<li>We can’t expect users (or even systems) to memorize a string of random numbers for every service they connect to.</li>
<li>And if we migrate our service to another server, its IP address may change—breaking all direct connections.</li></ul>

<hr/>

<h2 id="3-dns"><strong>3. DNS</strong></h2>

<p>Instead of relying on hard-to-remember IP addresses, we use something much more human-friendly: <strong>domain names</strong>.</p>

<p>But, we need a way to map a domain name to it’s corresponding IP address.</p>

<p>This is where <strong>DNS (or Domain Name System)</strong> comes in. It maps easy to remember domain names (like <a href="http://algomaster.io/" rel="nofollow ugc">algomaster.io</a>) to their corresponding IP addresses.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!s6fZ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a22b7b4-f51c-47c7-a9db-abb5ce56def7_1284x916.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!s6fZ!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a22b7b4-f51c-47c7-a9db-abb5ce56def7_1284x916.png" alt=""/></a></p>

<p>Here’s what happens behind the scenes:</p>
<ol><li><p>When you type <code>algomaster.io</code> into your browser, your computer asks a DNS server for the corresponding IP address.</p></li>

<li><p>Once the DNS server responds with the IP, your browser uses it to establish a connection with the server and make a request.</p></li></ol>

<p>You can find the IP address of any domain using the <strong>ping</strong> command. Just open your terminal and type ping followed by the domain name. And it’ll return the IP address currently assigned to that domain.</p>

<p><strong><a href="https://blog.algomaster.io/p/30-system-design-concepts?utm_source=substack&amp;utm_medium=email&amp;utm_content=share&amp;action=share" rel="nofollow ugc">Share</a></strong></p>

<hr/>

<h2 id="4-proxy-reverse-proxy"><strong>4. Proxy / Reverse Proxy</strong></h2>

<p>When you visit a website, your request doesn’t always go directly to the server—sometimes, it passes through a <strong>proxy</strong> or <strong>reverse proxy</strong> first.</p>

<p>A <strong>proxy server</strong> acts as a <strong>middleman</strong> between your device and the internet.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!i5Av!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff11ff886-4db5-4550-aa0f-5f232b094b03_887x477.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!i5Av!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff11ff886-4db5-4550-aa0f-5f232b094b03_887x477.png" alt=""/></a></p>

<p>When you request a webpage, the proxy forwards your request to the target server, retrieves the response, and sends it back to you.</p>

<p>Proxy hides your IP address, keeping your location and identity private.</p>

<p>A <strong>reverse proxy</strong> works the other way around. It intercepts client requests and forwards them to backend servers based on predefined rules.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!L8Gg!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3be42662-47d2-4165-b278-95d583e2d2be_904x517.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!L8Gg!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3be42662-47d2-4165-b278-95d583e2d2be_904x517.png" alt=""/></a></p>

<p>Allowing direct access to servers can pose <strong>security risks</strong>, exposing them to threats like <strong>hackers</strong> and <strong>DDoS attacks</strong>.</p>

<p>A reverse proxy mitigates these risks by acting as a controlled entry point that regulates incoming traffic and hides server IPs.</p>

<p>It can also act as a load balancer, distributing traffic across multiple servers.</p>

<p>If you want to learn about Proxy vs Reverse Proxy in more detail, checkout this article:</p>

<p><a href="https://blog.algomaster.io/p/proxy-vs-reverse-proxy-explained" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!-vu2!,w_140,h_140,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F005673b3-62cb-48a8-adc6-19fc69156a17_887x477.png" alt="Proxy vs Reverse Proxy (Explained with Examples)"/></a></p>

<h4 id="proxy-vs-reverse-proxy-explained-with-examples-https-blog-algomaster-io-p-proxy-vs-reverse-proxy-explained"><strong><a href="https://blog.algomaster.io/p/proxy-vs-reverse-proxy-explained" rel="nofollow ugc">Proxy vs Reverse Proxy (Explained with Examples)</a></strong></h4>

<p><strong><a href="https://substack.com/profile/83602743-ashish-pratap-singh" rel="nofollow ugc">Ashish Pratap Singh</a></strong></p>

<p>·</p>

<p><strong>October 30, 2024</strong></p>

<p><strong><a href="https://blog.algomaster.io/p/proxy-vs-reverse-proxy-explained" rel="nofollow ugc">Read full story</a></strong></p>

<hr/>

<h2 id="5-latency"><strong>5. Latency</strong></h2>

<p>Whenever a client communicates with a server, there’s always some delay. One of the biggest causes of this delay is <strong>physical distance</strong>.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!CL6W!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F664e76b6-37fa-4e36-b8a5-d41a5b0ad93a_1866x988.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!CL6W!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F664e76b6-37fa-4e36-b8a5-d41a5b0ad93a_1866x988.png" alt=""/></a></p>

<p>For example, if our server is in <strong>New York</strong>, but a user in <strong>India</strong> sends a request, the data has to travel halfway across the world—and then the response has to make the same long trip back.</p>

<p>This round-trip delay is called <strong>latency</strong>—the total time it takes for data to travel between the client and the server. High latency can make applications feel slow and unresponsive.</p>

<p>One way to <strong>reduce latency</strong> is by deploying our service across <strong>multiple data centers worldwide</strong>.</p>

<p>This way, users can connect to the <strong>nearest</strong> server instead of waiting for data to travel across the globe.</p>

<p><em><strong>Once a connection is made, how do clients and servers actually communicate?</strong></em></p>

<hr/>

<h2 id="6-http-https"><strong>6. HTTP/HTTPS</strong></h2>

<p>Every time you visit a website, your browser and the server communicate using a set of rules called <strong>HTTP (Hypertext Transfer Protocol)</strong>.</p>

<p>That’s why most URLs start with <code>http://</code> or its secure version, <code>https://</code>.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!8BgK!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46796792-f8b5-4e13-b4c2-6920b3d6e0e7_1942x1170.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!8BgK!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46796792-f8b5-4e13-b4c2-6920b3d6e0e7_1942x1170.png" alt=""/></a></p>

<p>Here’s how it works:</p>
<ul><li>The <strong>client</strong> sends a request to the server. This request includes a <strong>header</strong> (containing details like the request type, browser type, and cookies) and sometimes a <strong>request body</strong> (which carries additional data, like form inputs).</li>
<li>The <strong>server</strong> processes the request and responds with an <strong>HTTP response</strong>—either returning the requested data or an error message if something goes wrong.</li></ul>

<p>HTTP has a major security flaw, it <strong>sends data in plain text</strong>. This is a serious problem, especially for sensitive information like passwords, credit card details, and personal data.</p>

<p>That’s why modern websites use <strong>HTTPS (Hypertext Transfer Protocol Secure)</strong> instead. HTTPS encrypts all data using <strong>SSL/TLS,</strong> ensuring that even if someone intercepts the request, they can’t read or alter it.</p>

<p>But clients and servers don’t directly exchange raw HTTP requests and response.</p>

<p>HTTP is just a <strong>protocol</strong> for transferring data but it doesn’t define:</p>
<ul><li>How requests should be structured</li>
<li>What format responses should be in</li>
<li>or how different clients should interact with the server.</li></ul>

<p>This is where <strong>APIs (or Application Programming Interfaces)</strong> come in.</p>

<hr/>

<h2 id="7-apis"><strong>7. APIs</strong></h2>

<p>Think of an API as a <strong>middleman</strong> that allows clients (like web and mobile apps) to communicate with servers without worrying about low-level details.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!EH_B!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0c3a0e69-9134-4bd9-9d42-a898bc838e32_1574x504.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!EH_B!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0c3a0e69-9134-4bd9-9d42-a898bc838e32_1574x504.png" alt=""/></a></p>

<p>Almost every digital service you use—social media, e-commerce, online banking, ride-hailing apps—is built on APIs working together behind the scenes.</p>

<p>Here’s how it typically works:</p>
<ol><li><p>A <strong>client</strong> sends a request to an API.</p></li>

<li><p>The <strong>API</strong>, hosted on a server, processes the request, interacts with databases or other services, and prepares a response.</p></li>

<li><p>The <strong>API sends back the response</strong> in a structured format, usually <strong>JSON</strong> or <strong>XML</strong>, which the client understands and can display.</p></li></ol>

<p>APIs provide a <strong>layer of abstraction</strong>—the client doesn’t need to know <strong>how</strong> the server processes the request, only that it <strong>returns the expected data</strong>.</p>

<p>If you want to learn more about APIs, checkout this article:</p>

<p><a href="https://blog.algomaster.io/p/whats-an-api" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!f-cG!,w_140,h_140,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fae14a376-92a3-4ff6-a329-8e4f2a7ac9b5_1546x1074.png" alt="What&#39;s an API?"/></a></p>

<h4 id="what-s-an-api-https-blog-algomaster-io-p-whats-an-api"><strong><a href="https://blog.algomaster.io/p/whats-an-api" rel="nofollow ugc">What&#39;s an API?</a></strong></h4>

<p><strong><a href="https://substack.com/profile/83602743-ashish-pratap-singh" rel="nofollow ugc">Ashish Pratap Singh</a></strong></p>

<p>·</p>

<p><strong>January 21, 2025</strong></p>

<p><strong><a href="https://blog.algomaster.io/p/whats-an-api" rel="nofollow ugc">Read full story</a></strong></p>

<p>But, not all APIs are built the same. Different API styles exist to serve different needs. Two of the most popular ones are <strong>REST</strong> and <strong>GraphQL</strong>.</p>

<hr/>

<h2 id="8-rest-api"><strong>8. Rest API</strong></h2>

<p>Among the different API styles, <strong>REST (Representational State Transfer)</strong> is the most widely used.</p>

<p>A <strong>REST API</strong> follows a set of rules that define how clients and servers communicate over HTTP in a structured way.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!HBtO!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c704568-18c9-48e5-ba45-f8e6e1e25a6b_1554x896.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!HBtO!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c704568-18c9-48e5-ba45-f8e6e1e25a6b_1554x896.png" alt=""/></a></p>

<p>Rest is:</p>
<ul><li><strong>Stateless:</strong> Every request is independent; the server doesn’t store client state.</li>
<li><strong>Resource-Based:</strong> Everything is treated as a resource (e.g., /users, /orders, /products).</li>
<li><strong>Uses Standard HTTP Methods:</strong> Clients interact with resources using <strong>HTTP methods</strong> like:
<ul><li><strong>GET</strong> → Retrieves data (e.g., fetching a user profile).</li>
<li><strong>POST</strong> → Creates new data (e.g., adding a new user).</li>
<li><strong>PUT/PATCH</strong> → Updates existing data (e.g., changing user settings).</li>
<li><strong>DELETE</strong> → Removes data (e.g., deleting an account).</li></ul></li></ul>

<p>REST APIs are great because they’re <strong>simple, scalable, and easy to cache</strong>, but they have <strong>limitations</strong>, especially when dealing with complex data retrieval.</p>

<p>REST endpoints often return <strong>more data than needed</strong>, leading to inefficient network usage. If an API doesn’t return related data, the client may need to <strong>make multiple requests</strong> to retrieve all required information.</p>

<p>To address these challenges, GraphQL was introduced in 2015 by Facebook.</p>

<hr/>

<h2 id="9-graphql"><strong>9. GraphQL</strong></h2>

<p>Unlike REST, which forces clients to retrieve <strong>fixed sets of data</strong>, <strong>GraphQL</strong> lets clients <strong>ask for exactly what they need—nothing more, nothing less</strong>.</p>

<p>With a REST API, if you need a user details, user profile details along with their recent posts, you might have to <strong>make multiple requests</strong> to different endpoints:</p>
<ol><li><p><code>GET /api/users/123</code> → fetch user details</p></li>

<li><p><code>GET /api/users/123/profile</code> → fetch user profile</p></li>

<li><p><code>GET /api/users/123/posts</code> → fetch user’s posts</p></li></ol>

<p>With GraphQL, you can <strong>combine those requests into one</strong> and fetch exactly the data you need in a single query:</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!s0a7!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5b40446b-9156-4e7f-9107-654235bb1e53_2138x1806.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!s0a7!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5b40446b-9156-4e7f-9107-654235bb1e53_2138x1806.png" alt=""/></a></p>

<p>The <strong>server responds with only the requested fields</strong>, reducing unnecessary data transfer and improving efficiency.</p>

<p>However, GraphQL also comes with trade-offs—it <strong>requires more processing on the server side</strong> and isn’t as easy to cache as REST.</p>

<p>Learn more about REST vs GraphQL here:</p>

<p><a href="https://blog.algomaster.io/p/rest-vs-graphql" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!cEwV!,w_140,h_140,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc4d86506-cb5f-4c9f-a56a-257439ec46eb_1666x1210.png" alt="REST vs GraphQL"/></a></p>

<h4 id="rest-vs-graphql-https-blog-algomaster-io-p-rest-vs-graphql"><strong><a href="https://blog.algomaster.io/p/rest-vs-graphql" rel="nofollow ugc">REST vs GraphQL</a></strong></h4>

<p><strong><a href="https://substack.com/profile/83602743-ashish-pratap-singh" rel="nofollow ugc">Ashish Pratap Singh</a></strong></p>

<p>·</p>

<p><strong>March 11, 2025</strong></p>

<p><strong><a href="https://blog.algomaster.io/p/rest-vs-graphql" rel="nofollow ugc">Read full story</a></strong></p>

<p>When a client makes a request, they usually want to <strong>store or retrieve data</strong>.</p>

<p>But this brings up another question—<em><strong>where is the actual data stored?</strong></em></p>

<hr/>

<h2 id="10-databases"><strong>10. Databases</strong></h2>

<p>If our application deals with <strong>small amounts of data</strong>, we could store it <strong>in memory</strong>.</p>

<p>But modern applications handle <strong>massive volumes of data</strong>—far more than what memory can efficiently handle.</p>

<p>That’s why we need a <strong>dedicated server for storing and managing data</strong>—a <strong>database</strong>.</p>

<p>A database is the backbone of any application. It ensures that data is stored, retrieved, and managed efficiently while keeping it secure, consistent, and durable.</p>

<p>When a client requests to <strong>store</strong> or <strong>retrieve</strong> data, the <strong>server communicates with the database</strong>, fetches the required information, and returns it to the client.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!Xis-!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F57b338e3-3cc2-4bcf-86e4-0aa42cc2c2de_1060x884.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!Xis-!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F57b338e3-3cc2-4bcf-86e4-0aa42cc2c2de_1060x884.png" alt=""/></a></p>

<p>But not all databases are the same. Different applications have different <strong>scalability, performance, and consistency</strong> requirements, which is choosing the right type of database is important.</p>

<p>If you want to learn about different types of databases, checkout this article:</p>

<p><a href="https://blog.algomaster.io/p/15-types-of-databases" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!sdB1!,w_140,h_140,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F773d33c8-ab09-4d63-a6a4-ee55bd9ef0ff_1944x1148.png" alt="15 Types of Databases and When to Use Them"/></a></p>

<h4 id="15-types-of-databases-and-when-to-use-them-https-blog-algomaster-io-p-15-types-of-databases"><strong><a href="https://blog.algomaster.io/p/15-types-of-databases" rel="nofollow ugc">15 Types of Databases and When to Use Them</a></strong></h4>

<p><strong><a href="https://substack.com/profile/83602743-ashish-pratap-singh" rel="nofollow ugc">Ashish Pratap Singh</a></strong></p>

<p>·</p>

<p><strong>March 24, 2024</strong></p>

<p><strong><a href="https://blog.algomaster.io/p/15-types-of-databases" rel="nofollow ugc">Read full story</a></strong></p>

<p>In system design, we typically choose between <strong>SQL and NoSQL databases</strong>.</p>

<hr/>

<h2 id="11-sql-vs-nosql"><strong>11. SQL vs NoSQL</strong></h2>

<p><a href="https://substackcdn.com/image/fetch/$s_!3Es9!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa81088b7-a188-4089-845a-63936e930a71_1632x1076.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!3Es9!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa81088b7-a188-4089-845a-63936e930a71_1632x1076.png" alt=""/></a></p>

<p>SQL databases store data in tables with a <strong>strict predefined schema</strong> and follow the <strong>ACID properties</strong>.</p>
<ul><li><strong>Atomicity</strong> - A transaction is <strong>all-or-nothing</strong> (it either completes fully or not at all).</li>
<li><strong>Consistency</strong> – Data always remains <strong>valid</strong> and follows defined rules.</li>
<li><strong>Isolation</strong> – Transactions <strong>don’t interfere</strong> with each other.</li>
<li><strong>Durability</strong> – Once data is saved, it <strong>won’t be lost</strong>, even if the system crashes.</li></ul>

<p>Because of these guarantees, SQL databases are ideal for applications that require <strong>strong consistency and structured relationships</strong>, such as <strong>banking systems</strong>.</p>

<blockquote><p>Examples of popular SQL databases include: MySQL and PostgreSQL</p></blockquote>

<p>NoSQL databases on the other hand are designed for <strong>high scalability and performance</strong>.</p>

<p>They <strong>don’t require a fixed schema</strong> and use different data models, including:</p>
<ul><li><strong>Key-Value Stores</strong> – Fast lookups for simple key-value pairs (e.g., Redis).</li>
<li><strong>Document Stores</strong> – Store flexible, JSON-like documents (e.g., MongoDB).</li>
<li><strong>Graph Databases</strong> – Best for highly connected data (e.g., Neo4j).</li>
<li><strong>Wide-Column Stores</strong> – Optimized for large-scale, distributed data (e.g., Cassandra).</li></ul>

<p>So, <strong>which one should you use?</strong> It depends on the system requirements.</p>
<ul><li>If you need <strong>structured, relational data with strong consistency</strong> → <strong>SQL is a better choice.</strong></li>
<li>If you need <strong>high scalability, flexible schemas, or fast reads/writes at scale</strong> → <strong>NoSQL is a better choice.</strong></li></ul>

<p>Many modern applications <strong>use both SQL and NoSQL together.</strong></p>

<p>For example, an e-commerce platform might:</p>
<ul><li>Store customer orders in SQL (because they require strict consistency).</li>
<li>and store Product recommendations in NoSQL (because they need flexible and fast lookups).</li></ul>

<p>If you want to learn more about SQL vs NoSQL, checkout this article:</p>

<p><a href="https://blog.algomaster.io/p/sql-vs-nosql-7-key-differences" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!3Es9!,w_140,h_140,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa81088b7-a188-4089-845a-63936e930a71_1632x1076.png" alt="SQL vs NoSQL - 7 Key Differences You Must Know"/></a></p>

<h4 id="sql-vs-nosql-7-key-differences-you-must-know-https-blog-algomaster-io-p-sql-vs-nosql-7-key-differences"><strong><a href="https://blog.algomaster.io/p/sql-vs-nosql-7-key-differences" rel="nofollow ugc">SQL vs NoSQL – 7 Key Differences You Must Know</a></strong></h4>

<p><strong><a href="https://substack.com/profile/83602743-ashish-pratap-singh" rel="nofollow ugc">Ashish Pratap Singh</a></strong></p>

<p>·</p>

<p><strong>September 20, 2024</strong></p>

<p><strong><a href="https://blog.algomaster.io/p/sql-vs-nosql-7-key-differences" rel="nofollow ugc">Read full story</a></strong></p>

<hr/>

<h2 id="12-vertical-scaling"><strong>12. Vertical Scaling</strong></h2>

<p>As our <strong>user base grows</strong>, so does the number of <strong>requests hitting our application servers</strong>.</p>

<p>Initially, a <strong>single server</strong> might be enough to handle the load. But, as traffic increases, that single server can become a bottleneck, slowing everything down.</p>

<p>One of the quickest solutions is to <strong>upgrade the existing server</strong> by adding more CPU, RAM or storage.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!x4x6!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F491fc8b1-02e4-490c-bb5f-87286df10730_1086x556.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!x4x6!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F491fc8b1-02e4-490c-bb5f-87286df10730_1086x556.png" alt=""/></a></p>

<p>This approach is called <strong>Vertical Scaling (Scaling Up)</strong>—making a single machine more powerful.</p>

<p>But there are some major limitations with this approach:</p>
<ol><li><p><strong>Hardware limits →</strong> You can’t keep upgrading a server forever. Every machine has a maximum capacity.</p></li>

<li><p><strong>Cost →</strong> More powerful servers become exponentially more expensive.</p></li>

<li><p><strong>Single Point of Failure (SPOF)</strong> <strong>→</strong> if this one server crashes, the entire system <strong>goes down</strong>.</p></li></ol>

<p>So, while vertical scaling is a quick fix, it’s not a long-term solution for handling high traffic and ensuring system reliability.</p>

<p>Lets look at a better approach—one that makes our system more scalable and fault tolerant.</p>

<hr/>

<h2 id="13-horizontal-scaling"><strong>13. Horizontal Scaling</strong></h2>

<p>Instead of upgrading a single server, what if we <strong>add more servers</strong> to share the load?</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!aryi!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc8bba98b-abf7-42de-b699-0de1bcde0454_1180x426.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!aryi!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc8bba98b-abf7-42de-b699-0de1bcde0454_1180x426.png" alt=""/></a></p>

<p>This approach is called <strong>Horizontal Scaling (Scaling Out)</strong>—where we <strong>distribute the workload across multiple machines</strong>.</p>

<p>This approach is better because:</p>
<ul><li><strong>More servers = More capacity →</strong> The system can handle increasing traffic more effectively.</li>
<li><strong>No Single Point of Failure →</strong> If one server goes down, others can <strong>take over</strong>, improving reliability.</li>
<li><strong>Cost-effective →</strong> Instead of investing in a single, super-expensive machine, we can use multiple affordable ones.</li></ul>

<p>But horizontal scaling introduces a new challenge: <em><strong>how do clients know which server to connect to?</strong></em></p>

<p>This is where a <strong>Load Balancer</strong> comes in.</p>

<hr/>

<h2 id="14-load-balancers"><strong>14. Load Balancers</strong></h2>

<p><a href="https://substackcdn.com/image/fetch/$s_!DnSE!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Faebd5461-34f3-47c5-bc8e-5248bb9a92c8_1304x1050.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!DnSE!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Faebd5461-34f3-47c5-bc8e-5248bb9a92c8_1304x1050.png" alt=""/></a></p>

<p>A <strong>Load Balancer</strong> sits between <strong>clients</strong> and <strong>backend servers</strong>, acting as a <strong>traffic manager</strong> that distributes requests across multiple servers.</p>

<p>If one server crashes, the Load Balancer automatically redirects traffic to another healthy server.</p>

<p><em><strong>But how does a Load Balancer decide which server should handle the next request?</strong></em></p>

<p>It uses <strong>Load Balancing algorithms</strong>, such as:</p>
<ol><li><p><strong>Round Robin</strong> <strong>→</strong> Requests are sent to servers sequentially, one after another in a loop.</p></li>

<li><p><strong>Least Connections</strong> <strong>→</strong> Requests are sent to the server with the fewest active connections.</p></li>

<li><p><strong>and IP Hashing</strong> <strong>→</strong> Requests from the same IP address always go to the <strong>same server</strong>, which helps with <strong>session consistency.</strong></p></li></ol>

<p>Learn more about load balancing algorithms here:</p>

<p><a href="https://blog.algomaster.io/p/load-balancing-algorithms-explained-with-code" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!jpjg!,w_140,h_140,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54114fcc-cce5-4f0d-a678-47f8f0339fd4_1304x1076.png" alt="Load Balancing Algorithms Explained with Code"/></a></p>

<h4 id="load-balancing-algorithms-explained-with-code-https-blog-algomaster-io-p-load-balancing-algorithms-explained-with-code"><strong><a href="https://blog.algomaster.io/p/load-balancing-algorithms-explained-with-code" rel="nofollow ugc">Load Balancing Algorithms Explained with Code</a></strong></h4>

<p><strong><a href="https://substack.com/profile/83602743-ashish-pratap-singh" rel="nofollow ugc">Ashish Pratap Singh</a></strong></p>

<p>·</p>

<p><strong>June 2, 2024</strong></p>

<p><strong><a href="https://blog.algomaster.io/p/load-balancing-algorithms-explained-with-code" rel="nofollow ugc">Read full story</a></strong></p>

<hr/>

<p>So far, we’ve talked about scaling our application servers, but as traffic grows, the volume of data also increases.</p>

<p>At first, we can scale a database <strong>vertically</strong> by adding more CPU, RAM, and storage, but there’s a limit to how much a single machine can handle.</p>

<p>So, let’s explore other <strong>database scaling techniques</strong> that help manage large volumes of data efficiently.</p>

<hr/>

<h2 id="15-database-indexing"><strong>15. Database Indexing</strong></h2>

<p>One of the quickest and most effective ways to speed up database <strong>read queries</strong> is <strong>indexing</strong>.</p>

<p>Think of it like the index page at the back of a book—instead of flipping through every page, you jump directly to the relevant section.</p>

<p>A <strong>database index</strong> works the same way. It’s is a super-efficient lookup table that helps the database quickly locate the required data without scanning the entire table.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!ABC0!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe6543930-e7ce-4dda-b4b2-82db93223aa7_1330x788.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!ABC0!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe6543930-e7ce-4dda-b4b2-82db93223aa7_1330x788.png" alt=""/></a></p>

<p>An index stores column values along with pointers to the actual data rows in the table.</p>

<p>Indexes are typically created on <strong>columns that are frequently queried</strong>, such as:</p>
<ul><li>Primary keys</li>
<li>Foreign keys</li>
<li>Columns used in WHERE conditions</li></ul>

<p>But be careful—while indexes <strong>speed up reads, they slow down writes</strong> (<code>INSERT</code>, <code>UPDATE</code>, <code>DELETE</code>) since the index needs to be updated whenever data changes.</p>

<p>That’s why we should <strong>only index the most frequently accessed columns</strong>.</p>

<p>Learn more about Database Indexes here:</p>

<p><a href="https://blog.algomaster.io/p/a-detailed-guide-on-database-indexes" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!ucFq!,w_140,h_140,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe9d51c18-fd89-4320-af15-a1dfd9d4487f_1120x738.png" alt="Database Indexes: A detailed guide"/></a></p>

<h4 id="database-indexes-a-detailed-guide-https-blog-algomaster-io-p-a-detailed-guide-on-database-indexes"><strong><a href="https://blog.algomaster.io/p/a-detailed-guide-on-database-indexes" rel="nofollow ugc">Database Indexes: A detailed guide</a></strong></h4>

<p><strong><a href="https://substack.com/profile/83602743-ashish-pratap-singh" rel="nofollow ugc">Ashish Pratap Singh</a></strong></p>

<p>·</p>

<p><strong>May 5, 2024</strong></p>

<p><strong><a href="https://blog.algomaster.io/p/a-detailed-guide-on-database-indexes" rel="nofollow ugc">Read full story</a></strong></p>

<p>Indexing significantly improves <strong>read performance</strong>, but <em><strong>what if even indexing isn’t enough, and our database can’t handle the growing number of read requests?</strong></em></p>

<p>That’s where our next database scaling technique <strong>Replication</strong> comes in.</p>

<hr/>

<h2 id="16-replication"><strong>16. Replication</strong></h2>

<p>Just like we added more application servers to handle traffic, we can scale our database by creating <strong>copies</strong> of it across multiple servers.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!zLIG!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6d8f5734-fd0b-4591-a1a5-082cea0212ce_970x730.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!zLIG!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6d8f5734-fd0b-4591-a1a5-082cea0212ce_970x730.png" alt=""/></a></p>

<p>Here’s how it works:</p>
<ul><li>We have <strong>one primary database</strong> (also called the <strong>Primary Replica</strong>) that handles all <strong>write operations</strong> (<code>INSERT</code>, <code>UPDATE</code>, <code>DELETE</code>).</li>
<li>We have <strong>multiple read replicas</strong> that handle <strong>read queries (SELECT)</strong>.</li>
<li>Whenever data is written to the primary database, it gets copied to the read replicas so that they stay in sync.</li></ul>

<p>Replication improves the read performance since read requests are spread across multiple replicas, reducing the load on each one.</p>

<p>This also improves availability since if the primary replica fails, a read replica can take over as the new primary.</p>

<p>Replication is great for scaling read heavy applications, <em><strong>but what if we need to scale writes or store huge amounts of data?</strong></em></p>

<hr/>

<h2 id="17-sharding"><strong>17. Sharding</strong></h2>

<p>Let’s say our service now has <strong>millions of users,</strong> and our database has grown to <strong>terabytes of data</strong>.</p>

<p>A <strong>single database server</strong> will eventually struggle to handle all this data efficiently.</p>

<p>Instead of keeping everything in one place, we <strong>split the database into smaller, more manageable pieces</strong> and distribute them across <strong>multiple servers</strong>.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!17Jc!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6e67c3b7-92cd-4f84-854e-e71f45f7efa2_1902x668.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!17Jc!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6e67c3b7-92cd-4f84-854e-e71f45f7efa2_1902x668.png" alt=""/></a></p>

<p>This technique is called <strong>Sharding</strong>.</p>
<ul><li>We divide the database into smaller parts called <strong>shards</strong>.</li>
<li>Each <strong>shard contains a subset</strong> of the total data.</li>
<li>Data is distributed based on a <strong>sharding key</strong> (e.g., <strong>user ID</strong>).</li></ul>

<p>By distributing data this way, we:</p>
<ul><li><strong>Reduce database load</strong> → Each shard handles <strong>only a portion</strong> of queries.</li>
<li><strong>Speed up read and write performance</strong> → Queries are distributed across multiple shards instead of hitting a single database.</li></ul>

<p>Sharding is also referred to as horizontal partitioning since it splits data by rows.</p>

<p>If you want to learn more about Sharding, checkout this article:</p>

<p><a href="https://blog.algomaster.io/p/what-is-database-sharding" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!Zxuc!,w_140,h_140,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc8c20c76-23d3-40a9-af20-5c9b8957fe63_1210x1068.png" alt="What is Database Sharding?"/></a></p>

<h4 id="what-is-database-sharding-https-blog-algomaster-io-p-what-is-database-sharding"><strong><a href="https://blog.algomaster.io/p/what-is-database-sharding" rel="nofollow ugc">What is Database Sharding?</a></strong></h4>

<p><strong><a href="https://substack.com/profile/83602743-ashish-pratap-singh" rel="nofollow ugc">Ashish Pratap Singh</a></strong></p>

<p>·</p>

<p><strong>May 12, 2024</strong></p>

<p><strong><a href="https://blog.algomaster.io/p/what-is-database-sharding" rel="nofollow ugc">Read full story</a></strong></p>

<p><em><strong>But what if the issue isn’t the number of rows, but rather the number of columns?</strong></em></p>

<p>In such cases, we use <strong>Vertical Partitioning</strong>, where we <strong>split the database by columns</strong>. Let’s explore that next.</p>

<hr/>

<h2 id="18-vertical-partitioning"><strong>18. Vertical Partitioning</strong></h2>

<p>Imagine we have a <strong>User table</strong> that stores:</p>
<ul><li>profile details (name, email, profile picture)</li>
<li>login history (last_login, IP addresses)</li>
<li>and billing information (billing address, payment details)</li></ul>

<p>As this table <strong>grows</strong>, queries become <strong>slower</strong> because the database must scan <strong>many columns</strong> even when a request only needs a <strong>few specific fields.</strong></p>

<p>To optimize this, we use <strong>Vertical Partitioning where we split user table into smaller, more focused tables</strong> based on usage patterns.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!7S9u!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F75daa5e9-2a22-440f-bba3-f1336f613339_2282x1006.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!7S9u!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F75daa5e9-2a22-440f-bba3-f1336f613339_2282x1006.png" alt=""/></a></p>
<ul><li><strong>User_Profile</strong> → Stores name, email, profile picture.</li>
<li><strong>User_Login</strong> → Stores login timestamps.</li>
<li><strong>User_Billing</strong> → Stores billing address, payment details.</li></ul>

<p>This improves <strong>query performance</strong> since each request only scans <strong>relevant columns</strong> instead of the entire <strong>table</strong>.</p>

<p>It reduces <strong>unnecessary disk I/O</strong>, making data retrieval quicker.</p>

<p>However, no matter how much we optimize the database, <strong>retrieving data from disk is always slower than retrieving from memory</strong>.</p>

<p><em><strong>What if we could store frequently accessed data in memory for lightning-fast access?</strong></em></p>

<p>This is called <strong>caching</strong>.</p>

<hr/>

<h2 id="19-caching"><strong>19. Caching</strong></h2>

<p><strong>Caching</strong> is used to optimize the performance of a system by <strong>storing frequently accessed data in memory</strong> instead of repeatedly fetching it from the database.</p>

<p>One of the most common caching strategies is the <strong>Cache Aside Pattern.</strong></p>

<p><a href="https://substackcdn.com/image/fetch/$s_!y--V!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcc5992eb-9290-49c7-ad8d-594a7560ef41_2200x1102.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!y--V!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcc5992eb-9290-49c7-ad8d-594a7560ef41_2200x1102.png" alt=""/></a></p>

<p>Here’s how it works:</p>
<ol><li><p>When a user <strong>requests a data</strong>, the application first check the <strong>cache</strong>.</p></li>

<li><p>If the data is <strong>in the cache</strong>, it’s returned <strong>instantly, avoiding a database call</strong>.</p></li>

<li><p>If the data <strong>is not in the cache</strong>, the application <strong>retrieves it from the database</strong>, stores it in the cache for future requests, and returns it to the user.</p></li>

<li><p>Next time, the same data is requested, it’s served <strong>directly from cache</strong>, making the request much faster.</p></li></ol>

<p>To prevent outdated data from being served, we use <strong>Time-to-Live (TTL)</strong>—an expiration time set on cached data so it gets <strong>automatically refreshed</strong> after a certain period.</p>

<blockquote><p>Popular caching tools include <strong>Redis and Memcached.</strong></p></blockquote>

<p>If you want to learn more about caching strategies, check out this article:</p>

<p><a href="https://blog.algomaster.io/p/top-5-caching-strategies-explained" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!qQlQ!,w_140,h_140,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1b38086e-3798-4d29-8231-33c8e84ffbc6_665x468.png" alt="Top 5 Caching Strategies Explained"/></a></p>

<h4 id="top-5-caching-strategies-explained-https-blog-algomaster-io-p-top-5-caching-strategies-explained"><strong><a href="https://blog.algomaster.io/p/top-5-caching-strategies-explained" rel="nofollow ugc">Top 5 Caching Strategies Explained</a></strong></h4>

<p><strong><a href="https://substack.com/profile/83602743-ashish-pratap-singh" rel="nofollow ugc">Ashish Pratap Singh</a></strong></p>

<p>·</p>

<p><strong>October 24, 2024</strong></p>

<p><strong><a href="https://blog.algomaster.io/p/top-5-caching-strategies-explained" rel="nofollow ugc">Read full story</a></strong></p>

<p>Lets look at the next database scaling technique.</p>

<hr/>

<h2 id="20-denormalization"><strong>20. Denormalization</strong></h2>

<p>Most relational databases use <strong>Normalization</strong> to store data efficiently by breaking it into separate tables.</p>

<p>For example, in an e-commerce system:</p>
<ul><li>The <strong>Users table</strong> stores user details.</li>
<li>The <strong>Orders table</strong> stores their orders.</li>
<li>The <strong>Products table</strong> stores product details.</li></ul>

<p>While this <strong>reduces redundancy</strong>, it also <strong>introduces joins.</strong> When retrieving data from multiple tables, the database must <strong>combine them using JOIN operations</strong>, which can slow down queries as the dataset grows.</p>

<pre><code>SELECT o.order_id, u.name, u.email, o.product, o.amount
FROM orders o
JOIN users u ON o.user_id = u.user_id;
</code></pre>

<p><strong>Denormalization</strong> reduces the number of joins by combining related data into a single table, even if it means some data gets duplicated.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!RPfd!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F92757591-63fa-4a56-a7cc-e65b739b057a_1750x1398.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!RPfd!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F92757591-63fa-4a56-a7cc-e65b739b057a_1750x1398.png" alt=""/></a></p>

<p><strong>Example:</strong> Instead of keeping <code>Users</code> and <code>Orders</code> in separate tables, we create <code>UserOrders</code> table that stores user details along with their latest orders.</p>

<p>Now, when retrieving a user’s order history, we don’t need a <strong>JOIN operation</strong>—the data is already stored together leading to faster queries and better read performance.</p>

<pre><code>SELECT order_id, user_name AS name, user_email AS email, product, amount
FROM orders;
</code></pre>

<p>Denormalization is often used in read-heavy applications where speed is more critical but the downside is it leads to <strong>increases storage usage</strong> and more <strong>complex update requests</strong>.</p>

<hr/>

<h2 id="21-cap-theorem"><strong>21.</strong> CAP Theorem</h2>

<p>As we scale our system across multiple <strong>servers, databases, and data centers</strong>, we enter the world of <strong>distributed systems</strong>.</p>

<p>One of the fundamental principles of distributed systems is the <strong>CAP Theorem</strong>, which states that: No distributed system can achieve all three of the following at the same time:</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!-Rqf!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcd38a5f6-aca6-44e9-aa69-321ed4b04e03_1138x904.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!-Rqf!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcd38a5f6-aca6-44e9-aa69-321ed4b04e03_1138x904.png" alt=""/></a></p>
<ul><li><strong>Consistency ©</strong> <strong>→</strong> Every node always returns the <strong>most recent data</strong>.</li>
<li><strong>Availability (A)</strong> <strong>→</strong> The system <strong>always responds</strong> to requests, even if some nodes are down (but the data may not be the latest).</li>
<li><strong>Partition Tolerance (P)</strong> <strong>→</strong> The system continues operating even if there’s a <strong>network failure</strong> between nodes.</li></ul>

<p>Since <strong>network failures (P) are inevitable</strong>, we must choose between:</p>
<ul><li><strong>Consistency + Partition Tolerance (CP)</strong> → Ensures every request gets the latest data but may reject requests during failures. Example: <strong>SQL databases like MySQL</strong>.</li>
<li><strong>Availability + Partition Tolerance (AP)</strong> → Ensures the system always responds, even if some data is stale. Example: <strong>NoSQL databases like Cassandra and DynamoDB</strong>.</li></ul>

<p>To learn more about CAP theorem, check out this article:</p>

<p><a href="https://blog.algomaster.io/p/cap-theorem-explained" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!92J7!,w_140,h_140,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F78cd131d-3a18-4c4d-88e1-18834b191ef2_1138x904.png" alt="CAP Theorem Explained"/></a></p>

<h4 id="cap-theorem-explained-https-blog-algomaster-io-p-cap-theorem-explained"><strong><a href="https://blog.algomaster.io/p/cap-theorem-explained" rel="nofollow ugc">CAP Theorem Explained</a></strong></h4>

<p><strong><a href="https://substack.com/profile/83602743-ashish-pratap-singh" rel="nofollow ugc">Ashish Pratap Singh</a></strong></p>

<p>·</p>

<p><strong>July 31, 2024</strong></p>

<p><strong><a href="https://blog.algomaster.io/p/cap-theorem-explained" rel="nofollow ugc">Read full story</a></strong></p>

<p>In distributed NoSQL databases, achieving <strong>instant consistency</strong> across all servers <strong>is too slow</strong>.</p>

<p>Instead, we use <strong>Eventual Consistency</strong>—which means:</p>
<ul><li><strong>Not all nodes are updated instantly</strong>, but given enough time, they <strong>eventually</strong> sync and return the same data.</li>
<li>This allows the system to remain <strong>highly available and fast</strong>, even under extreme loads.</li></ul>

<p><strong>How Eventual Consistency Works:</strong></p>
<ol><li><p>A user updates data in <strong>one replica</strong> of the database.</p></li>

<li><p>The system <strong>immediately</strong> acknowledges the update, ensuring <strong>high availability</strong>.</p></li>

<li><p>The update is then <strong>propagated asynchronously to other replicas</strong>.</p></li>

<li><p>After a short delay, all replicas have the latest data, ensuring <strong>consistency over time</strong>.</p></li></ol>

<hr/>

<h2 id="22-blob-storage"><strong>22. Blob Storage</strong></h2>

<p>Most modern applications don’t just store text records, they also need to handle <strong>images</strong>, <strong>videos</strong>, <strong>pdfs</strong> and other <strong>large files</strong>.</p>

<p>But here’s the problem: <strong>Traditional databases are not designed to store large, unstructured files efficiently</strong>.</p>

<p><em><strong>So, what’s the solution?</strong></em></p>

<p>We use <strong>Blob Storage like Amazon S3</strong>—a <strong>highly scalable and cost-effective</strong> way to store <strong>large, unstructured files</strong> in the cloud.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!YYzQ!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcc338f10-3918-49f5-a828-fced5e905acb_298x306.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!YYzQ!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcc338f10-3918-49f5-a828-fced5e905acb_298x306.png" alt=""/></a></p>

<p>Blobs are the individual files like images, videos or documents.</p>

<p>These blobs are stored inside <strong>logical containers or buckets</strong> in the cloud.</p>

<p>Each file gets a <strong>unique URL</strong>, making it easy to retrieve and serve over the web.</p>

<blockquote><p><strong>Example:</strong> <a href="https://my-bucket-name.s3.amazonaws.com/videos/tutorial.mp4" rel="nofollow ugc">https://my-bucket-name.s3.amazonaws.com/videos/tutorial.mp4</a></p></blockquote>

<p>There are several advantages with using blob storage like:</p>
<ul><li><strong>Scalability</strong> → It can store petabytes of data effortlessly.</li>
<li><strong>Pay-as-you-go pricing</strong> → You only pay for storage and retrieval that you actually use.</li>
<li><strong>Automatic replication</strong> → Data is copied across multiple data centers and availability zones for durability.</li>
<li><strong>Easy access</strong> → Files can be retrieved using REST APIs or direct URLs.</li></ul>

<p>A common use case is to stream audio or video files to user application in real-time.</p>

<p>But streaming directly from blob-storage can be slow, especially if the data is stored in a <strong>distant location</strong>.</p>

<hr/>

<h2 id="23-cdn"><strong>23. CDN</strong></h2>

<p>For example, imagine you’re in <strong>India</strong> trying to watch a YouTube video that’s hosted on a server in <strong>California</strong>.</p>

<p>Since the video data has to <strong>travel across the world</strong>, this could lead to <strong>buffering and slow load times</strong>.</p>

<p>A <strong>Content Delivery Network (or CDN)</strong> solves this problem by <strong>delivering content faster</strong> to users based on their location.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!sN05!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa09696e9-f98e-47bd-9eb9-08a20c60464d_5667x2834.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!sN05!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa09696e9-f98e-47bd-9eb9-08a20c60464d_5667x2834.png" alt="Map of globally distributed servers serving content - What is a CDN" title="Map of globally distributed servers serving content - What is a CDN"/></a></p>

<p>source: <a href="https://www.cloudflare.com/learning/cdn/what-is-a-cdn/" rel="nofollow ugc">https://www.cloudflare.com/learning/cdn/what-is-a-cdn/</a></p>

<p>A CDN is a global network of distributed servers that work together to deliver <strong>web content</strong> (like HTML pages, JavaScript files, stylesheets, images, and videos) to users based on their <strong>geographic location</strong>.</p>

<p>Instead of serving content from a <strong>single data center</strong>, a CDN <strong>caches static contents</strong> on multiple <strong>edge servers located worldwide</strong>.</p>

<p>When a user requests content, the <strong>nearest CDN server</strong> delivers it instead of reaching all the way to the <strong>origin server</strong>.</p>

<p>Since content is served from the <strong>closest CDN node</strong>, users experience <strong>faster load times</strong> with minimal buffering.</p>

<p>To learn more about CDN, check out this article:</p>

<p><a href="https://blog.algomaster.io/p/content-delivery-networks" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!fUV6!,w_140,h_140,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc1ef9c83-5fd9-46e3-8469-c2a43f8b7dd2_1670x1046.png" alt="What is a Content Delivery Network?"/></a></p>

<h4 id="what-is-a-content-delivery-network-https-blog-algomaster-io-p-content-delivery-networks"><strong><a href="https://blog.algomaster.io/p/content-delivery-networks" rel="nofollow ugc">What is a Content Delivery Network?</a></strong></h4>

<p><strong><a href="https://substack.com/profile/83602743-ashish-pratap-singh" rel="nofollow ugc">Ashish Pratap Singh</a></strong></p>

<p>·</p>

<p><strong>March 4, 2025</strong></p>

<p><strong><a href="https://blog.algomaster.io/p/content-delivery-networks" rel="nofollow ugc">Read full story</a></strong></p>

<hr/>

<h2 id="24-websockets"><strong>24. WebSockets</strong></h2>

<p>Most web applications use <strong>HTTP</strong>, which follows a <strong>request-response</strong> model.</p>
<ol><li><p>The client sends a request.</p></li>

<li><p>The <strong>server</strong> processes the request and sends a response.</p></li>

<li><p>If the client needs new data, it must <strong>send another request</strong>.</p></li></ol>

<p>This works fine for <strong>static web pages</strong> but it’s <strong>too slow and inefficient for real-time applications</strong> like: live chat apps, stock market dashboards and online multiplayer games.</p>

<p>With HTTP, the only way to get real-time updates is through <strong>polling</strong>—sending repeated requests every few seconds.</p>

<p>But polling is inefficient because it increases server load and wastes bandwidth, as <strong>most responses are empty</strong> (when there’s no new data).</p>

<p>WebSockets solve this problem by allowing <strong>continuous, two-way communication</strong> between the <strong>client and server</strong> over a <strong>single persistent connection</strong>.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!5FHd!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F57a93491-9dae-4a6d-a661-81ff1437c256_1071x969.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!5FHd!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F57a93491-9dae-4a6d-a661-81ff1437c256_1071x969.png" alt=""/></a></p>

<p>Here is how WebSockets work:</p>
<ol><li><p>The <strong>client initiates a WebSocket connection</strong> with the server.</p></li>

<li><p>Once established, the connection <strong>remains open</strong>.</p></li>

<li><p><strong>The server can push updates</strong> to the client <strong>at any time</strong>, without waiting for a request.</p></li>

<li><p>The client can also send messages <strong>instantly to the server.</strong></p></li></ol>

<p>This enables real-time interactions and eliminates the need for polling.</p>

<p>To learn more about WebSockets, check out this article:</p>

<p><a href="https://blog.algomaster.io/p/websockets" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!yu18!,w_140,h_140,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7bde57be-682f-4254-944b-bcade624e544_1008x908.png" alt="What are WebSockets and Why are they Used?"/></a></p>

<h4 id="what-are-websockets-and-why-are-they-used-https-blog-algomaster-io-p-websockets"><strong><a href="https://blog.algomaster.io/p/websockets" rel="nofollow ugc">What are WebSockets and Why are they Used?</a></strong></h4>

<p><strong><a href="https://substack.com/profile/83602743-ashish-pratap-singh" rel="nofollow ugc">Ashish Pratap Singh</a></strong></p>

<p>·</p>

<p><strong>August 28, 2024</strong></p>

<p><strong><a href="https://blog.algomaster.io/p/websockets" rel="nofollow ugc">Read full story</a></strong></p>

<p>WebSockets enable real-time communication between a client and a server, <em><strong>but what if a server needs to notify another server when an event occurs?</strong></em></p>

<p>Example:</p>
<ul><li>When a user makes a payment, Stripe needs to notify your application <strong>instantly</strong>.</li>
<li>If someone pushes code to GitHub, a CI/CD system (e.g., Jenkins) should be triggered automatically.</li></ul>

<p>Enter Webhooks.</p>

<hr/>

<h2 id="25-webhooks"><strong>25. Webhooks</strong></h2>

<p>Instead of constantly <strong>polling an API</strong> to check if an event has occured, <strong>Webhooks</strong> allow a server to <strong>send an HTTP request</strong> to another server as soon as the event occurs.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!Bipe!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa833712f-f11d-4d48-b0b4-47e5616c8351_1890x766.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!Bipe!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa833712f-f11d-4d48-b0b4-47e5616c8351_1890x766.png" alt=""/></a></p>

<p>Here’s how it works:</p>
<ul><li>The <strong>receiver (your app)</strong> registers a webhook URL with the <strong>provider (e.g., Stripe, GitHub, Twilio)</strong>.</li>
<li>When an event occurs (e.g., user makes a payment), the <strong>provider sends an HTTP POST request to the webhook URL</strong> with event details.</li>
<li>Your app <strong>processes the incoming request</strong> and updates data accordingly.</li></ul>

<p>This saves server resources and reduces unnecessary API calls.</p>

<hr/>

<h2 id="26-microservices"><strong>26. Microservices</strong></h2>

<p>Traditionally, applications were built using a <strong>monolithic architecture</strong>, where:</p>
<ul><li>All features (e.g., authentication, payments, orders, shipping) are inside <strong>one large codebase</strong>.</li>
<li>If one part of the system <strong>fails or needs scaling</strong>, the <strong>entire system</strong> is affected.</li>
<li><strong>Deployment is risky</strong>—one bad update can take down the entire app.</li></ul>

<p><strong>Example:</strong> Imagine an e-commerce app where the <strong>order</strong>, <strong>payment, inventory</strong>, and <strong>shipping</strong> modules are all tightly connected in a <strong>single codebase</strong>.</p>

<p>If the inventory system crashes, the entire app could go down.</p>

<p>Monoliths work <strong>fine for small applications</strong>, but for <strong>large-scale systems</strong>, they become <strong>hard to manage, scale, and deploy</strong>.</p>

<p>The solution is to break down your application into smaller, independent services called <strong>micro-services</strong> that work together.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!N6tU!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9b2ef6df-0f55-4663-ad2a-7f8a11121753_2136x826.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!N6tU!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9b2ef6df-0f55-4663-ad2a-7f8a11121753_2136x826.png" alt=""/></a></p>

<p>Each <strong>microservice</strong>:</p>
<ol><li><p>Handles <strong>a single responsibility</strong></p></li>

<li><p>Has its <strong>own database and logic</strong>, so it can scale <strong>independently</strong>.</p></li>

<li><p>Communicates with other microservices using <strong>APIs or message queues</strong>.</p></li></ol>

<p>This way services can be scaled and deployed individually without affecting the entire system.</p>

<p>However, when multiple microservices need to communicate, direct <strong>API calls</strong> aren’t always efficient—this is where <strong>Message Queues</strong> come in.</p>

<hr/>

<h2 id="27-message-queues"><strong>27. Message Queues</strong></h2>

<p>In a <strong>monolithic system</strong>, functions call each other <strong>directly</strong> and wait for a response.</p>

<p>But in a <strong>microservices-based system</strong>, this approach is inefficient because:</p>
<ul><li>If one service is <strong>slow</strong> or <strong>down</strong>, everything waits.</li>
<li>High traffic can <strong>overload</strong> a single service.</li>
<li><strong>Synchronous communication</strong> (waiting for immediate responses) doesn’t scale well.</li></ul>

<p>A <strong>Message Queue</strong> enables services to <strong>communicate asynchronously</strong>, allowing requests to be processed <strong>without blocking</strong> other operations.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!uKwi!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3d148e2f-f1a4-4b94-8027-fa19ea6b5c61_1590x728.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!uKwi!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3d148e2f-f1a4-4b94-8027-fa19ea6b5c61_1590x728.png" alt=""/></a></p>

<p><strong>Here’s How It Works:</strong></p>
<ol><li><p>A <strong>producer</strong> (e.g., checkout service) <strong>places a message in the queue</strong> (e.g., “Process Payment”).</p></li>

<li><p>The <strong>queue temporarily holds the message</strong> until a <strong>consumer</strong> (e.g., payment service) is ready to process it.</p></li>

<li><p>The <strong>consumer retrieves the message</strong> and processes it.</p></li></ol>

<p>Using message queues, we can decouple services and improve the scalability and fault tolerance.</p>

<blockquote><p>Common message queue systems include: Apache Kafka, Amazon SQS and RabbitMQ.</p></blockquote>

<p>To learn more about Message Queues, check out this article:</p>

<p><a href="https://blog.algomaster.io/p/message-queues" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!hW_t!,w_140,h_140,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2a5f36fe-a10e-4b34-924c-5f32fd65adda_1194x764.png" alt="What are Message Queues and When to Use Them?"/></a></p>

<h4 id="what-are-message-queues-and-when-to-use-them-https-blog-algomaster-io-p-message-queues"><strong><a href="https://blog.algomaster.io/p/message-queues" rel="nofollow ugc">What are Message Queues and When to Use Them?</a></strong></h4>

<p><strong><a href="https://substack.com/profile/83602743-ashish-pratap-singh" rel="nofollow ugc">Ashish Pratap Singh</a></strong></p>

<p>·</p>

<p><strong>August 18, 2024</strong></p>

<p><strong><a href="https://blog.algomaster.io/p/message-queues" rel="nofollow ugc">Read full story</a></strong></p>

<p>Using message queues, we can prevent overload on internal services within our system.</p>

<p><em><strong>But, how do we prevent overload for the public APIs and services we deploy.</strong></em></p>

<p>We use <strong>rate limiting</strong>.</p>

<hr/>

<h2 id="28-rate-limiting"><strong>28. Rate Limiting</strong></h2>

<p>Imagine a <strong>bot starts making thousands of requests per second</strong> to your website.</p>

<p>Without restrictions, this could:</p>
<ul><li><strong>Crash your servers</strong> by consuming all available resources.</li>
<li><strong>Increase cloud costs</strong> due to excessive API usage.</li>
<li>and <strong>degrade performance</strong> for legitimate users.</li></ul>

<p>Rate Limiting <strong>restricts the number of requests</strong> a client can send within a specific time frame.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!H4QU!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd6c2b184-bb2e-421a-bf4d-03bd8bb80840_1704x436.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!H4QU!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd6c2b184-bb2e-421a-bf4d-03bd8bb80840_1704x436.png" alt=""/></a></p>

<p><strong>Here’s How It Works:</strong></p>
<ol><li><p>Every user or IP address is assigned a <strong>request quota</strong> (e.g., <strong>100 requests per minute</strong>).</p></li>

<li><p>If they <strong>exceed this limit</strong>, the server <strong>blocks additional requests temporarily</strong> and <strong>returns an error (HTTP 429 – Too Many Requests)</strong>.</p></li></ol>

<p>There are various rate limiting algorithms. Some of the popular ones are:</p>
<ul><li><strong>Fixed Window</strong> → Limits requests based on a fixed time window (e.g., 100 requests per minute).</li>
<li><strong>Sliding Window</strong> → More flexible version that dynamically adjusts limits to smooth out request bursts.</li>
<li><strong>Token Bucket</strong> → Users get <strong>tokens</strong> for requests, which <strong>replenish over time</strong> at a fixed rate.</li></ul>

<p>To learn more about rate limiting algorithms, checkout this article:</p>

<p><a href="https://blog.algomaster.io/p/rate-limiting-algorithms-explained-with-code" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!vVHH!,w_140,h_140,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0815d30e-dc9c-4ff4-9eb8-ac76d21ba52d_1048x684.png" alt="Rate Limiting Algorithms Explained with Code"/></a></p>

<h4 id="rate-limiting-algorithms-explained-with-code-https-blog-algomaster-io-p-rate-limiting-algorithms-explained-with-code"><strong><a href="https://blog.algomaster.io/p/rate-limiting-algorithms-explained-with-code" rel="nofollow ugc">Rate Limiting Algorithms Explained with Code</a></strong></h4>

<p><strong><a href="https://substack.com/profile/83602743-ashish-pratap-singh" rel="nofollow ugc">Ashish Pratap Singh</a></strong></p>

<p>·</p>

<p><strong>July 17, 2024</strong></p>

<p><strong><a href="https://blog.algomaster.io/p/rate-limiting-algorithms-explained-with-code" rel="nofollow ugc">Read full story</a></strong></p>

<p>We don’t need to implement our own rate limiting system – this can be handled by something called an <strong>API gateway</strong>.</p>

<hr/>

<h2 id="29-api-gateways"><strong>29. API Gateways</strong></h2>

<p>An API Gateway is a <strong>centralized service</strong> that handles <strong>authentication, rate limiting, logging and monitoring, and request routing</strong>.</p>

<p>Imagine a <strong>microservices-based application</strong> with multiple services.</p>

<p>Instead of exposing each service <strong>directly</strong>, an <strong>API Gateway</strong> acts as a <strong>single entry point</strong> for all client requests.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!_AFR!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb2f627d6-6141-413f-a528-4726d8869a0b_2218x1166.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!_AFR!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb2f627d6-6141-413f-a528-4726d8869a0b_2218x1166.png" alt=""/></a></p>

<p><strong>How API Gateways Work:</strong></p>
<ol><li><p>The <strong>client sends a request</strong> to the API Gateway.</p></li>

<li><p>The Gateway <strong>validates the request</strong> (e.g., authentication, rate limits).</p></li>

<li><p>It <strong>routes the request</strong> to the appropriate micro-service.</p></li>

<li><p>The response is sent back <strong>through the Gateway</strong> to the client.</p></li></ol>

<p>API gateway simplifies API management and improves scalability and security.</p>

<blockquote><p>Popular API Gateway solutions include <strong>NGINX, Kong and AWS API Gateway.</strong></p></blockquote>

<p>If you want to learn more about API gateways, checkout this article:</p>

<p><a href="https://blog.algomaster.io/p/what-is-an-api-gateway" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!_ZNj!,w_140,h_140,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcb0c8192-4b1b-4834-b068-20d8cfb65050_1856x1412.png" alt="What is an API Gateway?"/></a></p>

<h4 id="what-is-an-api-gateway-https-blog-algomaster-io-p-what-is-an-api-gateway"><strong><a href="https://blog.algomaster.io/p/what-is-an-api-gateway" rel="nofollow ugc">What is an API Gateway?</a></strong></h4>

<p><strong><a href="https://substack.com/profile/83602743-ashish-pratap-singh" rel="nofollow ugc">Ashish Pratap Singh</a></strong></p>

<p>·</p>

<p><strong>December 8, 2024</strong></p>

<p><strong><a href="https://blog.algomaster.io/p/what-is-an-api-gateway" rel="nofollow ugc">Read full story</a></strong></p>

<hr/>

<h2 id="30-idempotency"><strong>30. Idempotency</strong></h2>

<p>In distributed systems, <strong>network failures</strong> and <strong>service retries</strong> are common. If a user <strong>accidentally refreshes a payment page</strong>, the system might receive <strong>two payment requests</strong> instead of one.</p>

<p><a href="https://substackcdn.com/image/fetch/$s_!czYH!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F637607fe-3d2c-49db-be80-3d247bea6ab8_1212x394.png" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!czYH!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F637607fe-3d2c-49db-be80-3d247bea6ab8_1212x394.png" alt=""/></a></p>

<p>Idempotency ensures that <strong>repeated requests</strong> produce the <strong>same result</strong> as if the request was made <strong>only once</strong>.</p>

<p><strong>Here’s how it works:</strong></p>
<ol><li><p>Each request is assigned a <strong>unique ID</strong> (e.g., <code>request_1234</code>).</p></li>

<li><p>Before processing, the system checks if the request <strong>has already been handled</strong>.</p></li>

<li><p>If yes → It <strong>ignores the duplicate request</strong>.</p></li>

<li><p>If no → It <strong>processes the request normally</strong>.</p></li></ol>

<p>Idempotency prevents duplicate transactions and ensures data consistency in distributed systems.</p>

<p>If you want to learn more about idempotency, checkout this article:</p>

<p><a href="https://blog.algomaster.io/p/idempotency-in-distributed-systems" rel="nofollow ugc"><img src="https://substackcdn.com/image/fetch/$s_!yjks!,w_140,h_140,c_fill,f_auto,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F538d3bda-046f-48f0-b62d-730327bf3bd7_954x696.png" alt="What is Idempotency in Distributed Systems?"/></a></p>

<h4 id="what-is-idempotency-in-distributed-systems-https-blog-algomaster-io-p-idempotency-in-distributed-systems"><strong><a href="https://blog.algomaster.io/p/idempotency-in-distributed-systems" rel="nofollow ugc">What is Idempotency in Distributed Systems?</a></strong></h4>

<p><strong><a href="https://substack.com/profile/83602743-ashish-pratap-singh" rel="nofollow ugc">Ashish Pratap Singh</a></strong></p>

<p>·</p>

<p><strong>November 3, 2024</strong></p>

<p><strong><a href="https://blog.algomaster.io/p/idempotency-in-distributed-systems" rel="nofollow ugc">Read full story</a></strong></p>

<hr/>

<p>Thank you so much for reading!</p>

<p>If you found it valuable, hit a like ❤️ and consider subscribing for more such content every week.</p>

<p>This post is public so feel free to share it.</p>

<p><strong><a href="https://blog.algomaster.io/p/30-system-design-concepts?utm_source=substack&amp;utm_medium=email&amp;utm_content=share&amp;action=share" rel="nofollow ugc">Share</a></strong></p>

<hr/>

<p><strong>P.S.</strong> If you’re enjoying this newsletter and want to get even more value, consider becoming a <strong><a href="https://blog.algomaster.io/subscribe" rel="nofollow ugc">paid subscriber</a></strong>.</p>

<p>As a paid subscriber, you&#39;ll unlock all <strong>premium articles</strong> and gain full access to all <strong><a href="https://algomaster.io/newsletter/paid/resources" rel="nofollow ugc">premium courses</a></strong> on <strong><a href="https://algomaster.io/" rel="nofollow ugc">algomaster.io</a></strong>.</p>
]]></content:encoded>
      <guid>https://christova.writeas.com/system-design-concepts</guid>
      <pubDate>Wed, 29 Apr 2026 00:44:29 +0000</pubDate>
    </item>
  </channel>
</rss>