christova

#kafka

1. HTTP GET This retrieves a resource from the server. It is idempotent. Multiple identical requests return the same result. '

2. HTTP PUT This updates or Creates a resource. It is idempotent. Multiple identical requests will update the same resource.

3. HTTP POST This is used to create new resources. It is not idempotent, making two identical POST will duplicate the resource creation.

4. HTTP DELETE This is used to delete a resource. It is idempotent. Multiple identical requests will delete the same resource.

5. HTTP PATCH The PATCH method applies partial modifications to a resource.

6. HTTP HEAD The HEAD method asks for a response identical to a GET request but without the response body.

7. HTTP CONNECT The CONNECT method establishes a tunnel to the server identified by the target resource.

8. HTTP OPTIONS This describes the communication options for the target resource.

9. HTTP TRACE This performs a message loop-back test along the path to the target resource.

#HTTPMethods #HTTP #HTTPS

Top 9 Architectural Patterns for Data and Communication Flow

#SystemIntegration

Kafka was originally built for massive log processing. It retains messages until expiration and lets consumers pull messages at their own pace. Let’s review the popular Kafka use cases.

- Log processing and analysis

- Data streaming in recommendations

- System monitoring and alerting

- CDC (Change data capture)

- System migration

#kafka

Step 1 – The client sends an HTTP request to the API gateway.

Step 2 – The API gateway parses and validates the attributes in the HTTP request.

Step 3 – The API gateway performs allow-list/deny-list checks.

Step 4 – The API gateway talks to an identity provider for authentication and authorization.

Step 5 – The rate limiting rules are applied to the request. If it is over the limit, the request is rejected.

Steps 6 and 7 – Now that the request has passed basic checks, the API gateway finds the relevant service to route to by path matching.

Step 8 – The API gateway transforms the request into the appropriate protocol and sends it to backend microservices.

Steps 9-12: The API gateway can handle errors properly, and deals with faults if the error takes a longer time to recover (circuit break). It can also leverage ELK (Elastic-Logstash-Kibana) stack for logging and monitoring. We sometimes cache data in the API gateway.

Over to you: 1) What’s the difference between a load balancer and an API gateway? 2) Do we need to use different API gateways for PC, mobile and browser separately?

#APIGateway

#database

• 1. add(E element) – Adds the specified element to the end of the list.
• 2. addAll(Collection<? extends E> c) – Adds all elements of the specified collection to the end of the list.
• 3. remove(Object o) – Removes the first occurrence of the specified element from the list.
• 4. remove(int index) – Removes the element at the specified position in the list.
• 5. get(int index) – Returns the element at the specified position in the list.
• 6. set(int index, E element) – Replaces the element at the specified position in the list with the specified element.
• 7. indexOf(Object o) – Returns the index of the first occurrence of the specified element in the list.
• 8. contains(Object o) – Returns true if the list contains the specified element.
• 9. size() – Returns the number of elements in the list.
• 10. isEmpty() – Returns true if the list contains no elements.
• 11. clear() – Removes all elements from the list.
• 12. toArray() – Returns an array containing all the elements in the list.
• 13. subList(int fromIndex, int toIndex) – Returns a view of the portion of the list between the specified fromIndex, inclusive, and toIndex, exclusive.
• 14. addAll(int index, Collection<? extends E> c) – Inserts all elements of the specified collection into the list, starting at the specified position.
• 15. iterator() – Returns an iterator over the elements in the list.
• 16. sort(Comparator<? super E> c) – Sorts the elements of the list according to the specified comparator.
• 17. replaceAll(UnaryOperator operator) – Replaces each element of the list with the result of applying the given operator.
• 18. forEach(Consumer<? super E> action) – Performs the given action for each element of the list until all elements have been processed or the action throws an exception.

First let me explain what is an API in simple terms

Imagine you're a customer (an application or program) sitting at a table (a device), and you want to order food (request data or services). Instead of going into the kitchen (the system or server) yourself, you interact with the waiter (API), who takes your order and communicates it to the kitchen on your behalf. The kitchen then prepares the food (performs the requested task), and the waiter brings it back to you.

Similarly, an API allows different software applications to communicate and share information or perform specific tasks without knowing the intricate details of each other's internal workings. It acts as a bridge, enabling applications to request and exchange data or functionalities in a standardized way.

---

Ok now lets explore 6 common ways to build APIs.

1. REST: Architecture using standard HTTP methods for CRUD operations on resources.

2. GraphQL: Query language enabling clients to request specific data structures.

3. WebSocket: Full-duplex communication protocol for real-time applications.

4. gRPC: High-performance RPC framework for efficient communication between services.

5. MQTT: Lightweight messaging protocol for low-bandwidth, high-latency networks, common in IoT.

6. Serverless: Cloud computing model where developers focus on code, and the provider manages infrastructure.

#APIs

JUnit

As one of the most widely used frameworks, JUnit is synonymous with unit testing in the Java realm. Its simplicity and user-friendliness make it an indispensable tool for developers. Its seamless integration with IDEs and build tools enables continuous feedback on the code's health.

TestNG

TestNG extends beyond unit tests to cover a wider range of testing needs, including integration, functional, and end-to-end tests. Its flexibility and scalability cater to both simple scenarios and complex testing environments.

Selenium

When it comes to automation for web applications, Selenium stands tall. Its ability to simulate user interactions in web browsers across platforms is unparalleled. Its compatibility with JUnit and TestNG harnesses the strengths of these frameworks for comprehensive web testing.

Cucumber

Involving non-technical stakeholders in the development process is crucial, and Cucumber's Behavior-driven development (BDD) approach bridges this gap. It allows the description of software behaviors in plain language, ensuring everyone's understanding and contribution.

Mockito

Unit tests require isolation, and Mockito excels in mocking dependencies, allowing tests to focus on the code under test. It's a powerful ally in maintaining clean test code and offers seamless integration with JUnit and TestNG.

#java #testing

System design is a crucial aspect of software development, focusing on the architecture and integration of various components to ensure scalability, reliability, and efficiency.

In this post, we explore the top 12 building blocks of system design that every developer and architect should be familiar with.

1. Distributed Messaging Queues

A distributed messaging queue facilitates communication between different parts of a system by providing a reliable message broker that handles the exchange of information. Producers send messages to the queue, which are then consumed by various consumers, ensuring decoupled and scalable communication.

2. DNS (Domain Name System)

The DNS translates human-readable domain names (like www.example.com) into IP addresses that computers use to identify each other on the network. This process is crucial for routing traffic to the correct servers and ensuring that web services are accessible to users.

3. Load Balancer

Load balancers distribute incoming network traffic across multiple servers to ensure no single server becomes overwhelmed. This component enhances the reliability and efficiency of applications by balancing the load and improving redundancy.

4. Distributed Caching

Distributed caching stores frequently accessed data in memory, reducing the load on databases and speeding up response times. By caching data close to the client or the server, applications can handle higher loads and deliver content more quickly.

5. Database

Databases store and manage data for applications. They come in two primary forms: relational databases, which organize data into tables with predefined schemas, and document databases, which store data in flexible, JSON-like documents. Both types play crucial roles in different scenarios, depending on the nature of the data and the requirements of the application.

6. Distributed Task Scheduler

A distributed task scheduler manages the execution of tasks across multiple servers. It ensures tasks are executed efficiently and reliably, even in the face of server failures. This component is vital for applications that require regular and reliable task execution.

7. Observability

Observability refers to the ability to measure the internal state of a system by examining its outputs. It encompasses monitoring, logging, and tracing to provide a comprehensive view of system performance and health, enabling quick detection and resolution of issues.

8. Unstructured Data Storage

Unstructured data storage solutions, like BLOB stores, are designed to handle large amounts of unstructured data such as multimedia files, documents, and logs. These systems provide scalable and cost-effective storage options for data that doesn't fit neatly into a traditional database.

9. Scaling Services

Scaling services ensure that an application can handle increased load by adding resources dynamically. This can involve scaling up (adding more power to existing servers) or scaling out (adding more servers). Proper scaling ensures that applications remain responsive and reliable under varying loads.

10. Publish-Subscribe Model (Pub-Sub)

The Pub-Sub model allows for asynchronous communication between services. Publishers send messages to topics without knowledge of the subscribers. Subscribers receive messages from topics they are interested in, allowing for flexible and decoupled communication.

11. Unique ID Generator

Unique ID generators provide unique identifiers for entities within a system. These IDs are crucial for maintaining the integrity and consistency of data, especially in distributed systems where conflicts can arise if IDs are not unique.

12. Rate-Limiting

Rate-limiting controls the rate at which clients can make requests to a server. This protects the system from abuse, ensures fair usage, and maintains performance levels by preventing any single client from overwhelming the server.

Understanding these fundamental building blocks is essential for designing robust, scalable, and efficient systems.

Whether you're building a new application or scaling an existing one, incorporating these components will help ensure your system meets the demands of modern usage.

Stay tuned for more in-depth discussions on each of these building blocks and how they can be implemented effectively in your projects.

#DesignBuildingBlocks