How much does the ball cost?

The immediate answer might be 10 cents. If you jumped to that conclusion, well… you’re wrong.

If the ball were 10 cents, the bat would be 1.1 euros, making the total 1.2 euros, not 1.1 euros.

Let's dive into the math instead:

𝑥+(𝑥+1)=1.10

2𝑥+1=1.10

2𝑥=0.10

𝑥=0.05

So, the ball costs 5 cents, and the bat costs 1.05 euros.

This puzzle shows how easy it is to make wrong assumptions based on a superficial understanding. It mirrors the challenges we face as developers when underestimating the complexity of tasks. This isn't just a party trick. It's a powerful metaphor for understanding the pitfalls of estimations.

The 5-Minute Fix Myth

I bet you've experienced that moment when a new bug pops up. You quickly glance at the issue and happily say to yourself, “Oh, that’s easy. It’ll take 5 minutes.” You might even express this out loud, making a commitment to stakeholders.

Yet, what appears to be a simple fix often spirals into a complex ordeal, spanning multiple aspects of your domain.

Let's face it. It never is just a 5-minute fix.

Did you consider the time required to write tests for that bug to never happen again? Did you consider regressions? Could this fix potentially affect the performance of the application? Is it backwards compatible with previous versions of the application? Did you think about the pipeline's running time? The deployment time? What about documentation that needs updating?

Every developer has had the experience of diving into what seemed like a simple issue only to find themselves tangled in a web of dependencies and unforeseen problems. It's a rite of passage in the world of software development. We often underestimate the true scope of tasks because of overconfidence or a desire to demonstrate efficiency. But, as the bat and ball puzzle illustrates, our initial assumptions are often flawed.

Now, don’t get me wrong. I'm not advocating for discussing every minute task. Sometimes, a change is truly trivial.

I even have a counterexample puzzle to demonstrate my point.

Let's consider two trains that are 100 kilometers apart and move towards each other. One train travels at 40 km/h, and the other at 60 km/h. A bird starts flying from the first train at 90 km/h towards the second train, then turns around immediately upon reaching the second train and flies back to the first train, and so on, until the trains collide.

How far has the bird flown?

The first instinct is to sum up the infinite series of the bird's flights.

Instead, it's much simpler than that.

You just need to realize the trains take 1 hour to meet (100 kilometers / (40+60) km/h), and the bird flies for that hour at 90 km/h. So, the bird flies 90 kilometers.

This puzzle illustrates that sometimes, the problem isn't as complex as it seems. The key is knowing when to dig deeper and when to take things at face value.

Strategies to Mitigate Risks

So, how do we tackle this? It seems we’re stuck in between cognitive biases. Here are some strategies to mitigate the risks:

1. Challenge Your First Instinct

Conduct a brief risk assessment or a quick review with a teammate to uncover potential hidden complexities. Don’t underestimate the power of a second pair of eyes. This can help identify aspects that might not be immediately obvious and prevent underestimation of the task at hand.

2. Timeboxed Spikes

If the uncertainty is high, as it often is when starting new projects or integrating with new services, do a timeboxed spike. This means allocating a fixed amount of time to explore the problem space, gather information, and identify potential pitfalls. This helps in making more informed decisions and estimates.

3. Break Down the Task

Break down the task into smaller components. Eat the elephant one bite at a time. This way, you're more likely to reveal complexities that weren't apparent initially. Smaller tasks are easier to manage, track, and estimate accurately. It also helps in identifying dependencies and potential bottlenecks early.

4. Plan for Multiple Iterations

Recognize that the first fix may not be the final one. Plan for multiple iterations, including testing and feedback cycles. Continuous integration and deployment practices can help in catching issues early and often, reducing the risk of large, unforeseen problems.

5. Embrace Code Reviews and Pair Programming

Overconfidence in personal coding abilities might lead some developers to dismiss the possibility of errors in their own code, overlooking simple solutions. That’s where implementing practices such as thorough code reviews and pair programming comes in handy. These techniques foster continuous learning and humility, and also provide a fresh perspective on the problem, potentially uncovering issues that might have been overlooked.

6. Encourage Team Discussions

When a miscalculation happens, it’s important to realize it's not the end of the world. Encourage team discussions about mistakes and learning opportunities from past bugs to cultivate a more accurate assessment of debugging tasks. It's through sharing, how we grow the most. Learning from past experiences and sharing those learnings with the team can help in avoiding similar pitfalls in the future.

7. Document Your Findings

Effective documentation is crucial for maintaining clarity and continuity in software projects. When the entire team contributes to and utilizes documentation, it ensures comprehensive coverage and accessibility of information. Establish a documentation protocol where each team member contributes to the document creation and review process. Good documentation practices ensure that knowledge is not lost and can be easily transferred within the team.

8. Keep a Short Feedback Loop

Sometimes it's easy to drill deep and develop a new feature based on unconscious assumptions about what the end result should look like. By keeping the feedback loop short, you get the chance to clear them out. Regular check-ins and reviews with stakeholders can ensure that the development is on the right track and aligns with the requirements.

9. Foster a Culture of Continuous Learning

It’s important that both you and your colleagues engage in continuous learning. It can be anything, from the official training sessions, workshops or attending conferences, to more informal, brownbag sessions, reading club, or simple “this week I learned” trivia. Staying updated with the best practices in the industry oftentimes provides new insights and tools that can be applied to solve complex problems more efficiently.

10. Use Automated Tools and Techniques

If you've been following this newsletter, you probably saw this one coming. Leverage the power of unit testing, continuous integration, and continuous deployment tools to streamline the development process. Automated tools can help in identifying issues early and ensure that the codebase remains stable, reducing the time spent on manual testing and debugging.

Conclusion

When it comes to quick fixes, what seems like a 5-minute fix rarely is (if ever). By challenging your first instinct, conducting brief risk assessments, breaking down tasks, planning for multiple iterations, embracing code reviews, encouraging team discussions, documenting findings, and keeping a short feedback loop, you can better navigate the complexities and uncertainties inherent in development tasks.

Moreover, effective communication, fostering a culture of continuous learning, and leveraging automated tools can further enhance your team's ability to handle complex problems efficiently. By adopting these strategies, you can mitigate the risks associated with underestimating or overestimating tasks and ensure a smoother development process.

Pay it forward

Here are this edition’s knowledge nuggets that made me pause and think:

How Razorpay Scaled to Handle Flash Sales at 1500 Requests per Second - by

Latency is a silent killer - by

When Agile, isn't agile - by

Embracing Simplicity: A Look Into Stateless Design - by Nikki Siapno

Why Accelerate Deployment? - by

If you liked this post, share it with your friends and colleagues.

Share

Yet, unit testing, pivotal in software development, is fraught with misconceptions that can dilute its effectiveness. A major point of contention is the definition of a "unit", leading to two distinct approaches to testing, each profoundly shaping our methods and focus areas.

My favorite definition of a unit test comes from Vladimir Khorikov, and it’s a refinement of Kent Beck’s attributes of unit tests. In his book "Unit Testing Principles, Practices, and Patterns", Khorikov provides a concise breakdown for what unit tests should aim to achieve:

While the third point is straightforward, as everyone wants fast feedback loop from running the test suite, the first two warrant further discussion.

1. What’s a unit?

Jay Fields, who coined "solitary" and "sociable" unit testing, highlights that sociable tests aim to check broader behaviors, offering a stark contrast to the solitary tests that focus on smaller, isolated chunks of behavior.

The solitary approach, frequently identified with the London School of testing, views a unit as a single class or method, isolating it from external collaborators through the use of mocks. This method, while streamlining initial testing phases, tends to complicate maintenance as dependencies and system complexity grow.

Solitary unit tests focus on smaller behaviors — specific computations or orchestration of collaborators — using test doubles to isolate each unit and ensuring that mocked interactions adhere to predefined contracts.

However, this perspective can be restrictive and misleading, as it focuses attention narrowly on isolated functions rather than on the overall behavior of the system.

Contrastingly, the sociable approach, often aligned with the Detroit School of testing, defines a unit as a unit of behavior that encompasses interactions with its dependencies, unless external dependencies such as I/O operations demand isolation for efficiency.

Sociable unit tests embrace the real interactions between components, ensuring that the system behaves as expected in a more integrated environment.

This brings us to the second point.

What do we isolate?

The natural answer would be to isolate the unit under test.

If a unit is a class or a method, we want to test this unit in isolation from all external factors that might impact it.

All dependencies of that class should be replaced with test doubles (like mocks) in tests (except for immutable dependencies).

By doing this we are making sure that failures are localized to the unit itself rather than being influenced by external factors or the behavior of dependencies.

So… that means we will mock out any dependencies of that class away.

Our test landscape would then look like this:

What’s wrong with solitary unit testing?

A common complaint with this approach is that any modification in the codebase necessitates changes in the tests.

By focusing the unit on a specific method or class you automatically create a tight coupling between the tests and the implementation, rather than the behavior. This often entails extensive use of mocks to simulate interactions between the class under test and its external dependencies.

The core problem here is in fact precisely this: this kind of tests are heavily mocked. Mock-based tests are rigid. This coupling can manifest in tests that verify whether specific methods were called with particular arguments, rather than focusing on the outcomes those interactions should achieve. Any deviation from that implementation, regardless whether just a mere organizational change, will trigger some tests to fail.

Furthermore, by isolating each component and replacing dependencies with mocks, developers may inadvertently lock themselves into a particular design too early. Because this approach demands a deep understanding of the domain model upfront - which is not always the case -, thus often leading to premature design decisions.

Over-reliance on mocks and substitutes leads to brittle tests that are highly sensitive to changes in the implementation of dependencies, spiraling into a maintenance burden, as changes in the system necessitate widespread updates to the accompanying mocks.

Not only does this hamper the agility of the development process, but also contradicts the core purpose of unit tests — to document and make sure the code has the expected behavior.

No, the core purpose of tests is not to mirror the code structure. The core promise of tests is to help us, to make refactoring safe and easy to do.

Moreover, the argument that tests become stale over time typically reflects a misunderstanding of unit testing's role. If tests are designed to verify behavior, they evolve only when these behaviors change, thereby serving as a living documentation of system expectations.

The Problem with Deep Domain Testing

When tests drill too deeply into the domain, instead of focusing on the API or the service's external interfaces, developers end up with an inflated test suite where many tests offer little to no additional coverage. These overly detailed tests often duplicate the scenarios already covered by broader tests, adding maintenance overhead without any of the corresponding benefits. Moreover, when a new class is extracted from an existing behavior, the necessity to create additional tests and modify existing ones can dissuade developers from making beneficial refactorings.

What should trigger a new unit test?

Spoiler alert: it’s not adding a new class.

What should trigger the need for writing a unit test is implementing a requirement. This means the unit is a UNIT OF BEHAVIOR. It's about fulfilling a functional need, not just verifying implementation details. This functional need is what should drive test creation, because it reflects what the software is expected to do.

In contrast, writing a test simply to check if a method CalculateSum, correctly adds up two numbers, focuses too much on implementation details. Such details, albeit important, should not dictate the testing strategy.

Instead, testing should be based on higher-level objectives or user needs.

This is where your tests should be targeted, ensuring they are relevant and directly tied to the requirements of the software.

How do we implement sociable unit testing?

Focus on the public exports of your module, your facade. This preserves module encapsulation, crucial for maintaining flexibility.

Don’t focus on testing internals. These are implementation details.

Now, I know there’s a common practice to make internals visible to test projects. In C# we have the InternalsVisibleTo attribute. Refrain from using it, if you want to maintain the encapsulation of your module. It’s this encapsulation what actually preserves the system’s flexibility.

There’s one exception to this rule, and that’s when touching legacy code. There it’s perfectly fine to go ahead and break the encapsulation, as it is more valuable for that code to get under tests than to preserve its encapsulation. For new software, don’t use InternalsVisibleTo. This is what tweaks your development speed for the better in the long run.

Again, when we talk about sociable unit testing, the system under test is not just a class. The system under test are the exports of a module — its facade. This is your Domain in DDD that you’re testing. I’m not going to dive too much into this, as it’s a whole topic on its own, but simply think in terms of hexagonal architecture.

Allow your tests to be sociable. Allow them to communicate with real dependencies. The unit of isolation here is the test itself.

Circling back to Vladimir Khorikov’s three critical attributes of a unit test:

1. Unit of Software: This refers to the smallest piece of software that can be isolated for testing, which exhibits specific, meaningful behavior. This unit is not necessarily limited to a single method or class, but can involve any functional aspect significant from a business perspective.

2. Isolation: This attribute emphasizes testing the unit in a controlled environment where dependencies are either simulated or managed to ensure the focus remains on the unit itself, rather than on external interactions.

3. Quick Execution: The ability of a test to run swiftly complements the agile development process, enabling developers to receive immediate feedback and continuously refine the system with minimal disruption.

Code example

Let’s consider the domain of a Human Resources Management System (HRMS). This system is responsible, among others, for managing employee records and payroll processes. We'll focus on a domain model that involves the interaction between Employee’s payroll calculations, considering factors like overtime payments.

The Payroll business logic is implemented by the StandardPayRatePolicy, as follows: if it’s standard time, then multiply number of hours by standard rate, otherwise, if overtime, multiply the rate by 1.5 first, and then multiply by the number of hours worked.

In solitary unit testing, this would start with creating a test class for the Employee class. And since we want to isolate the unit under test, we will mock every dependency of that class - IPayRatePolicy in our case.

This ends up with twisted tested behavior. It’s a perfect depiction of how tests cluttered with mocks tend to focus too much on the interaction with mocks rather than the real outcomes, making it challenging to discern what is actually being tested and why tests fail when implementation details change.

Furthermore, frequent changes in the codebase will probably lead to outdated mocks, making it difficult to ensure that tests still reflect the true behavior of the system.

But what about PayRatePolicy?
For it, we would end up creating a new test class, which indeed would be focused on the behavior itself.

But what happens when we realize some of that behavior needs to be extracted away to keep our code base clean?

You guessed it. Tests will start failing and new tests will need to be written to accommodate the new implementation details. But did the business logic change? Not one bit.

Let’s now turn to sociable unit tests.

We end up with less tests overall, which are laser focused on our business logic. Regardless of how many tweaks and splits into separate classes we do, as long as the behavior stays the same, the tests won’t fail. Our trust in the test suite increases, and our overall development speed together with it.

General guidelines for writing sociable tests

Utilize specific domain entities

Start by using specific entities that are central to your domain. This helps ground your tests in the reality of your application's environment. You can then modify these entities according to the specific requirements of each test. It ensures that the tests are relevant and provide valuable insights into how the system behaves under different conditions.

Make asserts meaningful

In sociable unit testing, asserts are not just checks; they serve as documentation for the business logic they test. Each assert should clearly communicate what it’s testing and why that behavior is important. This makes tests easier to understand and maintain, and it turns the test suite into a form of documentation that can be useful for both developers and non-technical stakeholders.

Focus on the Behavior, not the Class

Shift your focus from testing specific classes to testing behaviors across classes. This change in perspective helps ensure that the tests assess the system’s functionality and user-facing behaviors rather than just the internal logic of its components. By focusing on behaviors, sociable unit tests can provide more meaningful assurances about the system's readiness for production.

Efficient execution strategies

Sociable unit tests can be more resource-intensive than solitary tests, primarily because they involve more components and require more setup. To manage this, consider not running all sociable unit tests all the time, especially during the early stages of development. Alternatively, you can run all the tests but do so in parallel to reduce the time impact. Modern CI/CD tools and test frameworks offer capabilities to run tests in parallel, leveraging cloud resources and multi-threading to handle the increased load efficiently.

Concerns and challenges

Breaking the testing pyramid

There is a common misconception that sociable unit tests blur the lines with integration tests, potentially breaking the traditional testing pyramid.

However, that’s not the case, as each type of tests serves distinct purposes:

• Unit Tests

  While they involve multiple classes, the focus of sociable unit tests is on the logic within a specific module or component, regardless of how it is organized. They are interested in the smallest units of behavior, which in this context, are the public exports of a module—essentially the functionalities exposed to the users or other modules. The key is what happens in the facade as a result of that logic, not just that the integration points work. We are testing internal logic (Class B, C, Z) through that facade (Class A).

• Integration Tests

  These tests are designed to verify the collaboration between such modules and ensure that various parts of the system work together as expected.

• End to End Tests

  They are responsible to make sure the whole system communicates well.

Addressing Architectural Challenges

Sometimes, creating a reusable setup for sociable unit tests proves too challenging, indicating a potential issue with the architecture itself. Difficulty in configuring tests for certain interactions might suggest that the components in question do not belong in the same context or domain. This difficulty can serve as a valuable indicator that it may be necessary to reevaluate and potentially redefine the architectural or domain boundaries. Such a reevaluation can lead to a more intuitive and coherent system architecture that better supports both development and testing.

The Role of Mocks and Test Doubles

Despite the emphasis on using real dependencies, the use of mocks and test doubles cannot be completely eliminated, especially when dealing with external services where non-determinism is a factor. Even staunch advocates of classical testing acknowledge their value in removing non-determinism, particularly in awkward collaborations.

Favor using fakes—simplified implementations of complex interfaces—over mocks. While mocks are useful for isolating behavior, they can lead to tests that are too closely tied to the implementation details of a component. Fakes allow for testing the interaction with dependencies in a way that is less brittle and more representative of actual interactions within the application.

When to Use Test Doubles

The decision to use test doubles should consider:

• Complexity of the SUT: In highly complex systems, using test doubles can help manage this complexity by simplifying the test environment. However, for systems where interactions are crucial to functionality, real dependencies might be necessary.

• Test Maintenance Effort: Test doubles can reduce initial setup time but may increase maintenance effort if the system evolves and the interfaces between components change.

Efficient Execution Strategies

Given the often extensive nature of sociable unit tests, it's impractical to run all tests frequently. Typically, only those tests that pertain to the code currently under development are run regularly. This targeted approach, which I refer to as the "compile suite," is essential for maintaining the balance between depth of testing and runtime

Advantages and Challenges of Sociable Unit Tests

Pros:

• Validate Refactoring: One of the significant advantages of sociable unit tests is their ability to validate the system's behavior after refactoring. Because these tests are less tied to implementation details, they allow for substantial code modifications without the need to rewrite the tests, provided the external behavior remains consistent.

• Simplified Maintenance:

  By reducing the reliance on mocks and stubs, there is less overhead in configuring and updating tests. This simplification leads to easier maintenance since changes in the codebase do not necessitate widespread updates across numerous mock setups.

• Behavioral Coverage: They are excellent for capturing emergent behaviors in complex systems that might be missed by more granular tests. These tests validate the integration of components, ensuring that the system works as expected in a setup that closely mimics the production environment.

• Reinforce Better Modularization: By testing how well components interact with their real dependencies, sociable unit tests encourage better system design and modularization.

• Indicate Boundaries: These tests help define and reinforce the boundaries of a component's responsibilities by clearly outlining how it should interact with other parts of the system.

• Enhanced Reliability:

  Using actual code interactions rather than relying on potentially flawed mock contracts increases the reliability of tests. Real interactions expose the system to more variable conditions and edge cases, improving the robustness of the software.

Cons:

• Verbose/Complex Setup: Setting up sociable unit tests can be more complex and time-consuming than solitary tests because they often involve configuring real system dependencies.

• (Can be) Slower: Because they interact with real components rather than mocks, sociable unit tests may execute more slowly, which could impact the speed of the development cycle. Whenever that really is the case, use test doubles.

Conclusion

By fostering tests that mirror real-world use rather than just code structure, sociable unit tests enhance not just the robustness but also the relevance of our testing practices. They prepare software not just to work, but to work in the way users truly need. This not only stabilizes the foundation of our applications but also streamlines the development process, making our software more adaptable and resilient to changes.

Pay it forward

Here are this edition’s knowledge nuggets that made me pause and think:

Server-Sent Event - by

Stateless Architecture - What's the Deal? - by

Why you need TDD - by

How an empty S3 bucket can make your AWS bill explode - by Maciej Pocwierz

If you liked this post, share it with your friends and colleagues.

Share

Let's understand why they're problematic and how to avoid them.

The Problem

We have a behavior we want to test, and there’s a complex setup to get to it. There might be similar edge cases that we want to cover, there might be some external dependency. The problem happens when we try to be smart and tackle uncertainty when writing a test, using conditionals.

Don’t be fooled to think that this is efficient. It’s not.

Don't overcomplicate things. The moment you introduce an if into a test, you're essentially inviting complexity. It's a sign that the test is trying to cover too much ground.

It's trading it all for a lack of clarity and a loss in focus. That’s not efficiency. Optimal tests maintain a laser-focused approach, examining one specific aspect or behavior at a time. Having such ifs blurs this focus, trying to encompass multiple scenarios in one go.

Not to mention, tests with conditional logic are much harder to maintain. They become riddled with paths that make understanding and updating them more difficult.

Yet, there is a deeper layer to this discussion, centering on the nature of the functions and methods we test. The ease or difficulty of testing a piece of code often boils down to its purity or impurity.

Pure and impure methods

Methods that are deterministic and side-effect-free, known in functional programming as pure functions, present a testing ideal.

Pure functions, by definition, always return the same output for the same input and have no side effects—meaning they don't alter any state outside themselves or rely on an external state.

This attribute not only aids in straightforward testing but also in enhancing the functions' reusability and integration into larger systems without complicating state management or dependencies.

On the other hand, impure functions, which depend on external states or cause side effects, complicate testing. Such functions can make the codebase harder to maintain by introducing dependencies and unpredictability.

While it's unrealistic to completely eliminate impure functions, their impact can be minimized by isolating them within the application.

This involves:

• Separating pure and impure functions

• Reducing the reliance on impure interactions

• Facilitating dependency injection or fakes for testing.

The Ideal approach

Aim for single responsibility tests.

Each test should validate one and only one condition or behavior. If you find yourself needing an if, it's likely that the test can be broken down into multiple, more focused tests or you need to break down your dependencies.

Avoid branching at all costs.

A test should be a straight path, not a branching tree. Instead of one test with an if statement, create multiple scenario-specific tests. Each tailored to a specific scenario or condition. It should follow a clear Arrange-Act-Assert (AAA) pattern without deviating into different scenarios.

Speaking of AAA, I personally find no use for // Arrange, // Act, and // Assert comments in code, as long as you can separate the sections by empty lines.

Besides, by clearly naming the system under test “sut”, when calling it, and storing the result in a “result” variable, it’s clear what’s being tested (act) and that everything before represented the setup (arrange).

var result = sut.PerformTask();

I won’t dive into the assertion bit, as it is self explanatory.

Target explicitness.

Tests should explicitly state what they're checking. When you read a test, its purpose and scenario should be immediately clear without needing to deep dive and decipher the logic.

That’s why I’m not necessarily keen on having strict test naming conventions such as [Given]_[When]_[Then] or [MethodUnderTestName]_[Scenario]_[Result].

Such conventions makes reasoning about tests difficult and often require you to look further into it in order to understand what’s actually being tested.

On top of that, especially the latter version, encourages you to focus on the implementation details and on isolating the method, instead of testing a behavior. I’ve already covered why you should strive to test units of behavior here.

I prefer a more natural, English-style phrasing, to depict the business case the test fulfils, as it makes reasoning about what’s being tested much easier.

For instance, instead of a generic name, I favor using something like user_login_with_valid_credentials_authenticates_successfully or user_login_with_invalid_credentials_fails_authentication.

Refactoring tests.

Regularly review and refactor your tests. As your codebase changes, ensure your tests remain relevant, clear, and focused. All code is a liability. Test code is still code, hence no stranger to this rule.

Diving into unit testing with xUnit

Let’s dive into some concrete code examples. For the purpose of this, I’m using C# with xUnit as my favorite playground.

As mentioned, we don’t optimize the code by crafting one test with multiple branching to cover multiple scenarios, but we craft specific tests for each case:

What if there are multiple edge cases to cover, all with a rather similar setup? That’s where [Theory] attribute comes to save the day.

Using xUnit's [Theory] attribute with [InlineData] allows for parameterized tests, covering multiple scenarios without resorting to conditional logic. Each data line represents a distinct, clear scenario, easy to pinpoint and debug. But the data is not so depictive, as it needs comments to make it clear. Let’s fix that.

Advanced Strategies for Complex Data

For such complex scenarios [InlineData(...)] reaches its limits. But xUnit offers two potent solutions: [MemberData(...)] and [ClassData(...)].

[MemberData(...)]

The [MemberData(...)] attribute allows you to reference a static property, field, or method within the test class or another class that returns an IEnumerable<object[]>, where each object[] contains a set of parameters for the test method.

This approach is particularly useful when your test data is too complex for inline definitions or when you want to share test data among multiple test methods.

[ClassData(...)]

For scenarios where even [MemberData(...)] might not be enough—perhaps due to complex setup that you want reused across multiple test classes or simply because you need data encapsulation, [ClassData(...)] steps in.

The [ClassData] attribute specifies a class that implements IEnumerable<object[]> to provide the data for the test method. This class must have a parameterless constructor.

Conclusion

Remember, when writing code and even more so in testing, less is often more.

Avoid the temptation to create complex, conditional tests. Strive for simplicity, clarity, and focus.

By keeping if statements out of your tests and adhering to the principle of single responsibility, your tests become more reliable, understandable, and maintainable.

Pay it forward

Here are this week’s knowledge nuggets that made me pause and think:

How Disney+ Scaled to 11 Million Users on Launch Day - by

STARTUP VS BIG TECH: What's the difference? - by

How to Self-Manage Even if You Have a Manager - by

Treat your career as a startup - by

If you liked this post, share it with your friends and colleagues.

Share

But let’s set the record straight.

Just because they can be used to announce something went wrong, doesn’t mean they should be used and abused as general purpose for error handling.

Exceptions are, as their name suggests, exceptional. They serve exceptional circumstances — unpredictable events that fall outside the normal operation of a program.

Business rules violations and data validation are expected flows. They’re conditions and scenarios that developers can foresee and thus, should plan for. There isn’t anything “exceptional” to them. They're anticipated pathways that require explicit handling within your application's logic. Using exceptions in such cases fundamentally misunderstands their purpose.

If you’re throwing exceptions expecting its invoker to catch it, then you need to rethink your error handling strategy. Such indiscriminate use of exceptions, especially for routine error handling, obfuscates control flow, degrades performance, and will make your code harder to understand and maintain.

I’m not the only one saying it.

Wikipedia defines exceptions as

anomalous or exceptional conditions requiring special processing – during the execution of a program. In general, an exception breaks the normal flow of execution and executes a pre-registered exception handler;

On Microsoft’s side, the general advice is to thread carefully into exception territory:

The C# language's exception handling features help you deal with any unexpected or exceptional situations that occur when a program is running.

For conditions that are likely to occur but might trigger an exception, consider handling them in a way that will avoid the exception.

Let’s have a closer look and understand the why behind the no-exception reasoning.

Performance costs

It might be really tempting to create exceptions, throw them and think you’ll deal with them later. But throwing exceptions incurs a significant performance cost due to the underlying operations involved - think stack unwinding.

This overhead is particularly problematic in performance-sensitive areas of an application.

Obscured logic and flow interruption

When exceptions are used liberally, the control flow of an application becomes fragmented and less intuitive.

Exceptions are a form of non-local control, acting as glorified go-to calls that interrupt the execution flow. This fragmentation complicates the task of tracing through code when you want to understand how errors are handled, and ultimately leads to inconsistent error management practices.

To put it more bluntly, exceptions cause jumps from one known point in our application to whatever point, in whichever state, often in ways that are hard to reason about. That is if we define that catch point somewhere.

When thinking about the business context of your problem, you should create abstractions that help understanding the domain and flow. Don’t be fooled to think that by crafting MyCustomException : Exception you’re building that abstraction over your domain. Exceptions act like a uno card that’s gonna pull a reverse on you later on. You think you got rid of the problem, but in fact you just made it more complex.

The False Security of exception handling

Relying on exceptions for error handling gives a false sense of security. The very act of throwing an exception for known error conditions, such as a missing dependency, although necessary in some cases, should be the exception rather than the rule.

By all means, if you really need the application to crash because you cannot continue, do so.

Just keep in mind to ponder really well whether that’s really the case. Don’t make code more complex and harder to maintain just because it’s “easy”.

And no, having a application-wide catch-all is not a great idea either. What are you catching? What are you forecasting to handle and how?

Deceiving nature of methods’ signature

In C# exceptions are hidden and make methods deceiving. When looking at the signature of a method, you’ll be easily tricked into expecting a certain return type. However, that signature is fooling you, by not giving you all the information.

There’s no proper way to know about the possible exceptions it might throw, except from the xml documentation a thoughtful author might’ve created (if any).

Let’s take the FileStream of Microsoft. We see there’s a bunch of exceptions presented as possible scenarios. But it makes for a horrible developer experience.

You now have to wrap chunks of code in different try statements, interrupting the execution of your function.

Should you have to worry about all these cases as errors?

Couldn’t they have been depicted any other way?

And even if you do your due diligence and cover all cases, what if tomorrow they decide to throw on a different exception?

Here we’re talking Microsoft, which is quite gentle with developers. What if we talk external public packages? What if we talk cross-team packages? What if we talk inner-team?

There’s no proper way for you as a developer to be warned at compile time that you haven’t handled this case. Nothing is forcing you to properly handle the method’s exceptions and what’s worse, unless you dive into each and every method, you’ll find about such cases a tad too late: at runtime.

And don’t get me wrong. This is not a C#-only problem. If you’re tinkering with JavaScript, you know that it too doesn’t have a native way to indicate the throws of a function, once you invoke it.

Promises with no catch will just log silent errors, leading to a terrible user experience. Ever caught yourself annoyed and frantically clicking a button, thinking it did nothing? Of course you did.

***

I think by now it’s became crystal clear why throwing exceptions is so problematic.

So what’s the alternative?

Enter Result Object

In contrast to the traditional exception handling model, adopting result objects for error management offers a more explicit, intuitive, and performant way of handling errors.

A result object encapsulates the outcome of an operation, clearly distinguishing between success and failure states. This model encourages a more deliberate handling of errors and makes the control flow within applications more transparent.

Enhanced Performance

Without the need to throw exceptions, the performance overhead associated with them is eliminated. Operations can return result objects directly, bypassing the costly process of exception throwing and catching.

Clarity in Control Flow

Result objects make the handling of errors a first-class concern in the application flow. This explicitness enhances readability and maintainability, as success and failure paths are clearly delineated within the code.

Flexibility and Consistency in Error Handling

Using result objects, developers gain flexibility in how and where errors are handled, promoting a consistent approach to error management across the application. This model facilitates easier debugging and testing, as error conditions are explicitly represented in the code.

Honest signatures

Result objects bring errors out of the shadows, depicting clearly the possibility of one to be returned. This explicitness enhances the self-documenting nature of code and aligns with principles of clean code that advocate for readability and maintainability.

Result object example

Incorporating result objects into your applications encourages a design that aligns closely with Domain-Driven Design (DDD) principles. By encapsulating success and failure within a single object, you can create a rich model of your application's domain, making the code more intuitive and aligned with business operations and logic.

Consider a trivial example where a method attempts to validate the properties of a user. Instead of throwing an exception when encountering an error, the method could return a result object indicating the failure and its cause:

This approach contrasts sharply with traditional exception handling, where the method might throw an exception for the same condition, complicating both the method's implementation and its consumption:

By using Result object, the method's signature truthfully communicates its behavior to callers, including the potential for failure, without relying on side effects like exceptions. This clarity and honesty in method signatures lead to safer, more reliable code.

Conclusion

The conventional wisdom surrounding exception handling is due for a reevaluation.

Exceptions should be reserved for truly exceptional, unpredictable events.

For routine error handling, especially for known error conditions, result objects offer a superior alternative that is explicit, efficient, and conducive to clearer, more maintainable code.

Adopting result objects not only improves the performance and reliability of applications but also enhances the developer experience by making error handling a more integral and transparent part of the programming model.

Pay it forward

Here are this week’s knowledge nuggets that made me pause and think:

Design is an Island - by

Geospatial indexes - by

How LinkedIn Uses Caching to Serve 5M Profile Reads/Sec? by

My learnings from the book "A Philosophy of Software Design" - by

Why should you care about web specifications? - by

If you liked this post, share it with your friends and colleagues.

Share

We move through life with an air of assurance. We wear an invisible robe that whispers:

"You're right. About this. About that. About almost everything."

Think about it. We are never wrong in the present.

We were wrong in the past, oh, naive selves as we were. But NOW we know better. In this present moment, the truth is clear and we know how the world works.

Or do we?

If we pause for a moment to think about it, this notion of near-omniscience is absurd. Yet it so often underpins our thoughts and actions.

This mindset is where fragility in our understanding of the world stems from.

I’m sure you’ve heard of the term "Black Swan".

It used to symbolize something utterly impossible or non-existent. The phrase implied a complete absurdity, a notion so contradictory it was beyond consideration.

For centuries, a black swan was deemed an impossibility.

Yet, in 1697, Europeans encountered black swans in Australia, shattering the long-held belief in their non-existence. This event exemplifies that even the most improbable scenarios can occur.

Despite this lesson, we continue to make numerous assumptions daily, considering them absolute truths.

These assumptions shape our perception of reality, often blinding us to the vast array of possibilities beyond our beliefs. Recognizing this tendency is the first step in acknowledging our own limitations and the vastness of what we don't know.

Okay, what does this have to do with stalling my career?

This sense of infallibility parallels the "Expert Beginner", also known as “Senior Junior”.

Throughout my career I interacted with a lot of “expert beginners”. Hell, I was one of them. Surely still am to some extent.

This trap eludes no-one.

The Expert Beginner land is the magical (read cursed) place where developers reach a certain level of proficiency and then stop progressing, believing they've learned all there is to know.

They stop seeking new knowledge or challenging their understanding, leading to a plateau in skills and awareness.

They assume their solutions and understandings are nearly always correct, mirroring the general human tendency towards assumed omniscience.

But this mindset is particularly detrimental in software engineering, where technologies and best practices are constantly evolving.

Just as the discovery of black swans challenged the universal belief in their nonexistence, breakthroughs in technology frequently overturn established norms in software development.

How do you get out of this trap?

1. Embrace Continuous Learning

Learning is a never-ending journey. In technology, as in life, there's always something new to discover.

Carve couple of minutes everyday to read and experiment with technology. If you're not constantly learning, then you're running behind.

2. Cultivate Humility

Recognize that no matter how much you know, there's always more to learn. The world is vast and complex, and our understanding is inherently limited. You are a student for life.

3. Seek Diverse Perspectives

Engage with others who have different skills, backgrounds and viewpoints. This will challenge your assumptions and expose you to new ideas and approaches.

Get out of your bubble or comfort zone. It’s true what they say. If you feel like the smartest person in the room, you need to switch rooms.

4. Reflect and Reassess

Regularly take time to reflect on your beliefs and practices. Consider how new information might affect your understanding and be willing to adjust your views accordingly.

5. Encourage Constructive Criticism

Create an environment where peers can offer feedback and constructive criticism. Being challenged is one of the most effective ways to grow and avoid complacency.

6. Experiment and Learn from Failure

Don't be afraid to try new things. Yes, you'll be awful at them initially. But don't fear being wrong or making mistakes. Failure is merely an iteration toward proficiency.

Conclusion

Understanding and mitigating our inherent belief in our own rightness is crucial for personal and professional growth.

That’s not to say I encourage you to chase after each and every buzzword and hype the tech world concocts on a weekly basis. Fundamentals trump these. However, I encourage you to constantly keep an eye out on what other professionals in the field have to say.

By embracing humility, continuous learning, and openness to new ideas, we can avoid the pitfalls of the "Expert Beginner" and the assumed omniscience, leading to a more nuanced and expansive understanding of the world and our place within it.

P.S. This train of thought was triggered by reading the book "Being Wrong" by Kathryn Schulz, which I highly recommend.

Pay it forward

Having said all these, I decided to introduce a new section to the newsletter."Pay It Forward" is where I'll share articles that piqued my curiosity and contributed to my "Today I Learned" moments in the current week.

Here are this week’s knowledge nuggets:

How Amazon S3 Achieves 99.999999999% Durability - by

Introduction to Pre-caching - by

Your ultimate self-confidence toolkit - by

Simplifying as much as possible is the way to go in the engineering industry by

If you liked this post, share it with your friends and colleagues.

Share

This principle finds a parallel in software design with Object Calisthenics, a concept introduced by Jeff Bay in "The ThoughtWorks Anthology". With this adaptation, it consists of a set of nine rules aimed at enhancing code's maintainability, readability, and overall quality by imposing specific constraints.

The initial idea of the paper was to create a pet project to follow these 9 rules to the teeth, yet the way I see it is that the end goal is to assimilate them as guidelines or, better yet, reflexes, to avoid common pitfalls, promoting a more thoughtful and refined approach to coding.

These rules encourage us all to write cleaner, more readable, and more maintainable code. Again, even if they may seem restrictive at first, adhering to them will significantly enhance your coding practices.

The rules are:

1. Use Only One Level of Indentation per Method

2. Don’t Use the ELSE Keyword

3. Wrap All Primitives and Strings

4. Use First-Class Collections

5. Use Only One Dot per Line

6. Don’t Abbreviate

7. Keep All Entities Small

8. Don’t Use Classes with More Than Two Instance Variables

9. Don’t Use Getters or Setters

Now let's break down these 9 rules:

1. Use Only One Level of Indentation per Method

This rule enhances readability and maintainability by ensuring methods are concise and focused on a single task — kind of like reading a document with clear and distinct paragraphs.

Each method (representing a paragraph) maintains a specific level of abstraction, guiding the reader through the code's logic without overwhelming them with details.

Adopting this approach not only improves code cohesiveness and reusability, but I also found it simplifies stack tracing and eases understanding by keeping unnecessary details out of a method's concern.

You might say that this strategy makes is harder to read the whole context of the class, because you have to hop in and out of methods to understand the context. I disagree with this idea, as you are normally reading the code as though it were a set of TO paragraphs, each of which is describing the current level of abstraction and referencing subsequent TO paragraphs at the next level down.

Sticking by this rule actually facilitates a smoother navigation through code by adhering to a single level of abstraction.

It's like cooking a lasagna; you don't need to know how to make bechamel sauce from scratch if you're not planning to make it, hence keeping the focus on assembling the lasagna layers.

Can you tell I’m hungry as I’m writing this?

2. Don’t Use the ELSE Keyword

Eschewing the else keyword not only clarifies code and reduces cognitive load, but it also helps in avoiding code duplication.

By favoring early return statements and leveraging polymorphism for conditional operations, code becomes more linear and less nested.

This practice leads to a cleaner, more intuitive code structure and reduces the cognitive load off our shoulders. Each decision point is straightforward, enhancing the code's readability and maintainability.

3. Wrap All Primitives and Strings

An int, on its own, is merely a scalar with no meaning. It relies solely on method names to express intent.

Wrapping primitives in objects not only provides the compiler (and the developers touching the code) with more context about the value's nature and its purpose, but it also facilitates semantically correct programming.

This encapsulation allows for type hinting and the incorporation of behavior into value objects, overcoming the issue of primitive obsession.

Through encapsulation, operations related to the value are centralized, enhancing code semantics and utility.

4. Use First-Class Collections

This rule specifies that any class containing a collection should not have any other member variables. It emphasizing that each collection deserves its own dedicated home (class).

Such encapsulation turns collections into first-class citizens within the domain, providing a clear home for behaviors related to the collection, such as filtering, aggregation, or applying rules to each element.

Wrapping collections thus not only gives them a more defined structural role in the domain, but also elevates domain-specific concepts, making collections integral and well-defined components of your software's architecture.

This principle is akin to wrapping primitives, but specifically tailored for collections, ensuring that behaviors have a designated home and that collections are treated with the same care and consideration as other domain-specific elements.

5. Use Only One Dot per Line

It can be quite daunting to decide which object should take responsibility for an action.

But a great strategy to identify misplaced responsibilities is to search for lines with multiple dots, as these often indicate a violation of encapsulation or a middleman object that knows too much.

This rule underscores the importance of adhering to the Law of Demeter, advocating for direct communication with an object rather than "poking around" its internals.

Adhering to this guideline significantly improves readability and debuggability, as it isolates actions to specific lines and makes it easier to pinpoint errors. To better picture this, imagine having a trace that shows an exception on a line with 4 dots.

Exceptions to this rule include Fluent Interfaces and the Method Chaining Pattern, where the goal is clarity.

But even then, a better approach is through a "taller" rather than "wider" code structure. This means separating one dot per line to make it easier to reason about and follow.

6. Don’t Abbreviate

Naming is one of the most challenging tasks programmers face.

Abbreviations unfortunately don’t make this any easier. They obscure meaning and indicate underlying issues like duplication or misplaced responsibilities.

Resist the urge to abbreviate, as this, most often than not, leads to confusion and hides larger design problems.

Consider the reason behind the abbreviation desire; it may reveal opportunities to eliminate redundancy or indicate that responsibilities might be better allocated.

At the other end of the spectrum, avoid over bloating names unnecessarily — context can often eliminate the need for repetition. For example, within a User class, ID clearly refers to the user’s ID without needing further specification.

Striking the right balance in naming fosters clearer communication, reveals deeper design insights, and enhances code readability.

7. Keep All Entities Small

This guideline advocates for no class exceeding 50 lines and no package exceeding ten files. Its end goal is to promote cohesiveness and ease of understanding.

Its main suggestion is that large classes often undertake multiple responsibilities, detracting from their clarity and reusability.

As a bonus, small classes benefit from fitting on a single screen, aiding quick comprehension.

Yes, I know, nowadays we have huge monitors. But I hate squinting, thus I keep my code editor’s zoom to obnoxious percentages.

Not gonna lie though, 50 lines can get quite challenging to implement. Yet having this goalpost in mind, helps in creating small, focused classes and leveraging packages for logically grouped behaviors. And this can significantly enhance code quality.

My personal flavor of this rule though is to focus on method-level, that is aiming for 10 to 15 lines max per method, and adhering to a guideline of no more than five methods per class. I found that this helps prevent violation of the Single Responsibility Principle (SRP), avoiding "god classes" and ensuring shorter dependencies.

8. Don’t Use Classes with More Than Two Instance Variables

This rule works as a caution against mixing responsibilities. Consider it a manifesto to preserve class cohesion.

It encourages limiting classes to two instance variables to foster an effective object model and SRP adherence. Adding more instance variables generally decreases cohesion, prompting a thoughtful decomposition into a hierarchy of collaborating objects.

This principle aids in simplifying complex object models, ensuring behavior aligns naturally with the appropriate class, bolstered by the compiler and encapsulation rules.

Again, here as well I find that the rule is quite strict, but it is so for the sake of the exercise — to get you in the right mindset when writing production code.

Nonetheless we must all recognize the necessity of flexibility. Instances where more variables are logically cohesive can deviate from this strict guideline.

My personal rule of thumb is to stick to 5 instance variables, but then again, it’s a rule of thumb, not something set in stone. It’s rather something to raise your awareness and start questioning whether that class still respects the SRP principle or not.

9. Don’t Use Getters or Setters

This guideline is built directly on the principle of strong encapsulation from the previous rule. It seeks to ensure behaviors remain closely aligned with their respective instance variables, reducing direct access to object states.

This approach aims to centralize behavior in the object model, leading to less duplication, fewer errors, and easier implementation of new features.

Emphasizing the principle of "Tell, don’t ask", it encourages designing objects that manage their own data, fostering strong encapsulation.

I find this particularly beneficial in Domain-Driven Design (DDD) for modeling domain entities. When working with DTO’s or contracts from APIs, I find it harder to apply this rule. Even so though, I try to stick to private setters instead of opening the gates to unwelcomed alterations, to maintain encapsulation without overly restricting data access.

As if all of these 9 rules weren’t enough, I feel at liberty of adding a 10th one:

10. Don’t Use Comments to Justify Bad Code

Comments should not be used as a crutch for poor code; instead, the code should be refactored to address underlying issues directly.

The only exceptions for comments are to clarify unconventional decisions or to provide summaries in public libraries.

TODO’s are accepted, as long as they don’t make it to the main branch.

If it’s something that cannot be done in the current feature, add a task or a story on your project’s backlog. I found that having such TODOs in the code is simply useless, as they will just get old, unprioritized and forgotten in the code.

Conclusion

These rules, as outlined by Jeff Bay, serve as an exercise to strengthen your coding reflexes and intuition, just like physical calisthenics enhance your body’s flexibility and mobility.

While there may be contradictions or challenges in applying these rules universally, the intent is to foster a mindset conducive to writing better code.

It's about finding a balance, using the rules to guide but not constrain and integrating them as they fit comfortably into your development practices, ultimately refining your approach to coding in real-life production scenarios.

P.S. If you liked this post, share it with your friends and colleagues.

Share

This reality closely mirrors an important economic principle: the Laffer Curve.

The Laffer Curve illustrates the relationship between tax rates and tax revenue.

It posits that starting from a 0% tax rate, increasing tax rates will also increase revenue.

But that happens only up to a point.

Contrary to what some believe, it’s not a straight line. It’s a curve.

Beyond this peak, further increases in tax rates actually decrease revenue, as they discourage earning or promote tax evasion to reduce overall revenue.

The curve demonstrates that there's an optimal tax rate that maximizes revenue without discouraging economic activity.

This goes to show that every system has its optimal point.

If we return to our initial optimization question, we can easily see the resemblance.

Too little optimization and your application drags.

Too much, and you’re in a quagmire of complexity.

Over-optimization

Sky-high tax rates will eventually halt economic activity.

Like setting tax rates without economic understanding, optimizing without profiling is flying blind. It can lead to wasted efforts on parts of your code that don’t actually need it.

Think premature optimization. Pushing for optimization at every turn decreases overall productivity and hampers software maintainability.

Under-optimization

Low tax rates miss out on important revenue.

Under-optimized code can lead to lackluster performance. Although speedy at first, code that is not cared for is guaranteed to become a big ball of mud.

How to find the sweet spot

Measure and monitor

Determining optimal tax rates demands understanding of the economy.

In the same vein, you need to dive into the domain first to find bottlenecks and hot points. Profiling tools and benchmarks help you to understand where the hot points, worth optimizing, are.

Understand the law of diminishing returns

The Laffer Curve hints at evaluating trade-offs.

In code there’s a balance between performance and maintainability. Aim for the right level of optimization that provides both speed and clarity. Not all effort is equally fruitful.

Consider maintainability

Too high or too low taxes lead to long-term economic damage.

Both overly optimized code and spaghetti balls of mud are hard to maintain and cause issues down the line.

Pushing for extreme performance can make your code unreadable. Like high tax rates stifle business, dense code stifles collaboration and future changes.

Iterate and refine

Tax policies often need tweaks and revisions based on changing economic conditions.

Similarly, as your software evolves and grows, revisit optimization decisions.

Conclusion

Just like economics, code is an ecosystem. You can think of your codebase like an economy. It has its resources (data), processes (algorithms) and agents (functions).

Overburdening any aspect, like with excessive optimization, can disrupt the harmony.

Just as policymakers use the Laffer Curve to inform decisions, let balance guide your coding strategy.

The best performance comes from equilibrium. Not extremes.

P.S. If you liked this post, share it with your friends and colleagues.

Share

Deploying new versions of applications or features is a crucial aspect of software development, but they don’t have to be a pain.

You just need to carefully choose the right strategy for your project.

The right deployment strategy minimizes downtime and bugs, enhancing user experience, while a poor choice can lead to significant issues, including customer dissatisfaction and money down the drain.

Here's an in-depth look at five key deployment strategies, highlighting how each manages risk, user experience, and feedback during the deployment process:

→ Big-Bang Deployment

→ Blue-Green Deployment

→ Rolling Deployment

→ Canary Deployment

→ Ring Deployment

Big-Bang Deployment

This strategy, alternatively known as the the Recreate deployment strategy, involves a straightforward approach to updating applications by sequentially shutting down the current version (Version A) and then deploying the new version (Version B).

It is characterized by its simplicity, but inherently implies a period of downtime for the service.

Process

1. Shutdown of Current Version

   The process begins with the shutdown of Version A, the current live version of the application. This step is critical and marks the start of the deployment's downtime.

2. Deployment of New Version

   Following the complete shutdown of Version A, Version B is deployed. The duration of the deployment, including the boot time for Version B, directly contributes to the total downtime experienced.

3. Service Restoration

   Once Version B is successfully deployed and operational, the service becomes available to users again, marking the end of the deployment process.

Advantages

Simplicity

The recreate strategy is straightforward to understand and implement, requiring fewer steps and less complexity in comparison to strategies that involve parallel environments or incremental rollouts.

Clear Transition

It provides a clean slate for the new version, eliminating potential conflicts or complexities that might arise from concurrent versions.

Challenges

Downtime

The most significant drawback of this strategy is the inevitable service downtime, the duration of which depends on the shutdown and boot times of the application.

Effective use of the recreate strategy necessitates careful planning to minimize the impact of downtime, possibly scheduling deployments during off-peak hours.

Risk Management

The ability to quickly rollback in case of issues with Version B is limited, as it would require repeating the process in reverse, further extending downtime.

Strategies to mitigate the risks associated with this approach include thorough pre-deployment testing of Version B and preparing a rollback version in advance, should an immediate switch back become necessary.

The Big-Bang Deployment strategy' simplicity makes it a viable option for certain scenarios, particularly those where service continuity is not critically time-sensitive or where application architecture does not support more complex deployment methods.

However, the inherent downtime and risk factors require careful consideration and planning to ensure a successful update process.

Blue-Green Deployment

This approach employs two identical production environments, named Blue and Green, to facilitate updates with zero downtime.

It ensures a seamless user experience by allowing one environment to be live while the other is prepared with the new version.

Process

1. Initial Setup

   The Blue environment serves as the active site, delivering the current live version to users, while the Green environment is set up but remains idle.

2. Deployment to Green

   The new application version is deployed to the Green environment, allowing the Blue environment to continue serving users without interruption.

3. Quality Assurance

   The QA team tests the new version in the Green environment, identifying and resolving any issues. This stage is crucial for ensuring the new version's readiness without impacting the live user experience.

4. Traffic Switch

   Once the new version is deemed ready, the load balancer redirects traffic from the Blue to the Green environment. This switch is instantaneous, transitioning users to the new version with no perceived downtime.

5. Safety Net

   Post-switch, the Blue environment becomes the idle environment, ready to be activated if a need for rollback arises due to unforeseen issues with the new version.

Advantages

Zero Downtime

Ensures continuous service availability during updates.

Seamless User Transition

Users are moved to the new version without any disruption to their service.

Instant Rollback

Provides a quick fallback to the previous version if the new deployment encounters issues, enhancing reliability.

Challenges

Resource Intensive

Requires double the infrastructure, increasing operational costs and resource needs.

Complexity in Management

The need to maintain two parallel production environments introduces complexity, especially in ensuring data synchronization and seamless transitions.

Unified Switch

The switch from Blue to Green applies to all users simultaneously, unlike strategies that allow for targeted user exposure.

Blue-Green Deployment optimizes for uptime and rollback capabilities, presenting a robust strategy for managing deployments with minimal user disruption.

Despite its resource intensity and the complexity it introduces, the high level of control and minimized risk make Blue-Green Deployment a favored strategy for achieving a smooth user experience and ensuring reliable rollbacks.

Rolling Deployment

This strategy, also referred to as the ramped deployment method, is tailored for environments with multiple instances, where updates are applied gradually by replacing old version instances with new version instances.

This method ensures continuous service availability and minimizes user disruption.

Process

1. Starting Point

   Initially, all instances run the old version (Version A) and are capable of handling user requests. This capacity is crucial to maintain service availability as instances are updated.

2. Introduction of New Instance

   A new instance running the updated version (Version B) is created and replaced in the pool, beginning to serve user requests alongside existing Version A instances.

3. Phased Replacement

   Subsequently, instances running Version A are gradually replaced with instances running Version B. This step is carefully managed to ensure that the system can support simultaneous operation of both versions without service degradation.

4. Continuous Operation and Testing

   Throughout this process, the remaining Version A instances must be capable of serving all user requests to prevent downtime. Continuous testing during the update ensures that the application performs as expected.

5. Completion

   The update concludes when all instances have been replaced with Version B and the application has been validated for proper functionality. Should any issues arise during the deployment, there's the option to roll back to Version A across all instances.

Advantages

Service Continuity

By updating instances incrementally, the application remains available, preventing downtime.

Version Compatibility

Supports the simultaneous operation of different application versions, facilitating a smoother transition.

Flexibility

Parameters such as the number of instances updated at a time can be tailored to the application's needs, offering control over the deployment process.

Challenges

Complex Version Management

Requires the application to support running multiple versions concurrently, which may introduce complexity in ensuring compatibility and seamless operation.

Deployment Duration

The phased approach extends the total deployment time, especially for applications with a large number of instances.

Utilizing automation tools can significantly streamline the rolling update process, ensuring efficient management of instance replacement and scaling.

Rollback Time

While rollback capabilities are inherent, the process to revert all instances to the previous version can be time-consuming.

The Rolling Deployment strategy offers a balanced approach to software updates, combining reduced downtime with increased control over the deployment process.

Its advantages in maintaining service availability and mitigating risks make it a favored choice for many applications, especially those requiring high availability.

However, the inherent challenges and complexity necessitate careful planning, coordination, and the use of sophisticated deployment tools to execute successfully.

Canary Deployment

This is a strategic methodology employed to introduce a new version of software to a limited subset of users or servers before a full-scale rollout.

This approach is named after the historical use of canaries in coal mines to detect toxic gases, serving as an early warning to detect potential issues with minimal impact.

Process

1. Initial Deployment

   A small portion of the production environment, let’s say 25%, is updated to the new version, serving a select group of users. This group, the "canary," is carefully chosen to represent the broader user base.

2. Monitoring and Feedback

   The performance, stability and overall user experience of the new version are closely monitored. Feedback during this phase is critical for identifying any issues early on.

3. Incremental Rollout

   Depending on the initial feedback, the deployment is gradually extended to more users: 50%, 75%... This step-by-step approach allows for continuous monitoring and assessment of the new version's impact.

4. Full Deployment

   If the new version proves to be stable and improvements are consistently positive, it is then rolled out entirely, completing the deployment process.

Advantages

Risk Reduction

The strategy allows for targeted selection of the initial user groups or servers, enabling a controlled and insightful early assessment.

Limiting the initial deploy to a small percentage minimizes the potential impact of unforeseen issues, making it easier to address problems without affecting the entire system.

Deployment Control

The pace and scope of the deployment can be dynamically adjusted based on real-time feedback and performance indicators.

Challenges

Segmentation and Monitoring Complexity

Effectively selecting a representative canary group and closely monitoring their experience requires sophisticated tooling and analytics capabilities.

Limited Initial Impact

While the strategy aims to mitigate risk, any issues in the new version can still negatively affect the experience of the canary group.

Rollback Strategy

Should the new version encounter significant issues, a rollback plan is essential.

This involves gradually decreasing the exposure of the new version, similar to the rollout but in reverse, ensuring a smooth transition back to the previous version.

By meticulously introducing a new version to a subset of instances and expanding based on successful performance and feedback, the Canary Deployment strategy offers a prudent approach to software updates.

This method prioritizes user experience and system stability, ensuring that any new deploy contributes positively to the product's overall quality and reliability.

Ring Deployment

Also known as phased or progressive deployment, this is a strategy that introduces a new version of software to users in stages, based on predefined groups or "rings".

Each ring represents a segment of the user base or infrastructure, with the update progressively reaching broader audiences after successful validation at each phase.

This method allows for detailed monitoring and targeted feedback, reducing the risk of negative impacts from new deploys.

Process

1. Segmentation

   The infrastructure is divided into distinct segments, each forming a ring. These segments can be based on various factors, such as user roles, geographical locations or system criticality.

2. Initial Deployment

   The new version is first deployed to the innermost ring, often consisting of internal users, testers or a small, controlled external group. This stage allows for close monitoring and rapid issue identification.

3. Gradual Expansion

   Following successful validation in the initial ring, the deployment expands to subsequent rings, gradually increasing the user base exposed to the new version. Each expansion is contingent on the successful performance and stability of the software in the preceding ring.

4. Final Rollout

   After passing through all the rings successfully, the new version is made available to the entire user base, completing the deployment process.

Advantages

Incremental Risk Management

By staging the rollout across different rings, potential issues can be isolated and addressed with minimal impact, enhancing overall system stability.

The number and criteria for each ring can be adjusted based on deployment goals and user feedback, offering flexibility in how the deployment is structured.

There is also a lot of flexibility in regards to the speed at which the deployment moves from one ring to the next can be controlled, allowing adjustments based on observed performance and user feedback.

Enhanced User Experience

Feedback is more manageable and actionable when collected from specific user groups, facilitating targeted improvements and adjustments.

Users benefit from a more reliable introduction of new features or versions, with issues likely identified and resolved before reaching the majority.

Challenges

Complexity in Management

Orchestrating a ring deployment requires detailed planning and coordination, increasing the complexity of the deployment process.

Like Canary Deployments, ring deployments facilitate easier rollback if issues are identified in later rings, minimizing potential disruption, but having such a plan in place is required.

Extended Deployment Time

The phased nature of ring deployments can extend the overall timeline for achieving full rollout, potentially delaying the availability of new features to all users.

Each phase of the deployment may require additional resources for monitoring and support, impacting overall resource allocation.

Ring Deployment stands out as a strategic approach designed to balance the introduction of new software versions with the need for stability and user satisfaction.

By leveraging detailed segmentation and phased rollouts, organizations can effectively manage the risks associated with software updates, ensuring that new deploys are both robust and well-received by users.

Conclusion

Selecting an effective deployment strategy depends on balancing your project's priorities: speed, risk, user impact, and resource allocation.

• Big-Bang Deployment offers a straightforward update process when downtime is manageable.

• Blue-Green Deployment is ideal for zero downtime and easy rollbacks, suited for resource-rich environments.

• Rolling Deployment offers a compromise between continuous service and efficiency, perfect for gradual updates across multiple instances.

• Canary Deployment allows for real-world testing on a small scale, optimizing for early issue detection.

• Ring Deployment provides detailed feedback through phased rollouts across segmented user groups, ideal for complex systems.

The choice should align with your operational capabilities and risk tolerance, ensuring updates enhance both user experience and system reliability.

P.S.

One important thing to note is that deploying (the process of implementing code in test or live environments) is not the same as releasing (the process of making the version available to end-users). But more on that in a future post.

P.P.S. If you liked this post, share it with your friends and colleagues.

Share

Here's why.

The Bus Factor poses the question:

Or, more specifically, whose absence would jeopardize the project?

While it may sound morbid (or hilarious, as many terms in software industry), the bus factor assesses the risk of having crucial knowledge and skills concentrated in a few team members.

A vital factor for project continuity.

It’s all about addressing those areas of code where the team replies:

"Oh, that feature?

It’s Johnny’s feature, only him knows how that works."

You might think being indispensable is beneficial.

After all, they can't fire you if you're the sole keeper of knowledge, right?

Right?

If you’re in this camp, there’s disappointing news ahead: hoarding knowledge will also trap YOU.

If you're irreplaceable, promotions will easily pass you by. You could miss out on new projects because your presence is crucial in the current one.

Additionally, imagine the chaos when something goes wrong while you're on vacation. You don't want that call to happen, do you?

For freelancers, this can affect client referrals. No client wants to risk their project stalling because they shared you with others.

So, what can you do?

→ Document Processes

Keep thorough documentation for critical tasks and processes.

→ Mentor and Train

Share your expertise through mentoring and training.

→ Encourage Collaboration

Foster a culture of regular knowledge sharing and collaboration.

→ Implement Pair Programming

Don't ever underestimate the effectiveness of this method for spreading knowledge.

→ Conduct Regular Reviews

Have team members explain various project aspects, highlighting knowledge gaps.

By sharing your knowledge, you enhance team resilience and equip them to face challenges.

Additionally, you help maintain a high bus factor, meaning a greater number of people would need to leave the team before the project is in jeopardy.

And don't mistakenly think this approach only benefits the team and the project. It also showcases your leadership qualities and commitment to collective growth.

Conclusion

Being indispensable might seem like job security. But in fact, it can hinder career progression.

Making yourself replaceable shows leadership, strengthens the team, and opens up personal growth opportunities.

Reducing the bus factor isn't just about protecting the project. It's about creating a dynamic, adaptable, and resilient work environment conducive to everyone's growth.

P.S.

I also read about a different, more positive analogy, used by Microsoft - the lottery analogy:

However, while definitely less morbid, I don’t think it depicts the concept of multiple people leaving the team on a reasonable scale.

Do you and your team mitigate the bus factor?

P.S. If you liked this post, share it with your friends and colleagues.

Share

But what exactly is a unit?

The definition of a "unit" has been such a polarizing question, that it led to the formation of two schools of unit testing:

• Detroit (Classicist) school

• London (Mockist) school

So what’s the difference?

The Detroit school (classicist) argues that

• Unit tests themselves need to be isolated, not necessarily the units.

• A unit under test is more about a unit of behavior, rather than just code.

• Only shared dependencies (those which can make tests affect each other) should be replaced with test doubles.

The London school (Mockist) believes that

• Units under test (often classes) should be isolated from each other.

• All dependencies should be replaced with test doubles (like mocks or stubs) in tests (except for immutable dependencies).

Deep-dive comparison

Classicist (Detroit School)

• State-Based Testing

  Focuses on the state of the application after a unit's execution, verifying outcomes based on the input and the final state.

  This approach is less about how the result is achieved and more about ensuring the end state is correct.

• Integration of Collaborators

  Rather than mocking dependencies, favors using real instances of objects or in-memory test doubles, aiming to test units in a context that closely mirrors how they operate within the full application.

  This can lead to tests that are more integrative and less isolated.

• Refactoring Flexibility

  A hallmark of the Classicist approach is its emphasis on refactoring.

  Since tests are not tightly coupled to the implementation details, developers can refactor code with confidence as long as the external behavior remains unchanged.

• Critiques of Mocking

  Detroit school argues that excessive mocking can lead to brittle tests that are overly tied to the implementation details, potentially leading to false positives and hindering refactoring efforts.

Mockist (London School)

• Interaction-Based Testing

  Prioritizes testing every class interaction meticulously, using mocks extensively to ensure that objects communicate correctly with their dependencies.

  This approach focuses on the pathways through the code rather than the final state.

• Isolation

  By mocking dependencies, it aims to isolate the unit under test, ensuring that failures are localized to the unit itself rather than being influenced by external factors or the behavior of dependencies.

• Behavior Verification

  Tests in the Mockist approach often verify that certain methods are called with the expected arguments, emphasizing the behavior over the state.

  This can make the tests more sensitive to changes in the code's structure.

• Refactoring Constraints

  While Mockist tests can be very precise in ensuring correct interactions, they can also become tightly coupled to the code's structure, making refactoring more challenging without breaking tests.

  This is a significant critique of the approach, as it may limit the ability to evolve the codebase.

• Simplicity in Mocked Behavior

  Mocks are designed to be as simple as possible, only providing the minimum required to make a test pass, which can lead to a mismatch if the behavior of the class changes but the mocks in other classes' tests are not updated.

• Increased Interfaces for Mocking

  May result in more interfaces in production code to facilitate mocking, tying tests closely to the code's architecture and potentially increasing the refactoring or rework cost.

• Maintenance Cost

  High, particularly if classes are tightly coupled. The approach promotes projects with loosely coupled, independent classes to keep maintenance manageable.

Choosing Between the Schools

• London School for Libraries: Well-suited for library development where classes are loosely coupled, and the behavior of public APIs needs clear definition and stability.

  The low maintenance cost and ease of use of mocks make it an attractive choice for ensuring API behavior contracts.

• Detroit School for Applications and Microservices: Ideal for applications where business objectives dictate rapid responsiveness to change. The emphasis on comprehensive behavior verification and less frequent need to alter test code in response to production code refactoring makes it suitable for projects with dynamic requirements.

Balancing Trade-offs

• False Positives and Refactoring: Mockist approaches may lead to higher instances of false positives in tests, where tests fail not due to bugs but due to changes in the implementation details. This can slow down development and make the test suite harder to maintain.

• Black Box vs. White Box Testing: The Classicist approach aligns with black box testing, focusing on the inputs and outputs without regard to internal workings. In contrast, the Mockist approach is akin to white box testing, with a deep understanding of the internal pathways of the code being necessary to write effective tests.

• Debugging and Design Freedom: Mockist testing can offer more granular insights during debugging but may restrict design flexibility due to its prescriptive nature. The Classicist approach, while less granular, offers more design freedom and facilitates easier refactoring.

Ultimately, the decision between these schools should be guided by the project's specific requirements and the development team's preferences. A hybrid approach might even be the most effective in certain contexts, leveraging the strengths of both methodologies:

• Combining Approaches for Comprehensive Testing

  Use Mockist techniques for unit testing individual components and Classicist strategies for integration and system tests to ensure overall behavior correctness.

• Evolving Strategies Over Time

  Start with a Mockist approach in the early stages of development when interfaces are more volatile, and transition to Classicist methods as the software stabilizes.

Conclusion

The debate between Classicist and Mockist approaches in TDD is not about which is universally better but about understanding the trade-offs and advantages of both.

What each school calls "units" should not merely represent units of code but units of behavior — actions or sequences that are meaningful within the problem domain and recognizable as valuable by stakeholders, including those in business roles.

The significance of a unit lies not in its size or the number of classes it encompasses but in its relevance and utility in solving a real-world problem.

Whether a behavior spans multiple classes, a single class, or merely a fraction of a method, the focus remains on its purpose and contribution to the application's overall functionality.

The key is not to adhere rigidly to one school but to understand the benefits and limitations of each approach and pragmatically use them based on the specific needs of the project.

P.S. If you enjoyed this post, share it with your friends and colleagues.

Share

What if I told you that the difference between an API that's merely good and one that's truly great could be measured?

Richardson Maturity Model is a measure used to gauge the maturity of a RESTful API.

This model, proposed by Leonard Richardson, is divided into four levels, from Level 0 to Level 3, with each level introducing more sophisticated RESTful principles, culminating in the concept of HATEOAS (Hypermedia as the Engine of Application State).

Let's go through RMM using the example of an online bookstore system.

I will illustrate how the system's web service evolves from Level 0 to Level 3, showcasing the benefits and changes at each stage.

Level 0: The Swamp of POX

At Level 0, the online bookstore API does not adhere to REST principles.

It has a single entry point for all interactions, such as fetching book information, placing an order, or updating user profiles. Clients communicate with the service using only POST requests.

There's no standard mechanism to manage state or leverage HTTP's built-in capabilities.

This level is characterized by its simplicity but also its limited use of HTTP's potential and is reminiscent of SOAP-based services.

Such APIs are still found in legacy systems where changing architectural styles would require significant refactoring.

However, for new developments, you should avoid it in favor of more RESTful approaches.

Level 1: Resources

This level introduces the concept of individual resources.

The bookstore service starts to differentiate between types of resources by assigning them unique URIs. For instance, individual books, users, and orders each get their own URI.

To get information about a book, a client now sends a GET request to a specific book URI.

The response structure remains similar but is accessed through a more specific, resource-oriented URL.

This marks a significant step towards RESTful design by recognizing that different data entities should have separate URIs.

Level 1 is widely adopted as the minimum standard for web APIs.

It simplifies the design by allowing direct access to specific resources but does not leverage the full power of HTTP.

The resources should be plural nouns. Using verbs in URI is bad practice.

Level 2: HTTP Verbs

At Level 2, the bookstore service begins to utilize HTTP verbs appropriately to define actions on resources.

This aligns the actions with the type of request being made, such as GET for retrieval, POST for creation, PUT for updates, and DELETE for deletions.

To place a new order, a client now uses POST to an orders resource, with a payload specifying the order details, such as the book ID and quantity.

To update a book, a client uses PUT, specifying the book ID and providing the new data.

This not only adheres to the principles of REST, but also enhances semantic clarity and the potential for caching.

Level 2 is considered a best practice for RESTful API design. It's commonly seen in web services, offering a balance between simplicity and the expressive power of HTTP.

Level 3: Hypermedia Controls (HATEOAS)

The highest maturity level introduces Hypermedia as the Engine of Application State (HATEOAS). At this stage, responses include hypermedia controls, guiding clients on potential actions through links within the response, making the API self-descriptive and fully navigable.

A response not only includes details about the book but also links for actions such as placing an order.

Despite its benefits, Level 3 has been slower to gain widespread adoption.

Challenges include increased complexity in client implementation and a lack of tooling support.

However, it's seen as the gold standard for RESTful design, promoting loose coupling and evolvability.

Strong APIs that require a high degree of discoverability and flexibility implement HATEOAS principles.

10 Questions & Considerations:

1. REST, RESTful or REST HATEOAS?

• REST is the architectural style underlying the web, focusing on stateless communication, resource identification, and the use of standard HTTP methods.

• RESTful APIs are those that adhere to REST principles, aiming for a stateless, cacheable, and layered system with uniform interfaces.

• HATEOAS (Hypermedia as the Engine of Application State) is a component of RESTful APIs, requiring responses to include hyperlinks to other actions and resources, allowing for dynamic client navigation and coincides with Level 3 of RMM.

2. What Are the Purposes of Different HTTP Verbs?

Understanding the purpose of HTTP verbs and their proper use is crucial in designing RESTful APIs and ensuring they are both intuitive and adhere to web standards. Here's a detailed look at the key HTTP methods, their intended actions, and guidance on some common edge cases.

GET

Purpose: Retrieve data from a server at the specified resource. GET requests should be idempotent, meaning multiple identical requests should have the same effect as a single one.

Actions: Used to request data without causing any side effects. This is the method used for searching or querying resources with specific parameters.

HEAD

Purpose: The HEAD method is used to retrieve the headers of a resource, without the body. It's essentially a GET request without the response body. This is useful for checking what a GET request will return in terms of headers, status code, or to verify if a resource exists before downloading it.

Actions: Used to check if a resource exists. It can verify if there have been modifications to a resource (using headers like Last-Modified or ETag). Can also check the size of a resource before downloading it, via the Content-Length header.

POST

Purpose: Submit data to the server to create or update a resource, or trigger a non-idempotent operation. It does not require the client to know the resource's URI beforehand.

Actions: Used for creating new resources, submitting form data, or triggering operations that change the server's state.

PUT

Purpose: Replace the target resource with the request payload.

It is idempotent, meaning subsequent identical requests should ensure the same state on the server.

Actions: Typically used for updating existing resources in their entirety or creating a resource at a specific URL if it doesn't exist.

PATCH

Purpose: Apply a partial update to a resource. Like PUT, it is intended for modifying resources but only changes the parts specified in the request.

Actions: Ideal for updating only a subset of a resource's data, making it more efficient than PUT for large resources with minor changes.

DELETE

Purpose: Remove the specified resource from the server.

Actions: Used to permanently delete a resource. It is idempotent, as the resource remains deleted after multiple identical requests.

OPTIONS

Purpose: Describes the communication options for the target resource. It's used to discover which HTTP methods are supported by a web server for a given resource. This can be particularly useful in RESTful APIs for supporting cross-origin resource sharing (CORS) preflight requests.

Actions: Used to determine the HTTP methods (GET, POST, PUT, PATCH, DELETE, etc.) that a server supports for a specific URL. Can check for CORS policies by revealing allowed origins, HTTP methods, and headers. It’s also useful in assisting in the exploration and debugging of APIs by providing information about how to interact with a resource.

3. Are HTTP Methods Idempotent?

GET, HEAD, PUT, DELETE and OPTIONS are considered idempotent, meaning repeating the requests multiple times has the same effect as making a single request.

POST is not idempotent, as repeating a POST request can create multiple resources or trigger the operation multiple times. Patch can be idempotent, but it’s not a guarantee.

4. Can GET Requests Have Bodies? Should They?

While technically possible, GET requests with bodies are not recommended and widely unsupported, as most servers ignore it.

It's not standard practice to send a body with GET requests, and doing so can lead to unpredictable behavior.

GET requests should convey requests via URI and query parameters, adhering to the HTTP specification and ensuring compatibility and predictability across web services.

This allows filtering or paging resources without changing the server's state, hence maintaining idempotency.

For complex queries that do not fit well within a URL's query parameters, a POST request to a search endpoint might be more appropriate, despite traditionally being a GET operation. This is an exception rather than the rule.

5. Is a soft delete a HTTP DELETE or not?

Soft deletes can be a DELETE request that, instead of removing the resource, marks it as inactive. Alternatively, a PATCH to update a status could be more semantically correct, depending on the implementation. Here consistency and team agreements come into play.

6. What Are HTTP Status Codes?

HTTP status codes are standardized codes in the Hypertext Transfer Protocol (HTTP) used by web servers to indicate the status of a request's response. These codes are part of the response message from the server and help the client understand the result of their request. They are grouped into five categories, each signifying a different type of response:

1. 1xx (Informational): Indicates provisional responses, requiring the requester to initiate a follow-up request to receive the final response.

2. 2xx (Success): Signals that the request was successfully received, understood, and accepted.

3. 3xx (Redirection): Means further action needs to be taken by the user agent to fulfill the request, typically a redirection to another URI.

4. 4xx (Client Error): Indicates an error on the client's part, such as a bad request or unauthorized access.

5. 5xx (Server Error): Signifies an error on the server's part, indicating the server failed to fulfill a valid request.

Each category has various specific codes, such as 200 OK for a successful request, 404 Not Found when a resource can't be located, or 500 Internal Server Error for general server failures.

7. How to Use HTTP Status Codes?

They are used to communicate the outcome of HTTP requests.

Clients, such as web browsers or API consumers, use these codes to determine the next steps in their process.

For example, a 200 OK status may lead to the display of content to a user, while a 401 Unauthorized status could trigger a login flow. They are essential for RESTful API design, providing a standardized method for indicating success, failure, and required actions to clients.

8. Should I Use Status Codes for Business Validation?

HTTP status codes are well-suited for indicating the technical outcome of a request (e.g., whether or not it was successfully processed by the server).

Sometimes they can also be used to some extent for business logic errors, provided there's a good match between the status code semantics and the business error condition. For common business validation errors, such as input data failing validation checks, 400 Bad Request is appropriate.

However, it's important not to stretch the semantics of HTTP status codes too far to fit business logic errors. Because in most of the cases, you need more than that.

So what do you do, return 200 OK with a Business Error in the Response body?

The best of both worlds is returning a general status code like 400 Bad Request or 422 Unprocessable Entity (for semantic errors in the request) and include a detailed error message in the response body.

This approach allows you to communicate specific business rule failures or validation messages in a structured format that clients can parse and handle accordingly.

9. Should Hypermedia Include Links to All Possible Endpoints?

Hypermedia does not necessarily mean exposing all possible endpoints directly, but providing a way to navigate the API dynamically.

It's about providing relevant links and actions based on the current resource state or context, thus maintaining a balance between discoverability and security.

For instance, a user resource might include links to related resources like a user's posts or profile settings, rather than exposing unrelated endpoints.

This approach reduces coupling and improves discoverability without necessarily exposing the full API surface.

10. How Does Versioning Fit in?

Versioning APIs is a critical aspect of designing and maintaining web services, ensuring that they can evolve over time without disrupting existing clients.

Versioning within RMM should be handled in a way that maintains the API's RESTful nature.

Techniques like using HTTP headers for version information or leveraging content negotiation allow the APIs to introduce versions without breaking HATEOAS principles or cluttering URIs with version numbers.

The Richardson Maturity Model provides a structured path towards fully RESTful services, emphasizing the importance of using HTTP verbs appropriately, considering hypermedia's role in API design, and integrating versioning thoughtfully.

By understanding and applying these principles, you can create more robust, scalable, and maintainable web services that offer greater flexibility and user experience.

P.S. If you enjoyed this post, share it with your friends and colleagues.

Share

ACID and BASE are two models that describe different approaches and guarantees in handling transactions and data consistency. Understanding these concepts is key to selecting the right approach for database design and choosing the right type of database system for your specific application needs.

ACID Model

Traditionally associated with relational databases (RDBMS), ACID is a standard for ensuring reliable transaction processing.

The pillars of the ACID are:

→ Atomicity

→ Consistency

→ Isolation

→ Durability

These four principles are not just theoretical concepts. They are the crucial foundations that ensure transaction reliability and integrity in every database.

Let's break these down like we would in a real-world project:

→ Atomicity

Think of this as an all-or-nothing approach. When you commit a transaction, either the whole thing goes through, or none of it does. It's like a team effort in a code sprint — if one part fails, the whole thing needs a rethink.

→ Consistency

This is your quality control. It ensures that every transaction transforms your database from one valid state to another, never breaking the rules you've set.

→ Isolation

Here's where things get tricky with concurrent transactions. Isolation means keeping these transactions independent of each other. It's like making sure your threads don't step on each other's toes.

→ Durability

Once your transaction commits, it's set in stone, even if a system failure occurs immediately after. This is your safety net, ensuring that what's done stays done.

Use Case

Ideal for systems where data integrity and consistency are non-negotiable, such as banking and financial systems.

BASE Model

BASE is often used in the context of distributed databases, particularly NoSQL databases, where achieving high availability and scalability is more critical than strict consistency. It is in sharp contrast with the traditional ACID (Atomicity, Consistency, Isolation, Durability) model of relational databases.

BASE acronym stands for:

→ Basically Available

→ Soft state

→ Eventual consistency

Let’s break down what BASE means and how it shapes the behavior and design of NoSQL systems.

→ Basically Available

We're talking Availability over Consistency.

NoSQL databases prioritize availability. The system aims to ensure that the database is available for read and write operations, even in the face of network failures or partitioning.

This leads to flexible downtime handling. While the system might not be 100% consistent at all times (unlike ACID systems), it ensures that basic operations can still be performed, albeit with some flexibility in terms of consistency.

→ Soft state

This translates to having state flexibility. The state of the database is "soft", implying that it can change over time, even without input. It is a departure from the rigid, stable state maintained in ACID databases.

It also implies having some tolerance to change. This aspect of BASE acknowledges that the database doesn’t always have to be perfectly consistent. Instead, it can tolerate some amount of temporary inconsistency or latency.

→ Eventually consistent

The system needs to assume some delayed consistency. Eventually, the database will become consistent. However, this consistency is not guaranteed at all times, especially immediately following write operations.

System experiences a real-world sync.

Think of it like updates propagating across various social media servers. While some users might see the latest content immediately, others might see it after a short delay.

Impact of BASE in NoSQL

The system has high scalability and performance. By relaxing the constraints on consistency, NoSQL databases designed with the BASE model can achieve higher levels of scalability and performance, particularly beneficial for large-scale, distributed systems.

Use Cases

The ideal candidates are applications where availability and partition tolerance are critical. But where it's also acceptable for data to be slightly out of date or inconsistent temporarily. Think caching systems, user-facing applications with high traffic like social networks or e-commerce platforms and distributed applications.

ACID vs BASE: A Comparison

1. Consistency and Availability:

   • ACID: Prioritizes consistency.

   • BASE: Prioritizes availability.

2. Flexibility:

   • ACID: Offers less flexibility. It maintains immediate consistency, which may restrict access during network or power outages. Transactions queue up, causing delays.

   • BASE: More flexible. Allows applications to modify records more freely, without strict queuing or waiting.

3. Performance:

   • ACID: The overhead of maintaining strict consistency, isolation, and durability can impact performance, especially in high-load scenarios or distributed environments.

   • BASE: By relaxing consistency requirements, BASE databases can offer better performance, particularly in distributed systems where latency and network overhead can be a factor.

4. Scale:

   • ACID: Traditionally, ACID-compliant databases are optimized for reliability over scale. Scaling vertically (adding more resources to a single server) is common, but horizontal scaling (across multiple servers) can be challenging due to the strict consistency requirements.

   • BASE: Designed with distributed systems in mind, BASE databases excel at horizontal scaling. They can handle large volumes of data across multiple servers, making them suitable for big data applications and cloud computing.

5. Synchronization:

   • ACID: Requires tight synchronization to maintain consistency across transactions. This can lead to bottlenecks in distributed environments.

   • BASE: Allows for more loose synchronization, leveraging eventual consistency. This approach reduces bottlenecks but may lead to temporary data inconsistencies.

6. System Design:

   • ACID: ACID transactions are less flexible in terms of schema and data model changes. The rigid structure ensures data consistency but can limit adaptability to evolving data requirements.

   • BASE: Offers greater flexibility in handling dynamic and evolving data models, often found in NoSQL databases. This is particularly useful in applications where the data schema may change over time.

When to use each

When to Use ACID

ACID databases are best suited for applications where data integrity and consistency are paramount. These include:

• Financial Systems: Banks and financial institutions rely on ACID databases for processing transactions where accuracy and consistency of data are critical.

• E-Commerce Transactions: For handling purchases, order processing, and inventory management, where each transaction needs to be reliably recorded.

• Healthcare Systems: Patient records and medical transactions require strict data accuracy and consistency.

• Enterprise Applications: Many business applications require reliable transaction processing to maintain data integrity in operations like payroll, human resources, and supply chain management.

In these cases, the potential performance trade-offs are acceptable given the necessity for strict consistency and reliability.

When to Use BASE

BASE databases are well-suited for applications where scalability and high availability are more important than strict consistency. These include:

• Big Data Analytics: Applications dealing with large volumes of data where the focus is on analysis and reporting rather than transactional integrity.

• Social Networks: Handling a massive, distributed set of data where real-time data consistency is not as critical as availability and scalability.

• E-Commerce Product Catalogs: For managing dynamic information like product listings and prices, where eventual consistency is acceptable.

• Content Management Systems: Where high traffic demands scalability and the data (like comments, posts, etc.) can tolerate some degree of eventual consistency.

In these scenarios, the flexibility and scalability of BASE databases offer significant advantages.

Can a Database be Both ACID and BASE?

As per the CAP theorem coined by Eric Brewer around 2000, it's challenging for a database system to simultaneously provide consistency, availability and partition tolerance properties. Generally, databases lean towards either ACID or BASE principles, but cannot be both.

• ACID Databases (like SQL databases): Prioritize consistency and reliability but may not offer the same level of availability or scalability as BASE systems, especially in distributed environments.

• BASE Databases (like many NoSQL databases): Focus on availability and partition tolerance but do not guarantee immediate consistency across all nodes.

Microsoft Azure Databases

Azure offers a wide range of database services catering to different needs, making the distinction between those that follow the ACID (Atomicity, Consistency, Isolation, Durability) model, typically associated with traditional relational databases, and those that adhere to the BASE (Basically Available, Soft state, Eventual consistency) model, common in NoSQL databases.

Understanding these options is key for selecting the right database service for your specific application requirements.

Here’s a brief overview of the most popular database options available on Azure:

ACID Compliant Databases

These databases prioritize transactional integrity and data consistency, following the principles of ACID:

1. Azure SQL Database:

   • Type: Fully managed relational database service.

   • Based On: Microsoft SQL Server engine.

   • Key Features: Offers advanced SQL capabilities, built-in intelligence for performance tuning, and global scalability.

   • Use Cases: Ideal for applications requiring complex transactions, rich data models, and strong SQL support.

2. Azure Database for MySQL:

   • Type: Fully managed database service based on the open-source MySQL Server.

   • Key Features: Built-in high availability, automatic backups, and scaling capabilities.

   • Use Cases: Best for applications that use MySQL as their database, such as web apps, content management systems, and e-commerce platforms.

3. Azure Database for PostgreSQL:

   • Type: Managed service for PostgreSQL.

   • Key Features: Offers built-in high availability, automatic backups, and scaling. Also provides advanced features like Hyperscale (Citus) for horizontal scaling.

   • Use Cases: Ideal for applications requiring advanced features of PostgreSQL, including geospatial capabilities, complex queries, and rich data types.

4. Azure Database for MariaDB:

   • Type: Managed service for MariaDB.

   • Key Features: Automated backups, built-in high availability, and easy scaling.

   • Use Cases: Suitable for businesses using MariaDB for their applications, especially when there is a need for open-source compatibility.

BASE Compliant Databases

These databases are designed for high availability and scalability, typically following the BASE model:

1. Azure Cosmos DB:

   • Type: Globally distributed, multi-model NoSQL database service.

   • Key Features: Supports document, key-value, graph, and column-family data models. Offers turnkey global distribution, multi-region replication, and strong consistency models.

   • Use Cases: Suitable for large-scale applications needing high availability, low latency, and support for various data models and types.

2. Azure Table Storage:

   • Type: NoSQL service that stores large amounts of structured, non-relational data.

   • Key Features: Offers a key/attribute store with a schema-less design. Cost-effective for storing massive amounts of non-relational data.

   • Use Cases: Ideal for applications requiring flexible data schema with quick lookups and large-scale storage.

3. Azure Cache for Redis:

   • Type: Fully managed in-memory data store based on Redis.

   • Key Features: Provides high throughput and low latency data access. Ideal for caching frequently accessed data.

   • Use Cases: Primarily used for high-performance caching, Redis can support different models depending on its configuration. While it's typically used in a manner aligning with BASE for caching scenarios, it can be configured for data persistence and some degree of transactional support.

Conclusion

The choice between ACID and BASE depends on the specific requirements and nature of the application.

• ACID is the go-to for applications requiring strong consistency and reliability.

• BASE is ideal for scenarios where availability, flexibility, and scalability are paramount.

Understanding the strengths and limitations of each model is crucial when designing systems that effectively meets the client’s needs and constraints.

P.S. If you enjoyed this post, share it with your friends and colleagues.

Share

Let’s face it, as systems evolve, they become chattier and chattier and it is critical to ensure their reliability while also mitigate costs.

To achieve this, it's crucial to understand the differences between queues and topics, events and commands, and to know what the right messaging service for your system is.

In this article, I'll cover all these aspects, along with an in-depth look at all the Azure messaging services.

Let's explore them one by one.

Queue or Topic?

Both are used in messaging systems for communication between different parts of a system. But they have distinct characteristics:

→ Queues

They follow a point-to-point model. Messages sent to a queue are processed by only one consumer.

Each message in a queue is processed once and only once. It's a method of ensuring that a particular task is executed only once.

Some use case examples would involve processing a specific job such as generating a report or sending an email.

→ Topics

Are based on a pub-sub model, where messages are published to a topic and multiple subscribers can listen and receive these messages.

A single message sent to a topic can be received by multiple subscribers.

This is useful for broadcasting messages.

Some use case examples would involve updating multiple microservices about a change in the state of a user’s order.

Comparison

The choice between using queues or topics depends on the specific requirements of your system.

Queues are better suited for situations where a specific action needs to be taken, with each message processed only once.

Topics are ideal for situations where a message or notification needs to be broadcasted to multiple parts of the system without expecting a direct response.

Command or Event?

While they both are types of messages in a system, they serve different purposes and imply different behaviors.

→ Commands

A command is a directive to perform a specific action. It's an instruction to the system to change its state, usually named in the imperative mood (e.g., CreateOrder, ShipItem).

Commands are active. When a command is sent, it's expected that the receiving component will act upon it. It is a form of request from one part of the system to another.

Commands are targeted, usually sent to a specific component or service responsible for handling them.

Use Cases

Commands are used when the intention is to cause a change or perform an action. For instance, a ProcessPayment command would instruct the system to execute the steps associated with processing a payment.

→ Events

An event is a notification that something has happened.

It's a statement of fact and is typically named in the past tense (e.g. OrderCreated, ItemShipped).

Events are passive. When an event is sent out, it does not expect a direct action to be taken in response.

Can be broadcast to multiple listeners or handlers who may choose to act upon them.

They help in decoupling the system. The publisher of the event does not need to know who is consuming or listening to the event.

Use Cases

Events are used for notifying the system about changes in state or other occurrences. For example, an OrderCreated event might trigger various actions like updating inventory, notifying the user, or initiating a billing process.

→ Key Differences

1. Intent

Commands request that something should happen.

Events inform that something has happened.

2. Direction

Commands are directed at a specific target with the expectation of a response or action.

Events are a broadcaster, notifying the broader system and don't care much about the response.

3. Payload

Commands have a rich payload usually, containing all the necessary data for the specific component to be able to fulfil the request.

Events are slimmer, not knowing what data the interested parts of the system might need to perform their respective actions.

4. Coupling

Commands can imply tighter coupling between the requester and the performer of the action.

Events tend to support looser coupling between components (publishers don't need to know about subscribers)

How to decide what to choose?

1. Evaluate the nature of the interaction.

If the interaction is about notifying other parts of the system about changes or occurrences without expecting a direct action, use events. If it's about explicitly requesting an action or change, use commands.

2. Consider the system design.

Assess how tightly coupled your components are and how much you want them to know about each other. Events can help in creating a more decoupled architecture.

3. Ponder complexity and scalability.

Commands can make workflows and user actions more explicit and traceable. Events can be more scalable and can simplify complex interactions by decoupling components.

Comparison Events are about notifying the system of occurrences and allowing independent components to react as needed, promoting loose coupling. Commands are about explicitly requesting actions or changes, often implying a more tightly coupled interaction.

Azure Messaging Services

Azure offers a variety of messaging services tailored to different needs, from simple queues to complex event routing. Understanding the features, use cases, and differences between these options is crucial for designing an effective architecture. Here's a comparison of Azure's primary messaging systems:

1. Azure Queue Storage

• Overview: A simple, cost-effective service for storing large numbers of messages that can be accessed from anywhere via authenticated calls using HTTP or HTTPS.

• Use Cases: Ideal for simple tasks such as background tasks, job scheduling, and batch processing.

• Pros: Easy to use, highly scalable, and economical.

• Cons: Limited to simple scenarios, lacks advanced features like topic-based subscriptions or filtering.

2. Azure Service Bus

• Overview: A more advanced enterprise-level messaging system that supports queues, publish-subscribe (topics), and more complex integration patterns.

• Use Cases: Suitable for high-value enterprise messaging where topics, sessions, transactions, and ordering are required.

• Pros: Offers advanced features like FIFO (First-In, First-Out) messaging, duplicate detection, and deferred messages.

• Cons: More expensive than Azure Queue Storage and can be overkill for simple needs.

3. Azure Event Hubs

• Overview: A highly scalable data streaming platform and event ingestion service, capable of receiving and processing millions of events per second.

• Use Cases: Ideal for big data scenarios, telemetry, data analytics pipelines, and application logging.

• Pros: Exceptional performance and scale, capable of handling massive amounts of event data.

• Cons: Focuses more on event streaming than message operations like dead-lettering or message deferral.

4. Azure Event Grid

• Overview: A fully-managed event routing service that uses a publish-subscribe model for uniform event consumption.

• Use Cases: Perfect for reactive programming, integrating with various Azure services, and building event-driven architectures.

• Pros: Provides filtering, multicasting, and event fan-out capabilities. It's deeply integrated with Azure services and supports custom webhooks.

• Cons: Primarily focused on event signaling rather than data transfer and doesn't support long polling or other typical message queue functions.

Comparison and Considerations

1. Scalability

While all services offer high scalability, Event Hubs is the go-to for massive-scale event ingestion, and Service Bus is better for enterprise-level messaging.

2. Complexity and Features

Service Bus offers complex features and patterns, suitable for intricate messaging needs, whereas Queue Storage is for simple scenarios. Event Grid excels in routing and handling events from various sources.

3. Pricing

Costs can vary significantly. Queue Storage is generally the most economical, while Service Bus and Event Hubs might incur higher costs due to their advanced capabilities.

4. Integration

Consider how well each service integrates with your existing architecture. Event Grid and Event Hubs offer seamless integration with many Azure services.

Selecting the right messaging option in Azure depends on the specific requirements of your application, such as the nature of the data, required throughput, desired features, and cost considerations.

• For simple queue-based messaging, Azure Queue Storage is straightforward and cost-effective.

• For more complex enterprise messaging, Azure Service Bus offers robust features.

• For massive-scale event processing, Azure Event Hubs is unparalleled.

• For event routing and reactive programming, Azure Event Grid provides excellent capabilities.

Understanding the strengths and use cases of each service allows you to make an informed decision that best suits your application's needs.

P.S. If you enjoyed this post, share it with your friends and colleagues.

Share

At first glance, you might think they’re one and the same, akin to twins, but look a little closer and you'll find they're more like distant cousins.

Here's why.

CQS (Command Query Separation): General behavioral principle Coined by Bertrand Meyer in his book 'Object-oriented Software Construction' (1988, Prentice Hall), CQS is a principle asserting that an object's methods should either be Commands or Queries, but never both.

• Commands perform an action.

• Queries solely return data.

If you consider the codebase to be a bustling city, CQS applies the traffic laws to ensure everything flows smoothly.

It keeps the streets (methods) clear of accidents (unexpected side effects) by preventing the same method from making changes AND broadcasting those changes.

Let's look closer at this traffic management system. Let's say a method is responsible for both registering a vehicle's speed and determining if it's exceeding the speed limit.

This violates the Command Query Separation principle as it combines a side effect (recording the speed) with information retrieval (checking for a speed limit breach).

Let's fix that simply by separating the state-changing behavior from the state-retrieving one.

This separation clarifies the responsibilities of each method, making the code easier to read, understand, and maintain. It also prevents unexpected side effects, which are common in methods that combine querying and state modification. By creating this distinction you’ll be much more confident to mix and match queries tailored to your needs. With commands you’ll still need to be thoughtful.

Now, of course that doesn’t mean we need to be complete purists when it comes CQS, especially when it comes to returning a value from a command.

Exceptions occur, such as when you’re creating a new entity and you return an id. But while commands returning a value is not the end of the world, it’s crucial you don’t alter the state in a query.

All in all, when applied at the method or class level, CQS is all about clarity in design and making your code easier to reason about.

CQRS (Command Query Responsibility Segregation): The Architectural Evolution

CQRS takes the essence of CQS and expands it into the realm of system design.

Here, the game changes.

We're no longer talking about methods. We’re talking abut segregating the entire system into two parts:

• one for commands (writes)

• one for queries (reads).

Think of your system using two separate databases:

• one for reading the data.

• one for writing data in.

Why the split?

To allow independent scalability and optimization.

With CQRS, you can scale your read and write workloads independently. You're free to optimize and modify the read side without touching the writes, and vice versa.

It's having two specialists handling what they do best.

Now, of course it’s not compulsory to have two databases in order to apply CQRS. You can begin implementing CQRS starting with a logical separation between your read and update models in your system and grow from there, balancing scalability and simplicity.

Because with great power comes...you guessed it: complexity.

Implementing CQRS isn't a walk in the park.

You're dealing with separate models, tackling consistency concerns, and often venturing into eventual consistency territory.

It's a trade-off, a balancing act between scalability and simplicity.

Event Sourcing: Time Travel Made Possible

• CQS:

  • Mostly on the sidelines for this play.

  • It doesn’t inherently interact with or leverage event sourcing.

• CQRS:

  • A frequent collaborator with event sourcing.

  • It elegantly stores changes as a breadcrumb trail of events, allowing systems to rewind, replay, and predict.

Consistency: A Balancing Act

• CQS:

  • Operates in a world where consistency is a given, not a luxury.

  • It’s all local, straightforward interactions here.

• CQRS:

  • Walks a tightrope with consistency, especially in distributed systems.

  • Immediate consistency isn't always a guarantee, making way for its more relaxed cousin, eventual consistency.

Integration with Other Patterns:

• CQS:

  • It is a principle that’s more internal and localized.

  • It doesn't necessarily dictate or heavily influence the broader architectural decisions and is often orthogonal to other patterns.

  • It’s about making the individual components of a system (the methods) more understandable and manageable, reducing complexity where functionality and data retrieval are concerned.

• CQRS:

  • It’s a top-tier player in system design, often used in concert with other high-level patterns.

  • It pairs exceptionally well with event sourcing, where the system's state changes are logged as a series of immutable events.

  • This relationship allows for advanced capabilities, like system snapshots or an audit log with "undo" functionality.

Complexity and Suitability:

• CQS:

  • Given its method-level governance, adopting CQS doesn't require a significant overhaul of the system architecture.

  • Suitable for systems of various sizes.

  • It demands a cultural shift in how developers approach writing methods/functions, aiming for pure, side-effect-free functions.

• CQRS:

  • Implementing CQRS is no small feat.

  • It involves a fundamental restructuring of how the system handles data.

  • It's ideal for large-scale, enterprise-level applications where scalability, high performance, and reliability are paramount.

  • Its trade-off is increased complexity in data consistency and transaction management, potentially leading to higher initial development and maintenance costs.

Conclusion CQS and CQRS, though stemming from similar principles, serve two different purposes.

CQS excels in method-level clarity and directness.

CQRS tackles system-wide segregation, scalability, and embraces complexity.

They are not just acronyms, but approaches tailored for specific paths in software engineering.

P.S. If you enjoyed this post, share it with your friends and colleagues.

Subscribe now

Latency is so important to master that Gmail’s creator, Paul Buchheit, even coined a rule in regards to it: the 100ms rule.

But more on that later. Let’s first start with the basics.

What is latency?

It refers to how fast a signal or data packet travels from one point to another in a network or system. It's essentially a delay measured in milliseconds (ms) or even microseconds (µs). Latency significantly impacts system performance and user experience.

When we talk latency, we can split it into different types:

• Network Latency: The time taken for a data packet to travel across a network from sender to receiver.

• Disk Latency: The time it takes for a disk to complete a read or write operation.

• Processing Latency: The time it takes for a system to process a given input.

Now circling back to the 100ms rule.

It states that every interaction should be quicker than 100ms.

Why?

Because 100ms is the threshold “where interactions feel instantaneous.”

But how fast is 100ms really?

The chart illustrates various computer and network events with their latencies, presented in actual time (nanoseconds to seconds) and a scaled, more relatable version for our common understanding.

While the actual latencies might seem incredibly short, the scaled version provides a perspective on their relative impact.

Notes:

• Actual Latency: Given in nanoseconds (ns), microseconds (µs), milliseconds (ms), or seconds (s). Represents the real-time it takes for the event.

• Scaled Latency: A metaphorical translation to a human-understandable timescale, showing how we might perceive these latencies if they were stretched out.

Key Network Events and Latencies

• CPU Cycle: The base of all computations. Lower cycle times mean faster processing but understanding the latency from memory access is crucial for optimizing performance.

• Cache Access: Different levels of cache have varying access speeds. Optimizing which data is stored in each can vastly improve performance.

• Memory Access: Accessing RAM is significantly slower than accessing CPU cache. Knowing this helps in designing systems that minimize memory access bottlenecks.

• Disk I/O: Both SSDs and HDDs have latencies orders of magnitude higher than memory access. Systems designed to reduce disk access can greatly benefit.

• Network Requests: The latency involved in sending data over a network can be unpredictable and varies greatly. Efficient network protocols and data caching strategies are vital.

• Reboots and Timeouts: The longest latencies come from system reboots and timeouts, scaling to hundreds and even thousands of years, underscoring the relative slowness of these operations in the digital realm.

Why Latency Matters?

It's crucial across all tech sectors. For example:

Here are some examples:

• Cloud Computing and Data Centers

  Amazon found that every 100ms delay in web page loading time can lead to a 1% loss in sales. This means that for a company like Amazon, with sales of $514 billion in 2022, a mere 100ms delay could potentially cost around $5.14 billion in sales annually.

• Online Gaming

  Riot Games reported that when they improved the latency for "League of Legends" players in North America by moving servers, there was a notable increase in player engagement. Players in affected regions played up to 7% more games when they experienced a 10-30ms improvement in latency.

• Streaming Services

  Netflix's research revealed that a delay of 2 seconds in video start-up time increased the chance of abandonment by 6%. Moreover, for every 0.5 seconds of additional buffering, there are 1% fewer starts of any videos on their platform.

• Financial Trading

  In the world of high-frequency trading, a 1 millisecond advantage in trading applications can be worth $100 million a year to a major brokerage firm, as reported by TABB Group.

• Healthcare and Telemedicine

  According to a study by the American Telemedicine Association, the use of telestroke services (rapid response video consultations for stroke victims) in rural areas, where quick response is critical, resulted in as much as a 25% decrease in patient disability. This is highly dependent on low-latency connections for real-time video consultations.

• Automotive and Transport

  For autonomous vehicles, a study suggested that reducing latency from 100ms to 10ms in vehicle communication systems can decrease collision rates by 20-30%. This could potentially save thousands of lives considering there were an estimated 38,800 traffic fatalities in the US alone in 2019.

• Virtual and Augmented Reality

  In a study conducted by Nvidia, they found that reducing latency from 50ms to 20ms in VR systems reduced user discomfort significantly, with reports of nausea dropping by over 50%. This can lead to longer, more engaging VR experiences and higher user retention rates.

How can you mitigate latency

• Profiling and Monitoring: Regularly profile and monitor applications to understand where latencies are occurring.

• Caching Strategies: Implement effective caching to reduce the need to access slower storage.

• Asynchronous Programming: Use asynchronous operations to prevent application blocking during long operations.

• Optimize Data Transfer: Minimize data sent over the network and use efficient serialization methods.

• Choose the Right Tools: Use databases, libraries, and frameworks known for performance and suited for your specific needs.

• Optimize Code: Ensure that your application code is efficient and not causing unnecessary delays.

• Upgrade Hardware: Use faster processors, more RAM, or SSDs instead of HDDs to reduce processing and disk latency. Yes, that translates to faster resources in the cloud.

• Content Delivery Networks (CDN): For web applications, use CDNs to cache content closer to users, reducing network latency.

• Load Balancing: Distribute traffic evenly across servers to avoid overloading any single resource.

• Network Optimization: Use network optimization techniques like compression or faster protocols to reduce data transmission times.

Conclusion

While inherent in all systems, understanding and reducing latency can greatly enhance performance and user satisfaction. By recognizing the sources of latency and implementing strategies to mitigate it, you can ensure that your system remains efficient, responsive, and scalable.

Whether you're a network administrator, a software developer, or a system architect, managing latency effectively is a key part of optimizing the performance and reliability of your operations.

P.S. If you enjoyed this post, share it with your friends and colleagues.