Run! The Distributed Systems are coming!

You can avoid the fate of those that fell before you

This is a letter to my fellow engineers, specifically those who operate in the front-end world. This letter is sent with love, but carries a message of deep foreboding. It is a warning and yet it is a message of hope, that there is a chance to maintain peace and happiness in your work life.

This letter is inspired by a question recently posed by the talented Andy Pimlett – with whom I had the pleasure of serving alongside the lovely people of Mando. See Andy Pimlett on LinkedIn: JavaScript Front-End Architecture. Therein Andy poses a question that has been keeping my mind in constant churn: ‘what does the line between the UI and the backend platform look like now?’. More importantly to me however is raising awareness of the quite frankly ridiculous levels of complexity that hide behind the innocent term ‘Microservices’, as I shall lay out here…

Lurking in your Jam

The devil doesn’t come dressed in a red cape and pointy horns. He comes as everything you’ve ever wished for.

Tucker Max – Apparently

Jamstack is exactly the sort of revolution the front-end world has desperately needed for too long. It promises to reduce the long-standing pain resulting from complexity and bloat that has emerged out of the constant over-engineering introduced by well intending technologies such as npm, React and Angular.

I’ll not cover the ‘Client’ end definition of Jamstack as there are plenty of perfectly good resources for that already including the Jamstack website itself: https://jamstack.org/. To be absolutely clear on my position here, I have been blown away by the power of Jamstack. This really is the tech that the web needs now. You want to put your money into new web technology? I’m telling you it is Jamstack and its global community; Web3 can get in the bin (really, forget Web3, it’s another last-ditch Ponzi-scheme from the crypto bros to try and keep their awful blockchain investments on life-support for another year).

That said, the threat to your peace and happiness that I wish to warn you about lies, unbeknownst to many, innocently nestled away in a corner of a diagram featured in many of the Jamstack articles, videos and websites that talk about this otherwise wonderful new technology:

Buried in these tempting looking simple diagrams lies a terrible, terrible beast that wears a red coat and charges through your peaceful villages shooting anyone who fails to kneel before the might of the King.

(Let’s forget for a moment that all the diagram shows is that the app server on down has been moved into a small little innocent tidy-looking box alongside CDN)

A Microservice by any other name will be just as difficult

Scenario: You’re building an application… You are writing some logic that calls a method on a class in another area of the application.

Question: Would calling the method be much improved by sending it on a trip along a network?

The only sensible answer, surely, would be: “are you mad?”. And, yet, here we are, some 10 years on since the term ‘Microservice’ was first bandied about, with half the planet decomposing their applications into separate ‘Microservices ‘ and having them directly call each other synchronously across a network. This antipattern now even has its own name: ‘distributed monolith’ and it is known to give all the pain of a monolith plus the pain of a distributed system and basically none of the benefits.

Why would anyone willingly triple the complexity of an application for no tangible benefit? Well… first of all… I guess… we’re social creatures, if our peers and those we look up to are doing something then we naturally take an interest. If everyone in your industry is talking about a thing then it becomes so well known that you may even be getting pressure from your boss or clients to do that thing.

Secondly… we’re techies, nerds, geeks, we like the challenging things. Hey, if Netflix do it and it gives them great results then the complexity must be worth it and it will feel great to wield complexity and win fortune for the business!

Finally, I think the name has a lot to do with it. How much harm can come from something called a ‘microservice’? Well, would you willingly embark on the path of a Service Oriented Architecture for your UI tier? Because that is what you are doing  whether you realise it or not. Microservices are a slant on the classic ‘Distributed System Architecture’. It has all, if not more, of the complexity of your classic Distributed System, but the name ‘microservice’ doesn’t really give you that feeling. Sure a Microservice must be an easy service, easy to deploy, easy to build, easy to integrate? No. No. No. Read on…

Distributed Systems destroyed my technical career

If ever there was a clear way to demonstrate how complicated Distributed Systems really are, it lies in my career trajectory. During the first half of my career I always imagined I was destined to become a technical architect. That is until I encountered technical architecture in Distributed Systems design. Now I am a hands-off manager. I find that managing teams of people and managing projects with all of the pressures and complexities that come with those responsibilities is far easier and more rewarding than programming Distributed Systems. But of course some people love this stuff! I’m not saying it is bad or a terrible choice, just that I do not have the technical chops to survive a career in this area and I don’t consider myself to be very dim. I certainly wouldn’t want to wander into this quagmire by accident!

So, what, exactly, are you trying to protect me from?

I get it, by now you’re probably sick of hearing me describing how difficult I find Distributed Systems. So let me tell what I know that engineers must know to be able to deliver effective Distributed Systems, or Microservices, or SOA, or Cloud Native Architecture, or MACH or whatever you want to call it – they all have the same fundamental concerns.

  1. Loosely coupled architecture – You need it. Its numero uno for any distributed system. Without it you have a distributed monolith which isn’t a distributed system at all. Loosely coupled means embracing asynchrony and with that comes eventual consistency, which your application must also embrace. Asynchrony is different from ‘async’ calls. Instead your tools in this space include message queues, service buses and brokers – more specifically for cloud native you will include events and streams.
  2. SRE – Distributed Systems are so complex that the industry consensus is that you should assume your system is in a failure state and go from there. I’m not even kidding. See Software Reliability Engineering, you will likely need someone skilled in this on your engineering and operational teams. You need to make sure your system is consistent, available and durable. This really is a career path in its own right!
  3. Deployment Strategies – You need to be able to safely update your microservices, components and so on without disrupting service. See the Google DORA research in this area for the different strategies you will need to familiarise yourself with: https://cloud.google.com/architecture/application-deployment-and-testing-strategies
  4. Observability: Modern cloud native systems will often have hundreds of components interacting with each other to achieve common business goals. How do you know if the system is healthy or not? How do you know the lifecycle of your customer’s data through your system for compliance with data protection standards? Enter Application Performance Monitoring and Distributed Tracing. See: https://opentracing.io/docs/overview/what-is-tracing/

Hey, I would always default to a single application until such time that the technical demands necessitate decoupling. Monoliths are NOT a dirty word, they have many benefits in terms of maintainability. A helpful rule here is thus: if your components need to call each other directly to achieve their goals then keep them together in a monolith and save yourself a world of pain. Most UIs that call down into a distributed architecture will probably be fine with a single UI aggregation tier or at most BFFs (Sam Newman) https://samnewman.io/patterns/architectural/bff/). Introducing loosely coupled architectures into your UI strikes me as a one-way trip to pain.

Your options have been laid before you…

Know that it is not a technical decision, it is a business decision and often closely tied to your business architecture.

It is too late for many of us in backend/platform engineering but I am worried for those of you responsible for client tier architecture. The concept of ‘Microservices’ is a creeping menace and I hope that these lessons serve to protect your sanity and stress levels.

So don’t feel pressured into a Distributed Architecture just because it’s a name you’ve heard (Microservices, MACH) or part of a wider architecture that you already get some benefits from (i.e. Jamstack). Just be aware that those little boxes on those diagrams belie decades of computer science that will likely never become as simple as the tempting diagrams want you to believe.

Understand whether you truly need microservices and avoid them if at all possible! But! But just because I struggle to imagine many scenarios where a UI tier would warrant a distributed architecture does not mean you don’t need it. Netflix and Amazon need it but so do some smaller businesses. Hell, my current employer has carefully considered its technical strategy, weighed the pros and cons and has decided that the increase in complexity of a distributed system is worth it for their commercial strategy. Know that it is not a technical decision, it is a business decision and often closely tied to your business architecture.

If you wish to stand and fight then be prepared to wield your weapons of Loosely Coupled Architecture, SRE, Deployment Strategy and Observability. But be sure, no one will blame you should you choose to consolidate your efforts in something far simpler; something that protects your daily life, bringing joy to you and your customers.

Run! The Distributed Systems are coming!

Babeling in defence of JavaScript

And so it goes, the eternal question “What is wrong with JavaScript?” and the inevitable, inescapably droll, reply:

Oh, ho ho ha ha haaaaaaaaaaah… The gag never gets less funny. I need to be clear that Scott Hanselman is one of my favourite people in the public eye. I hold him to be an industry treasure and I’m fully aware of him just poking fun here but we’ve all seen this dialog before and we all know it is not always so lighthearted.

At the end of the day, these scenarios showing how ‘broken’ JavaScript is are almost always bizarrely contrived examples that can be easily solved with the immortal words of the great Tommy Cooper:

Patient: “Doctor, it hurts when I do this”
Doctor: “Well, don’t do it”

Powerful Facts

Lets be absolutely clear that JavaScript is an incredibly powerful language. It is the ubiquitous web programming language. Of course it currently has a monopoly that ensures this status. That does not change the fact that JavaScript runs on the fastest, most powerful and most secure websites. So clearly it does exactly what is needed when in the right hands.

JavaScript is free with a very low barrier to entry – all you need is a web browser.

JavaScript through its node.js guise powers Netflix, LinkedIn, NASA, PayPal… The list goes on and on.

Furthermore it is easy enough to learn and use that it is a firm favourite for beginners learning programming. It is in this last point that we observe some particularly harmful industry attitudes towards JavaScript.

What’s The Damage?

So now that we can all agree that Tommy Cooper has fixed JavaScript from the grave and now that we’re clear about just how seriously capable JavaScript is as a language, we can get onto the central point: industry attitudes to JavaScript are damaging. While many languages such as SQL and PHP are common targets of derision and it seems to me that each case has it’s own unique characteristics and nuances, there is something notably insidious about the way JavaScript is targeted.

One of the more painful examples of JavaScript’s negative press can be observed in the regular reports from those learning programming that they feel mocked for learning JavaScript. This is, quite frankly, appalling. We work in an industry that is suffering from a massive global undersupply of talent and we’re making potential recruits feel like crap. Well done team! Even globally established personalities such as Miguel de Icaza of Xamarin fame can’t help but fan these flames. What chance do new recruits have?

The JavaScript Apocalypse?

Moving on to the issue that prompted me to start writing this article; WebAssembly is here. It has a great website explaining all about it: webassembly.org. It even has a logo! It also has a bunch of shiny new features that promise to improve the experience of end users browsing the web.

WebAssembly logo
Of course WebAssembly has a logo!

From distribution, threading and performance improvements to a new common language with expanded data types, WebAssembly offers a bunch of improvements to the web development toolkit. I’m all for these changes! JavaScript and the web programming environment are far from perfect and these are another great step in the right direction.

Of course WebAssembly’s common language also promises to open up the web client for other programming languages. “Hurrah!” I hear many cheer. I’m seeing countless messages of support for the death of JavaScript at the hands of the obviously infinitely superior quality languages of C#, Rust and Java 🙄 Yeah… I’m not so sure…

Nah!

Like most programming languages, JavaScript is a product of its environment: namely, the web browser. It did have competition in the early days with VBScript back in IE4/5… I think… It was a long time ago. But otherwise it has developed on its own in response to demand from the web developer community and in response to the changing web landscape. The modern incarnations of JavaScript (ECMA Script 6/7/8) are incredibly powerful, including modern language features such as an async programming model, functional capabilities and so on. In many ways modern JavaScript resembles the languages to which it is so frequently compared but it also lacks many language features that are less relevant to web client programming such as generics and C#’s LINQ. It’s loose typing system make it well suited for working with the HTML DOM. Overall it would appear, as you might expect, that JavaScript is made for web client programming and is in fact the best choice for this task.

Even the WebAssembly project agrees, confirming on the project website that JavaScript will continue to be the ‘special’ focus of attention and you know what? This is a good thing!

Babel

Look, we already have other languages that compile for the web client but I don’t see any existential threat from the (albeit beautiful) CoffeeScript or from the (misguided) TypeScript. Sure, WebAssembly will make this more effective but the reasons that TypeScript hasn’t already taken over the web development world will still apply to C# and WebAssembly. We have seen a similar battle play out in the database world where NoSQL was lauded as the slayer of the decrepit 1970’s technology we all know as SQL. That was until NoSQL databases started to implement SQL. Turns out that SQL is hard to beat when it comes to querying data, which is unsurprising when you consider its 50-odd years of evolution in that environment and the same rule will apply to any JavaScript challengers. Personally I suspect a large part of JavaScript’s alternatives failing to take hold is that web client programming doesn’t need the added static typing, etc.; in my experience all these challengers do is introduce compiler warnings and complexity that waste time. Ultimately I don’t have all the answers here but it is fair to say that it would take a serious effort to out-web the language that has evolved for the web environment.

The Tower of Babel (from WikiPedia)

Where my real concern lies is in the well known problems that are brought about by having too much choice when it comes to communicating. We use human readable programming languages so that we can communicate our programs to each other. With that in mind it is clearly more effective in the long run if we all learn to talk the same language. The story of The Tower of Babel shows us that for a long time we have considered too much choice to be a very bad thing when it comes to communication.

It would be a frustrating situation indeed if we were to end up having to consider and manage the overhead of multiple languages for a single task all because of some daft attitudes towards JavaScript. Furthermore, businesses that are struggling to find web developers don’t now also need to worry about whether these developers are Rust, Java or C# web developers. JavaScript is the right tool for the job so lets stop wasting time with all the JavaScript bashing and get on board with an incredibly powerful language we can all understand!

Babeling in defence of JavaScript

A functional solution to interfacitis?

/ˈɪntəfeɪsʌɪtəs/
noun
noun: interfacitis
inflammation of a software, most commonly from overuse of interfaces and other abstractions but also from… well… actually it’s mostly just interfaces.

An illness of tedium

Over the years my experience has come to show me that unnecessary abstractions cause some of the most significant overheads and inertia in software projects. Specifically, I want to talk about one of the more tedious and time consuming areas of maintaining abstracted code; that which lies in the overzealous use of interfaces (C#/Java).

Neither C# or Java are particularly terse languages. When compared to F# with its Hindley-Milner type inference, working in these high-level OO languages often feels like filling out forms in triplicate. All too often I have experienced the already verbose syntax of these languages amplified by dozens of lengthy interfaces, each only there to repeat the exact signature of it’s singular implementation. I’m sure you’ve all been there. In my experience this is one of the more painful areas of maintenance, causing slowdowns, distraction and lack of focus. And I’ve been thinking for some time now that we’d probably be better off using interfaces (or even thin abstract classes) only when absolutely necessary.

What is necessary?

I like to apply a simple yard stick here: if you have a piece of application functionality that necessitates the ability to call multiple different implementations of a component then you probably require an interface. This means situations such as plugins or provider-based architectures would use an interface (of course!) but your CustomerRegistrationService that is called only by your CustomerRegistrationController will not. The message is simple, don’t start introducing unnecessary bureaucracy for the sake of it.

There are, I admit, cases where you might feel abstraction is required. What about a component that calls out to a third party system on the network? Surely you want to be able to isolate this behind an interface? And so I put it to you; why do you need an interface here? Why not use a function? After all, C# is now very well equipped with numerous, elegant functional features and many popular DI frameworks support delegate injection. Furthermore if you are following the SOLID practice of interface segregation then chances are your interface will contain only one or two method definitions anyways.

An example

So, for those times when you absolutely must abstract a single implementation, here is a simple example of an MVC controller using ‘functional’ IoC:

public class RegistrationController : Controller
{

    private readonly Func<string, RegistrationDetails> _registrationDetailsQuery;

    public RegistrationController(Func<string, RegistrationDetails> registrationDetailsQuery)
    {
        _registrationDetailsQuery = registrationDetailsQuery;
    }

    public ActionResult Index()
    {
        var currentRegistration = _registrationDetailsQuery(User.Identity.Name);

        var viewModel = ViewModelMapper.Instance
            .Map<RegistrationDetails, RegistrationDetailsViewModel>(currentRegistration);

        return View(viewModel);
    }
}

 

Addendum

13-March-2018: It has been pointed out to me that a further benefit of this approach is that static providers may also supply IoC dependencies whereas instances are required for interface-based IoC. What are your thoughts on this approach?

A functional solution to interfacitis?

Things I wish I knew 10 years ago: Abstractions

We need to talk about abstractions

The main reason I decided to start this blog is that I have begun working for a company that has genuinely challenged many of my assumptions about how software should be developed. I have spent much of my career learning from the more prominent voices in software development about how to write software effectively. I have learned, practiced and preached the tenets of clean code, TDD, layered design, SOLID, to name a few of the better known programming practices and had always believed that I was on a true path to robust, maintainable software. Now I find myself in a position where over the space of just one year I have already questioned many of the practices I had learned and taught in the preceding decade.

I hope to share on this blog much of what I have discovered of late but for my first entry discussing programming practices I want to talk about abstractions. In particular I want to call into question what I have come to understand as overuse of abstractions – hiding implementations away in layers/packages, behind interfaces, using IoC and dependency inversion – as often encountered in the C#/.NET and Java world.

Abstractions?

I have been wondering lately if I have simply spent years misunderstanding and misapplying abstractions, but I have seen enough code written by others in books, tutorials, blogs, sample code and more diagrams than I can bear to know that I have not been alone in my practices. Furthermore, I have found myself on a few occasions of late in discussions with developers of similar experience who have come to share a similar feeling towards abstractions.

The all too familiar layer diagram
The all too familiar layer diagram. © Microsoft. https://msdn.microsoft.com/en-us/library/ff648105.aspx

A typical layering structure
A typical layering structure

So what do I mean by abstractions and what is the point of them, really? The old premise and the one that I would always reiterate is that abstractions help enforce separation of concerns (SoC) by isolating implementation details from calling code. The reasoning being that code of one concern should be able to change without affecting the code dealing with other concerns, supposedly because code dealing with one concern will change for different reasons and at different times than the code dealing with other concerns. Of course we mustn’t forget that one of the more natural causes of abstractions is the isolation of logic to enable Unit Testing. Ultimately the result is that software is written in such a way that the different code dealing with different concerns is kept separate by abstractions such as interfaces and layers while making use of IoC and Dependency Injection to wire the abstractions together. Furthermore it is worth me stating that the usual separate ‘concerns’ touted by such advocacy frequently includes Presentation/UI, Service/Application Logic, Business Logic, Data Access Logic, Security, Logging, etc.

[Authorize]
public class StudentController : Controller
{

    private readonly IStudentRepository _repository;
    private readonly IStudentService _service;
    private readonly IUnitOfWork _unitOfWork;

    public StudentController
    (
        IStudentRepository repository, 
        IStudentService service, 
        IUnitOfWork unitOfWork
    )
    {
        _repository = repository;
        _service = service;
        _unitOfWork = unitOfWork;
    }

    public ActionResult UpdateStudentDetails(StudentDetailsViewModel model)
    {
        if (ModelState.IsValid)
        {
            var student = _repository.Get(model.StudentId);

            student.Forename = model.Forename;
            student.Surname = model.Surname;
            student.Urn = model.Urn;

            _service.UpdateStudentDetails(student);

            _unitOfWork.Commit();
        }

        return View(model);
    }
}

Abstracted code, obscurity through indirection.

YAGNI!

I am not about to start claiming that everything should just be thrown together in one Big Ball of Mud. I still feel that SoC certainly is worth following but it can be effectively achieved by applying simple encapsulation, such as putting more repetitive and complex logic of one concern within its own class so that it may be repeatedly invoked by code dealing with other concerns. An example of this would be the code to take an entity key, fetch and materialize the correlating entity from a data store and return it to the caller. This would be well served in a method of a repository class that can be called by code that simply needs the entity. Of course packages/libraries also have their place, in sharing logic across multiple applications or solutions.

Where I see problems starting to arise is when, for example, the aforementioned repository is hidden behind an interface, likely in a separate layer/package/library and dynamically loaded by an IoC infrastructure at runtime. Let’s not pull any punches here, this practice is hiding significant swathes of software behind a dynamic infrastructure which is only resolved at runtime. With the exception of some very specific cases, I see this practice as overused, unnecessarily complex and lacking in the obvious transparency that code must feature to be truly maintainable. The problem is further compounded by the common definition of the separate concerns and layers themselves. Experience has shown me that when coming to maintain an application that makes use of all of these practices you end up with a voice screaming in your head “Get the hell out of my way!”. The abstractions don’t seem to help like they promise and all of their complexity just creates so much overhead that slows down debugging and impedes changes of any significant proportion.

With one exception I have never spoken to anyone who has ever had to swap out an entire layer (i.e. UI, Services, Logic, Data Access, etc.) of their software. I’ve personally been involved in one project where it was required but it was a likely eventuality right from the start and so we were prepared for it. I have rarely seen an example of an implementation of an abstraction being swapped or otherwise significantly altered that did not affect its dependents, regardless of the abstraction. Whenever I have seen large changes made to software it very rarely involves ripping out an entire horizontal layer, tier or storage mechanism. Rather it will frequently involve ripping out or refactoring right across all layers affecting in one change the storage tables, the objects and logic that rely on those tables and the UI or API that relies on those objects and logic. More often than not large changes are made to a single business feature across the entire vertical stack, not a single conceptual technical layer and so it stands to reason that should anything need separating to minimise the impact of changes it should be the features not the technical concerns.

Invest in reality

So my main lesson here is that: The reality of enforcing abstractions through layering and IoC is very different from the theory and usually is not worth it, certainly when used to separate the typical software layers. With the exception of cases such as a component/plug-in design I am now completely convinced that the likelihood of layered abstractions and IoC ever paying off is so small it just isn’t worth the effect that these abstractions have on the immediate maintainability of code. It makes sense in my experience not to focus on abstracting code into horizontal layers and wiring it all up with IoC but to put that focus into building features in vertical slices, with each slice organised into namespaces/folders within the same project (think MVC areas and to a lesser extent the DDD Bounded Context). Spend the effort saved by this simplification keeping the code within the slices clear, cohesive and transparent so that it is easy for someone else to come along, understand and debug. I’d even go so far as to try to keep these slices loosely dependent on each other – but not to the point that you make the code less readable, i.e. don’t just switch hard abstractions of layers into hard abstractions of slices. I don’t want to offend anyone, I’m just putting my experience out there… why not give this a try… I promise you probably won’t die.

Vertical slices with MVC Areas
Vertical slices with MVC Areas

Take a look at the following updated controller action. You know almost exactly what it is doing just by looking at it this one method. This contains ALL of the logic that is executed by the action and to anyone first approaching this code they can be confident in their understanding of the logic without having to dig through class libraries and IoC configuration. Any changes that are made to the action would simply be made here and in the DB project, so much more maintainable! Being completely honest, even recently, seeing code written like this would rub me up the wrong way so I understand if this gets some others on edge but I’ve come full circle now and am pretty convinced of the simplified approach. And its this dichotomy I’d like to discuss.

[Authorize]
public class StudentsController : Controller
{
    public ActionResult UpdateStudentDetails(StudentDetailsViewModel model)
    {
        if (ModelState.IsValid)
        {
            using (var context = new StudentsContext())
            {
                var student = context.Students.Single(s => s.Id == model.StudentId);

                student.Forename = model.Forename;
                student.Surname = model.Surname;
                student.Urn = model.Urn;

                SendStudentDetailsConfirmationEmail(student);

                context.SaveChanges();
            }
        }

        return View(model);
    }

    private void SendStudentDetailsConfirmationEmail(Student student)
    {
        ...
    }
}

Transparent, maintainable, intention-revealing code and no need for IoC!

This is just an opening

So this has been my first attempt to open up some conversation around the use of abstractions in software. I’ve tried to keep it brief and in doing so I’ve only just scratched the surface of what I have learned and what I have to share. There is still so much more for me to cover regarding what I and others I know in the community have been experiencing in recent years: Should we abstract anything at all? What is maintainable if not SoC via IoC? How do we handle external systems integration? What about handling different clients sharing logic and data (UI, API, etc.)? How does this impact self/unit-testing code? When should we go the whole hog and abstract into physical tiers? I could go on… So I intend to write further on this subject in the coming weeks and in the meantime it would be great to hear if anyone has any thoughts on this, good or bad! So drop me a line and keep checking back for further posts.

Things I wish I knew 10 years ago: Abstractions