Build your own software

I built this blog in an uncommon way, so I wanted to share. This post will be about the why behind my choice to build another static site generator. The next post will be about the what with the technical details.

There are so many tools for generating static sites, why build another? In short, I am biased towards building my own software systems.

Hidden risks

My thinking has grown up on Rich Hickey talks. I mention this not for the content of any single talk he has given but the general philosophy of software design he shares. Daniel Higginbotham has tried to capture some of Hickey's philosophy here.

I have developed a healthy fear given the risks of depending on systems I don't understand, written by someone who's goals and biases of which I'm unaware and with unknown future development path. Not to mention the real legal risks of different software licenses. I have been burned plenty after deeply investing in learning a new software system and building my own software on it's abstractions, only to discover that the abstraction makes something simple difficult or the implementation has a bug that was difficult to anticipate then to debug or fix. It's disempowering to need to dig into the guts of a new abstraction when your goal is to just move past it.

With SaaS services, there are risks at every moment of outages, network failures or data loss as your system has hidden dependencies on many teams of people, legacy architecture choices and their own SaaS dependencies. It's frustrating when your systems are down and you're left without recourse.

As David Foster Wallace discusses in his speech "This is Water", the longer and deeper you submerge your mind in an environment (or programming language, or software tool) then the more your thinking begins to resemble the facts and limits of that environment. As you become more invested in someone else's software ideas, you may forget that some challenges you're facing are not common or miss that entire classes of errors exists in your environment that don't exist elsewhere.

Ken Thompson is modern, again

At this point in my journey, I think about software in terms of a composition of simple and reliable tools (ala Ken Thompson's UNIX philosophy). Investing in building with a small set of tools & abstractions is slow in the short term but carries significant long term benefit. Earlier in my career, I wasted many hours learning broad but not deep, chasing the new popular language or framework but not sticking with any of them long enough to achieve mastery. Economies of scale exist in building deep knowledge (true in other disciplines, too!).

I'm not convinced that software mastery can result in perfection but practice does find improvements. The more small tools you build, the more ideas and code you have to draw upon when you need to compose something bigger. I haven't found that larger software systems have the same economies of scale; they tend to have their own specific quirks and norms.

Further reading on Rich Hickey's philosophy of software development

I'd highly recommend reading or listening to Rich Hickey's talks but if you're short on time then Daniel Higginbotham has to captured some of Hickey's philosophy here.

Published: 2020-11-24

Tagged: dsl clojure blog