Macroexpand the Mind

Analysis Paralysis

There are lots of things that can kill a project before it starts: lack of funding, lack of time, lack of specification, lack of applicable talent, the list goes on and on. Many of these problems can be alleviated with creative resource management and a bit of elbow grease. There is, however, a far more subtle mental phenomenon that can be as destructive to the creator as it is to the fledgling project:

Analysis paralysis or paralysis of analysis is the state of over-analyzing (or over-thinking) a situation so that a decision or action is never taken, in effect paralyzing the outcome. A decision can be treated as over-complicated, with too many detailed options, so that a choice is never made, rather than try something and change if a major problem arises. A person might be seeking the optimal or “perfect” solution upfront, and fear making any decision which could lead to erroneous results, when on the way to a better solution.

Source: Wikipedia – Analysis Paralysis

It’s incredibly easy to start second guessing every decision you’ve made (or are going to make) before you’ve made your first commit. In the early stages of any project, one has to consider all the possible options available that could be used to accomplish the task at hand and, from that list, choose just “the best” one. To be sure, decisions made at this stage can greatly affect the outcome of the project. But here’s the rub: none of this will matter if you never actually ship the project. It’s always preferable to pick a path that turns out to be wrong versus picking the path that leads to nothing at all.

So here’s my advice to anyone setting out on a new project:

Just pick something, anything, to get the ball rolling. Your requirements will change, there will be unplanned roadblocks, and that’s OK. Worry about those things as they come up, not before. And most of all, do not let your analysis paralysis get in the way of being useful!

Struggling With Technical Debt

Every company I’ve ever worked at has struggled with the concept of Technical Debt. If you’re not familiar with the term

Technical debt (also known as design debt or code debt) is a neologistic metaphor referring to the eventual consequences of poor or evolving software architecture and software development within a codebase.

Source: Wikipedia – Technical Debt

A “move fast and break things”-esque culture, can make it hard to take a step back and make sure the frankenhack you’ve created even qualifies as an MVP. When the pressure is on and product expected delivery yesterday, I think we all have a tendency to shutout that little voice in our head telling us to do it the Right Way ™. But at some point crunch time will end, and you’re left maintaining something that’s hardly passable as a product, much less a paramount of engineering prowess.

Process to the… rescue?

In my experience, processes like Scrum and XP serve to exacerbate rather than alleviate the problem if not managed carefully. Here’s how it usually goes:

  1. Start working on a [poorly scoped/written/sized/etc] story
  2. Discover something ugly that needs to be re-worked, but isn’t in the scope of your current story
  3. Write a fixit chore in your project tracker with a pithy title and high priority label
  4. Watch as that ticket is immediately prioritized under the “real” work by the product team

This pattern carries on ad infinitum until the aforementioned frankenhack buckles under its own complexity. Usually at this point someone utters the word “re-write”, much to the horror of everyone else in the company. Yes, we’re pushing out code at a break-neck pace, but at what cost? Would this level of flippancy about our craft be tolerated in any other engineering profession?

Dash of plot-thickening agent

Almost without fail, when the topic of technical debt comes up with product or businessy people around, someone presents this golden insight

Why don’t you just fix it as you see it?

There are at least two obvious reasons why this is deeply misguided at best, and positively destructive at worst. First, refactoring tends to be a recursive process with no clear base case. You might innocently start by ripping out a function or cleaning up the logic of some helper class, but then you have to track down all the tests for that bit of code. Oh, and while you’re at it, track down all the clients of that code which are hopefully well-factored already (unlikely) and tested in a way that doesn’t depend directly on the implementation of that thing you just rewrote (also unlikely). This process continues recursively until that one class or function you were refactoring turns into a multi-hundred line commit across tens of files.

The second problem is far more insidious at a process level. Ask yourself this question: do the above refactorings really fall under the scope of the current story? If the answer is no, then your 1 point story can easily turn into a 3 or 5 pointer in no time. (I’m using sizing jargon here loosely; translate the points to whatever unit of work your organization uses.) This high degree of churn can be absolutely demoralizing to a team that’s used to shipping on a regular basis. The high level of busy work without forward progress can create stress and burnout throughout the product and engineering teams.

Lights flicker in the distance

Many of these problems seem to compound and intensify as the engineering team grows. There are no silver bullets, and process alone isn’t going to get you very far (it might even drag you backwards along the path). This is a problem I reckon most engineering organizations struggle with and it is our duty as working professionals to come up with a viable long-term solution.

For the record, I don’t really know what that solution looks like, but here are some strategies I’ve used as both a manager and developer over the years with varying degrees of success:

  • Stop compromising so much! It’s your job to ship features, yes. It is also your job to give other stakeholders (product, bizdev, marketing, etc.) the appropriate feedback and transparency so they can properly scale their expectations. There is of course an implicit requirement here that those stakeholders actually care what you have to say. Without that, many organizations are doomed to failure before they have a chance at success.

  • Be noisy about debt. Don’t let those fixit stories sink into the backlog abyss. Product-focused processes like Scrum drive the development of user-facing features often at the cost of engineering discipline. Engineering should strive to push their priorities through with the same level of commitment and force as the other stakeholders.

  • Perfectionism != Debt-reducing. In my younger days, I was absolutely obsessed with perfection. Not to say everything I wrote was perfect (far from it), just that I spent an exorbitant amount of time thinking about the Right Way ™ to do something. In the end, especially when working in a group of more than 1 or 2 people, the outcome will be the same (or worse). You’ve spent so much time building a perfect house of cards, you can’t remember exactly how it all fits together. This is a classic example of optimizing for the wrong thing.

  • Set time aside on the roadmap. If all else fails, create “cleanup sprints”. These are usually 1 or 2 week sprints where the engineering team doesn’t build any new features and instead focuses on rebuilding existing code to meet higher quality standards. These sprints need to be carefully managed as they can easily turn into partial rewrites (often generating even more spaghetti code).

Especially in the start-up world, I think the problem of Technical Debt can easily reach crisis levels. In an environment when you’re already strapped for time and money, how can you possibly slow down to fix the things you’ve broken? There isn’t an easy solution, but we owe it to ourselves and our organizations to start the discussion today.

Learning to Blog by Blogging

I really have been meaning to start a blog for quite some time. Every couple of days I think of another idea for a post I’d love to write, a neat piece of code I’d love to share, or a fun little whatever I think is awesome. So why haven’t I done it until now you ask? I’ve gone through a large number of rationalizations, not the least of which are:

I don’t want to maintain it.

or

I don’t want to design and style it.

or even

I don’t want to waste time when I could be doing other things.

But these are just things I tell myself; little lies to keep my emotions in congruence with my actions. The real reason is far more simple: fear. I have an irrational fear of blogging.

I’m not afraid of rejection. Some people absolutely cannot stand it when others disagree with their opinion. Years of IRC and active forum usage have taught me that all opinions are controversial. Some people will hate you for no other reason than it’s in their very nature to do so.

I’m not afraid of public speaking. Blogging, in my mind, is this generation’s most common form of public speaking. But I present to groups on a regular basis, I’ve always been outspoken in academic and professional settings, and I love having open heated debates about whatever the topic du jour happens to be. I absolutely love leading the discussion and helping to drive people to new ideas or levels of understanding.

I’m not afraid of confrontation. Well in the physical sense I suppose I am, but that’s a different matter entirely. If programmers are known for anything, it’s their willingness to bikeshed incessantly about things that fundamentally do not matter. We are used to being the masters of our code and want to know something about everything. We yearn to feel important, especially amongst our peers.

Well ok, these are all lies. I’m human; it’s in my very nature to avoid sources of discomfort. But in reality, they pale in comparison to the most terrifying possibility of all: loneliness. What if I spend all this time pouring out my heart, only to be met with no audience at all. No hate mail, no dissent, no nothing. My words are just quiety ignored and swallowed into the infinite abyss of the Internet.

So I’ve made a decision: I’m writing these words for me. I’m writing them because I want to be a better writer. I’m writing them because I want to be better at expressing and communicating my thoughts and ideas. I’m writing them because my ideas matter. I’m writing them because I matter. I’m writing this because I want to contribute to the narrative of our generation.

The thoughts and ideas contained within this blog are a part of me. They are a small glimpse into my reality – my human experience.

This is my narrative, allow me to share it with you.