The Linear Developer Experience

Many complex systems exhibit a phenomenon known as a tipping point. A tipping point is a non-linear event, when a small quantitative change triggers a significant qualitative change that is often irreversible. These moments of rapid and dramatic change play a prominent role in diverse fields of study, such as physics, sociology, economics, epidemiology, as well as software engineering. Knowing about tipping points, understanding where and when they occur, helps organizations and individuals anticipate and mitigate the significant consequences for teams and the applications they work on.

As history repeatedly demonstrates, the complexity of software continues to grow as software becomes more broadly and deeply integrated into every aspect of our lives. Companies are quickly recognizing the need to provide a good developer experience to manage that complexity in order to boost productivity and increase developer satisfaction. While the need for a good developer experience is widely recognized, there is no consensus on its definition.

However, a common sentiment among software developers is that a project’s developer experience starts out delightful but eventually and suddenly becomes dreadful.

Let’s take a Django web application as an example: After you set up a Django project and generate and configure the initial application, development follows a straight path. First, you create models, controllers, views, and unit tests. Django does the heavy lifting to create an admin interface as well as database migrations when your models change. As the application evolves, you add more models, more controllers, more views, and more unit tests. Django makes updates to the admin interface and performs the database migrations.

We could describe the Django developer experience as linear: While the application grows quantitatively (more models, controllers, views, etc.), it remains the same qualitatively (the architecture of the application is still expressed as models, controllers, views, etc.). In other words, even though your application gets bigger, your developer experience does not change. Hence, the application's developer experience grows linearly.

Until you hit the tipping point! Just one more requirement, just one more feature, a small quantitative change that triggers a significant qualitative change, the straw that breaks the camel’s back, the last drop that overflows the barrel.

There are many factors that push us over the tipping point, but some are consistently at play: Implementing a multi-step process that spans multiple database transactions or orchestrates multiple services, a process that outlives a single request-response lifecycle, or time-based actions.

For example, Django, like most web frameworks, supports only single-step processes well. Per request, the Django framework executes one request handler, which is a plain function. In turn, that request handler invokes one database transaction. However, how do we approach a task that does not solely execute one database transaction or one service call, or a task that outlives the duration of a single request? How do we implement a task that requires two, five, or ten transactions, that orchestrates two, five, or ten services, or that runs for minutes, hours, days, weeks, months, or years?

Every project I’ve worked on hits a tipping point. We’ve all had the meeting where the tipping point becomes apparent. Sometimes we were junior engineers, and quietly listened. Other times we acted as senior engineers, and hotly debated terms such as orchestration, choreography, commands, events, job queues, job scheduling, or outbox patterns. Our arguments flooded the room, but regardless of our role, the imminent turn for the worse quickly became apparent. We had arrived at the tipping point!

At this point, Django no longer feels like my ally. Django feels like my adversary. Instead of empowering me, Django holds me back. I find myself needing to work around restrictions and limitations. Databases, queues, and timers are the duct tape holding my application together. Sagas, commands, and events are the crutches on which I drag myself through the day.

I have two options: I can embrace the tipping point as inevitable, or I can reject the tipping point as inevitable.

By embracing the tipping point, I can either declare defeat now or later. If I am willing to declare defeat now, I begin building my application as if I’ve reached the tipping point already. From day one, I implement my application in terms of sagas, commands, and events, I orchestrate and choreograph, and I enqueue and dequeue in anticipation that eventually I will have to go down that path anyways. Well, if my developer experience is dreadful from day one, at least I can’t be missing what I once had. If I am willing to declare defeat later, I begin my application as if no tipping point looms on the horizon. I soothe myself with euphemisms, I tell myself that if I don’t have to re-architect and re-implement my applications later, I am making a mistake now. KISS—keep it simple, stupid. YAGNI—you ain’t gonna need it. I can find comfort in an ecosystem armed with acronyms assuring me all of this is fine.

However, it’s not. If I embrace the tipping point, whether I declare defeat now or later, eventually quality, productivity, and satisfaction will suffer. Engineers will be writing scores of code to “plug into” hunting and fixing heisenbugs in the edge cases of the application, barely able to predict or reason about the consequences of releasing that bug fix. Unexpectedly, I will “lose” the best engineers on the team, and they will no longer be doing creative work on the application; instead, the best engineers will now be doing creative work on libraries, frameworks, and platforms.

Nobody will be delighted.

Instead, I should reject the tipping point as inevitable. At this point in time, when most software runs in the cloud and on the edge, integrates with multiple and various third-party APIs, and executes processes that run from a few seconds to many days, I should demand libraries, frameworks, and platforms that offer a linear developer experience—one that I will not outgrow as my project scales.

One that starts delightful and stays delightful.

Tipping points can be a devastating event causing a sudden and consequential decline in the experience of your team, leading to a decline in quality, productivity, and satisfaction. By knowing about tipping points and understanding where and when they occur you are able to anticipate, mitigate and outright avoid their consequences.

Don’t let a tipping point derail your project. Be proactive and be aware of your developer experience. Adopt libraries, frameworks, and platforms that start delightful and stay delightful.

Look for a Linear Developer Experience.

Learn more in Temporal's on-demand webinar Introducing the Linear Developer Experience.