Embracing the development of CRUD

When we haven't got the basics right, we don't have a right to call it "Boring"

As a traveller of forums of all things related to Software Engineering, there is a common thread of thought I see from time to time. CRUD apps are boring work. CRUD apps aren't "real" software projects. Companies purporting to change the world are building "just" CRUD apps so it's just glorified speak.

Well, the last statement is somewhat accurate but not for the reason of building "just" CRUD apps.

The point is, there's a trend to treat the idea of developing CRUD apps as something boring. Something unglamorous. And worst of all, to treat it as if it sin't real engineering. But there's a problem with that. And the result of these thoughts are destructive both to individuals in software engineering and to the practice of software engineering itself.

Before we get into how important and widespread CRUD is, there's a question that bears answering. What is CRUD? In software CRUD stands for Creating, Reading, Updating, and Deleting. It's used to describe applications that are used to create information, retrieve it at a later stage for viewing, update it when information needs to change, and delete if the information is no longer required. It sounds simple. And that's the underlying reason for people to treat the idea of CRUD as boring. You take in information, an instructions, and you create, retrieve, update, or delete the information based on the instruction. Those who refer to this as boring will generally paint a picture of this as being a repetitive cycle done till the end of your miserable life as a CRUD developer.

The thing is, at a deeper level, CRUD is everywhere. Everywhere where humans are. Games are probably the most distant from this but even at the deepest level, it's CRUD. You create new games, you retrieve status from players and the inputs, you update what is being shown, and you delete what's not needed. Most people would bristles at the idea of this being referred to as CRUD. But ultimately, it is what it is. Facebook, Twitter, Instagram. All CRUD. Well. We aren't sure if Facebook implements the Delete part of anything but let's not go there. The point is, that beyond the shiny recommendation engine, ultimately the network is all about creating content (even ads), displaying it, changing it, or deleting it. Every single application that interacts with humans is ultimately a CRUD application.

The point is, CRUD goes beyond taking an input and modifying a database. The truth is that the human interactions that lead up to that action are connected in an inseparable way to that final Database change. Human interactions are what ultimately make up the majority of work in any CRUD application. And there's nothing easy about that.

When we think of CRUD application like this, the resulting complexities and challenges in CRUD are suddenly more visible.

Consider project management and the tools built for it. Depending on the team and project size, there are any number of project management methods that might apply. But even if we make two team the exact same size, and change their makeup to be exactly the same, there is no agreed management style that has been given in the history of Project Management. Taking that further, even if the two teams agreed to use the same Project Management method/tool, for example Kanban, the teams would choose entirely different ways of using it. Maybe it would be how many columns they want to set in, or how they want to use labels vs boards for categorisation of tasks. Each team may have different choices around how they want to enforce privacy and permissions. Just the manager of each team having a different personality style could change a huge part of the core logic of such an application.

This is all despite the fact that there's nothing "technologically advanced" going on here. Just inputs and outputs. It's as pure CRUD as it can get. Forget scaling. Forget efficiencies. Even to satisfy the basic input output of a system like this is an incredibly complex and difficult task. Proof that the real work around CRUD happens far away from the database.

Project management is still a nascent field one might say. Something very human. So let's take accounting for example. A practice that is basically the younger brother to the history of commerce. The entire industry agrees upon the one true practice of double entry bookkeeping to rule them all. And even though there's an indsutry behemoth of a software tool (Quickbooks), there's still no consensus on which tool is the "right" tool to use. That's because there is none.

Once again forget AI, fraud detections, scaling, etc. Just throw in human variables. Is the team large or small? Does it have outside clients or is it purely internal accounting? Is the team remote or based in a single location? Is the CFO a control freak? Do other teams need to have access to accounting data? Again we see that the interactions leading to CRUD matter more and that modelling these into the application is complex. More importantly, it's impossible to disconnect the two.

Point is, you cannot separate human interactions from CRUD applications.

So let's talk about "hard" non CRUD problems. What are these things that the industry considers "hard" problems that makes CRUD look positively pedestrian? And how valid are these problems against a world without human interaction?

• Scaling and infrastructure problems. These are indeed non CRUD in a traditional sense. But having the world's best scaling mechanisms and custom infrastructure won't matter squat if no one is using the software that's running on it. On the flipside, working on chipsets that enable people to go faster and do more is an actual valid truly non CRUD type of work. And we'll always want to go faster and always want more, so there's no question of utility.
• Machine learning and AI. The illustrious words of our generation. Deep learning algorithms. In many cases these tools are used to enhance the CRUD applications that are in place. On their own they'd be useless. That said, there are actual non CRUD ways in which these areas of software are useful. In medicine for example I believe AI is such a powerful force. And I don't think the act of searching for ways to fold proteins is really CRUD.
• Algorithms and architecture. Both of these find the CRUD/non CRUD rules applied in the exact same ways that they are applied to scaling and infrastructure problems. People can research and discover completely new ways to structure frameworks or prove algorithms that allow for better compression without ever having to apply it to a CRUD application. And they can be very useful at some point.

All this to say that non CRUD problems exist. Not everything in the world is a CRUD problem. And not being CRUD isn't an indication of it not being useful. The problem though is our separation of CRUD vs non CRUD as "easy, boring, mundane" vs "hard, interesting, rewarding".

All software engineering is hard. But when we keep deriding CRUD work we are actually causing deep problems for the software industry and the work it produces as a whole.

I remember in my early years being part of the group that became convinced that CRUD is boring and that my time would be better spent working on "real" problems. Maybe I was a failure for having never touched Machine learning. These thoughts are depressing and problematic for many engineers. For engineers whose heart isn't in solving these non CRUD problems, some of them might end up burning out as they keep chasing the wheels of change in the world of research and non CRUD work. Some engineers might just drop out of the engineering work as a whole. Convince them that the work they are doing doesn't fulfill anything valid and eventually they'll want to move on from it. When that happens we contribute to a body of people that don't want to become experts in solving human interactions. Even worse, we create a negative impression of the people who do want to keep working on it and that will finally end up in a self serving prophecy.

The biggest fallout from all of this is that the software industry will continue to grow further and further away from wanting to solve human problems. So the end users of the products we build are the ones that will suffer the most. And this is already happening. Be it through algorithms that are being applied to treat unique individuals as part of a statistic, or ad exchange systems that invade your privacy, or project management tools that aid toxic management while claiming to do the opposite, the negative effects of forgetting human problems is already out there.

What do we do then?

I'm a big fan of doctrines and manifestos. They are great guidelines for someone to look at and see what they are truly invested in. And when people talk to me about what work to do in software, I try to explain the sides of software in terms of doctrines rather than "hard"/"not hard" problems. If someone wants to know if they should take a job in mobile development or data pipeline administration, it's a lot easier to explain things this way.

The way I like to explain it is "human" vs "non human" problems. On the human side, if teasing out human interactions and figuring out how to model these quirks inside of a system are of interest to you, CRUD applications would probably be a great match. If instead you feel that you want to work on areas of research where you'll come up with ideas not knowing what might work and what might not and you want to work on it without worrying about modelling for human interactions, then working on non CRUD problems might be for you.

Generally the conversations I have are more complex than the above paragraph. People have questions, BUT starting it off like this allows for a real conversation. Because which person wants to say they aren't working on "real" problems?

At the end of the day though, life is CRUD. Eveything we do is an interaction ultimately revolves around creating, reading, updating, and deleting information wthat we have. Whether we are creating art, watching a movie, driving a car, or buying fruits from a supermarket, everything we do involves adding, retrieving and changing information that we've accumulated from the time we were born.

And despite that, we still have no agreed way on how to model any of this when developing software. And that is unlikely to ever change. Books will be written. Gold standards will emerge. Consensus never will. Probably. That's because CRUD development when considered as what it really is, modelling human interactions, is a really really hard problem.

We, the software industry, deride the importance and value of development of CRUD applications at our own peril. We praise those who work on shiny "hard" stuff just because they've worked on shiny "hard" stuff. We try to pad company descriptions to make it sound like we are working on anything but the actual human interaction parts. We throw in fancy words like "we work on hard problems and using new technology and building amazing scalable architecture" as if saying "we make getting accounting information easier for people" is a mark of shame. We value knowledge of new technology over expertise in threading together human interaction. We perpetuate a cycle of fear of missing out if you aren't working on the latest and greatest. And we need to stop.

Because if we really want to build a future in which software is useful in all the ways we expect it to be, we have a long way to go even in just understanding how we should approach solving human requirements.

So don't shy away from CRUD. It's real hard work and should be respected as such. Embrace it. Embrace the fact that your primary duty is modelling human interactions and then putting it into software. Even if you work on a team where all you do is complete JIRA tickets based on someone else's modelling, learn from it. Learn what you think is right and wrong. Try it out on your own. Once you learn how to view it from this perspective, I promise you that there's a life filled with learning ahead of you.

Posted on March 22 2018 by Adnan Issadeen