Evaluating Software Quality
In software development, it is a common mantra that the highest priority of the development team is the quality of the software. Every developer I've ever worked with has said something to that effect. However, it really does seem like everyone has their own definition of what good quality is and how to measure it. The decisions made by the team are the key to determining how valued this priority truly is. If quality truly is the highest priority of the team, then decisions that compromise the development, and by extension the quality, can not be made ad hoc.
How is Quality Typically Measured?
I had a conversation with my dad, who has been a Scrum Master for large teams working on enterprise-scale projects. One of the things I was really able to identify from that conversation is that quality, from a broad standpoint, is measured by properties that I would generally characterize as 'functionality'. This includes how well defined use cases and requirements are, how good performance of the code is, and what the experience is for the customer of your product, whether that be another development team integrating your sdk or an end user’s experience with a service. These are the tangible results from the development of the code. All of these things are what can be seen by QA, the customer, or the end-user by the time the project is finished.
There is another, emerging, part of this conversation. As more companies move to agile methodologies, and as more developers get experience working with it, the ability to add new components to existing code becomes an additional criteria towards evaluating quality. Developers have to be forward thinking about how new problems will affect old solutions, and how new chunks of code will fit with the original implementation. As a result quality tends to be measured not only by what is visible, but the ability to modify an existing chunk of code quickly to accommodate new requirements.
How Can We Evaluate Quality More Consistently?
My friends and coworkers know that I really like unit testing as a part of software engineering. It might actually be the most satisfying part of the software development process to me. Generally, when people learn this, it strikes them as odd, and definitely unusual. The fun, but more importantly the value, that I see in unit testing is as a way to evaluate, alter, and improve the code that has been written. At the heart of everything, I would say that my passion within software development lies in the pursuit of quality, and I believe that the ability to write concise and comprehensive unit tests is likely the most simple way to evaluate the quality of your software.
One of the biggest pieces of misinformation that developers have internalized, is the idea that comprehensive unit testing allows the company to save money by finding bugs early. Any developer on any large project can tell you that more money is wasted over the lifetime of a project as a direct result of poor decisions made on bad assumptions at a high level early in the project’s life. I've worked on projects or features that have hemorrhaged value and money because of all the wasted time trying to shoehorn in a shoddy feature demanded by someone who made a decision without consulting anyone with domain knowledge. Is it cheaper to find bugs early? Yes. Is it easier to identify breaking changes if you have a large suite of tests? Yes. Are these the reasons WHY you should write your tests? No. The value of the tests lie in what they can tell you about your code.
Unit tests are a tool which can be used to evaluate the quality of the software. From a developer's standpoint, they can even be the first major step to identifying a code smell or a bug, and they can reveal problems in the code base that might not have even been considered during the implementation.
How Can Unit Tests Be Used to Evaluate Quality?
If you are able to write comprehensive unit tests, you can evaluate the cohesion of your code, the relationship between classes and objects, and the integrity of your business logic. Even more than simply checking the box of requirements in order to open your Pull Request, you can use your unit tests to evaluate the architecture and structure of your code. Are you mocking classes from sdks you're using? You need to wrap those classes to reduce coupling to the sdk. Are you mocking dependencies that you never set up or verify in your tests? You might have a class that is doing too much that can be split into multiple classes with smaller responsibilities. Do you need to call a function before another function as part of the set up for your test? You might need to reevaluate your state management in the class! Writing unit tests for your code can reveal pitfalls in the assumptions made during development, and allow you to see warning signs of code smells and future complications even easier.
When the option of deferring unit testing is preferable to reducing overall scope of a feature or postponing minor functionality in order to meet a deadline, you are being dishonest about your commitment to quality. Yes, unexpected things will happen during the development of a project, deadlines exist, and sometimes it is necessary to make things fit for now and find a way to fix them later. However, a consistent pattern of omitting unit tests indicates that they are not being evaluated as part of the scope of work, so the team's commitment to quality specifically omits a key aspect in improving and evaluating it.
Why Would Omitting Unit Tests Reduce Quality?
Let's imagine a project. We've planned the work, defined the requirements and features, and we've established the deadline for the delivery. 5 months pass, and now we're 3 weeks away from our release deadline, with some known issues, one major feature left on the table, and time ticking off the clock. How do we hit that deadline with as little change in scope as possible? Likely, what gets left out are things like unit testing, because they take time to do and it is more important to be feature-complete than to have 100% unit test coverage. So, the remaining unit tests that need to be written will become incurred technical debt, issues to be created, groomed, and prioritized. Realistically, or maybe pessimistically, what will likely happen is that since the original work was feature-complete, the next project lands on the table, and the minor lack of test coverage is pushed down on the backlog until there is a point in time where it makes sense to do this work. In a better world, the second that release is cut, the tech debt is prioritized, tackled and resolved. In this world, the incurred tech debt would be noted as omitted scope. This is work that SHOULD have been done but wasn't, because of the approaching deadline. The actual development effort took longer than the expected deadline.
But what happens when we look at this project at this moment in time and start looking towards the future? We know that we will likely need to build on this functionality, and that those changes will require somewhere between a partial to full refactor of the code. So, we've tossed some tech debt on the backlog that we can address immediately, or we can push it back a bit and incur the work of that tech debt as part of the development work of the new feature. The perspective is that time spent unit testing or refactoring code for a functionally complete feature that will be removed or heavily rewritten soon is just wasted time. Logically, we don't want to commit resources to what would largely amount to a wasted development effort. It would be work to verify and test code that would become outdated the moment we start working on the next feature. However, it’s not verifying a complete feature, it’s finishing incomplete work.
In a perfect world, the unit tests would have been done when the code was being written the first time. The highest priority work would have been completed exhaustively, and perhaps the least critical parts of the feature would have been missed by the deadline, or delayed till a hotfix or patch. The work was never actually completed, it was just left half complete. It's almost like adding a button because the requirements specified it, but making the button do nothing because in the next release you know you're going to remove it. What's worse is that without unit testing the code, you've compromised your ability to evaluate the quality. The tests weren't just there to make sure that your feature-complete implementation was exhaustive, it was your opportunity to evaluate the architecture and the decisions made while implementing your code.
Without the followup work you may be building on top of an unstable framework, and the end result could be more code, of increased complexity, and with compromised quality.
How Can We Enforce Our Commitment To Quality?
Admittedly, my approach to software engineering is rooted in a sense of idealism. Often, developers understand the limitations of the business well before discussions around development work even begins. So when planning and estimations start, developers don't necessarily argue for what they think should be done in order to maintain a high standard of quality, they instead offer the middle ground of a compromise between the business's goals and the team's necessities. Planning should happen with the technical goals and the business goals being laid out side by side and evaluating what work needs to happen first. The team should be willing to fight for the technical goals, instead of undermining them before the discussion happens, if the team wants to maintain a high standard of quality.
It's also a matter of taking pride as a software engineer. When we, as a team, need to take shortcuts to hit a deadline, and we do so by cutting out things like unit testing and refactor work, then we are selling short our commitment to quality . If corners need to be cut, we should be cutting corners in a transparent way that makes it easy for us to communicate what changed in scope and where the estimation failed. It's easier to show the change in scope when you can point at a necessary bug fix that didn’t make the deadline than it is to point towards the incurred tech debt, like missed unit testing and refactor work. The former is a clear indication of scope change, while the latter looks like a wish list of low priority items. They might ask if it really is necessary since you were able to get our release done anyways.
To borrow an analogy, if software was carpentry, we would still have built a house that stands. You could even tour the house and live in it comfortably for a short while, but eventually the fact that there’s no insulation in the drywall, switches turn off outlets in other rooms, and the garage door opens when you use the microwave will all become apparent as a result of the corners cut, and there will be no cheap solution to any of these problems. Once the house is built, it’s a lot harder to fix some of the underlying problems.
Conclusion
All engineers and every development team will say that quality is their highest priority. No one will ever say outwardly that they don’t care about the quality of their code but it happens anyway. When a team expresses a commitment to quality, but discards unit testing and other methods of code validation at crunch time, it emphasises how weak this commitment to quality truly is. It also makes it harder for the team to honor their commitments to their end users and stakeholders, by allowing a lower standard of quality in the product. Actions will always speak louder than words, and making sure that code is written to be complete is fundamental to being able to evaluate its quality. This is what the value of unit tests really are. They are the tool we can use to improve our quality, evaluate our decisions, and set ourselves up for success for future work. When the team doesn’t truly care about quality, then it is inevitable that unit tests will be among the first on the chopping block of cut corners.
For those who don’t have a strong background in writing unit tests, or who don’t know where to start looking to strengthen their commitment to quality, I am trying to create a guide to writing unit tests from the perspective of evaluating quality, that I hope to put out soon.