In Joel Spolsky’s blog post “The Joel Test: 12 Steps to better Code”, he describes a test composed of twelve simple yes-no questions. For a yes you get one point. 10 points are acceptable and 12 are perfect. If you have less than 10 points, you will get in trouble with your software – sooner or later.
For a quick self-check, these are the original questions:
- Do you use source control?
- Can you make a build in one step?
- Do you make daily builds?
- Do you have a bug database?
- Do you fix bugs before writing new code?
- Do you have an up-to-date schedule?
- Do you have a spec?
- Do programmers have quiet working conditions?
- Do you use the best tools money can buy?
- Do you have testers?
- Do new candidates write code during their interview?
- Do you do hallway usability testing?
Although Joel’s Test is still an excellent indicator for good software development and engineering, 20 years have past and many game changing technologies have emerged like mobile apps, the public cloud and in general better tooling is available. The success of git and github changed how we develop software. In this article I want to extend Joel’s test with contemporary questions:
- Do you enforce a common code styleguide?
- Do you write tests?
- Do you conduct code reviews?
- Do your developers write documentation?
- Do you focus on code health?
- Do you practice continuous integration?
- Do you have a mentoring program?
- Is your infrastructure reproducible?
- Are you doing your best to keep your engineers?
- Do you provide the best technology for your developers?
- Do you focus on the four key metrics?
- Do you empower your developers?
The extended test consists of 24 yes-no questions. As with Joel’s Test, for a yes you get one point. The ranking is:
- <= 20 points, you must improve
- 21 points, you are ok
- 22, 23 points, you are a high-performer
- 24 points, you are best-in-class
Further I want to emphasis that sustainablity is my main intention for the test. Many questions contribute directly or indirectly to a sustainable and healthy codebase which is crucial for a successful long-term software project and in general for a successful software company. Titus Winters defines a sustainable codebase as:
Your organization’s codebase is sustainable when you are able to change all of the things that you ought to change, safety, and can do so for the lifetime of your codebase.
13. Do you enforce a common code styleguide?
Consistency is one of the most important properties of a codebase. It bolsters readability and maintainability which are essential for sustainable code. A consistent codebase is easier to grasp and makes onboarding new developers faster. New programmers are guided by the prevailing style and can adapt quickly to it. Consistency is also an indicator for coders’ discipline, clearly you don’t want to have dead code, unused imports, wrong indentations, and other intricacies in your codebase. The desired consistency can be achieved by a code styleguide.
At best you enforce the rules of the styleguide via tooling like static code analyzers, linters and autoformatting tools. Often these tools are integrated into the build or are executed before a commit. Further there are also manually measures like code reviews to enforce a common style.
A consistent style increases productivity, e.g. linters prevent sloppy programming errors, autoformatters leave no room for useless (sometimes religious) discussions about indentation and formatting rules. All code looks the same. Developers’ taste and ego take a back seat.
14. Do you write tests?
Writing automatic test is a major trait of a sustainable codebase. There are many kind of tests but the best known classification comes with the Test Pyramid.
- Unit Tests
- Service Tests
- User Interface Tests
Particularly unit tests build the foundation and give developers confidence to move fast and not to break existing functionality. Unit testing is a major pillar of a fast feedback loop. This keeps developers happy and the quality high. In general, tests act as a safety net, prevent new bugs from being introduced and old bugs from reoccurrence.
Without automatic tests your codebase will erode and only long-term developers will be capable to make changes. Onboarding new developers will take months or will never succeed at all. Over time developer speed will slow down and finally come to a complete halt. Heavily relying on manual testing before a release is a clear indicator of missing automatic tests and extends the release cycle by days or weeks. High performers deploy on a daily basis which is not possible with manual testing phases. Therefore manual testing should be reduced to a minimum or completely avoided.
Establishing a good testing culture is especially important. E.g.
- no code changes without a corresponding test
- no bugfix without a test demonstrating the bug is indeed fixed
- unit test should be fast, so developers run them continuously
- unit test code coverage should be at a reasonable level like ~70%
At Google, they practice the Beyonce Rule “If you liked it, you shoulda put a test on it!” This rule inverts responsibility, e.g. if someone breaks a feature and there was no test, the original author of the broken feature “shoulda put a test on it!”.
15. Do you conduct code reviews?
Code reviews are a critical step in your software engineering process. Not only they prevent entering bugs into your mainline but they are a major tool for knowledge transfer, learning and mentoring. The code review process fosters a common understanding between reviewers and author and offers a platform for discussions about trade-offs and design decisions. Reviews are not only focused on correctness but also on readability, performance and other non-functional properties.
All of that will lead to better solutions. Further reviewers practice their code reading skill which is as important as code writing. Besides compiling, linting and running tests, code reviews form a major step in a developers feedback loop. Code should never be committed into mainline without a proper code review.
Because code reviews can conjure up heated discussions, reviewers should comply to some code review guidelines in order to guarantee a flawless experience.
16. Do developers write documentation?
Documentation starts with the code. Code comments or a good
description of a pull request are good examples. Thereby good
documentation focuses on why something was done. An extensive
README.md acts as the “front-page” of a project and should contain
its purpose and instructions for developers to set up their local
environment for development, e.g installing prerequisites, building
the project, running the tests.
Additionally a variety of documents with different purposes exist:
- Design Docs (showing alternative solutions, why was one approach chosen over the others?)
- Architecture Diagrams (System overview, showing coherence between components)
- Operational Playbooks for Software Reliability Engineers (SREs) (operational instructions for fighting outages)
All these documents should be written by developers, operators or other technical people. Living, up-to-date documentation makes a project more understandable and long-term project members are capable of answering questions why things were done in the past – in the majority of projects, the top answer is “this is historically grown”. The only way to get real insights is conducting time consuming face-to-face interviews. Documentation helps to keep an overview over an ever-growing project, to facilitate the start for new developers and to build a searchable knowledge base. Past decisions should be transparent through good documentation and not hidden in people’s heads.
17. Do you focus on code health?
A healthy codebase is a major criteria for developer happiness. If your developers working on a shitty codebase, they adapt to the poor quality or leave. The existing codebase act as a role model. For the purpose of high quality code, it is important to continuously focus on code health. The best coders are repelled by bad code and attracted by healthy code. But what is a healthy codebase?
A codebase is healthy when:
- you have fast builds
- you have an easy development setup
- you have fast and maintainable tests
- you have clean, readable, loosely coupled and consistent code
- you can easily debug the system
- you continuously tackle technical debt
You can find a much more exhaustive explanation of code health in Google’s Testing Blog about Code Health.
Signs of bad code are:
- complicated developer setup
- hard to debug, missing monitoring, noisy garbage logs
- long build times
- inconsistent code (dead code, unused imports, different formatting styles, no code styleguide)
- large merge conflicts due to long running feature branches, broken mainline
- no tests, flaky tests, hard-maintainable tests because of mocking overuse
Never trade dirty code or workarounds due to time or release pressure for code health. You will end up very badly in the long run. Worse yet, you get in a vicious cycle because bad code slows you down and in order to fulfil the next release you add more dirty workarounds. So always prioritize code health, even when it looks counterintuitive at first sight.
18. Do you practice continuous integration?
Nowadays Continuous Integration is hopefully commonplace. At best, you work with trunk-based development and your mainline is always releasable, preferably with feature toggles. Highest priority is to keep the mainline green and a broken build should be fixed immediately. Small and frequent releases prevent bugs or even outages which happen when large releases are done only intermittently.
CI helps to prevent tedious merge conflict resolutions because your developers regularly commit into mainline. Additionally you will get rid of time consuming integration problems at the end of your implementation phases.
“Agile”’s main goal is to identify risks as early as possible and not to postpone them till the end of a project. CI supports exactly that. With CI, you will detect integration and design problems early. This allows you to make course correction on the way.
19. Do you have a mentoring program?
Mentoring is a great way to coach new employees. With a mentoring program, you not only foster a consistent engineering culture with best common practices like software methodologies, code styleguides, code reviews, testing culture and other policies across the whole organization but it is mandatory for molding new recruits into a productive team.
Mentored developers are more likely to stay. They feel approved and treated as first-class employees because you invest time and effort in them from the beginning. It is a fantastic way to give raise to great and loyal engineers.
20. Is your infrastructure reproducible?
Today, public cloud providers like AWS and Azure provide APIs to create, update and manage your infrastructure. With the usage of APIs, it is natural to store your whole Infrastructure as Code (IaC).
Since we are speaking about code, all previous points apply for infrastructure too. You should keep it under version control, conduct code reviews, run tests in you CI pipeline and keep your codebase clean via formating and linting tools. Known tools for infrastructure are Kubernetes manifests, Helm charts, Terraform or AWS Cloudformation.
A good indicator if your infastructure is reproducable is how you treat your servers, like cattle or pets? Recreation of a failing server should be easier than duct-taping a failing one. Beware of snowflake servers!
21. Are you doing your best to keep your engineers?
I’ve built a lot of my success off finding these “truly gifted” people, and not settling for ‘B" and “C” players, but really going for the “A” players. … I found that when you get enough “A” players together, when you go through the incredible work to find five of these “A” players, they REALLY like working with each other because they have never had a chance to do that before. And, they don’t want to work with “B” and “C” players, and so it becomes self-policing, and they only want to hire more “A” players, so you build up these pockets of “A” players, and it propagates …"
Steve Jobs
Hiring good engineers is very important but keeping them is too! The best engineering teams share a common mindset. Engineers want to work with other engineers sharing the same interests and approaching the same goals. This creates a virtuous circle, engineers motivate each other in order to achieve the best possible outcomes. Be careful though, a single new hire can disturb an A-player team’s balance. That’s why the hiring process is very important.
You should treat your engineers as first-class employees. Companies, recognizing their value, provide tech-specific careers with compensations comparable to high-management salary level.
Further, engineering managers play an essential role to keep your engineers (happy). They should know how to code and share the same engineering mindset. If your engineering managers are not technical skilled, your engineers will not take them seriously. Worse yet, they will leave at some point in time because they feel misunderstood.
You should strive to hire A players and to keep them.
22. Do you provide the best technology for your developers?
This points extends Joel’s original ninth point “Do you use the best tools money can buy?” Joel focused mainly on software tools running on your local machine like IDEs, editors, compilers, debuggers and also local hardware like a second monitor. All this make your programmers happy and productive. While this is still true, nowadays it takes more than local tooling to keep your developers happy.
A major pillar of productivity is going to the public cloud like Amazon AWS or Microsoft Azure. Public clouds offer a great work experience for your engineers. The self-service approach is particularly important. Engineers can automate their application stack via APIs and scripts, use the newest technology and operate the complete system without writing tickets and getting approvals from superiors. It enables your developers to take responsibility for the full lifecyle of the application from design, to implementation, to release and to operation. It’s a great productivity booster. If you cannot go to the public cloud due to legal or other restrictions, beware of the on-prem enterprise non-cloud. The enterprise non-cloud is a treacherous thing, it offers modern software stacks like Kubernetes or Prometheus but is fails to deliver the main advantages of the public cloud like real self-service with APIs, elasticity, scalability and a pay-per-usage price model.
Other kind of tools are equally important, e.g. modern collaboration tools like Google Docs or Office 365 which support parallel collaboration at one document with multiple users, modern communication tools like Slack and a contemporary version control platform like github with integration for code reviews, for CI pipelines and for the public cloud.
23. Do you focus on the four key metrics?
The best way to measure if your engineering department is top-notch, are the four key metrics, defined in fantastic book Accelerate.
High performers strive for:
- Short lead time
- High deployment frequency
- Short mean time to restore (MTTR)
- Low change fail percentage
Some of the above metrics seem contradictory like high deployment frequency and low change fail percentage. But according to the studies of the DevOps Report the best companies perform in all key metrics excellently.
If you look for improvements, it is critical to focus on these four key points permanently. Don’t fall prey to expensive agile transformation methodologies like SAFe. Usually they promise a magical and fast performance boost but often the impact turns out as insignificant or even worsens the situation.
24. Do you empower your developers?
Many companies treat their developers as pure delivery teams, i.e. they are used to implement features based on roadmaps defined by leadership and product management departments. Why this is a bad idea, see Marty Cagan’s post “why products fail?”. With this approach you only getting half of their value. Further companies think developer teams can be easily replaced or even outsourced. They are treated as mercenaries but as we know from John Doerr
we need teams of missionaries, not teams of mercenaries.
But for an outsourced IT, it is almost impossible to work in missionary mode.
In order to get the most of you developer teams, you must hire missionaries and empower them. Empowered developers are first-class employees. Your developers are the best source of innovation because they know the enabling technology. But it is hard to attract, identify and to keep top talent. See, how to hire (11. from Joel’s test) and keep the best.
Empowered developers should not only implement predefined features from stakeholders but are invited to come up with their own solutions to satisfy the customer needs. Therefore we should give developers problems to solve, not features to implement. It is mandatory that developers understand the business context. A quick self-check if you have empowered developers:
- Can developers identify themselves with the customers?
- Do they attend user tests regularly?
- At best, do your developers use the application themselves?
- Do they know the pains of the customers or are they shielded away by a wall of stakeholders, business departments, program managers and never have the chance to get in contact with customers?
- Do your developers drive their own features or do they only get them at the sprint-planning?
Finally empowered engineers means a mindset change in the whole company, see Marty Cagan:
(the company is) moving from a model where the technology teams exist to “serve the business” to one where they exist to “serve the customers, in ways that work for the business.
Conclusion
Digital companies like Google, Amazon, Facebook, Microsoft or Netflix have these traits in their blood and they live by these standards already for decades. For them, “software” is their core product, they cannot afford piling up technical debt since they would cripple and risk their own future. This is well understood by them. In contrary to non-digital companies with oldschool enterprise IT departments and outsourced IT.
Non-digital enterprises never considered “software” as a competitive advantage. But slowly even they understand that “software eats the world” and they want to do develop software in-house again. In such situations, missing know-how and “wrong” people with lack of skills are common. This article wants to be a simple and easy applicable guideline for them to establish a good coding mentality and culture with good practices. I hope you can apply the full test and gain some insights about strengths, weaknesses and possible improvements in your company or team.