By Barry Briggs
[JUST A DRAFT RIGHT NOW!!]
Over the past few months I’ve been working with some old friends at the International Association of Software Architects (IASA) to try to figure out some way to quantitatively measure the value of software architecture. We’re trying to come up with answers to the following questions:
- Why is software architecture good (i.e., why do you need software architects?)
- How can you quantitatively assess an application or service?
- What makes a good software architect?
These are difficult questions, particularly when you compare software architecture with other fields. For example, it’s relatively easy to quantify the value of a Six Sigma process-improvement organization: you measure time, resources required, and costs of a process before optimization, and then after, and you have a solid measurement of value – one that is simply not possible with software architecture.
Why?
Well, on a net-new project, architecture is applied at the very beginning, so it’s difficult to know if the lack of it would have made any difference. Arguably, on a rewrite of a project, one could compare against some set of criteria how much better the new version works vis-à-vis the old one – but there are usually so many other factors in such a project that it’s essentially impossible to separate out the contribution architecture makes. For example, faster hardware or just plain better coding might be the reason the new app runs faster, not the fact that the new design is factored more effectively.
The Army Barracks
Perhaps an analogy can help us tease out how to think about these questions. Software architecture is often compared (poorly) against physical, building architecture – but let’s try to make the analysis a bit more constructive (pun intended).
Consider something as mundane as an army barracks. How would we measure the quality of its architecture?
I suppose there are lots of ways, but here are mine.
First and foremost, does it do the job for which it was intended? That is, does it provide enough room to house the required number of soldiers, does it provide appropriate storage, bathrooms, and showers for them? Is it well insulated and heated? In other words, does it meet the immediate “business need?” If not – well, you certainly couldn’t assess its architecture as good in any way.
Then we could ask many other questions, such as:
- Compliance with laws and standards, that is, building codes, Army regulations, local standards, and so on. Like business need, this one’s binary: if not compliant, no need to perform any additional evaluation.
- How resilient is it? Can it withstand a power failure, a Force 5 hurricane or (since this is a military installation) a direct hit by an artillery shell?
- How much load can it take? If there’s a general mobilization and much more space is needed, how many extra beds can it hold? 2x? 5x? 10x, in a pinch?
- New workloads. The Army mandates that barracks become coed. Can the facilities be quickly adapted – if at all – to support separate sleeping areas, bathrooms, etc.?
- How easy is it to add new features? For example, does it require a teardown to add air conditioning or can existing heating ducts be reused in the summer? How hard is it to install wi-fi hubs?
- What about new components? Say the Army mandates that every barracks has to have a ping-pong table, which entails a building addition. Can such a thing be done quickly with minimal disruption?
- Business continuity. Say the barracks does fall down in a storm. Are there sufficient facilities on the base – or on other bases – that the soldiers can rehoused?
- Aesthetics. OK, maybe this isn’t a good one for a barracks, but for other types of buildings – think I.M. Pei or Frank Lloyd Wright – aesthetics drive our view of good architecture.
You get the idea, and, hopefully, the analogy. In this case the value of good design – of architecture – is readily apparent.
Assessing Software Architecture
When we think about software architecture, we can apply similar criteria.
Business Need
If the software doesn’t satisfy business requirements, then – as we said above – it by definition cannot be “well-architected.” Determining how well software meets the need, however, can be an interesting and challenging discussion. For years, software development began with requirements documents, which could stretch to tens, hundreds, even thousands of pages; and managers would simply tick off the features that were implemented. (And as often as not by the time all the documented requirements were met, the business environment had changed, and the app was behind.)
With agile development, users are much more involved in development from the start, tweaking and mid-course-correcting the product during the development process. If there is a requirements document, it represents the starting point rather than a final statement – and this is good, because as the product takes shape, opportunities always present themselves, both to users and developers.
Still, how do we assess how well the product meets the need? Of course, one way is to ask users if they have the features they need; if not, something’s obviously missing.
But that’s not all.
Every line of code, every non-code artifact (e.g., images) should be traceable back to the business requirement. If there is a feature, somebody should be using it. Monitoring tools can help track which features are exercised and which are not. (The Zachman Framework was an early approach to documenting traceability.)
This applies to infrastructure as well. As infrastructure is increasingly documented through Infrastructure-as-Code (IaC) these Terraform or ARM or CloudFormation configurations should justify their choices: why – from a business perspective – this or that instance type is required because of expected load, SSD storage is needed because of anticipated IOPS.
Standards and Compliance
Like satisfying the business need, complying with relevant standards is binary: the software does or it doesn’t, and if it doesn’t, you’re done.
Now by standards we don’t mean “best practices” – we’ll talk about those in a moment. Rather, ensuring that personal data is anonymized in order to comply with GDPR, or that two-factor authentication against a central corporate provider (such as Active Directory) is used, or that only certain individuals have administrative privileges: where such standards are in place, they are mandatory, not complying places the organization at considerable risk, and thus the system cannot be assessed as well-architected.
However, best practices can be more flexible. For example, a cloud governance team may mandate the use of a particular cloud provider, a certain set of landing zones, a particular relational database, and so on. In rare cases exceptions may be granted. Here the goal of such guidelines is intended to speed development and ease operations, by removing the need for every development team to waste time selecting the appropriate provider or service and for operations teams to learn them all.
Granting such exceptions must be intentional, that is, careful analysis should uncover the core need for the exception; it should be documented and possibly, the best practice should be updated.
Defining Your Software Architecture Strategy
As is true with best practices, the definition and importance of other aspects of software architecture will necessarily vary from organization to organization. When developing architecture assessments, organizations should consider what their goals regarding software architecture are. For example, what are the relative priorities of:
- Application performance
- Application scalability
- Developer productivity
- Business continuity, including RTO/RPO
- Application visibility (observability) and self-healing
- Software extensibility
- Ease of upgrade
- Usability (e.g., is it mundane/functional or beautiful?)
For example, for non-multi-national organizations georedundancy or multi-regional replicas may not be necessary. Others may decide that the expense of active-active BC/DR solutions is too high.
Moreover, different applications will attach different levels of importance to these criteria. For example, an intranet application that shows cafeteria menus need hardly be georedundant or be built with microservices – it wouldn’t hurt, but perhaps resources could be devoted elsewhere!
Strategy to Principles to Assessment
Having defined the organization’s strategic goals from software architecture – i.e., what is good software architecture and why it’s necessary – actionable principles can be developed. By “actionable” we mean that developers can look at them and understand what must implemented, and perhaps even how.
For example, if a key strategic goal is that applications should be extensible, then a principle – that a developer can use – is that apps should have a REST API, documented with OpenAPI or the like.
A good starting point can be popular industry principles, such as the The Twelve-Factor App originally intended to guide the development of SaaS applications but in fact is very broadly applicable (shown below, via Wikipedia).
| # | Factor | Description |
| I | Codebase | There should be exactly one codebase for a deployed service with the codebase being used for many deployments. |
| II | Dependencies | All dependencies should be declared, with no implicit reliance on system tools or libraries. |
| III | Config | Configuration that varies between deployments should be stored in the environment. |
| IV | Backing services | All backing services are treated as attached resources and attached and detached by the execution environment. |
| V | Build, release, run | The delivery pipeline should strictly consist of build, release, run. |
| VI | Processes | Applications should be deployed as one or more stateless processes with persisted data stored on a backing service. |
| VII | Port binding | Self-contained services should make themselves available to other services by specified ports. |
| VIII | Concurrency | Concurrency is advocated by scaling individual processes. |
| IX | Disposability | Fast startup and shutdown are advocated for a more robust and resilient system. |
| X | Dev/Prod parity | All environments should be as similar as possible. |
| XI | Logs | Applications should produce logs as event streams and leave the execution environment to aggregate. |
| XII | Admin Processes | Any needed admin tasks should be kept in source control and packaged with the application. |
We can learn several things from 12-Factor:
Principles Must be Easy to Understand, and Actionable
There are many ways of framing principles, of which 12-Factor is just one. What is key is that developers should intuitively understand what it means to implement them. For example, in 12-Factor, “any needed admin tasks should be kept in source control” easily translates to putting IaC artifacts in a GitHub repo.
Another common approach to documenting principles is called PADU, which stands for Preferred, Acceptable, Discouraged, and Unacceptable. PADU is attractive because it enables a range of options. For example, a “Preferred” approach to project management might be the use of an online Kanban board; “Acceptable” might be a form of Agile; use of waterfall methodology might be “Discouraged;” and using Excel for project management would be “Unacceptable.” Governance bodies (or the teams themselves) can then score themselves on a 0-3 basis and require a minimum score to deploy.
Principles Must Evolve
Organizations must recognize that owing to technical advances the principles may – and must – change over time. For example, the sixth “factor” above mandates that processes should be stateless; yet in today’s world it is increasingly possible, both from a technical and cost-effectiveness point of view to maintain state in business logic in certain circumstances.
Organizations Must Have Their Own Principles
Again, organizations may interpret industry principles according to their priorities and needs. Moreover they can – and should – add their own. For example, 12-Factor does not mention building zero-trust computing ecosystems and for many, if not most, this is essential.
Assessing Software Architecture
Having created a robust set of principles, it’s relatively straightforward to measure the degree to which a given product or service adheres to them. Many organizations use scorecards to rate software in an architecture review process, with minimum passing grades.
The Value of Software Architecture
A not-so-obvious conclusion from this exercise is that there are fundamentally three value propositions of applying software architecture strategies, principles, and assessments:
Usefulness, in other words, ensuring that the software does what its users want it to do, in terms of features, availability, and scale, to name a few.
Risk mitigation. Compliance with regulations and standards helps reduce the probability of a business or technical disaster.
Future-proofing, that is, enabling the product to grow both in terms of new features and the ability to exploit new technologies.
It’s exceedingly difficult to quantify the value of architecture (and architects), however. Yet it is intuitive that software cost estimation models such as Cocomo (Constructive Cost Model) which base estimates on line of code (specifically, e=a(KLOC)b) could benefit — i.e., improve their accuracy — by including coefficients for architectural influence.
Many thanks to Miha Kralj of EPAM Systems, Jim Wilt of Best Buy, and Bill Wood of AWS for their comments and suggestions. Errors of course are my own.
Comments are closed.