Looking to the manufacturing industry, despite relative hesitancy in past years, it has changed more recently to embrace the cloud. Manufacturers are adopting the cloud for range of uses including high-performance computing (HPC), enterprise resource planning (ERP), Internet of Things (IoT), big data, predictive analytics, and more.
While we have several techniques for code analysis, I want to introduce a new set of tools for everyone. I recently had a challenge to track down which pieces of code eventually touch an external service -- database, REST service, etc. This is easy enough for some simple code, or to find the actual interface points. But if you want to find which service each REST endpoint touches, for example, it can become easy to miss some one-off places. So, like nearly everything else we do, we can automate quite a lot of this!
Companies of all sizes and industries vary in software development approach. Company culture and attitudes towards security will determine the quality of software and applications produced. A quality and security focused development culture will help drive good process that protects against common security vulnerabilities and will make product development more predictable and enjoyable. So, how do you know if your organizational culture is affection software and cybersecurity?
Performance testing is part of any healthy IT strategy. It’s a given that any application with high-volume usage will undergo regular performance testing. These tests expose faulty assumptions about how apps handle high volume, ensure infrastructure scaling works as expected and identify load-related defects. Performance testing’s ability to detect flaws occurring under high load can help improve applications regardless of scale.
Open source frameworks are where most code is written today. So is open source the root cause of security breaches? Or is it the organization’s IT strategy, development process and work culture that create the weak spot? And, how should businesses control and maintain the software they produce?
Dev9 has a mantra for on-premises application projects: “start with the deployment." We call this idea, “Deployment-First strategy.” In this strategy, a team starts the project by deploying software to a production environment. Don’t start with code or tests. Start with a single deployment to production.
Cloud Business Intelligence (BI) is rapidly growing as applications that leverage powerful cloud services promise users the tools needed for better business decision making. It offers IT, scalability elasticity, and ease of use. Business users will also find Cloud BI software’s multi-source data discovery and data visualization capabilities appealing.
Since taking off in 2013, Docker has been making waves in technology. Self-described as “lightweight, stand-alone executables package(s),” Docker provides businesses with “standardized units for development, shipment, and deployment.” With proper usage, containers can be an immense Cloud adoption acceleration tool that increases uptime and production.
Today’s fast paced and ever-changing world isn’t slowing down. Organizations and their technology departments are increasingly data-centric and striving for agile practices, such as continuous delivery (CD) and microservices. And while the technological benefits of CD and microservices are obvious and direct, enterprise-level benefits is a topic just starting to pick up traction. People are starting to ask: “Why are microservices important to my business as a whole?”
Think of all the decisions that make up a software development project. Countless are being made around language, framework, container, provisioning, testing philosophies and more.
The process is intricate enough with one project. Now try to apply this complex web to microservices where each service may have different architecture patterns. While Enterprise Architects may guide Solution Architect teams on some high-level decisions, each deliverable that follows requires decision-making. To ensure project transparency, it’s important to create a historical record of architecture decisions. These records give Architects visibility to project lifecycle, evolution, and decision-making rationale, allowing them to troubleshoot when needed.
To capture the numerous decisions that occur in a project lifecycle, Dev9 adopted a process called Architecture Decision Record (ADR). From a consulting angle, ADRs are extremely important – they’re a powerful tool to guide clients on how to own, maintain, and manage the solutions we help build. ADRs effectively communicate architecture decisions.
The rise of lean development and microservices further complicates the ability to communicate architecture decisions. While these concepts are not inherently opposed to documentation, their processes often fail to effectively capture decision-making processes and reasoning. Another possible inefficiency when recording decisions is bad or out-of-date documentation. It's often a herculean effort to keep large, complex architecture documents current, making maintenance one of the most common barriers to entry.
While a large system diagram or a surplus of sequence diagrams can reveal some of the "what" of the architecture, they're often too abstract to communicate any of the "why." In addition, some architectural decisions cannot be cleanly represented in a graphic format. For example, the decision to use universal unique identifiers (UUID) as a database’s primary key instead of an auto-incrementing INT type, cannot be represented on a diagram. It can be inferred from the code; but, eventually new people will join the project and think, "why did they do it like that?" That's the challenge we address with lightweight ADRs.
ADRs are not new. They've been around since at least 2011, possibly longer. The balancing act involves making ADRs easy enough that people implement them, without compromising useful information.
Reduce Mental Barriers First
Writing documentation can be frustrating. It's one of the least-favorite tasks amongst developers, so it often gets done hastily. This haste results in bad or inaccurate ADRs. To ensure proper adoption of ADRs, simplify the process as much as possible - it should be dead simple to add an ADR. You’ll also need to limit the information captured. If they start turning into industrial-strength specifications, then you're right back to waterfall development.
Considering this, we suggest the following principles to capture architecture decisions using an ADR framework:
Store with Code
In microservices, architecture decisions are made at many granular levels. Keep your ADRs in direct chronological sync with the project. That way, it's easy to browse decisions and get insight when examining code. This practice writes ADRs along with evolving code, giving visibility to the robust history that is software development.
Document Decisions, Not State
You want to capture the chains of thought that contribute to your software’s current state for historical context. For example, switching from SQL to NoSQL should have a decision document. ADRs explain all the reasoning and considerations behind certain decisions. This gives new and seasoned team members ready-to-read documents to answer any questions.
Files should be able to be sorted chronology to give visibility to software development steps. New folks should be able to read the chronology and walk away with a reasonable concept of the architecture and the big decisions involved without jumping through hoops.
Proposed ADR Structure
We've done a bit of research that considers all the above points, and we think the following structure will satisfy them all. Keep in mind that one of the biggest barriers is getting developers, leads and architects to contribute and maintain files. To ensure successful ADR adoption, prioritize these efforts:
- Store all documents in a directory at the root level of the project, called `arch`.
- Files named in YYYYMMDD-NN-Short-Description.md pattern.
- Multiple files made on the same day use NN as a sequence indicator
- Files made on days with only one file can choose to keep a 00/01 sequence indicator, or omit it.
- Files never deleted, even when decisions superseded. The historical record is of importance.
- Use a lightweight text formatting engine. We highly recommend Markdown since GitHub renders these nicely in the browser.
A sweet spot between minimalist document structure and enough relevant content is needed. Information should capture decision strategy, but should be concise enough to read quickly and easily.
Each ADR needs to have a title. Again, keep it simple (i.e. "Database Storage Choice" or "Dropwizard over Spring Boot.")
This section lets us share the key factors in the decision - context, technology, politics, availability, expediency, and more. Avoid opinionated text, these should just be the facts.
Here we will put a few sentences describing the decision that we made and the rationale behind it.
Finally, document any potential issues or technical debt that this decision affects, positively or negatively.
At Dev9, we believe that historical records are important for project transparency. It also gives Architects visibility to project lifecycle, evolution, and decision-making rationale, making troubleshooting smoother. We’ve implemented this as a process to ensure that every development effort runs smoothly, but provides for an easy hand-off to the teams who follows us. If you’re interested in further reading, please take a look at a couple of examples below.
Dropwizard over Spring Boot
Dev9 would generally prefer to use Spring Boot, but existing infrastructure at [client] makes this difficult. [Client] has an existing IaaS solution with pre-built starter projects in Dropwizard. More work would be required to get us going on Spring Boot, while Dropwizard would let us start writing application logic sooner. [Client] IT staff have said we're free to use Spring Boot if we package it correctly into a Docker container; but it would not be supported by them.
We are adopting Dropwizard over Spring Boot. It is politically expedient, it gets us up and running significantly faster, and it's well-supported by [client] IT teams. It also brings a lot of operational readiness factors, such as log aggregation, stat collection and aggregation, and predefined build pipelines. Even though the team's strengths are in Spring, we believe the time spent learning Dropwizard's quirks will be significantly less than the time we'd spend getting Spring Boot functional in this environment.
Some additional time needed to come up to speed. May need additional time for onboarding new team members who are likely more familiar with Spring.
Cucumber as Testing Framework
[Client] Project Manager is very familiar with Cucumber and Gherkin. She already writes stories in the manner of Given/When/Then, and she agreed to be more exacting when she writes them going forward. Team has no attachment to any BDD-style frameworks, and the idea of executable documentation sounds exciting.
We are choosing Cucumber as our BDD framework for our acceptance testing. Our definition of “done” for stories will also include a Cucumber Feature File with all the glue code implemented.
While this is a nice framework, we have a couple risks. We don't want too much testing done at the acceptance level. We still want the testing pyramid to be an effective and accurate model of our efforts. We also don't have a plan for how we'll expose these cucumber files so others can easily read them. We suspect there's a way to publish them to Sonar, but we'll prioritize that work in sprint planning.