While there are many challenges today with building web applications, there are also many options to address the issues we face with technology, process, and people, allowing us to reap the benefits of the web as an application platform.
Technology solutions
While many of the challenges with today’s web applications come from the vast array of technologies that are available, there are clear strategies that can be employed to turn those same issues into advantages that can make building applications easier. The key is to use a technology portfolio that allows applications to be modular, simple, and isolated from any instability in the underlying platform. Another critical aspect of each member of this portfolio is that it must be able to maintain those abilities at the scale at which the application will be built.
Modularity
Modularity is the ability of a library or framework to spread its functionality across multiple components that are highly cohesive, but loosely coupled. It is especially important for JavaScript applications because of its extreme reliance on the network for performance, and relatively underpowered web browser environments. It also strongly encourages efficient and flexible code reuse.
Modularity has evolved over the years starting with custom built solutions such as CommonJS and AMD, to today, where it is now included in the JavaScript language. While there are still major features that still need to be finalized with regards to the loading of those modules, it is possible today to use excellent approaches to authoring native modular source code while still insulating against a platform that has not finalized how to load them.
Reducing complexity
In order to reduce complexity, we need to understand that just because something is possible does not mean it is necessarily the right choice or the only choice. Many technologies allow high levels of abstraction to be introduced in order to keep code as isolated as possible, but sometimes we need to take a step back and think about how to make things simpler and easier to manage and maintain, rather than adding multiple layers of complexity to solve a fairly easy problem. By choosing technologies that allow abstractions to be introduced, but combining that with a disciplined approach to using those abstractions, complex problems can be solved with clean, maintainable code.
Architectural insulation
One of the most crucial decisions developers can make when dealing with instability and change is architectural insulation, a strategy for defining an approach that insulates your code from the portions of the platform that are in flux or changing.
In the early days, web developers focused on the very big challenge of working around very fragile, broken, or incomplete pieces of core browser functionality. Today, it can mean wrapping modern technology that is either changing or where best practices or performance may not be optimal with an API that works well and is flexible.
Tool selection: Beyond the proof of concept
When evaluating a technology, it is imperative that the investigation move beyond a basic proof of concept and give the technology under consideration a thorough analysis.
This requires having the patience to fully understand the right way of using a technology, giving it a solid evaluation. This also requires not being biased based on something being seemingly simple out of the box, or mistaking a nice looking website for something that will be a powerful tool for creating impressive web applications.
One of the most popular approaches used today to compare JavaScript frameworks is the ToDoMVC project. While it is a good first step in this direction, the ToDo app is far too simple to really exercise the power and understand the benefits and limits of a technology. If you do not know the limitations or weaknesses of a technology, then it can fail at a critical point in the development process. No solution is perfect, so knowing its limits and weaknesses is essential to providing a thorough and practical analysis of a technology.
Organization and process solutions
While technology is a primary source of the challenges and solutions related to modern software development, the processes that govern the use of those technologies can also serve to hinder or help development projects. Some of the primary processes that can benefit a project are code transpilation, effective source code management strategies, code quality metrics and automated processes for testing and continuously integrating changes to the application.
Transpilation
Many efforts have been made over the years to improve JavaScript, ranging from improving the core language (ES5, ES6, ES7, etc.), extending the language (TypeScript), or providing alternatives to the language (Elm, Clojure, CoffeeScript, Dart). In the words of Brendan Eich, “The web must evolve, or die. So too with JS”.
One of the best ways to future proof your code today is through transpilation, a form of compilation that takes one version of a language and converts it into another.
By integrating this into a development process, it is far easier for developers to embrace new additions or extensions to the JavaScript language, without waiting for all browsers to support every new feature or syntax.
In the past, developers would have to wait years to leverage significant improvements to the platform. Now, developers can author code following modern best practices, and then convert it to code that runs on all platforms today.
Source code management and strategy
Previously, most traditional source control systems made it rather difficult to efficiently maintain branches of code and easily review changes. While it is easy to believe that all of these systems are the same, modern tools, such as Git support processes that reduce the risk in breaking the master code line.
With the ability to efficiently merge and branch, it is easier to review changes before they are merged with the main code line.
The most popular tool to solve these problems today is Git, which has brought many benefits of distributed source control to the mainstream, but it is not the only solution.
Organizations should create, maintain, and follow internal code standards and have a regular review process to make sure that source code meets a certain standard and works as expected. And this should happen before it lands in the source control master.
There are many tools and approaches to facilitate review, but it requires a team of people that demand excellence, accept criticism, and understand that the goal is to create the highest quality, most consistent source code to deliver a robust and maintainable web application.
Automated testing
In a world with many different browsers and platforms, automated testing is more important than ever. This usually begins with an approach to writing code that is testable at all times (TDD, BDD, or some similar methodology). By integrating the creation of tests into the development process, the quality of the code base can be measured more clearly and the maintainability will also increase, due to the ability to quickly assess if a change causes failures in existing features. Automated tests are normally grouped into two types: unit tests and functional tests.
Developers should typically begin with unit testing, which allows a team to test each portion of a codebase, in isolation, with minimal dependencies. Complex interactions may then be mocked or stubbed with a simple interface.
Functional testing is then leveraged to test the approach of an end user. Functional tests are inherently much more difficult to author, so the more that a development team can test with unit tests, the easier an application is to test and maintain.
Continuous integration and code coverage
There are many platforms and services to automate testing. Continuous integration (CI) notifies a team if any automated tests failed with each additional commit made to the application source tree, or with any potential commit that is being considered for acceptance into the codebase.
Code coverage analysis tells a team that, not only did their tests pass, but verifies that they were careful enough to thoroughly test all of their source code. CI also includes information about code coverage, and other relevant details. There are many CI tools on the market, and flexible testing tools can be used within any of these environments.
Testing tools and continuous integration may also be used to test other metrics, including performance regressions, user interface changes, and accessibility.
An optimized CI environment helps organizations track changes in test compliance, code coverage, performance, and other metrics with each change to their source code.
Code quality
There are many ways to measure code quality, but consistency is the first step towards creating a codebase that is easy to follow and maintain. Beyond consistency, enforcing certain patterns and excluding anti-patterns is essential to preserving good software architecture. These same linting tools may be configured and customized to verify these patterns are followed.
Various code linting tools such as ESLint, TSLint, and JSHint exist to help teams verify that their code is structured to their own internal standards.
While tools can catch many things, peer reviews of source code will also help in areas that are not easy to capture with tooling.
Compared to just a few years ago, far better development tools are provided by web browsers and within Node.js for debugging and introspecting applications.
People
Engaging and managing your team
Robust architecture and a solid development process helps solve many challenges. This provides a solid foundation for then working with your team to make sure they’re able to deliver solid solutions.
It is important to encourage teams to think enough up front to reduce the likelihood that they will need to constantly rewrite. Remember that agile does not mean that you no longer need to think or explain what you’re trying to achieve, just that it means that there is ongoing flexibility to adjust and iterate on ideas over time.
Another challenge an enterprise may have to manage is that engineers often think in terms of agile, but other elements of the business do not when it comes to contracts and deadlines.
The solution to this challenge is not always easy, and varies widely per organization. Fundamentally it requires organizations to find a balance between agility and predictability that is right for them.
Reducing choices and complexity
With so many powerful technologies being continuously released, it is tempting to forget that adopting too many technologies can work against the creation of a robust and scalable application. By reducing some of the options, the complexity of your application architecture can be greatly reduced. It may be helpful to think about how well the person that needs to maintain your application in two years will do with what you have created, even if that person may be you.
Documenting decisions
It is important to not only document your source code, but also major decisions that have been made and why. This reduces the likelihood of having the same debates over and over again, especially with each new hire.
It also makes it easier to actually revisit a decision should your assumptions change. For example, at one point the Dojo team ruled out a reliance on SASS as a CSS preprocessor because it was created with Ruby, and one goal for the Dojo team was to reduce the complexity by relying wherever possible on JavaScript-based solutions. Recently someone asked to revisit the solution on SASS. If the decision had not been documented, the team may have started the debate from scratch, or may have rejected it because the decision had already been made. However, it was noted that SASS had released a new version based on Node.js, which was in contrast to the original reason for rejecting the technology. The Dojo team looked at the decision, realized that the main reason for initially rejecting it was because of the reliance on Ruby, and then decided to revisit the decision.
Giving people a reason to stay
Software developers today have many opportunities to grow their careers. This can lead to high turnover rates in some organizations. This leads to difficulty in preserving quality and architecture over time as no one can remember why things are done the way they are (unless they are well documented!).
Therefore, it’s important to give people a reason to stay. This can be done by creating a positive environment to create good software, compensating people fairly, removing obstacles that get in the way of being productive, reducing artificial pressure such as arbitrary deadlines, and encouraging people to do their best work.
InnerSource and open source
Organizations often find themselves in silos where teams do not share code across groups, and reinvent features and applications without consistency across the enterprise.
An emerging trend called InnerSource brings many of the lessons learned with large open source projects to internal enterprise efforts. In many cases these are social and quality issues, the same challenges that successful open source projects solve, to encourage reuse and collaboration among teams, while solving problems that would normally prevent this from occurring.
Organizations may also benefit from open sourcing some of their work. Each organization is different, but either opening up some software, or contributing fixes and improvements to key open source dependencies, helps keep the ecosystem that most organizations depend on improving all the time. Many organizations find that contributing to open source improves team morale, helps recruit talented engineers, and finds and solves issues more expediently.
Conclusion
We have covered many possible solutions to challenges with technology, process, and people when building web applications.
In the final installment of this series, we will expand on some of SitePen’s approaches and choices to how we actually solve these problems today.