Nodules: Better Module/Package Handling for Node.js

By on September 15, 2010 5:25 am

Nodules is a module loader for Node that provides a powerful set of features for application development and distribution. Nodules was written to solve the two major missing pieces of functionality I needed for efficient application development: flexible dependency resolution and module reloading. Nodules runs on top of Node’s simple base module loader, and nicely compliments the base loader with additional functionality. Furthermore, it provides this functionality using idiomatic asynchronous techniques, a smart package layout, and powerful module reloading.

Dependency Resolution

The first motivation for Nodules was to provide a means for distributing packages that are dependent on other packages without requiring manual download of dependencies. You should be able to easily distribute packages without having to include every dependency and without requiring users to find and download every dependency. Nodules relies on the CommonJS package mapping specification as the format for defining dependencies. With package mappings, one can define how module identifiers are mapped to dependency modules by URL. You can easily map to and include virtually any JavaScript module using package mappings. Nodules handles these mappings and automatically downloads dependent modules on demand. Using URL-based definition for dependencies follows the paradigm from the browser for loading modules and makes it simple to depend on any publicly available module.

Setting up dependencies is very simple with package mappings. You can simply include a mapping like this:

	"mappings": {
		"commonjs-utils": "",

And now when you run applications through Nodules, you can refer to modules from commonjs-utils with:

var jsonSchemaModule = require("commonjs-utils/json-schema");

The first time you run your application, it will download the commonjs-utils package, and from then on it will remain cached and available for local execution. It’s as easy as that. No need to worry about package availability or bugging authors about uploading their packages, or working with forks. Nodules gives you dependency resolution that just works.

Nodules has numerous other capabilities for package mappings including engine-dependent overlays, single module references, code compiler/translator definitions (so you can use CoffeeScript or other code translators), but these are covered in detail in the Nodules documentation. For now we’ll focus on introducing Nodules.

The concept of dependency mappings allows us to utilize URLs as the primary identifier of packages, providing a clear and unambiguous identification, and then aliasing that mapping. This URL-based approach fits perfectly with the future EcmaScript modules system that is based on URLs as module resource locators as well. Mappings decouples the concerns of dependency resolution, module/archive repository mirroring, and package installation, allowing you to use the right tool for the right job. It also means you can use any utilize virtually any package in existence, it merely needs a URL. Packages may be forks, branches, or just a zip file available for download. Also, Nodules is not mutually exclusive to a package manager. You can use Nodule’s dependency resolution aware module loader for dependency handling and a package manager for package installation (which may involve much more than simply satisfying dependencies).

Module Reloading

The second key capability of Nodules is module reloading. This is absolutely essential for productive development. The write-code/save/debug loop for developers is the most critical path in development, and any extra steps or delays in the process have a huge impact on productivity. By providing module reloading, this process is optimized and developers can focus on coding and see the results of code changes as quickly as possible. I firmly believe that any developer who has to wait more than one second to see the effect of their code change isn’t coding in an optimal development environment. This is more than just a nicety, it is critical for focused, rapid development.

Correct module reloading can actually be a non-trivial task for a module loader. Simply reloading the module that was altered is usually insufficient since other modules may still have references to the older version of the module. Reloading the entire runtime (like the DJ module reloader for restarting Node) is very inefficient which can slow down development, especially for larger applications. Reloading the runtime can create other problems, such as losing debugging connections (a good debugging tool with breakpoints, watches, etc. is another key component of productive development), frequent re-execution of initialization code, and losing connections to clients (particularly painful for Comet applications).

Nodules is able to solve these problems and intelligently react to module changes by doing dependency tracking of modules to ensure all modules have correct references. Whenever a module is changed, that module is completely recompiled/reloaded (from source), and then any dependent modules (modules that have require’d the changed module) are re-executed (does not require recompilation, it uses cached compilation). This ensures correct, safe references with minimal compilation and re-execution of code. No unnecessary code or initialization code needs to be executed, and the runtime can continue as quickly as possible. Nodules also provides the ability to define code blocks that execute on re-execution. This is particularly useful for when you need to make sure that a HTTP server remains active while you swap out a different handler on reload. This is described in more detail in the documentation on module reloading.

Nodules provides safe, intelligent, fast module reloading without requiring server restarts which break connections for users and for debugging, and has negligible module monitoring costs (less than 1% CPU with even hundreds of modules on my system).

Modules Everywhere: To the Browser

Nodules helps facilitate the vision of consistent broad access to modules by giving you the ability to access a module from any location from within Node, using a CommonJS specification that can be implemented by anyone. Furthermore, Nodules is also designed to help deliver modules to the browser. Nodules includes several methods to facilitate retrieving modules by source and sending them to the browser. These methods can be used directly or they can be used in conjunction with Transporter, which will handle wrapping the modules with CommonJS module wrappers for async loading in the browser. On the browser side, one can then request and load the modules with RequireJS or Yabble.

Other platforms

Nodules is written using abstractions that make it very suitable for adaption to other platforms. In fact Nodules also runs on Narwhal (Rhino) and based on demand and interest, it will likely be ported to other engines. Narwhal developers have also discussed implementing the CommonJS package mappings proposal which would give it a compatible approach for dependency resolution.


Nodules provides a comprehensive module loader for Node with all the tools for efficiently building large scale libraries, tools, and applications and distributing them in the most portable manner possible.

Other posts in the series