Introducing dojo-amd-converter

By on April 3, 2013 9:30 am

AMD offers many benefits over the legacy Dojo module syntax. While Dojo is forwards-compatible, to take advantage of the benefits of AMD, developers are often faced with the challenge of migrating and refactoring large quantities of boilerplate code from the legacy dojo.provide/dojo.require syntax to AMD’s define and require. As we make the upgrade to AMD in our projects, we also usually want to leverage the latest version of our evolving APIs. With early work beginning on Dojo 2.0, we definitely need to work together as a community to automate as much of the upgrade process as possible.

Introducing the dojo-amd-converter

We have created an alpha quality release of dojo-amd-converter, a tool for helping you make a one-time migration of your existing source code to a newer version of Dojo.

Be warned, the tool handles only 70-80% of the manual conversion process, and generally does better with more standard usage of Dojo. Normally we wait a bit longer before announcing our projects, but this should be useful in its early alpha state. We know this tool has quite a few open issues that need to be fixed and we invite you to help us improve it.

Our hope is that this conversion tool, which is today useful for converting pre-AMD code to AMD code, could also form the basis for a community effort to make it possible to migrate code bases more efficiently from Dojo 1.x (pre-AMD or AMD) to Dojo 2.x. That work has not yet started, since Dojo 2.0 is not yet an actual thing.

What does it do?

Based on a collection of rules, and leveraging Node.js, libxml.js, and esprima, the parser will take an existing Dojo source tree, and convert the following:

  • non-AMD modules to AMD modules
  • namespace references to module references
  • legacy HTML custom attributes to HTML5 compliant data-dojo-*
  • legacy i18n bundles to AMD bundles
  • older Dojo APIs to their refined replacements

For example, if before conversion you had a file like this:

dojo.provide("my.namespace.i18n.Bundle");
dojo.requireLocalization("dijit", "common");
dojo.requireLocalization("foo.package", "Bundle");
dojo.requireLocalization("bar.package", "Bundle2");

var foo = dojo.i18n.getLocalization("foo.package", "Bundle"),
	bar = dojo.i18n.getLocalization("bar.package", "Bundle2", foo.bar),
	baz = {
		locale: "en-lol",
		i18n: dojo.i18n.getLocalization("bar.package", "Bundle2", this.locale)
	};

You would end up with this, after conversion:

define([
	"dojo",
	"dojo/i18n",
	"dojo/i18n!bar/package/nls/Bundle2",
	"dojo/i18n!dijit/nls/common",
	"dojo/i18n!foo/package/nls/Bundle"
], function (dojo, i18n, i18nBundle2, i18ncommon, i18nBundle) {

	var foo = i18nBundle,
		bar = i18n.getLocalization("bar.package", "Bundle2", foo.bar),
		baz = {
			locale: "en-lol",
			i18n: i18n.getLocalization("bar.package", "Bundle2", this.locale)
		};
});

Usage

To use the dojo-amd-converter, Mac and Linux users should follow these simple steps:

  1. Clone the repo using Git:
    git clone --recursive https://github.com/SitePen/dojo-amd-converter.git
  2. Modify the provided config.js file to specify some basic settings for your application, such as your source and destination directories.
  3. From within the repo, run the provided shell script on your configuration file:
    ./parse.sh config.js
If you receive a permission denied error, you may need to chmod the shell script:

chmod +x parse.sh

config.js options

  • excludePaths – An array of paths that will be excluded from processing. Each path should be a regular expression
  • srcDir – Root directory of the files to process.
  • distDir – Output directory of the processed files.
  • printOutput – Print output instead of sending it to the output directory.
  • makeDeclareAnonymous – Make declare calls anonymous instead of leaving the first argument as-is. This will avoid creating any global variables with declare.
  • removeUnusedDependencies – Remove dependencies that are not referenced in a converted module.
  • onlyProcessTemplates – Only process HTML files inside template directories.

The AMD converter does not currently work on Windows, due to the dependency on libxmljs.

Getting Involved

This project is in the early stages, and our hope is that it will also be used for work to convert to Dojo 2.0. We have many open issues for items we have already identified in our work using this tool. The dojo-amd-converter tool is licensed under the same terms and conditions as other Dojo Foundation projects.

To get involved, you will need to have a Dojo Foundation CLA on file for us to consider your code change. Please get involved if this tool is of interest to you now or in the future, as we think it is a much needed improvement for making it easier to upgrade between Dojo releases.

Getting help

We have handled many large scale application conversions from legacy to modern, AMD-based Dojo. Contact us to discuss how we can help with your application.

Comments

  • Ron M

    I think this is a great project and definitely needed. Our company developed a similar tool that did most of the work for us, although did not automate things like converting HTML files or referencing templates, etc. The rest we did by hand – and are still not all the way done, so this tool could still help us out.

    One thing it did do though was automatically add module dependencies for our own modules to the files and give us the opportunity to configure how they were named going forward. The configuration file format basically followed the tables you have in the livedocs 2.0 migration guide which denote the old syntax (dojo.addClass), module (dojo/dom-class), and new syntax (domClass.add). We did it slightly differently and split up the last column into two, the module id (domClass) and the function (add).

    This configuration file gave us the ability to convert not just Dojo APIs, but our own APIs as well. In the brief look I took at the source for this tool, it looks like you are taking a different approach in handlers.js that looks tailored specifically for Dojo functions and does not currently have a way to configure more functions – at least not in config.js.

    Is something like this planned for this tool? For example, we had cases such as:

    dojo.provide(“A.B”);
    dojo.require(“A.C”);
    dojo.declare(“A.B”, A.C, {
    func: function() { A.D.foo(); }
    });

    Would this tool convert this to the following? And would it allow renaming D to something else, much like you rename “window” to “winUtils”?

    define([“dojo/_base/declare”,”A/C”,”A/D”], function(declare,C,D) {
    return declare(C, {
    func: function() { D.foo(); }
    });
    });

    We wouldn’t necessarily require “A.D” because we knew it was included elsewhere, so why require it again. I know – not great coding practice, but it happened. That is one reason AMD is so nice.

    In this case, a basic configuration file would look like the below, but could easily be changed to handle more complex cases:

    [

    [“A.D”,”A/D”,”D”,”foo”]

    ]

    If you do allow for this kind of configuration, I could easily see this tool being used for future migrations when there are API changes. Dojo could even provide such a configuration file with each new release.

    Thanks.

  • Hi Ron,

    https://github.com/SitePen/dojo-amd-converter/blob/master/handlers.js provides a lot of the logic for what is currently supported.

    At the moment, something like dojo.require(“mypackage.mywidget”) becomes “mypackage/mywidget”. There’s not currently a way to map the local name, though that’s a good suggestion (either a config or handler function for mapping names, etc.). Could you please open an enhancement issue at https://github.com/SitePen/dojo-amd-converter/issues?state=open ?

  • Samurai Jack

    What are the version requirements for Node.js and libxmljs for the converter to work. I have tried installing Nodejs v0.10.18 and libxmljs v0.8.1. But every time I try to run parse.sh I get the error “libxmljs” not found. This tool can save us huge amount of time IF I can make it work somehow.

  • I’ve just tested using node 0.10.18 and libxmljs 0.8.1 and it works fine. The error you describe means that either libxmljs did not install or you ran npm install in the wrong directory; it should be ran from the root directory of the converter.

  • TacomaGranny

    I am struggling to make this work. My dojo is embedded in an html file. Could you maybe provide a little more detail in the example for us relative dojo newbies.

  • sudatta

    I am trying the converter to work. But when I am trying to run
    ./parse.sh config.js I am getting this below error.
    parse.sh: line 7: node: command not found

    Please help asap , as its very urgent. I am working on a project where we need to migrate dojo 1.3 to dojo 1.10 version.

  • Ramana

    Hi Dylan,
    I am a dotnet developer ,My application is using DOJO1.5 and running on windows.
    I want to upgrade to latest version dojo(1.10.3).Is there any way to upgrade to convert my legacy code to AMD automatically?

  • Hi Sudatta, Unfortunately I didn’t see your comment previously. The best thing I can suggest is asking under a support plan ( http://sitepen.com/support ), but it sounds like you didn’t have Node.js installed

  • The dojo-amd-converter project will get you about 70-80% of the way there, in terms of upgrading boilerplate and converting some legacy APIs to modern APIs. If you need hands-on help, we’re happy to offer it under support ( http://sitepen.com/support )

  • Chinta Renukeswar

    even i tried to install the same versions , but i am not able to run parse.sh file .. Will that work with new versions of node js

  • @chintarenukeswar:disqus I know we have not needed to use it in over a year, so it’s entirely possible that it doesn’t work with more recent versions of Node.js. Could you please file an issue on the GitHub project and we’ll look into it?