In addition to the panel discussion and keynote which the Ajaxians were gracious enough to let me participate in. I also gave a “regularly scheduled” talk on Dojo 1.0, many of the lessons we’ve learned leading up to it, and the philosophy of construction (and why it benefits you and your users).

The talk was packed, with folks actually standing at the back of the room. Thanks to everyone who turned up and especially to everyone who asked questions. Getting the kinds of focused, thoughtful questions that were asked always makes talks like these better for everyone.

Update: after my talk Andreas of the excellent qooxdoo framework made the point that my examples of how to use Dojo 0.9/1.0 widgets didn’t clearly enunciate how fully we support programmatic creation of all Dojo widgets. While beginning Dojo users may see markup-driven examples like:

<span dojoType="dijit.ProgressBar"
   progress="20%"
   ...>
...

Dojo 1.0’s method of building widgets programmatically is hugely simplified from the 0.4.x days:

var pb = new dijit.ProgressBar({ progress: "20%" });
dojo.body().appendChild(pb.domNode);
...

Both examples create an equivalent widget, and every property that can be set via markup is also available to be passed via the configuration property bag that every widget expects when calling its constructor function via the “new” keyword. This lightweight convention, in conjunction with the alternate script types, allows for full feature parity between markup and programmatic construction of widgets. Both are first-class citizens in Dojo 0.9/1.0 and you can build your entire app with programmatic construction if you so choose.

I can only imagine what kind of a political nightmare it must have been for the IE team to pull this off, but they deserve praise for finally liberating their browser from the clutches WGA (whose “advantage” has always been unclear).

When it became clear that IE 7 was going to be tied to WGA in an early beta, it left a lot of angry developers scratching their heads. Perhaps smart moves like this are enough to counter the lack of public communication about the future of IE and the stunning paucity of debugging tools for Redmond’s browser (among other things), but anything that cycles IE 6 out of circulation faster is good for the web. Kudos to the IE team for doing the right thing by their users.

Dojo 0.9 is a different beast than any Dojo before it. It’s smaller, it’s (much) faster, and it “feels” different in many ways. As Jon Sykes noted, “everything is familiar yet strangely different”. I could go on for hours about why dojo.query() is completely indispensable or how having dojo.behavior as a part of Core changes everything, but there’s some fun stuff up the stack too. In particular, new parser bears mention, and a bit of explanation. In addition to now letting you build instances of any class (not just widgets), it also lets you configure their behavior as well as their properties.

For a long time systems like Flex and Laszlo have had a corner on naturally mixing behavior with markup. HTML’s native <script> tag doesn’t provide any context, and worse, can’t provide a way to be triggered by a particular event or action (outside of proprietary extensions). Several use-cases are important:

• Scoped execution
• Event-driven execution
• Attachment vs. replacement

In Dojo 0.9 we took a hard look at them and devised <script type="dojo/method"> and <script type="dojo/connect">. Lets take a quick tour and show how it makes writing event handlers natural, building new widgets a snap, and over-riding built-in behavior trivial.

(more…)

I’m excited to announce that SitePen is hiring in our R&D department, which roughly translates into “do research on how to improve the Open Web, make it a reality through Open Source software, and get paid to do it”. If that sounds like your dream job, we want to talk to you.

Here at SitePen, we’ve got some pretty deep experience with building apps for mobile devices and having seen the issues first-hand, I firmly believe that Apple is doing a great thing (in the medium term) for the world by making the Open Web the way to deliver apps to their phone. In the short term, they’ll be doing millions of people a huge favor by letting them replace their Treos with a device that doesn’t reboot when exposed to air, sunlight, or text messages.

There are interaction problems yet to be solved (how do you “thingify” a web app for the iPhone? offline? etc.), but fundamentally the iPhone puts the onus on the browser to mediate hardware access better. This is the right race to be running since it forces every one else to one-up Apple on openness and immediately cuts out the idiotic middle man situations that Brew and even J2ME create through oligopoly and ineptitude, respectively.

(more…)

This past Friday, we pushed Dojo 0.9 Milestone Release 2 out of the nest. This is the last milestone before Beta and the system is starting to take a recognizable shape. Only thinner.

Here’s what’s new and awesome in M2:

• Dijit has landed! Holy cow is it fast. Stay tuned for themes and more widgets.
• Layered builds. Slice and dice your builds any way you like to achieve maximum performance
• Style code is now even faster
• Lots of new modules, bug fixes, and quality APIs

(more…)

As Chief R&D Monkey here at SitePen, my work primarily revolves around ensuring that Dojo rocks for our customers and that we take what we learn from building apps and ensure that the toolkit evolves to encompass those lessons. Early this year it became clear that for folks building Dojo apps, the question of “what is Dojo?” was becoming increasingly hard to answer. Each individual user might be working with a different subset and therefore they don’t share a common definition, which makes sharing what you know about the system harder than it needs to be.

In response, we’re in the middle of a huge undertaking: rethink the entire API surface area of Dojo from the ground up and split it up into 3 different top-level projects. With R&D dollars and time coming from all corners of the Dojo universe, 0.9 is shaping up to be smaller, faster, more coherent, and easier to understand. I’ll be doing a regular “where are we now?” set of posts here on the SitePen blog for folks who don’t have time to follow all our mailing lists and forums or watch the commit flow.

So, where are we now?

(more…)

At last month’s Open Ajax Alliance meeting in New York, a good chunk of time was dedicated to revisiting what, if anything, OAA should do about “Mobile Ajax”.

I’m something of a pessimist about what most people mean when they say “Mobile Ajax”, but the OAA meeting included several bits and pieces that gave me a lot of reasons to be less grumpy. Firstly, the device manufacturers are starting to get it. One of the attendees made the prescient point that the content devs, the phone vendors, and users alike are all trying to route around the OpCo’s. This suddenly put a lot of things in perspective for me: is wifi a stupid thing to put in a phone? You bet. But it beats the OpCo network. The list of cat-and-mouse features like this goes on and on. At some point something’s gotta give.

At the meeting someone was passing around a very slick Nokia device that has done more to convince me that that the web on phones isn’t dumb than anything else I’ve seen in the last year. Clearly, the UI issues are horrendous, but good screen + beefy CPU + wifi + thumboard makes a lot things bearable. Better yet, there was word that many of the issues I’ve raised about what’s useful input and output in a mobile web context (hint: text is no longer king) is starting to be addressed by the phone vendors. Better yet, there’s rewnewed work at the W3C to address some of the most pressing issues. In fact, it seems that the mobile browser vendors might be attacking the problem of rich content types better/faster than their desktop counterparts. An 18-24 month handset turnover rate will help make up for a lot of lost time.

But I mentioned Ajax and not just the web.

Ajax/DHTML lives in the nexus between a lot of the various web standards which means that it’s subject to the vagaries of all of them. No matter what the specs say, the lowest common denominator of what the clients ship is what the “standard” really is. The mobile browser market has been hugely fragmented for a very long time, but that looks to be changing. I think it’s pretty safe to say that in the next 3 years we’ll be down to three major rendering engine “flavors” in smartphones and that KHTML/Webkit derived browsers will probably carry the day for folks not on MS or Blackberry devices. Frankly, I didn’t expect it to happen this quickly, but I’m gald of it. Clearly, there are major issues still to overcome, but relative standardization on browsers brings the mobile web into alignment with a world that desktop web devs can approach. The sooner good browsers trickle down, the sooner we can start the painful process of forgetting what we think we know about doing webapps and building better idioms for mobile apps.

I got back late in the week from a trip to the ISOC-IL annual conference where I gave a full-day tutorial on Ajax and Dojo (with a little bit of Comet mixed in for fun). The conference organizers were tremendously kind and thoughtful and the whole trip was a lot of fun. What was best, though, were the great questions that folks had during the tutorial. It’s always a hit-and-miss on this kind of thing (big room, low lights, different country, potential language barrier, etc.) and it can go very badly indeed if people aren’t curious about the topic(s). The ISOC-IL conference drew great folks with wonderful and pointed questions that helped make everything go faster. It’s rare that conferences are that tightly run, well-attended, and have such good food and caffeine. Conference size can’t beat quality and the ISOC-IL conference proved it.

The day after the conference, I was lucky enough to get to visit some of the Zend hackers in Tel Aviv who were also tremendously kind and had lots of great feedback on how we can improve Dojo. Work aside, this was my first trip to Israel and I can say with some confidence that I’ll be back. My trip left me thinking that there’s a lot more to Israel than news accounts in US papers would imply.

Dylan’s last post on performance is only one in a series we’ll be running on the topic, and as promised this post is all about the tools of the trade for doing Ajax app performance tuning and how to use them. Here at SitePen, we often get called into a project when the heat is really on: after most of the code is written and just before the hoped-for (or worse, already slipped) ship date. Needless to say, we like this situation even less than our clients do. We always prefer to be involved early enough in the development process to be able to steer clients towards architecture decisions that will scale and perform better and still fit the budget, but sometimes the damage is done. What then?

Let the data guide you.

It sounds simple and naive, but most developers who have been through their share of performance tuning crunches will have horror stories of hours or days lost to phantom performance “problems” that turned out to be nothing more than the hunch of one developer. It’s no use optimizing your database configuration or adding more expensive storage systems if the bottleneck is at the JavaScript or HTTP levels. Likewise, tuning your HTTP server to the hilt may have no effect if the bottleneck is storage contention or TCP fragmentation. Getting to a root cause requires defining the goal of a tuning project, testing each change to isolate causality, and keeping a log book handy and up-to-date.

It may be necessary to write custom tools to help you diagnose problems in some environments, but there’s a stable of tools that we always seem to fall back on here at SitePen and I’m going to do a quick run-through of what we do at each step when we’re analyzing webapp performance and scalability problems (note: they’re not the same thing!). Here’s the short list, and why we can’t live without them:

• Firebug 1.0 Beta
• To users, perceived performance is the only thing that matters, and that means that investigation should examine the system from the user’s perspective and work backward from there. There was a time when the Firefox TamperData extension ruled the roost for this, but no more. Now that page loading requests can be graphed inside of Firebug, things are getting a lot easier. Not only can the graph view show 404 requests and slow responses, it often lays bare the synchronous nature of script execution and requests and the 2 HTTP connection limit. Generating “before” and “after” evaluations for clients has never been so easy.
• Venkman and Firebug 1.0 Beta
• Now that Firebug includes some profiling and debugging support, Venkman may finally be on the way out, but whichever tool you use it’s highly valuable to be able to profile in-browser JavaScript performance at a function level. HTTP and server-side problems are often a source of perceived latency, but simple testing with full caches can easily point to client-side performance issues. Any logging or profiling system will impact overall page performance, but you should be using these tools to get relative timing data. There’s a special mode for the Dojo package loader that can be used to get accurate function names and line numbers. While the timing information may not translate 100% across FF, IE, Opera, and Safari, the relative timings tend to be in line.
• dojo.profile
• The dojo.profile module lets you do tic/toc timings of JavaScript code and provide a table showing averages and total timings. We use this to verify relative timings across browsers once Venkman/Firebug point out bottlenecks and to validate fixes in a cross-browser way.
• Tsung and Apache Bench
• As I noted earlier, HTTP-level performance problems can seriously impact application latency. Neither tool can pinpoint fundamental problems like outbound bandwidth saturation (making the system more scalable doesn’t matter if you can’t send more data across your link), but when the problem is one of scale and not instantaneous performance, these tools let you begin to validate assumptions. Apache Bench is great for testing balls-to-the-wall concurrency of a single script, but very often you’re more interested in full-app performance under more realistic workloads. While there are commercial tools available that can do this kind of load testing in a “real world” way, Tsung provides a highly-capable “replay” proxy mode that will generate workloads that can be used to monitor system performance from a variety of angles. Since we’re most often interested in “how many users can it handle?” rather than “how many times a second can I request foo.php?”, Tsung is an invaluable ally. As a downside, Tsung often requires a proxy and an Erlang build.
• bonnie++
• Databases and web servers alike need good I/O performance, and bonnie++ lets us determine if we’re getting anything like the theoretical disk performance out of a system. Knowing the “shape” of your workload is essential, but I find that when remediating disk I/O issues bonnie++ usually finds its way into my analysis.
• Please, please, please make sure that your file systems are running with noatime set.
• “EXPLAIN” statements and slow-query logs
• SQL is the ubiquitous abstraction that most of the web runs on, and every database system today provides information on how well it’s returning what you request. A thousand other things can niggle your SQL server performance to death, but nothing should get done without logs and EXPLAIN output to guide you.

System development and tuning need to go hand-in-hand, and expert help can clearly make a huge difference. The tools above are some of the most visible artifacts of the process, but it’s discipline in the process itself that’s of the most paramount importance. Let the data guide you and everything else is likely to work out…assuming you know where the goal line is.

Performance tuning can easily drive you crazy should you not have a goal in mind. Without a goal, there will always be another tweak, another 3% to be eeked out of the system. Combined with the marathon sessions that seem to lead nowhere, it’s important that developers doing performance work remember to keep their eye on the ball and to take a walk or a nap or just stop for the day when there isn’t forward progress for an hour or so. That, of course, means having a ball to keep an eye on. So before you start your tuning adventure (or call us in to help), you need to know what your budget is, what your responsiveness goals are, and what your scalability targets are.

For more thorough treatments of how to build things that both perform well and can be made to scale, I strongly recommend Cal Henderson’s “Building Scalable Web Sites”, Theo Schlossnagle’s “Scalable Internet Architectures”, and Jeremy Zawodny’s “High Performance MySQL”.

Next time: why the Dojo build system matters, why the x-domain package loader is awesome, and other stupid HTTP tricks.