Blog

Jun 19

Dojo + Koa

By on June 19, 2015 7:03 am

Dojo and its AMD loader provide outstanding tools for structuring a Web application on the client-side. However, the notion of “writing a JavaScript application” has widened in definition over the past few years with the increased popularity of Node.js. Though Dojo can be used in a Node.js environment with the AMD module pattern, other key frameworks have gained prominence in the server-side JavaScript space, including Express, Flatiron, Sails.js and the Dojo Foundation’s very own Persevere. These frameworks give structure and handle common tasks such as routing, template rendering, and content negotiation. Still, since most operations on a Node.js server are asynchronous, server-side JavaScript can be a complex, treacherous mess of callbacks. Enter Koa, a Node.js framework that attempts to save us from this callback hell by using ECMAScript 2015 Generators. Using Dojo on the client-side and Koa on the server-side makes for a robust, clean, and expressive application. In this post, we’ll explain what generators are and how to use Koa with Dojo for ultimate code cleanliness.

Gena-what?

Generators are an exciting part ES2015. In fact, some browsers don’t support them yet. Despite their currently-lacking browser support, generators are exciting because of their power and ability to aid in how asynchronous code is written. Before understanding what a generator is and why they are useful, let’s first understand how callbacks become unmanageable quickly.

Callback hell

Let’s define a simple async function for example purposes:

	function async(delay, done) {
		setTimeout(done, delay);
	}

This code asynchronously waits for a specified amount of time then executes a callback function. Calling this function is just like calling any other asynchronous function, such as making a request – you provide a callback function:

	// ...
	async(1000, function () {
		console.log('done!');
	});

So far so good. But say we want to call this function ten consecutive times; quickly, the code gets complex and ugly, as we’d end up with ten nested callbacks:

	// ...
	async(1000, function () {
		async(1000, function () {
			async(1000, function () {
				async(1000, function () {
					async(1000, function () {
						async(1000, function () {
							async(1000, function () {
								async(1000, function () {
									async(1000, function () {
										async(1000, function () {
											console.log('done!');
										});
									});
								});
							});
						});
					});
				});
			});
		});
	});

Generators to the rescue!

A generator is a fancy type of iterator: it has a next method that accepts one argument and returns {done, value} tuples and it also has a throw method. Generators allow functions to be “exited” and then “re-entered” at a later time, with state persisted across each re-entrance:

	function* oneToThree() {
		yield 1;
		yield 2;
		yield 3;
	}

	var iterator = oneToThree();

	console.log(iterator.next().value);
	console.log(iterator.next().value);
	console.log(iterator.next().value);
	// '1'
	// '2'
	// '3'

The yield keyword is like a “pause and return” flow control statement. When the next method is repeatedly called, the generator is partially executed, advancing through the code until a yield keyword is hit. This simple yet powerful API can be used to provide an alternative to traditional nested callbacks. Let’s revisit our async example from earlier with ten nested callbacks and write it using generators instead:

	function run(generatorFunction) {
		var iterator = generatorFunction(resume);
		function resume(val) { iterator.next(val); }
		iterator.next()
	}

	run(function* asyncExample(resume) {
		console.log(yield async(1000), resume);
		console.log(yield async(1000), resume);
		console.log(yield async(1000), resume);
		console.log(yield async(1000), resume);
		console.log(yield async(1000), resume);
		console.log(yield async(1000), resume);
		console.log(yield async(1000), resume);
		console.log(yield async(1000), resume);
		console.log(yield async(1000), resume);
		console.log(yield async(1000), resume);
		console.log(yield async(1000), resume);
		console.log(yield 'done!');
	});

This code will execute exactly the same as our nested callback code from hell. Note that we wrapped our async function calls in a generator function called asyncExample. We also wrote a special function, run, that executes a generator function in an iterative manner. This wrapper generator function and runner function together allow us to ditch the callback ugliness.

Koa, front and center

Because generators help us avoid callback hell in an asynchronous environment, they make our server-side JavaScript much cleaner. Koa is a Node.js framework designed specifically for this purpose: to clean up server-side JavaScript code by eliminating callbacks. This also makes error handling much easier as an added bonus.

A Koa server is an Application object containing an array of middleware, each of which is a generator function. Let’s look at a server that responds with “Hello World” for every request, adapted from the Koa website:

	var koa = require('koa');
	var app = koa();

	// response

	app.use(function *(){
		this.body = 'Hello World';
	});

	app.listen(3000);

Each middleware has a Context via this that allows you to access the Request and Response objects as well as get/set other information. In this example, we set the response body to ‘Hello World’.

Koa middleware functions cascade in a very expressive, elegant manner, something that was hard to accomplish with Express and non-generator frameworks. Based on the order in which middleware functions are registered, control is yielded “downstream” then flows back “upstream” before returning to the client. For example, let’s add a x-response-time middleware to our hello world server:

	var koa = require('koa'),
		app = koa();

	// x-response-time

	app.use(function *(next) {
		var start = new Date;
		yield next;
		var ms = new Date - start;
		this.set('X-Response-Time', ms + 'ms');
	});

	// response

	app.use(function *() {
		this.body = 'Hello World';
	});

	app.listen(3000);

Before responding with the expected ‘Hello World’, the request flows through our x-response-time middleware: the start time is saved, control is yielded to the response middleware, then control flows back to the response time function to pick up where it left off, setting the X-Response-Time header using the stored start time.

Dojo + Koa = clean code

Let’s walk through a simple example application that uses Dojo on the client-side and Koa on the server-side (download here). We’ll create a form that allows a user to enter his or her name and submit it; the name is then persisted in a MongoDB instance. Here is what it will look like when we are done:

Demo Dojo + Koa Application

First, let’s start with what we know: Dojo on the client-side. We’ll create a index.html HTML page that includes Dojo and references our application JavaScript file (via dojoConfig.)

<!DOCTYPE html>
<html>
<head>
	<meta charset="utf-8">
	<title>Dojo, Koa, and Mono
	<link rel="stylesheet" href="app.css?1">
</head>
<body>
	<div class="content">
		<form action="/users/add" method="post">
			<input type="text" name="name" placeholder="Enter your name..." autocomplete="off">
			<button type="submit" id="addButton">Add
		</form>

		<div id="users"></div>
	</div>

	<script src="//ajax.googleapis.com/ajax/libs/dojo/1.10.4/dojo/dojo.js" data-dojo-config="deps:['./app.js']">
</body>
</html>

Next, let’s create an app.js file alongside our index page. This file needs to do one thing: request all users so we can list them on the index page. We’ll use dojo/request to make the XHR request and dojo/dom-construct to create an element for each persisted user.

define([
	'dojo/request',
	'dojo/dom-construct'
], function (request, domConstruct) {

	document.querySelector('[name="name"]').focus();

	request('/users', {handleAs: 'json'}).then(function (users) {
		var userList = document.querySelector('#users');

		users.forEach(function (user) {
			domConstruct.create('div', {
				innerHTML: user.name
			}, userList, 'first');
		});
	});
});

At this point, we have a complete client: we have an HTML page with a form with a method set to “/users/add” and some JavaScript that requests users from “/users”. All that’s left is to create a server to service these two requests. And of course, let’s use Koa. Let’s create a server.js alongside the index page and application JavaScript:

var serve = require('koa-static'),
	route = require('koa-route'),
	parse = require('co-body'),
	monk = require('monk'),
	wrap = require('co-monk'),
	db = monk('localhost/koa-dojo'),
	users = wrap(db.get('users')),
	koa = require('koa'),
	app = koa();

Except for the MongoDB dependency, these are all different koa components, as it is architected extremely modularly. We are requiring middleware for routing, body parsing, etc. The usage of these modules will be clear once we fill in the rest of the file. Next, we need to serve our application and set up our routes:

// ...

// Serve the application code statically
app.use(serve('.'));

// Allow users to be added or filtered via an API
app.use(route.post('/users/add', addUser));
app.use(route.get('/users', filterUsers));

Above, we’re using Koa’s provided static middleware to serve our application. Next, we use Koa’s provided route middleware to set up two routes: a POST to ‘users/add’ and a GET to ‘/users’, the two routes our client will make requests to. Lastly, we need to define these two routes and start the server:

//...

function *addUser() {
	var user = yield parse(this);
	yield users.insert(user);
	this.redirect('/');
}

function *filterUsers() {
	this.body = yield users.find({});
}

app.listen(3000);

console.log('listening on port 3000');

Here is where the magic happens. First, note how the addUser and filterUsers functions are actually generator functions, meaning we can use yield within them. Additionally, note how we are using monk, a MongoDB wrapper that supports generators. Inserting a user, which normally would involve a database operation with nested callback, is a one line operation; the same is true for finding a user and sending these results to the client.

With that, our demo application is complete. We have a simple Dojo client that lists users and allows users to be added, and we have a simple generator-powered Koa server that handles these requests.

Conclusion

As Web developers, we inherently enjoy writing JavaScript, so Node.js’s increased popularity is a good thing. While Express and other libraries have served a great purpose to date, libraries such as Koa that make use of new language features as they become available will allow more elegant and clear coding. Dojo works well with such a library and can easily serve as a robust client in this client-server architecture.

Learning more

Want to learn how to use modern ES2015 or TypeScript features in your application? Looking for advice on how to architect an application that leverages JavaScript or TypeScript on both the client-side and server-side, based on your specific functional and technical requirements? We’re happy to help. Contact us for a free 30 minute consultation to discuss your application.

Jun 17

JavaScript in the Enterprise: Where do your developers turn for JavaScript Support?

By on June 17, 2015 8:45 am

The Situation

Deadlines are looming and it looks like it’s is going to come down to the wire. A developer has hit a roadblock while trying to integrate code from another team and connect it to a third-party API. He can’t figure out where things are going wrong. Is it his code? Is it the other team’s code? Is it the API? A few hours in and he’s going in circles — with no answer in sight.

‘Who can I even ask to help me with this?’

The rest of the developers on the team are heads down trying to complete their own tasks, and googling for an answer has proven fruitless due to the number of moving parts involved.

After pinging a couple of the developers on the other team and getting the runaround, the developer is completely stuck. Due to confidentiality and the rapidly approaching deadline, going to a public forum is out of the question and, even if he could, he’s unlikely to get an answer – let alone the right answer – in time.

This leaves him with with only a couple of viable ways to solve his problem.

Option 1: Keep hacking on it and hope the answer comes.

Option 2: Escalate to a development manager who will need to stop what they’re doing, review the issue, and work to solve it.

At this point, one thing is certain, the project is delayed and multiple tasks are stopped while this bug remains at large. The developer did the best he could with what he had to work with. But what if there was an…

Option 3: SitePen Support

The developer logs into his SitePen Support account, provides the details of his issue and asks for an answer. A SitePen engineer who is familiar with the developer’s source code, reviews the issue and works with the developer to get him unstuck. Meanwhile, the rest of his team continues on their tasks, uninterrupted. The project is saved!

With SitePen Support, each developer on your team has instant access to their very own dedicated help desk, staffed by SitePen’s expert JavaScript engineers. Inevitable problems like bugs, bad implementations, questions, and indecision are constantly wreaking havoc on project timelines and there are dozens of these issues popping up, around the clock, that kill team efficiency and threaten a project’s success. Choose OPTION 3 and eliminate problems quickly and efficiently by having SitePen’s team of senior JavaScript engineers on call to extend the knowledge, manpower and expertise of your development team.

Learn how SitePen Support can help your team!

Jun 10

Dojo FAQ: How can I sequence asynchronous operations?

By on June 10, 2015 8:01 pm

Dojo’s Deferred module provides a convenient way of managing asynchronous operations. If you’re new to deferreds, you can get a good introduction by reading our blog post and some tutorials on dojotoolkit.org: Getting started with Deferreds and Dojo Deferreds and Promises.

There are a few features of the then method on a promise that are important to understand and remember:

  • then always returns a promise
  • the promise returned by then resolves to:
  • the value returned by the callback passed to then
  • OR

  • if the callback returns a promise, the value that promise resolves to
Jun 1

Multi-Platform Distribution with TypeScript

By on June 1, 2015 10:47 am

Over the past several years, JavaScript has grown to be relevant not only for rich browser applications, but also for server and console applications. Many types of JavaScript libraries can be useful on both ends of this spectrum. Dojo 2 is no exception, and one of our goals is therefore to make it as easily distributable and consumable across environments as possible.

Module Compilation in TypeScript Today

TypeScript can already help toward this goal by compiling to both the AMD and CommonJS module formats. For example, given the following simple TypeScript module:

export function myFunction() {
    // ...
}
May 1

SitePen Podcast – Ep. 1 The Pilot

By on May 1, 2015 2:07 pm
SitePen Podcast

At SitePen, we’ve always thought it would be fun to do a podcast, we just didn’t know what to talk about. My idea of doing a show about Fantasy Football was universally panned as were topics such as speedrunning SNES games, Birding and ‘just talk about the weather’. Finally the idea came to us, we should talk about JavaScript!

It turns out, we are all very passionate about this topic and quickly agreed! So we gave it a shot and created a podcast. Is the name the result of too many brainstorming by committee meetings? Marketing gone wrong? Or did we simply mess up the name and decide to keep it? Listen to the podcast and find out!

Let us know what you think

This episode is a pilot which means that we need to show the big studio execs that people are interested. At least I think that’s how pilots work in podcasting. I actually don’t know. Either way, let us know if you’re interested in hearing more!

Apr 17

dstore 1.1 released

By on April 17, 2015 12:21 pm

dstore

We have released dstore version 1.1, which features a new set of stores for local DB storage. This feature provides the ability to store data locally in browsers, and use high-performance querying capabilities through the disparate technologies of IndexedDB and WebSQL (and localStorage), through the consistent dstore interface. Complex queries can be formulated in your application, and data can retrieved from local database storage with the same APIs and query format as is currently supported with other stores. Queries can then be automatically translated to SQL or cursor operations, depending on the underlying technology available.

Apr 16

We’re Contributing to Dojo 2!

By on April 16, 2015 2:28 pm

At the end of 2014, we looked ahead to determine where to focus SitePen’s open source efforts in the coming year. After our successful contributions to dgrid and the new dstore package, it became increasingly clear that contributing to the future of Dojo was where our team would have the biggest impact and the most fun. So, we’re happy to announce that we are and will be actively contributing to the planning and development of Dojo 2!

SitePen In Action

So far, We have contributed our knowledge and expertise in developing enterprise web applications to what Dojo 2 needs to be; a modular, collection of packages that fit together to deliver high-performance web applications that are well-designed, highly performant and maintainable.

We’ve also poured our hearts (and design resources) into the overhaul of the Dojo Website, which was in dire need of a modern design and cleaner information architecture.

roadmap

So Much Excitement!

A few of the reasons we’re looking forward to contributing to Dojo 2:

  1. the final result will be a tight collection of packages that feel familiar to Dojo developers while leveraging the capabilities and improvements of the modern web platform.
  2. It’s going to be authored in TypeScript! As a superset of JavaScript, TypeScript makes it simpler to support many of the recent ES2015 improvements and provide support for types and interfaces.
  3. Helping to reshape and grow the Dojo community!

Don’t delay! Check out the new Dojo Web site, read the Dojo 2 vision and roadmap and when you’re spun up like us, hit the contribute button and tell us that you want to be part of the future of Dojo!

Apr 14

Minimizing Dijit Instances in dgrid

By on April 14, 2015 11:22 am

dgrid

Dijit and dgrid provide a powerful set of user interface components, allowing for fast construction of sophisticated web applications with excellent performance and interactivity. However, one particular configuration of dgrid that can impact memory and performance: heavy use of persistent Dijit editors within grid cells. The dgrid’s Editor plugin makes it very easy to leverage various Dijits for editing cells. However, using the default “always-on” mode, where a Dijit is instantiated for every editable cell, in combination with a several columns with numerous rows can quickly consume a lot of memory and create a noticeable drag on your application’s performance.

Dijit editors can help provide some fantastic functionality, with powerful widgets for auto-completion, date input, and validation. In this post, we’ll review two approaches to achieving advanced data input in dgrids that are designed for heavy data manipulation without compromising performance or memory consumption.

Mar 26

Dojo FAQ: How do I upload a CSV file, then access that data in my application?

By on March 26, 2015 12:24 pm

DojoFAQ

While a more recent advancement allows us use the HTML5 file API to retrieve contents from files, this approach is not universally supported in web browsers as yet. Instead, we will access data from user-uploaded CSV files using the following steps:

  • Upload a file to the server
  • Retrieve the file from the server
  • Load the data into an easy-to-use format