Someone managed to upload a document that broke Moxie — I’ve removed this document from the Moxie database and cleaned up most of the test docs people added to Moxie, leaving just a few. Go ahead and give Moxie + Dojo Offline a try if you haven’t before.

Sorry about getting this up a little late in the day; I’m actually in Redmond, Washington right now to attend a Microsoft research event at their R&D labs. I just got situated at my hotel and found a WiFi network to work from to start hacking on code and post this status report.

Last Week

Last week we put up a screencast showing Dojo Offline and Moxie in action; created a blog post discussing our latest Dojo Offline deliverable and source code; put up our first official release of Dojo Offline (the JavaScript API layer + Moxie — this is an alpha release that does not include the client-side proxy yet); and got a live demo of Moxie running on our servers.

Coding was a bit slow later in the week since I was working on a speech for Yahoo.

This Week

The big task is to continue to refine the local client side proxy. This includes improving linkage between the JavaScript layer and the local proxy to make sure we work correctly when the network is present or gone; pulling and adding resources from the local proxy cache even without HTTP/1.1 headers sent from the server if we are offline; and more. We might also begin to create installers/uninstallers for Windows, as well as code to register the local web proxy for startup when the system starts.

Also, the Moxie demo has been used by scores of people, with lots of interesting sample documents added by folks, and someone found a way to upload some content that breaks when we download it through our JSON data structure (I believe it is a SCRIPT tag that is closed, but I am still isolating the bug). Until then the downloading phase of Dojo Offline for Moxie spins continuously; this is a Moxie bug and not a Dojo Offline bug, which I will isolate this week and fix.

Instead of doing a status report this week I have created a screencast and blog post: Screencast of Dojo Offline + Demo + Release Download

Last Week

The JavaScript layer for Dojo Offline, including the Moxie demo, were brought to functional completion. This means we have syncing working, including local data storage — Moxie now illustrates and shows both. Dojo Offline (the JavaScript libraries) will work even without the smal client-side proxy being finished, though it will not be reliable enough for deployed apps without the client-side proxy.

This Week

Now that the JavaScript layer is functionally complete, it is now time to turn our attention to Polipo and the small client-side download.

Three big tasks this week. First, I will push an alpha release of Dojo Offline that has the JavaScript layer functionally complete to my web server; this will let folks download and play with the code created so far, and will include a short blog post and Moxie demo posted on my site as well. Second, I am reviewing C for the client-side installable, since I have not worked in C for awhile; the third big task is to review and study Polipo, the local, client-side web proxy, and become proficient with its source base.

Overview

Every week we will be sending out a status report on the Dojo Offline Toolkit project to keep folks abreast of what we have accomplished the last week and what is planned for the week ahead.

Last Week

Last week the focus was on syncing and local data storage. Just as in previous big tasks we grounded things by focusing on real world applications, we began work on the JavaScript API for syncing and local data storage with a real application: Moxie.

Moxie is a demo application created for Dojo Storage in the past, and is a web-based word editor and word-processor. At the beginning of last week, Moxie only did persistent local storage using Dojo Storage with no server-side component. To make sure I’m not wandering off into architecture astronautics, and to also always have a real-world demo application for Dojo Offline, I wanted to turn Moxie into the kinds of applications that the Dojo Offline Toolkit will be embedded into; one’s with a server-side system, some kind of web-servicey way for the client to interact with the server (REST, XML-RPC, etc.), and a relational database for storing data.

I modified Moxie to use a MySQL database for storing it’s documents on the server-side. Then, I created a simple data layer for accessing these documents in Java using JDBC; I didn’t necessarily need to use a relational database or a simple facade when dealing with the database (I could have done everything as a serialized hashtable into the filesystem, for example), but I imagine that most applications that will embed the Dojo Offline Toolkit will be constructed in a similar way, and I want to catch any synchronization or offline issues that might be common. All of this was done in Java.

Next, I exposed a simple RESTian API for Moxie to get documents, post new documents, delete documents, get a list of documents, etc. using a Java servlet . REST is a simple alternative for web-based services based on HTTP itself, as opposed to XML-RPC, SOAP, etc.

The next step was to modify Moxie’s client-side to now interact with this server-side, using the RESTian APIs to get documents, post new ones and updated ones, etc.. At this point, Moxie was now a traditional web application — it did not save it’s data into Dojo Storage, but rather loaded and saved its data and list through the server-side’s RESTian API.

Jumping back to the server, I created a full synchronization framework, loosely based on the API that was in our Dojo Offline API blog post a few weeks ago. I got this sync framework to take in the JSON client sync info like we laid out in that blog post, and got the server to correctly process this JSON and produce it’s own JSON response. I hand fed the client-side’s JSON into this server-side sync system, and the server would correctly respond with the right sync responses. I also made sure that this sync infrastructure is generic, so that any Java server-side apps that need sychronization with the Dojo Offline Toolkit — it works generically with different item types, such as “documents”, “tasks”, etc. I won’t describe it in detail, because, as you’ll see in our next section, I found something better.

This Week

After building the server-side sync framework, I realized some important facts: this was just too hard to build, and there’s a simpler way to do things. While I was able to roll a generic Java-based server-side syncing framework, what about someone working in PHP or Ruby? Are they going to roll custom support for Dojo Offline’s JSON-based syncing protocol? It’s just too difficult.

I then realized that we don’t need any real support from the server-side for syncing — in fact, the JSON-based Dojo Offline sync protocol is actually a premature optimization that’s not needed. When I created the server-side Moxie, I created RESTian based APIs that allowed a traditional web-client to communicate with the server, then I created the SyncServlet which allowed a web-client to do syncing through a completely different avenue. If you crack open our Dojo Offline sync protocol, you will basicly just see a list of commands, such as “create this item”, “delete this item”, etc.

This lead me to realize that I can ditch our sync protocol and the need for server-side support, and instead have the client-side be a little more involved. This is great, since it lowers the bar even more for folks to be able to use Dojo Offline in their applications without heavy re-engineering.

On the client-side, in the Dojo Offline Blog post, I described some of the data structures that a programmer writes into so that their data is persisted on the client-side and is synced:

// called when the page is finished loading and the offline toolkit

// is ready to be used

offline.onLoad = function(){

// define where to put our offline user data; these

// data structures will automatically be kept in sync

// so we can use them in our application, and will

// automatically be persisted in local client-side

// storage

contacts = offline.getDataStore("contact");

}

function addContact(contact){

contacts.newItem(contact);

}

When you use a DataStore, such as the ‘contacts’ data structure, you use methods like newItem(), deleteItem(), etc. to work with it — this causes a command log to be updated in the background transparently. In the old sync design we would essentially serialize this into JSON and send it to the server, where the server would deserialize and replay the log, running each of the commands, such as “update” and “create” on the server-side. Why should we do this, though? Why not just allow the application programmer to place their own hooks on the client-side, and simply replay the command log on the client-side? The command log consists of a bunch of CREATE, UPDATE, DELETE, etc commands — the client-side can just have a callback to know when a create command has occurred, for example, and simply call a pre-existing server-side API that exists for creating that kind of item, updating that kind of item, etc. Most application’s will already have these kinds of APIs, since they are needed by Ajax application’s anyway, so why not leverage them?

For example, in the above ‘contacts’ DataStore, the client-side can place a function on to contacts.sync.onCreateCommand that will be called whenever a create command is inside the command log. This method could then take the command data, which would be a new contact, and call a pre-existing server API for creating a contact.

This week I will build the client-side using the approach described above. Changing our sync approach won’t create any new work for me, since I hadn’t tied the server-side syncing into the client-side yet — plus, this sync approach is simpler, so it should be simpler to implement.

Overview

Every week we will be sending out a status report on the Dojo Offline Toolkit project to keep folks abreast of what we have accomplished the last week and what is planned for the week ahead.

Last Week

We finished the UI portion of the Dojo Offline Widget, blogged it, and put a demo release up.

Last week we also finished:

• dojo.dot.files is finished and works now — this code refreshes any files that need to be available offline. Moxie, our demo app, was modified to correctly use dojo.dot.files to indicate what resources should be available offline.
• Dojo Offline now features a background network ‘thread’ that is testing every 15 seconds to see if a web application’s web site is available or not, shifting our on- or off-line status appropriately.
• Manually moving on- and off-line now work.
• Moxie was updated to have this new functionality. We also modified Moxie to use a customized version of the Dojo Offline Widget UI to make sure that UI customization works.
• The infrastructure to know when Dojo Offline is truly loaded and ready to be used was coded and tested.
• The infrastructure for Dojo Offline to load and save core data in local persistent storage was finished.

Another new release of Moxie with this functionality was pushed out . Expect a short blog post today with details on this functionality as well.

This Week

The first task today is to create a short blog post on our new release that has the features indicated above.

Next, we are going to jump into the guts of syncing and application-level data persistance and manipulation.

Overview

Every week we will be sending out a status report on the Dojo Offline Toolkit project to keep folks abreast of what we have accomplished the last week and what is planned for the week ahead.

Last Week

After a bunch of thought to ensure an easy-to-use and powerful API, we blogged what the Dojo Offline API will look like. See the blog post. Thanks to everyone who helped.

Last week we also began coding the JavaScript API for Dojo Offline. One nice highlight is that folks will be able to begin playing around with Dojo Offline even before the offline proxy is ready.

This Week

The default Dojo Offline, as seen in the DOT mockups from a few weeks ago, is almost done; this UI is driven off of the Dojo Offline framework, and is easily customizable. Expect to see it done today or tomorrow, with a demos that allow you to play around with the UI portion of Dojo Offline.

Once the UI portion is done, the next step is to fill out the framework itself, starting with the dojo.dot.file portion of the API.

Overview

Every week we will be sending out a status report on the Dojo Offline Toolkit project to keep folks abreast of what we have accomplished the last week and what is planned for the week ahead.

Last Week

The big task last week was finishing the Dojo Offline API. Lots of great people gave lots of great feedback, and the design for how programmers will work with DOT is done; expect a blog post today or tommorrow with details.

This Week

The first task this week is to document the API we have come up with. Next, its time to start coding. The first task will be to implement the API in JavaScript. A nice side-effect is the API will actually work without having us to finish the local proxy immediately, since it can just cache it’s UI files using standard HTTP/1.1 caching inside the browser cache. This means folks can get started prototyping apps quickly; the downside is that your UI files might get blown from the browser cache, which is what the small, downloadable web proxy is meant to solve.

Overview

Every week we will be sending out a status report on the Dojo Offline Toolkit project to keep folks abreast of what we have accomplished the last week and what is planned for the week ahead.

Last Week

The big task last week was finishing mockups of what potential offline-enabled web applications might look like. The goals here are to make sure we are building the right technology to support offline-needs and to find a common UI that can be turned into a template usable by developers. We created three offline mockups: Gmail, Blogger, and a corporate portal named Adenine. You can see a full blog post on the SitePen blog with these mockups here: http://www.sitepen.com/blog/2007/01/09/28/

Once the mockup was finished mid week, we moved on to defining what Dojo Offline’s API would look like. A lot of great suggestions came in from folks like Dustin Machi, Brendan Eich, Julien Couvreur, and more. We also talked on the phone with Dave Camp, a Mozilla developer working on offline access in the web browser itself, in order to identify common needs and find similar APIs.

On Friday we gave a presentation at the Dojo Developer Day on the Dojo Offline Toolkit, which was received well. We also had nice several nice writeups in the mainstream computer press, including eWeek ( http://www.eweek.com/article2/0,1895,2080295,00.asp ) and InfoWorld ( http://weblog.infoworld.com/techwatch/archives/009606.html )

This Week

The big goal this week is to finish the Dojo Offline API and document it, then start coding.