[Note: This blog post is out of date. For up to date information on Dojo Offline please see the official web page.]
Introduction
The best way to start development on a programming framework is to ground it in the kinds of user interfaces it will be used in. This ensures that you don’t create astronaut architectures that have no real world use.
The first step in creating the Dojo Offline Toolkit is to therefore figure out what offline web applications might look like. This will help us determine what to include in the Dojo Offline API and what to leave out, and will also be bundled as a simple HTML/CSS template that developers can easily drop into their applications.
SitePen kicked this off by creating offline-enabled mockups of three popular and useful web applications: Gmail, Blogger, and a corporate portal named Adenine. For all three the goal is to find a consistent, simple user-interface for offline web applications.
Gmail
We start with a mockup of an offline-enabled Gmail (click on any of the images to view them full-size):
The first thing to notice is the addition of a new widget on the left-hand side of the page, the Offline Info widget. This widget encapsulates all of our offline functionality for Gmail.
If the Dojo Offline Toolkit is installed and you sign into Gmail, Gmail will automatically download your 100 newest emails into the offline cache. Now you can access Gmail offline to read your newest emails or compose new ones. When you sign back on your emails will be automatically synced and uploaded:
What happens if the Dojo Offline Toolkit is not installed? Here’s what you see:
Simple! Clicking the ‘Learn How’ link takes you to a page with the following getting started text:
- Download the Dojo Offline Toolkit, a small (200K) utility that helps this web site work offline. This tool is secure and safe for your machine, and should take just a few seconds to download. It will work across Windows, Linux, and Mac OS X.
- Double-click the installer on Windows and Mac OS X to install. For Linux, the installer is a small Firefox extension — approve the install.
- To access this website even when offline, drag the following link to your desktop: Run Gmail.
- Double-click the link on your desktop to start this web application, even if offline.
This page makes it easy to get going with DOT — the ‘Download’ link will automatically select the correct download of DOT for your platform. The ‘Run Gmail’ link can be dragged to your desktop, providing an easy icon to double-click to run Gmail even when you’re offline. This will open your web browser and show the Gmail UI, along with your offline data.
Odds & Ends
Don’t forget that our goal is to find a relatively generic UI for offline web apps, so SitePen has to define the edge conditions for some of this and provide a bit more detail here.
Here’s the full sequence of syncing messages. First, we download the web application’s user-interface into the offline cache if that has never been done before, or update parts that are new:
The status messages give the percentage of completion so far, along with various messages. Next, we upload anything that is new in the offline cache, like new emails:
Then, we fetch the new stuff:
Finally, we either succeed:
Or we fail:
For an error, the little ‘details’ link will pop up a window with error details returned by the server; Dojo Offline apps can return any appropriate string here.
What happens if there are conflicts between local and remote data? This won’t happen for Gmail, since it is basicly a single-user application, but you could imagine it for Google Docs, an online word processor. For the first version of Dojo Offline we make an important decision: the server must make a defacto choice for data merges and conflicts, without a “merge” user-interface for the end-user. This is application specific, so a Google Docs might simple merge a user’s local document with the remote one. If there is a conflict, it could make an application-specific choice to simply use the data from the newest version of the document.
The decision to not support a complicated conflict/merge UI in Dojo Offline currently is an important one to reduce the complexity of this first iteration. Quite alot can be done by having the server simply choose how to merge and decide between conflicts in a non-UI way; if the server chooses, it can save both versions of the data for later presentation for the user, with two different document filenames, for example.
If automatic merging or conflict resolution occurred on the server, the user is notified of this:
If a user clicks the “(details)” link, a popup window will appear with text returned by the server that indicates how merging or conflict resolution might have occurred, similar to the log the Palm Pilot software uses after a sync (which also does automatic merges, btw). This should be terse and not confusing, such as “The document ‘foobar’ was changed on the server, but your version is newer — your’s was used.”
There is also a timestamp on when the last sync occured, which in this case is “Yesterday at 9 AM”:
Under this are the number of unique, new or modified items that might be in the user’s local cache but which are not available remotely; this is shown with the ’3 modified offline items’ text. This will clear once a successful sync has occurred.
A user can also elect to synchronize manually, by pressing the Synchronize button themselves; this will kick off a synchronization session.
The Configure screen is also important; clicking the ‘Configure’ link will show configuration inline:
You can turn off offline caching completely by de-selecting ‘Enable offline’, and can force all synchronization to happen manually by de-selecting ‘Synchronize automatically’. You can imagine that some offline web apps will turn both of these off by default, or perhaps force all syncing to be done manually, such as for a web based book reader like the OpenLibrary where you can download books for offline reading, but manually.
After these two options offline web applications can place their own, simple, specific configurations; for Gmail, we can select how many new emails to download during synchronization.
The “Empty” link will empty your cache, while “Storage Settings” will show storage configuration for your data. For Dojo Storage, if hidden Flash is being used to persist the users data, for example, then this will show the Flash Storage configuration dialog.
Blogger
An offline-enabled version of blogger has basicly the same widget as GMail’s Offline Info, but on the right-hand side of the page in the Blogger Dashboard:
On page load, Blogger would automatically download the last 10 blog posts, including images used in those posts. Individual blog posts indicate next to them if they have been locally modified (“modified offline”). If a blog post is not downloaded offline yet, the link “Download offline” appears, which when clicked causes the Offline Info widget to show download progress information, just like a sync. If you have local items that have not been synced, and have elected to sync manually, a small ‘(recommended)’ label will appear next to the Synchronize button.
The rest of the offline UI is pretty much the same, including the DOT download page, the syncing UI, and the configuration UI. The configuration UI might possibly have a small number of application-specific configuration, such as “[10] Newest Posts Synced,” where [] is a text field that can take a number.
Corporate Portal (Adenine)
Adenine is a web-based corporate portal; it was selected since it is indicative of how many corporate portals are used and structured. In Adenine, you have a standard portal dashboard; in this case, we have added the Dojo Offline widget:
Like the offline-enabled Gmail and Blogger, this application would auto-download whatever material is appropriate on page load, such as your recent calendar data, recent workflow tasks, and any custom databases you usually work with.
If you are working with a custom database, such as a Recruiting database that allows you to have a list of potential recruits, you can manually download a database or an item within that database; notice the ‘Download offline’ link in the right-hand corner of the web-spreadsheet.
Once clicked, ‘Download offline’ would cause the application to select an appropriate subset of this database to download locally, with a small DHTML popup window:
Here’s whats great about this UI: the ‘Download offline’ link and popup window are a generic way to indicate that you want to manually download an ‘item’ for offline access. The DHTML popup is a generic way to show syncing progress when the Offline Info widget is not visible for some screen. This could be applied to calendar events, specific blogs in an RSS reader, specific contacts in a sales database, etc.
Conclusion
With these mockups, SitePen now has a relatively generic set of templates for offline access for web applications. The Offline Info widget is a generic ‘overview’ area for offline operations; the ‘Download offline’ link is useful for manually bringing specific items offline; and the Sync DHTML popup is useful for providing offline download status when the Offline Info widget is not visible.
The next step is to use these UIs to help create an appropriate API for the Dojo Offline Toolkit, which is what SitePen will be doing the rest of this week. Tune in next week to see our proposed offline API.
Special Thanks
Special thanks to the following applications and libraries that were studied for this work:
- SalesForce.com
- Scrybe
- Zimbra
- ClickTime
- Intel’s Occasionally Connected (OCC) Framework
- Palm Pilot Desktop and Conduits
Thanks to Julien Couvreur and Dustin Machi for feedback on the UI mockups.
You have illustrated some excellent example uses for this powerful web extension. I can’t wait to see some of the creative new projects in which this undoubtedly becomes a crucial component.
[...] Brad Neuberg created some examples of offline projects in order to find a common UI or offline web applications for the Dojo Offline Toolkit. [...]
This is amazing… I can’t wait to see it action… brilliant vision… now the Iphone… this could change how GMAIL users are able to sync their emails with Thunderbird or Mail software…on a PDA… Keep kickin ass…
Well done Brad. Even a novice like me can understand what’s involved. Now how to solve it all is your work.
Love ya;
D@D
[...] sitepen has an interesting post on how offline web-applications implemented with the Dojo Offline Toolkit could look & work like. including detailed mock-ups of a – fictional – offline Gmail-prototype. [...]
Hi Brad,
How could we help you in this great work ?
JB
Hi JB! Thanks for the nice comment; there are several ways to help:
* Are you a developer? Think about some of the things you would want from an offline API
* Are you a C programmer? Are you available to hack on the offline proxy for Linux, Windows, or Mac OS X?
* Are you a Ajax web programmer? Can you hack together some demo apps as we develop Dojo Offline to validate it’s UI, API, and architecture?
* Are you a communicator? Can you write tutorials, articles, blog posts?
Do any of these fit within your interests and skill sets?
Best,
Brad
sorry but just impossible to find the download (your links point to your blog…)
Ouriel: What download? There’s no download yet — the toolkit is in development :) These are just mockups to figure out the API and UI requirements at this point; the toolkit will be developed over the next 3 months, and will be available at the end of that time (though betas and early iterations will be available sooner than 3 months). It will all be open source.
Best,
Brad
Really good stuff Brad, cant wait to use it in my projects.
Keep rocking!
Thanks Diego!
Best,
Brad
[...] The silver lining for those of us in the Ajax space is that we will have a captive audience of users hungry for innovative 3rd party software, and there’s no other way to deliver it than through Safari. We have the opportunity to create powerful new browsing experiences for mobile users, potentially using techniques such as Brad Neuberg’s incredibly cool work on the Dojo Offline Toolkit to provide seamless operation when a connection is unavailable. [...]
OK Brad,
What I could do :
- some tricks in dojo.storage to create new storage drivers (as I did for FF2)
- blog posts and documentation for french community
- give my opinion about the API
It would be great if you published a white paper (a wiki, a google doc ?) about the API so that we could give you some feedback about it.
Hi dude! What’s up?
It will be a excelent tool, but do u have any idea when u will finish this tool?
I’m looking forward to starting to use this powerful tool.
Cya.
Brad, I think that in offline mode you will have limited functionality on this websites.
This sounds more than promising..!
I would be interesting in helping, in any way i can, but up to late February i don’t have spare time at all :(
Contact me for more details if you want
[...] <p>The big task last week was finishing mockups of what potential offline-enabled web applications might look like. The goals here are to make sure we are building the right technology to support offline-needs and to find a common UI that can be turned into a template usable by developers. We created three offline mockups: Gmail, Blogger, and a corporate portal named Adenine. You can see a full blog post on the SitePen blog with these mockups here: <a href=”http://www.sitepen.com/blog/2007/01/09/28/”>http://www.sitepen.com/blog/2007/01/09/28/</a></p> [...]
Ideally, wouldn’t synchronization happen automagically? Let’s not make users have to know ahead of time when they’re going to be offline, as often that’s an unpredictable event (wireless router dies, etc.).
[...] The dojo folks are working on a toolkit to allow offline use of web apps. It’s starting, interestingly, with a set of mockups. Worth looking at. [...]
JB, thanks so much for the offer of help! The biggest way you could help would be to hit the most critical bug that exists for Dojo Storage right now, since I want to get this solved for Dojo Offline:
http://trac.dojotoolkit.org/ticket/2232
I have trouble reproducing the bug, but I have definently encountered it. Can you work to find a repeatable way to reproduce this bug, since I’m concerned about it? Once we know how to repeat it, we can find a solution.
Following this bug, there are a number of bugs, tasks, and enchancements to Dojo Storage that are important; here are all my open tasks/bugs for Dojo Storage:
http://trac.dojotoolkit.org/report/7
I’d say getting Dojo Storage smaller is the most important one after the bug above ( http://trac.dojotoolkit.org/ticket/2228 ), as well as doing some more benchmarking on Dojo Storage to know how many individual records it can handle in terms of performance ( http://trac.dojotoolkit.org/ticket/2284 )
If you can work on this stuff it would _hugely_ help me hit my 3 month target.
Best,
Brad
[...] Last week we reported on addressing usability for offline access, with offline mockups of popular web apps. Usability is just as important for programmers as it is for end-users, it just takes a different form: the API, or Application Programmer Interface. Getting the API right is just as important as the UI; programmers need the love too. [...]
[...] We have posted up the default user-interface widget for Dojo Offline; this is all coded up now in JavaScript and is complete. This means all the default user-interface code, which lives in dojo.dot.ui, is done. The UI is ‘driven’ by the rest of the Dojo Offline framework, which includes dojo.dot and dojo.sync. Note that there is no real code inside of dojo.dot and dojo.sync; these are just stubs for now and return suitable ‘fake’ data to drive the UI. The next step is to actually code the inside of dojo.dot and dojo.sync, which actually persist data and do syncing. [...]
Hi,
I want to help (at least testing) you with offline Gmail. How to do that?
Hi, can i get the link of the Corporate Portal Adenine.
It looks good, so i’m interested to see an online demo!
Thanks.
keyhell: There is no offline Gmail. Those are just mockups created to help us figure out generic offline UIs :)
Nabil: Note that like the offline Gmail, the offline mockups of Adenine are just mockups — the real web based Adenine does not support offline access. These were created to help us understand the UI and requirements of offline web apps better to focus the Dojo Offline Toolkit on real world concerns. I unfortunately can’t find my original link for Adenine; I just searched on the web for representative corporate internal interanets to base mockups on. There are lots out there.
[...] Dojo Offline Toolkit [...]
What work was done with salesforce.com ?
Did you have a look at their (PC-only) Offline Edition ?
Does your stuff work on Mac OS X ?
Rup
[...] Last week the focus was on syncing and local data storage. Just as in previous big tasks we grounded things by focusing on real world applications, we began work on the JavaScript API for syncing and local data storage with a real application: Moxie. [...]
[...] This leads from the announcement of the Dojo Offline Toolkit, which sitepen is going to use to provide offline Gmail and Blogger functionality. [...]
[...] Dojo, un Framework JavaScript puissant a fondé dans le support d’applications en mode hors connexion avec la fonction de stockage Dojo. Trois solutions sont jusqu’aujourd’hui supportés Flash, WHATWG (qui sera disponible bientôt dans la version 0.4.2) utilisant les capacités de stockages offertes par Firefox 2, et finalement un stockage utilisant le système de fichier natif. Les solutions jusque là semble bien être disponible “logicielles” et même indépendamment des navigateurs. Sitepen est l’un des excellents exemples de voir Dojo Offline Toolkit en action avec deux exemple pratique de GMail et Blogger. [...]
[...] add to del.icio.us · Slashdot it!· Stumble it!· Reddit it!· Netscape it!· Digg it!· Adobe Apollo officially launchedNo YouTube for IranThe future of operating systemsTechnoratire-launches WTFWeb 2.0 applications vs. desktop applications [...]
[...] Zimbra – web mail client with new offline features, when will this happen with Gmail, offline is in vogue. [via TC] [...]
[...] Migrating more and more of my daily use to web applications such as Gmail I have also experienced the difficulties going through for exampel email when offline. Today I thought about it again and did a quick search at Google, and the result represents what I love with Internet and the open-source community. Guess what, of course there are other people thinking about it and trying to solve it. Firstly, there exists solutions to use Google Desktop to at least search for emails offline since it cache them locally on your computer. But there was also a really exciting project going on. Something that is necessary and absolutely the next step, the ability to run web applications offline in your browser. Gmail, Google calendar, collaboration portals and other web applications offer great advantages compared to have them installed on your desktop or laptop, but there is always a need of Internet connection. At Sitepen.com they have since january been working on a project called Dojo Offline Toolkit. It is a toolkit for enabling web application to run offline and sync when online. They have started by focusing on three popular and useful web applications: Gmail, Blogger, and a corporate portal named Adenine. Their blog offers a great insight into the development of an open-source project. Eweek, among many, mention this project as well. [...]
Hi,
I was googling for a solution for using GMail offline and I just got here. I was very excited and downloaded Dojo Offline, installed it, ran GMail and… nothing happened! I mean, the link posted here for accessing GMail offline leads to sitepen blog. What’s happening?
@Lucas: Dojo Offline is a toolkit for developers to integrate into their web applications — a developer must go the extra mile of actually coding it into their web site for it to work. We used a mockup of GMail to help create the software and figure out its direction, but we don’t integrate into GMail.
[...] Trillion astra, NewsGator online, and Attensa Online (right-click functionality), Box, Zoho (desktopize), ThinkFree, Gmail and Blogger hack(Dojo), ShareMethods, Calgoo (Google Calendar), Tubes, FireFox 3.0, Apollo, Silverlight, Wallnote, Sharpcast, Zimbra, Scrybe, Qube, and egnyte. [...]
It would be interesting to alter the UI to reflect the online/offline state.
For example, “online” gmail looks normal, but in “offline” mode the blue toolbar becomes gray and the black text become something less black (like #666 or something). Maybe even a page watermark.
The idea is that you would use the whole UI to remind the user that the app is offline, vs. relying on a single area or widget.
I often use entourage in offline mode quite often and I can’t count the times that I have forget that I wasn’t connected.
[...] Eu já havia lido sobre isso em outros locais, que o Google iria lançar uma versão offline para o GMail, inclusive que outras empresas de software já haviam desenvolvido toolkits que se utilizam do Google Gears para efetuar tal coisa, mas ainda não havia procurado um toolkit que o fizesse. Hoje resolvi procurar e encontei um que me pareceu interessante. Primeiro encontrei isso no SitePen Blog depois de ler atentamente resolvi seguir a indicação do post e acessar a página do Dojo Offline Toolkit. O Dojo Offline é um toolkit open source que facilita sua vida possibilitando a criação de aplicações web offline. Para utilizá-lo e poder criar versões offline para aplicações como GMail, Blogger, etc, basta baixar sua última versão em http://dojotoolkit.org/offline e no mesmo site procurar mais informações. [...]
[...] Isto mesmo, é a solução apresentada por uma aplicação feita em DOJO. O Dojo off-line Toolkit, habilita uma opção para editar mensagens do Gmail mesmo quanto voce esta offline, isto é um ganho enorme para quem utiliza notebook, e sempre precisa responder alguns e-mails mas não tem uma acesso a internet disponÃvel no momento, com o Dojo off-line Toolkit voce simplesmente, edita a mensagens e depois que tiver o acesso a internet sincroniza com o servidor do gmail. [...]
[...] The big task last week was finishing mockups of what potential offline-enabled web applications might look like. The goals here are to make sure we are building the right technology to support offline-needs and to find a common UI that can be turned into a template usable by developers. We created three offline mockups: Gmail, Blogger, and a corporate portal named Adenine. You can see a full blog post on the SitePen blog with these mockups here: http://www.sitepen.com/blog/2007/01/09/28/ [...]
[...] week the focus was on syncing and local data storage. Just as in previous big tasks we grounded things by focusing on real world applications, we began work on the JavaScript API for syncing and local data storage with a real application: [...]