More Query and Caching Power for Data Stores: ClientFilter, JsonQuery, and JsonQueryRestStore

By on December 18, 2008 1:02 am

Dojo includes several new modules which open up new querying and caching capabilities for Dojo data stores. is available in Dojo 1.2, and provides the ability to cache queries, update the query results and cached queries based on local modifications, and performs sub-queries and sorting on the client-side. The JsonQuery is a new mixin class for Dojo 1.3 that converts Dojo query objects and sort objects to JSONQuery expressions; JSONQuery can be used for delivering complex queries to the server. JsonQueryRestStore is a new module (for Dojo 1.3) that extends the JsonRestStore and combines the ClientFilter and JSONQuery such that any query can be executed on the client or the server automatically based on what data has been locally cached on the client-side, utilizing dojox.json.query to execute queries on the client-side when appropriate.


ClientFilter is a base class that can be extended by other stores, to add client-side querying and sorting functionality. ClientFilter uses these capabilities for two ends. First, queries can be cached and the cached queries can later be used for other queries and sorting operations as long as the query contains the same or superset of the data required by a subsequent query. In order to use the query cache, a ClientFilter subclass should call ClientFilter’s cachingFetch method from its fetch method. Also, to cause queries to be cached, the argument passed to the fetch method should have a queryOptions property set to {cache: true}. For example,

   fetch: function(args){
      var deferred = this.cachingFetch(args);
myStore = new MyStore();

The ServiceStore and its subclasses (which includes JsonRestStore and it’s subclasses) will automatically use ClientFilter (and cachingFetch) for you if it has been loaded.

myStore = new{target:"/data"});

Widgets, like the grid, can take queryOptions for passing to the store:

table dojoType="dojox.grid.DataGrid" queryOptions="{cache:true}" ...

Alternately, you can set the cacheByDefault to true on the store object, and all queries will be cached:

myStore.cacheByDefault = true;

When the fetch is executed the results will be placed in the query cache. If the request is repeated the cached version can be retrieved rather than requiring another request to the server. Also, if the the same request is made with different sort parameters (the result of the clicking on a column in the grid), the cached result set can be used and sorted on the client-side to satisfy the request.

The second goal of the ClientFilter is to update result sets after local modifications have taken place. If a new item was added or an item was updated, the display of the data should usually be updated, but a store user, like a widget, may not know how to update the collection that is being displayed. Often widgets simply add new items to the end of a list and if an item is modified, no re-sorting is done. However, the ClientFilter can actually update previous query results based on object modifications; new items and modified items being added, moved, or removed to the correct place in a result set based on the query and sorting parameters. For example, suppose we performed a query:

var requestArgs = {sort:[{attribute:"firstName"}], onComplete:function(results){
   fetchResults = results;

And then we modified one of the items:

myStore.setValue(fetchResults[1], "firstName":"John");

The fetchResults was originally sorted by firstName, but after this modification, the sorting may now be incorrect. With a ClientFilter powered store, we can update this result set:

myStore.updateResultSet(fetchResults, requestArgs);

This will iterate through all the recent modifications and update the fetchResults array, moving, adding, and removing items as necessary to ensure the result set is correct. Furthermore, this can be very valuable for new queries while items are dirty. With the item modification we made above, if we haven’t yet called save(), the server will not be aware of this change. If we did another fetch that actually made a server request, the server may return the wrong results based on the sorting or query parameters due to its lack of knowledge of the current state of client-side data. ClientFilter can solve this problem as well: it will automatically apply the updateResultSet method to all queries, updating the results based on any local modifications (dirty items) that haven’t been sent to the server.

The ClientFilter can perform client-side queries based on name-value pair query objects, and sort parameters. However, out of the box, the ClientFilter does not understand string-based queries; the Dojo Data API specifies that string queries are store specific so it is impossible for ClientFilter to know how to interpret all string-based queries. However, you can implement two simple methods to enable caching and client-side querying of a string-based queries. clientfilter.png isUpdateable(request) should be implemented, taking the same argument as the fetch method and returning a boolean indicating whether or not the query can be updated (handled on the client-side). matchesQuery(item, request) should also be implemented, taking an item and returning a boolean indicating whether or not the item should be included in the query providing by the second argument. By implementing these functions, ClientFilter can be made to work with string-based queries. Or alternately, you can use the JsonQuery module to implement these functions for JSONQuery based queries.

JsonQuery is a module that can convert Dojo Data query objects and sort attributes to JSONQuery expressions for the server and also implements isUpdateable(request) and matchesQuery(item, request) for JSONQuery. JSONQuery can be used to describe substantially more complex queries than simple URL query parameters. JSONQuery supports a full range of operators (<, >, !, &, |, =, etc.), sorting, recursive search, and more. The JsonQuery module is an abstract mixin class, so you can add the JSONQuery capabilities to a store:

dojo.declare("JsonQueryEnabledSomeStore", [SomeStore,]);

JsonQueryRestStore extends JsonRestStore and combines the power of ClientFilter and JsonQuery. With JsonQueryRestStore, JSONQuery can be used to formulate queries that can be executed either on the client or the server. JsonQueryRestStore utilizes the JsonQuery module to transform name-value object queries to JSONQuery format and integrates with the ClientFilter module such that queries and sorting are executed on the client-side when cached results are available, and sent to the server as needed. Note that the queryOptions.cache must still be set to true to cache queries with the JsonQueryRestStore.


ClientFilter provides a foundation for optimizing data stores, using cached data as appropriate to avoid server-side requests for lower latency updates. JSONQuery is a powerful query language which can easily be used via the new Dojo Data JsonQuery module (in Dojo 1.3). JsonQueryRestStore is a new JsonRestStore subclass which provides a complete solution for consistent client and server-side querying, automatically handling the use of query caches for optimum performance.