Introducing dstore

By on November 17, 2014 1:24 pm

dstore 1.0

Dojo has long distinguished itself with a robust and complete application architecture. And the foundation of this architecture has been the store interface, providing clean separation and consistent interface between presentation and data sources. We are committed to continuing to improve this architecture, and in this pursuit, we now releasing dstore, a next generation object store interface and set of data modeling components.

dstore is an exciting new package that provides a number of significant improvements over the prior data store framework, including a more fluent querying API, improved event notification, advanced cross-store filtering, and a more modular mixin system that allows us to easily combine components with alternate format support, advanced querying support, and more. In this post, we want take a brief look at these new features and improvements. From there, you can explore the tutorials and documentation that we have written for this new package.

Mixin-Based Composition

mixinThe dstore package contains several store components with supplementary functionality that can be added to stores. This supplementary functionality is now mostly implemented as mixins (rather than store wrappers), making it simple to compose stores with different combinations of functionality. Some of the store mixins that are available in dstore include:

  • Trackable – This mixin adds support for tracking data changes.
  • Cache – This adds caching functionality, with support for caching both objects and query results/collections.
  • SimpleQuery – This provides client side querying (filtering and sorting) functionality. This is used by the Memory store, but can also be used to add client side querying functionality to other stores.
  • Csv – This enables parsing and serializing CSV data, and demonstrates alternate format support.

Stores can be configured with different combinations of functionality by using Dojo’s declare to compose stores from base stores with mixins added. For example, if we wanted a Rest store that will parse incoming data as CSV, and handle data notifications, we could write:

	define(['dstore/Rest', 'dstore/Csv', 'dstore/Trackable'],
			function (Rest, Csv, Trackable) {
		// compose the new store from the Rest store and mixins
		var CsvRestStore = declare([Rest, Csv, Trackable]);
		// create an instance of it
		var myStore = new CsvRestStore({
			target: '/Path/'
		});

Chained Querying

The dstore stores now feature the ability to incrementally query data. Querying typically involves a few keys parts. We can filter a collection of objects down to a desired subset. We can sort (and resort) those objects to achieve the proper order. With dstore, this type of query can be described with distinct method calls, rather than a single query() method call. This has several advantages. First, this facilitates a cleaner, more readable API. Each aspect of a query is clearly denoted in a query sequence. For example, a query might look like:

resultingCollection = store
	.filter({color: 'red'})
	.sort('price');

Here we are clearly specifying how the objects are filtered and what to sort on. Intermediate collection objects can be reused without requiring the whole query to be re-specified.

Chained querying also allows us to better encapsulate query results. Often we want to filter a store for certain objects, and then delegate sorting and paging responsibility to a grid or other list component. In the past, a grid would need to know the query and the store, so it could query the store itself. But with dstore, this query can be encapsulated into a single collection entity that can be passed to the grid, which can then be sorted and paged without needing any extra parameters.

Finally, incremental querying helps to have a better model for observing changes in collections. In the past, we have setup notifications around individual pages, but this can lead to some complicated issues at page boundaries. By being able to listen for notifications on the entire collection instead of individual pages, we can greatly simplify real-time responsive widgets.

Data Notifications

The notification system in dstore has been improved to make monitoring data changes easier. First, data notifications are now available, by default in all stores (or any store that properly extends the base class, dstore/Store). Notifications also follow a more standard event interface, listeners can be registered through the on() method, with different forms of notifications designed by distinct event types: add, update, and delete.

The dstore/Trackable mixin (mentioned earlier), now is used to track the index position of objects, and add this to events. If you don’t need index position to be tracked, than the Trackable mixin doesn’t need to included or loaded (but you can still monitor for events). And, because page requests can be retrieved from Trackable tracked collections, we can monitor paged data, without having separate notification and index tracking for each individual page.

Advanced Filtering

dstore provides the ability to build sophisticated filters, for more comprehensive standardized querying across different stores. Previously in Dojo object stores, only simple objects with name-value pairs could be used across all stores (custom functions could be used, but only with certain stores). dstore includes a Filter constructing that can define various conditions like equality, greater than, less than, non-equality, contains. These conditions than combined by using and and or conjunctions, for a large range of possible filtering permutations. For example, to create a query that finds all products with a price between $10 and $20:

var tenToTwenty = new store.Filter().gt('price', 10).lt('price', 20);
var filteredProducts = store.filter(tenToTwenty);

These filters can then be applied in client-side/memory stores, and can also be serialized to standard URL query parameters for simple queries or RQL queries (a superset of URL query parameters) for more advanced filters.

Working with dgrid

Working with dgrid The dstore release has been coordinated with the release of a new version of dgrid (0.4) that is dstore-driven. Having dgrid use the new dstore interface, simplifies the dgrid interface, no longer requiring queries to go through dgrid. The newest version of dgrid with dstore also improves notification handling in the paged queries that are used by dgrid/OnDemandGrid.

Transitioning

With the new store infrastructure, interaction with existing components and modules is very important. The dstore package includes an adapter for working with Dojo Charts (dstore/charting/StoreSeries), as well as legacy adapters for Dojo object stores. This DstoreAdapter makes it possible to use a dstore with existing components that expect object stores, and the StoreAdapter allows existing object stores to work with new components expecting a dstore.

We have a number of other exciting additions coming up in future dstore releases that we are working on as well. dstore represents the active and thriving progress of the Dojo ecosystem, as we move forward with building and evolving the most advanced web development infrastructure available.

Comments

  • Pingback: dgrid 0.4 released! | Blog | SitePen()

  • Ajaxiome

    Is there a way to have a concrete Api Documentation ?
    What are the methods available on this dstore ?
    I search but unable to find

  • @patrickdorio:disqus I’ve opened an issue at https://github.com/SitePen/dstore/issues/62

  • Ajaxiome

    Thanks

  • Alan

    I’m having trouble figuring out what the dgrid/Rest store expects from the web server. Googling for “dstore rest service” is not panning out for me.

    I want to transition from dojo/store/JsonRest, and I’d like to figure out how I need to modify my service to work with dgrid/Rest.

    From http://dgrid.io/js/dgrid/test/Rest.html, I can tell that there is at least one new header (“X-Range” in addition to “Range”) and a new URL parameter (“limit”).

    Is it as easy as just continuing to use Range, ignoring X-Range, and letting “limit” override the calculated limit? Do I prefer X-Range over Range? Do they interact?

  • Chris Foradas

    My JSON store (django rest framework) returns keys for “count”, “next”, “previous”, and “results”.

    “count” is the number of rows available.

    “next” is the url for the next page of results (e.g. ids 26-50).

    “previous” is the url for the previous page of results (null in this case since this is the first page of results).

    The “results” key contains the actual data objects I’d like to display in the OnDemandGrid.

    How do I connect the “results” to the grid?

    The returned JSON ( collection: new Rest({target: ‘/api/events’?format=json’),}) ) looks like this:

    {“count”:1411,”next”:”http://localhost/api/events/?format=json&page=2″,”previous”:null,”results”:[{“id”:1,”event_type”:”02″,”event_at”:”2015-03-31T12:53:41Z”,”machine_id”:1,”revs”:4342,”color”:5,”heads_info”:”using http”,”tag”:1,”hidden”:false},{“id”:2,”event_type”:”02″,”event_at”:”2015-03-31T12:53:41Z”,”machine_id”:1,”revs”:4342,”color”:5,”heads_info”:”using http”,”tag”:1,”hidden”:false},

    {“id”:25,”event_type”:”02″,”event_at”:”2015-03-31T12:54:01Z”,”machine_id”:1,”revs”:4342,”color”:5,”heads_info”:”using
    http”,”tag”:1,”hidden”:false},

    ]}

  • Ken Franqueiro

    It looks like you also posted this on SO (which is a much better format for this question) so I gave a response there: http://stackoverflow.com/a/30606317/237950

  • Pingback: Quick tip: dstore with ArcGIS API for JavaScript - odoenet()