The NoSQL movement continues to gain momentum as developers continue to grow weary of traditional SQL based database management and look for advancements in storage technology. A recent article provided a great roundup of some of the great new technologies in this area, particularly focusing on the different approaches to replication and partitioning. There are excellent new technologies available, but using a NoSQL database is not just a straight substitute for a SQL server. NoSQL changes the rules in many ways, and using a NoSQL database is best accompanied by a corresponding change in application architecture.
The NoSQL database approach is characterized by a move away from the complexity of SQL based servers. The logic of validation, access control, mapping querieable indexed data, correlating related data, conflict resolution, maintaining integrity constraints, and triggered procedures is moved out of the database layer. This enables NoSQL database engines to focus on exceptional performance and scalability. Of course, these fundamental data concerns of an application don’t go away, but rather must move to a programmatic layer. One of the key advantages of the NoSQL-driven architecture is that this logic can now be codified in our own familiar, powerful, flexible turing-complete programming languages, rather than relying on the vast assortment of complex APIs and languages in a SQL server (data column, definitions, queries, stored procedures, etc).
The REST architecture has become increasingly recognized for its value in creating scalable, loosely coupled systems. REST is presented as a network interaction architectural style, not a programming methodology. However, the principles of REST can actually be very meaningfully and beneficially applied in the programming realm. We will look at how the resource oriented approach of REST can be applied as principles for programming language usage and design. The motivation for looking at REST should be clear. Little in history has been as ridiculously successful and scalable as the web, and REST is a retrospective look at the principles that were employed in designing the core technologies of the web, particularly HTTP. Applying such proven principles to our application design will certainly be beneficial.
Roy Fielding‘s REST architecture is broken down into seven constraints (and the four sub-constraints of the uniform interface). The individual concepts here are certainly not new, but collectively looking at these concepts as resource oriented programming may provide an interesting new perspective. I will also look at how these principles are exemplified in Persevere 2.0 in its object store framework, Perstore, and its web stack Pintura.
REST is a powerful architecture because of its loose coupling design that facilitates a high level of interoperability. However, even the dissertation that defines REST states that one of the trade-offs of REST is that it “degrades efficiency, since information is transferred in a standardized form rather than one which is specific to an application’s needs”. That is, you might be able to achieve more efficient operation using a mechanism that goes beyond REST’s uniform interface, that is more specific to your application. This comes at the price of a loss of interoperability and should only be used if necessary, but with this in mind, let’s look at a good way to use RPCs that integrate with Dojo‘s REST module, the JsonRestStore. This approach builds on the REST architecture, while allowing exceptions as needed.
Dojo includes several new modules which open up new querying and caching capabilities for Dojo data stores. dojox.data.ClientFilter is available in Dojo 1.2, and provides the ability to cache queries, update the query results and cached queries based on local modifications, and performs sub-queries and sorting on the client-side. The JsonQuery is a new mixin class for Dojo 1.3 that converts Dojo query objects and sort objects to JSONQuery expressions; JSONQuery can be used for delivering complex queries to the server. JsonQueryRestStore is a new module (for Dojo 1.3) that extends the JsonRestStore and combines the ClientFilter and JSONQuery such that any query can be executed on the client or the server automatically based on what data has been locally cached on the client-side, utilizing dojox.json.query to execute queries on the client-side when appropriate.
There is growing support in browsers for offline capabilities with the HTML 5 specification for local storage, offline notifications, and offline application cache, but adapting an application to store changes locally and do synchronization when connectivity is restored remains a major challenge for developers. Dojo 1.2′s new dojox.rpc.OfflineRest module automates the local storage of data and synchronization by leveraging the Dojo Data and REST abstractions. The OfflineRest module augments the JsonRest service in Dojo such that requests are cached in local storage for offline access, and modification requests (put, post, and delete) modify the cache and are recorded for delivery to the server; immediately if online, otherwise when connectivity is restored. Furthermore, JsonRest is the core engine used by JsonRestStore. Consequently, you can simply use the standard Dojo Data API with the JsonRestStore and effortlessly add offline capability with no modifications to your data interaction code. The underlying Rest service automates the handling of caching, storing data locally, and syncing changes. In addition the new OfflineRest module has no dependency on plugins, but rather progressively utilizes offline features that are available, while still operating properly on legacy browsers without offline support.
Non-trivial data often has structures that cannot be well-defined with normal linear, acyclic data descriptions. Data that consists of cycles, many-to-one relationships, and non-local references often requires custom strategies for serialization and transfer of the data over JSON. Dojo 1.2 has added support for JSON referencing with the dojox.json.ref module. This module provides support for several forms of referencing including circular, multiple, inter-message, and lazy referencing using id, path, and combined referencing forms.
Complex database driven widgets can be utilized with minimal coding effort. RESTful JSON is an increasingly popular database interface, and in later posts we will look at how JsonRestStore can be used with Amazon S3, CouchDB, Persevere, and Ruby on Rails. The JsonRestStore fully implements the Dojo Data read, write, notification, and identity interfaces. Therefore it can be used with any widget that can utilize these data stores including widgets with editing and live auto-updating features.