Evolving Schemas with Persevere

Traditional relational databases have strict data schemas; all records in a table must be of the same type. With new non-relational database technologies, some databases feature schema-free tables, each record is free to have any structure desired. This fundamentally different approach to the structure of persisted data is one of the core differences between relational and non-relational databases. However, is strict-schema or schema-free always the best approach for all the components in an application, throughout the entire life cycle of the application? Persevere provides a more flexible approach, allowing tables to start schema-free, and type constraints can be gradually added as desired or needed, with the freedom to partially constrain, defining types for some properties and not for others.

The strict schemas of traditional databases can be beneficial for maintaining data integrity. All entities in a table are guaranteed to have the same structure with each column having a defined type. On the other hand, this structure lacks flexibility. Adding new properties to entities requires a schema change for the whole table, and it is simply impossible to dynamically allow additional properties on objects to be saved in a database row.

Schema-free databases have the opposite pros and cons. The entities in the tables have unlimited flexibility for storing data with any desired properties, but it is difficult to create consistent data integrity guarantees. Furthermore, the flexibility and reduced upfront overhead with schema-free databases may be perfect for the prototype stage of application development, but for production applications, the data integrity guarantees may be more important. classschematable.png

Persevere bridges this gap by allowing tables to have schemas with any level of constraint desired. New tables are created with empty schemas, effectively the equivalent of a schema-free table where entities can have any properties, with minimal upfront overhead to persisting data. The quick table setup is well-suited for the rapid prototyping phase of application development. Schemas can then be augmented with constraints on data using JSON Schema as the schema structure. Individual properties can be given different type information, including primitive type, length, default value, enumerations, and more.

Schemas are completely flexible in what they define, entities can have some constrained properties, and other properties can be allowed to have any value, or other properties can still be dynamically added and removed per entity. This approach complements the object model for a comprehensive integrated persistence and class system in Persevere.

Getting Started

In Persevere, every table has a corresponding class and schema. The class for a table is synonymous with the schema, the class follows the structure of JSON Schema for defining the structure of its instances, and the class can be used for object instantiation and instanceof checking in the JavaScript environment. To create a new table/schema, we open up the object browser/database explorer and click the new button, enter a class name (we will be using “Product” in this article if you want to follow along), and choose save. This can also be done with the HTTP/REST interface (you can use RestTest or Poster to create requests):

POST /Class/

We can now immediately begin using our new table/class. To create new objects we can use the browser/explorer or we can also begin creating new instances using the standard HTTP/REST interface:

POST /Product/
  "name": "MacBook Pro",
  "price": 1999

These new objects can have any properties added to them, and are then saved in the Persevere database. You can also create new instances from the Persevere JavaScript environment. It is easy to try this from the Persevere console:

// properties can be defined on the constructor/class
var obj = new Product({name:"Snow Shovel"});
// or by normal property setting
obj.price = 12.99;

So far we have not made any schema definitions for the Product class, and therefore every instance can have any properties with any values.

Let’s define constraints on the objects of our new class now. Following the structure of JSON Schema, we define types of properties in a properties object. The properties object holds property definitions that specify the type and other constraints on the property values for instances. For example, we will define that the name property should be a string, and that the price property should be a number. This could be done from the Persevere console (you may want to switch into multiline mode):

// properties can be defined on construction
Product.properties = {
     name: {type: "string"},
     price: {type: "number"}

Note that for continuing development of your class, it may be easier to create a class definition in .js file in the WEB-INF/jslib directory. We could define our class there:

  properties: {
     name: {type: "string"},
     price: {type: "number"}

(As changes are made to this file, the class will be automatically updated.)

Now that we have a constraint on Product, if we attempt to create a new object that violates the constraint it will be rejected:

POST /Product/
  "name": "Chevrolet Corvette",
  "price": "high"

This request will be denied because the price value is not a valid number. The same constraints will be applied if we attempted to modify an existing object with a PUT request, or programmatically:

var aProduct = load("Product/1");
// this will throw a TypeError
aProduct.price = "going up";

With its object persistence integration with schemas, Persevere effectively adds a form of typing to JavaScript. Persevere’s data schema is analogous to the gradual typing concept in language typing. The records in the Persevere database exist as JavaScript objects in the JavaScript runtime and therefore any property change that has schema constraints can be simultaneously viewed as a type constraint from the language perspective and as a column constraint from the database perspective.

We can continue to progressively modify the schema for the Product class. We can do this from the REST interface as well. Here we will add a length constraint on the product name, so that it cannot be longer than 40 characters, and we will set a maximum price of one million:

PUT /Class/Product.properties
   name: {type: "string", maxLength: 40},
   price: {type: "number", maximum: 1000000}

This operation will not only add additional constraints on future data, but it will verify and modify any of the existing Product instances. If they have a name greater than 40 characters it will be truncated and if it has price greater than a million it will be set to a million. If you change the type of property Persevere will perform coercion to ensure that all the legacy objects are modified and their properties are coerced to the correct type. Generally, the coercion follows the JavaScript rules for primitive coercion (parsing for numbers, toString() for strings), and if a type become an object, array, or another class in the system, new objects are created to fill the properties. After any schema change, Persevere should update all existing instances such that all these object instances conform.


Persevere’s object model supports inheritance as well. This can be used for inheriting methods as well as inheriting property definitions from super classes. When a class (subclass) inherits from another class (super class), all instances of the subclass are also instances of the super class, and therefore all instances of the subclass must be valid by the schema for the super class as well as its own schema. This means that the subclass is inheriting the constraints of the super class. For example, we will create a subclass of Product called Toy. A Toy will still require name and price properties, but we will require minAge and maxAge:


POST /Class/
   "id": "Toy",
   "extends": {"$ref": "Product"},
   "properties": {
       "minAge": {"type": "number"},
       "maxAge": {"type": "number"}

Note, that this time we created the properties object directly in the creation of the class. The properties object can still be updated later for this object as well.

The Toy class inherits from the Product class, therefore any Toy object will require a minAge, maxAge, name, and price. The Toy table will consist of all instances of Toy, all conforming to the Toy schema, and the Product table will consist of all instances of Product, which includes the Toy instances. We can get a list of all the Product objects (including Toys) using the HTTP/REST interface:

GET /Product/

The flexible schema approach in Persevere gives developers the benefits of schema-driven same-type records and schema-free object databases. Developers can quickly get started, rapidly developing and prototyping applications without needing to define hard schema constraints, and having the flexibility of adding additional information and properties to objects at will using the properties pattern. Further along in the project life cycle, developers still have the power of defining schemas to enforce data constraints, providing data integrity as needed.

Learn more about how SitePen can be your partner

SitePen is a strategic consultancy committed to achieving technical objectives, solving critical business problems and helping our customers build web applications the right way, the first time.