This guide covers the basics of SproutCore’s model layer. By referring to this guide, you will be able to:
- Understand the anatomy of records.
- Define your application specific models and relations between them.
- Create and manage records.
- Understand how the store manages your data.
- Query the store for data.
endprologue.
In SproutCore the model layer is the lowest application layer and holds all your data as well as the business logic of your application. The controller layer calls into the model layer to modify data and retrieves data from the model layer. Generally, controllers use bindings to perform these functions.
The model layer is also responsible for talking to your server, fetching and committing data when necessary. The server communication aspect of the model layer is not covered in this guide, but in the guide about using data sources.
Models are a blueprint for your data, defining the data schema of your application. This data schema is generally similar to the data schema of your back-end application.
In SproutCore, models are defined by subclassing SC.Record. When you actually want to create a data record from one of your blueprints, you use SC.Store to create an instance of a SC.Record class. Your application’s store manages the lifecycle and the data of your records in a central place. When you retrieve or update a property from a record, the record actually uses the store to access the underlying data hash.
All the classes of SproutCore’s model layer are located in the datastore folder inside the main sproutcore folder. Have a look at the source code there if you want to have more in-depth information. The code has plenty of inline documentation and can be a valuable resource to gain deeper insights in how the store works.
A SproutCore record consists of four main components:
- Store key
- Id
- Status
- Data hash
Each record has a unique store key which is assigned when the record is created. The store key is a unique record identifier in the whole store and is mainly used internally to relate ids, statuses and data hashes to each other in an unambiguous way. The store key is the only one of the four components which is actually a property of SC.Record. The other three components are stored centrally in the store and mapped to the individual records using the store key.
All records of a certain type have a unique id as usual in relational database systems. In fact the ids of SproutCore records usually are the same as the primary keys of your data in the backend. Therefore, unlike the store key, the id is not automatically created but it is your responsibility to assign a unique id when creating or loading records.
The status of a record represents its current state with respect to the corresponding record on the server. The store uses the status property to determine which operations can be performed on the record, for instance, if a record can be edited safely and which records need to be commited back to the server.
Last but not least, the actual data of a record is stored in a plain JSON data hash. When you get or set a property on a record, the value of this property is read from or written to the data hash.
There are five primary record status codes in SproutCore:
- SC.Record.EMPTY
- SC.Record.READY
- SC.Record.BUSY
- SC.Record.DESTROYED
- SC.Record.ERROR
The names of these states are pretty self explanatory: EMPTY indicates a non existing record. READY indicates that the record can be safely edited. BUSY indicates that the record is currently locked for write operations, mostly because of an ongoing commmunication with the server. Finally DESTROYED is the state of destroyed records and ERROR indicates that something went wrong while processing this record.
The three main states READY, BUSY and DESTROYED have several substates. You will learn more about these substates below when you actually start working with records. You can also refer to the complete overview of record states in the last section of this guide.
Defining a model in SproutCore is as easy as subclassing SC.Record:
App.Contact = SC.Record.extend({
});
You just have created your custom App.Contact model class. However, this empty model is only of limited use, so let’s add some record attributes.
App.Contact = SC.Record.extend({
firstName: SC.Record.attr(String),
lastName: SC.Record.attr(String),
age: SC.Record.attr(Number)
});
WARNING: Property names defined on SC.Record itself are reserved names, meaning they cannot be used for custom record attribtutes. Please refer to the documentation of SC.Record for a list of all reserved names.
We have used the SC.Record.attr helper to add the firstName, lastName and age attributes with the type of each attribute as first argument. The optional second argument of SC.Record.attr is an option hash. E.g. we can add some default values to our attribtues:
App.Contact = SC.Record.extend({
firstName: SC.Record.attr(String, { defaultValue: ‘Unspecified’ }),
lastName: SC.Record.attr(String, { defaultValue: ‘Unspecified’ }),
age: SC.Record.attr(Number, { defaultValue: 0 })
});
Whenever you specify a defaultValue option on an attribute it will return this default value if it’s null or undefined.
NOTE: The defaultValue will not be written to the underlying data hash and therefore not committed back to the server.
If the name of the model’s attribute property differs from the name you want to use in the data hash, you can specify a custom key for each attribute which will be used to access the data hash:
App.Contact = SC.Record.extend({
firstName: SC.Record.attr(String, { key: ‘first_name’ }),
lastName: SC.Record.attr(String, { key: ‘last_name’ }),
age: SC.Record.attr(Number)
});
All basic JavaScript data types can be used as attribute types:
- String
- Number
- Boolean
- Array
- Object
Additionally SproutCore comes with a predefined attribute helper for date and time values.
App.Contact = SC.Record.extend({
dateOfBirth: SC.Record.attr(SC.DateTime, { format: ‘YY-mm-dd’ })
});
For a reference of how to specify your custom date format check the documentation of SC.DateTime#toFormatedString.
In SproutCore you don’t define the primary key property of your models explicitly like you defined your custom attributes above. The records’ primary keys are managed by the store, so every record inherently has an id property. However, you can specify the identifier of this id property. This is where it can become a bit confusing at first… but let’s clear it up step by step.
First of all, by default SproutCore uses the identifier “guid” for the primary key. You can change this identifier by defining a primaryKey property in your model:
App.Contact = SC.Record.extend({
primaryKey: ‘uid’
});
NOTE: If you want to use your custom id identifier in all your models, you can make your life a bit easier and your code more maintainable by defining a custom record base class, where you define the primaryKey property. Then you can subclass this custom base class to create your models.
However, this primary key identifier is only used to identify the id property in the underlying data hash, but not to get or set the id on a record. For example if you create a record and pass a hash with initial values, then SproutCore will now look for a property called “uid” in the hash when you don’t explicitly specify an id. If you want to get or set the id of a record though, you always use id independent of the +primaryKey+’s value:
myRecord.get(‘id’); // note: NOT ‘uid’
myRecord.set(‘id’, 1);
WARNING: You should never change the id of an existing record using set() like above unless you know what you are doing.
It is a best practice to never include the id in the data hash, because then you end up with two ids: the id property in the data hash and the id managed by the store. If you receive a JSON data hash from the server (where the id is necessarily included) then you should extract and delete the id from this hash before using it to load the record into the store.
Often models don’t exist completely independent of each other but are related to other models. For example one or more addresses could belong to the Contact model we created above. So let’s define the Address model first:
App.Address = SC.Record.extend({
street: SC.Record.attr(String),
number: SC.Record.attr(Number)
});
If we only need one address to be associated with each contact, then we can use the toOne relation helper:
App.Contact = SC.Record.extend({
address: SC.Record.toOne(
‘App.Address’,
{ isMaster: YES, inverse: ‘contact’ }
)
});
App.Address = SC.Record.extend({
contact: SC.Record.toOne(
‘App.Contact’,
{ isMaster: NO }
)
});
Notice the isMaster and inverse options used with the toOne helper. The isMaster: YES option on the address attribute makes sure, that the Contact record actually gets marked as changed when you assign a different Address record to it. You should always set the isMaster option to YES on one side of the relation and to NO on the other to control which record is committed back to the server when you alter the relation.
The inverse option specifies the property name of the inverse relation on the associated model and should be set on the side of the relation where isMaster is set to YES.
In the underlying data hash a toOne relation is simply represented as the id of the associated record.
NOTE: It is not mandatory to define both directions of the relation if you don’t need it.
If we want to associate multiple addresses with a certain contact, then we have to use the toMany relation helper:
App.Contact = SC.Record.extend({
address: SC.Record.toMany(
‘App.Address’,
{ isMaster: YES, inverse: ‘contact’ }
)
});
App.Address = SC.Record.extend({
contact: SC.Record.toOne(
‘App.Contact’,
{ isMaster: NO }
)
});
The only thing that changed compared to the one-to-one example above is the toMany keyword in the Contact model. The isMaster and inverse options apply to toMany relations in the same way as they do to toOne relations.
In the underlying data hash a toMany relation is represented as an array of ids of the the associated records.
NOTE: It is not mandatory to define both directions of the relation if you don’t need it.
If we not only want to relate multiple addresses to one contact, but also relate one address to multiple contacts, we have to use toMany on both sides of the relation. SproutCore’s toMany helper manages many-to-many relations without a join table, which you would use in a relational database:
App.Contact = SC.Record.extend({
address: SC.Record.toMany(
‘App.Address’,
{ isMaster: YES, inverse: ‘contact’ }
)
});
App.Address = SC.Record.extend({
contact: SC.Record.toMany(
‘App.Contact’,
{ isMaster: NO }
)
});
Again the only thing that changed compared to the one-to-many example from above is the use of the toMany helper in the Address model.
Since a many-to-many relation effectively is constructed by using toMany on both sides, it is represented in the underlying data hashes of both sides of the relation as an array of record ids.
NOTE: It is not mandatory to define both directions of the relation if you don’t need it.
Any property defined on a model class not using SC.Record.attr is a transient property. This means that its value is not passed through to the data hash of the record and therefore is neither committed back to the server nor loaded from incoming JSON data.
App.Contact = SC.Record.extend({
// transient property
isContact: YES,
});
NOTE: If you use the set method on an undefined property, SproutCore by default will pass the value through to the underlying data hash. You can turn this behavior off by setting ignoreUnknownProperties: YES in your model classes.
Now that we have defined our Contact and Address models it’s time to actually create some records. All records are managed by the store, so we have to make sure first that we have an instance of SC.Store available. Usually the store is instantiated somewhere in your application’s core.js file:
App = SC.Application.create({
store: SC.Store.create().from(SC.Record.fixtures())
});
The example above shows creating the store with fixtures as data source. You can read more about fixtures and using other data sources in the respective guides.
You can create records of a previously defined record type like this:
contact = App.store.createRecord(App.Contact, {});
The first argument of the store’s createRecord method is the record type. The second argument is a hash with optional initial values. Furthermore you can specify the record id as third argument:
contact = App.store.createRecord(
App.Contact,
{ firstName: ‘Florian’, lastName: ‘Kugler’ },
99
);
Usually you will not specify an id like this, because either you get the record id from the server, or you want to use some kind of temporary id on new records until they get comitted to the server, which then can return the id of the persisted record. So let’s use a temporary id:
contact = App.store.createRecord(
App.Contact,
{ firstName: ‘Florian’, lastName: ‘Kugler’ },
– Math.random(Math.floor(Math.random() * 99999999))
);
NOTE: Ids are not limited to numbers, but can also be strings.
When you create a record its status will be READY_NEW, indicating that the record is editable and does not exist on the server yet.
When creating associated records, you first have to create the records and afterwards establish the connection between the records.
contact = App.store.createRecord(App.Contact, {/…/}, 1);
address = App.store.createRecord(App.Address, {/…/}, 1);
// for a toOne relation
contact.set(‘address’, address);
// for a toMany relation
contact.get(‘address’).pushObject(address);
In this case we’re adding the Address record to the Contact record, because Contact is defined as master in this relation and has the inverse property set. It is important to add the non-master record to the master record in order to set up the connection between these records properly.
Updating record attributes is as easy as calling the set method:
contact.set(‘firstName’, ‘Jack’);
In order to be able to update record attributes the record has to be in a READY state. If you update an attribute of a newly created record, the status will still be READY_NEW. If you update an attribute of a record that was previously loaded from the server or committed to the server, then the status will transition from READY_CLEAN to READY_DIRTY.
NOTE: Dirty states always indicate that the record needs to be committed back to the server.
To delete a certain record, just call the destroy method on it:
contact.destroy();
Equally to updating a record, the record has to be in a READY state to be able to be destroyed. If you destroy a newly created record (which was not yet committed to the server) the status will transition from READY_NEW to DESTROYED_CLEAN, indicating that there is no need to tell the server about the destroy, since it never knew about this record in the first place. If you destroy a record loaded from the server, then the state will transition from READY_CLEAN (or READY_DIRTY if you changed it before) to DESTROYED_DIRTY, indicating that the server needs to be notified about this destroy action.
You can get the id, the store key and the status of a record by calling the get method on the respective properties:
id = contact.get(‘id’);
storeKey = contact.get(‘storeKey’);
status = contact.get(‘status’);
To test if the record is currently in a certain state, use JavaScript’s binary operators:
status = contact.get(‘status’);
// checks if the record is in any READY state
if (status & SC.Record.READY) {
}
// checks if the record is in the READY_NEW state
if (status === SC.Record.READY_NEW) {
}
NOTE: For a complete list of record state constants see the documentation of the SC.Record class.
Because the store manages all records in memory, you can query it for records of a certain type, records with a certain id or more complex search criteria.
If you know the type and the id of the record you want to retrieve, you can just hand these two parameters to the store’s find method:
contact = App.store.find(App.Contact, 1);
This statement returns the record of type App.Contact with the id 1. If the record does not exist, then the return value will be null.
WARNING: When find is called with a record type and an id as arguments, it only looks for records of exactly this type. It will not return records which type is a subclass of the specified record type.
To find all records of one record type, just pass it to the find method:
contacts = App.store.find(App.Contact);
If you want to find all records of several record types, pass an array of record types to the find method:
contactsAndAddresses = App.store.find(
[App.Contact, App.Address]
);
You can also find all records of a certain type and all its subclasses:
allRecords = App.store.find(SC.Record);
The above statement returns all records in your application, because we are asking for all records of type SC.Record, which is SproutCore’s base model class.
Internally find converts the specified record types to a query, it’s just a convenient method to save some characters of typing required to create the query yourself. Read on in the next section how to do this and to learn more about the return type of find.
SproutCore features a SQL-like query language to facilitate more complex queries to the store. However, let us first translate the find calls of the previous section to using queries, like find does internally. To build a query which looks for all records of a certain type, you just call SC.Query.local with this record type as argument and pass this query to find:
query = SC.Query.local(App.Contact);
contacts = App.store.find(query);
As you can see, the method from the previous section of directly passing the record type to the find method just saves you the call of SC.Query.local. Querying for multiple record types or all records follows the same pattern:
query = SC.Query.local([App.Contact, App.Address]);
contactsAndAddresses = App.store.find(query);
query = SC.Query.local(SC.Record);
allRecords = App.store.find(query);
Whenever you call SC.Store’s find method with a query (or using one of the convenient ways from the previous section) it returns a SC.RecordArray. As the name already indicates SC.RecordArray implements SC.Array and therefore you can use it like a normal read-only array. For example:
contacts.firstObject(); // returns first result
contacts.objectAt(3); // returns fourth result
contacts.lastObject(); // returns last result
Please refer to the documentation of SC.Array to learn more about the array access methods.
NOTE: If the query was not yet fetched from the server, the store automatically forwards it to the data source to load the data from the server.
NOTE: SC.RecordArray objects automatically get updated by the store when you add or remove records to or from the store which match the corresponding query.
You can limit the results of a query to match certain conditions:
query = SC.Query.local(App.Contacts, {
conditions: ‘firstName = “Florian”’
});
results = App.store.find(query);
The above query returns all records of type App.Contacts and subclasses of this type where the firstName attribute matches the value “Florian”. You can combine several conditions using the logical operators AND, OR and NOT as well as parentheses ( and ) for grouping:
query = SC.Query.local(App.Contacts, {
conditions: ‘firstName = “Florian” AND lastName = “Kugler”’
});
query = SC.Query.local(App.Contacts, {
conditions: ‘(firstName = “Florian” AND lastName = “Kugler”) OR age > 30’
});
However, you will want to not only hard-code the query conditions, but to make use of variables containing the desired values. For this you can use query parameters.
SproutCore handles two different types of query parameters: sequential and named parameters. Lets rephrase the above query using sequential parameters:
query = SC.Query.local(App.Contacts, {
conditions: ‘(firstName = %@ AND lastName = %) OR age > %’,
parameters: [‘Florian’, ‘Kugler’, 30]
});
The elements of the parameters array will be inserted sequentially at the positions of the %@ placeholders. Now lets do the same with named parameters:
query = SC.Query.local(App.Contacts, {
conditions: ‘(firstName = {first} AND lastName = {last}) ’ + ’OR age > {age}’,
parameters: {
first: ‘Florian’,
last: ‘Kugler’,
age: 30
}
});
Which of these methods you use is mainly a matter of personal preference and the complexity of your query.
The arguments inside the query conditions can be of the following types:
- Attribute names of the record type queried for.
- null and undefined.
- true and false.
- Integer and floating point numbers.
- Strings (single or double quoted).
Furthermore you can use the following comparison operators:
- =, !=, <, <=, >=.
- BEGINS_WITH (checks if a string starts with another one).
- ENDS_WITH (checks if a string ends with another one).
- CONTAINS (checks if a string contains another one, or if an object is in an array).
- MATCHES (checks if a string is matched by a regexp, you will have to use a parameter to insert the regexp).
- ANY (checks if the thing on its left is contained in the array on its right, you will have to use a parameter to insert the array).
- TYPE_IS (unary operator expecting a string containing the name of a model class on its right side, only records of this type will match).
To obtain ordered query results you can simply add the orderBy option to your query:
query = SC.Query.local(App.Contacts, {
conditions: ‘age > 30’,
orderBy: ‘lastName, firstName ASC’
});
In this case the results are sorted in an ascending order, first by last name and second by first name. If you omit the ASC keyword, the results are by default sorted in an ascending order. To sort them in descending order, put the keyword DESC after the name of the property.
NOTE: If you need a custom sorting order, you can register your own comparison operator for a specific model attribute using SC.Query.registerComparison. Please refer to the documentation for further details.
All the queries you used until now will cause the store to match all records in memory with the query’s conditions. You can also build one query on top of another to construct more efficient query trees:
query1 = SC.Query.local(App.Contacts, {
conditions: ‘age > 30’,
});
aboveThirty = App.store.find(query1);
query2 = SC.Query.local(App.Contacts, {
conditions: ‘lastName BEGINS_WITH “K”’
})
results = aboveThirty.find(query2);
The second query is based on the first one by calling find on the RecordArray of the first query instead of App.store. The second query matches the results of the first query against its own conditions. In this case it would return all Contact records where age is greater than 30 and the last name starts with the letter “K”.
NOTE: Scope queries can be thought as chained queries using the AND logical operator.
You will have noticed the keyword local in the SC.Query.local call we used until now to create the queries. Actually the keyword local is somewhat confusing, because local queries do not act exclusively on the in-memory store but also call the data source to fetch records from the server. The main characteristic of local queries is that the store automatically updates their results whenever the contents of the local in-memory store change.
Remote queries (build with SC.Query.remote), on the other hand, return a SC.RecordArray which is not updated automatically. However, “remote” doesn’t mean necessarily that the results have to be fetched from a remote server. They could also be loaded from a local browser storage. It’s admittedly a bad choice of names.
WARNING: You should use local queries in almost all cases unless you know what you’re doing.
If SproutCore’s built-in query operators are not sufficient for your use case, you can easily extend the query language. For example by default there are no bit-wise operators, so lets implement a BITAND operator which evaluates to true if the bit-wise and of the two arguments is unequal to zero:
SC.Query.registerQueryExtension(‘BITAND’, {
reservedWord: true,
leftType: ‘PRIMITIVE’,
rightType: ‘PRIMITIVE’,
evalType: ‘BOOLEAN’,
});
We call SC.Query.registerQueryExtension to register the new operator with the name BITAND as first argument. The key components of the hash passed as second argument are evalType and evaluate. evalType is either BOOLEAN (if you return a boolean value in evaluate) or PRIMITIVE (if you return e.g. a number or a string). The actual operation is implemented in the evaluate function after the operands are retrieved by this.leftSide.evaluate(r,w) and this.rightSide.evaluate(r,w).
NOTE: Look at the source of SC.Query for more examples of how to implement query operators.
We have covered the primary record states and the occurrence of some of the possible substates above. If you want to gain a complete understanding of what record states exist and what they mean, you will find a short description of all states below as well as complete state transition diagrams.
SC.Record.READY_NEW: A record in this state was created in memory but has not yet been committed back to the server. If you commit changes on your store, this record will be sent to the data source automatically. Likewise if you destroy this record it will simply be removed from memory without notifying the server. The data source also cannot modify records while they are in this state since they have local changes that would be lost.
SC.Record.READY_CLEAN: This is the default state of a record once it has been loaded from the server or its changes were committed. A record in this state exists on the server and has not been modified locally. If you commit changes on your store, this record will NOT be included. Likewise, if your data source receives a notification from the server that this record has changed, it can modify this record without any kind of error since no changes would be lost.
SC.Record.READY_DIRTY: This is the state of a record that exists on the server but has since been modified in your SproutCore application. It has pending changes that will be sent to the server when you commit changes in your store. If your data source tries to modify this record, it will throw an error since local changes would be lost.
SC.Record.BUSY_LOADING: When you first get a record from the store, it will usually be in the BUSY_LOADING state. This means that the record did not exist in the store and the server has not yet returned any data. All properties will be empty at this point. When data is loaded for this record, it will transition to READY_CLEAN.
SC.Record.BUSY_CREATING: A record in this state was newly created in the store. We are now waiting on the data source to confirm that it has been created in the server as well. Once this completes, the record will become READY_CLEAN. If the create fails, the record will return to READY_NEW.
SC.Record.BUSY_COMMITTING: A record in this state was modified in the store and it is waiting on the data source to confirm the change is saved on the server as well. Once this completes, the record will become READY_CLEAN.
SC.Record.BUSY_REFRESH_DIRTY: A record in this state has local changes but you asked the data source to reload the data from the server anyway. When the server updates, it will replace any local changes with a fresh copy from the server. If the refresh fails, the record will return to its READY_DIRTY state.
SC.Record.BUSY_REFRESH_CLEAN: A record in this state has no local changes but you asked it to reload the data from the server anyway. When the server finished, success or failure, this record will return to the READY_CLEAN state (unless there is an error).
SC.Record.BUSY_DESTROYING: A record in this state was destroyed in the store and now is being destroyed on the server as well. Once the destroy has completed, the record will become DESTROYED_CLEAN. If the destroy fails (without an error), it will become DESTROYED_DIRTY again.
SC.Record.DESTROYED_CLEAN: A record in this state has been destroyed both in the store and on the server. A record enters this state, for example, if your data source destroys the record or if you destroy the record in memory then commit that change successfully to the server.
SC.Record.DESTROYED_DIRTY: A record in this state was destroyed in the store by the application, but it has not yet been destroyed on the server. A record in this state cannot accept changes from the server unless the server also puts it into a destroyed state. Committing changes on your store will ask the data source to send this destroy to the server.
- February 6, 2011: initial version by Florian Kugler
- March 2, 2011: added filenames and small fixes by Topher Fangio
- March 2, 2011: minor corrections by Florian Kugler