Missing data in OnDemandGrid using JsonRest store

[tonyabo]

I am trying to test the new dgrid in my existing application. I am using OnDemandGrid and a JsonRest store. I am finding that the grid displays as expected and pulls more data when I move the scroll bar. It is not however showing all of the rows, and I cannot figure out why.

I am using what I pulled from the master branch yesterday, with the dojo 1.7.3 release.

The grid is defined as follows:
require(["dojo/_base/declare", "dgrid/OnDemandGrid", "dojo/store/JsonRest", "dojo/store/Observable", "dojo/sto
re/Cache", "dojo/store/Memory", "dgrid/Selection", "dojo/domReady!"],
function(declare, Grid, JsonRest, Observable, Cache, Memory, Selection){
thisApp.popupStore = new JsonRest({
target: thisApp.sessionUrl + "?Action=GetRows",
// idProperty: ??
});
thisApp.grid = declare([Grid, Selection])({
columns: thisApp.columns,
store: thisApp.popupStore,
selectMode: thisApp.selectionMode,
}, "popup");
}
);

Note extra stuff in require() for playing and testing but not necessarily being used.

thisApp.columns will resolve to:
{ c0: { label: "", className: "left" }, c1: { label: "", className: "left" } }

thisApp.selectionMode is the string "single"

When the grid loads, it issues a GET request using the specified URL and header "Range: items=0-24".

The response has headers:
Content-Range: items 0-24/52
Content-Type: text/json; charset=ISO-8859-1

and the data looks like this:

[{"c0":"AL", "c1":"Alabama"},{"c0":"AK", "c1":"Alaska"},{"c0":"AZ", "c1":"Arizona"},{"c0":"AR", "c1":"Arkansas"},{"c0":"CA", "c1":"California"},{"c0":"CO", "c1":"Colorado"},{"c0":"CT", "c1":"Connecticut"},{"c0":"DE", "c1":"Delaware"},{"c0":"DC", "c1":"District of Columbia"},{"c0":"FL", "c1":"Florida"},{"c0":"GA", "c1":"Georgia"},{"c0":"HI", "c1":"Hawaii"},{"c0":"ID", "c1":"Idaho"},{"c0":"IL", "c1":"Illinois"},{"c0":"IN", "c1":"Indiana"},{"c0":"IA", "c1":"Iowa"},{"c0":"KS", "c1":"Kansas"},{"c0":"KY", "c1":"Kentucky"},{"c0":"LA", "c1":"Louisiana"},{"c0":"ME", "c1":"Maine"},{"c0":"MD", "c1":"Maryland"},{"c0":"MA", "c1":"Massachusetts"},{"c0":"MI", "c1":"Michigan"},{"c0":"MN", "c1":"Minnesota"},{"c0":"MS", "c1":"Mississippi"}]

note that there are 25 objects in the array.

A nicely formatted grid containing the first 16 rows and a vertical scrollbar is rendered.

Then, when I pull the scroll bar down I see a new request with header "Range: items=24-51", and response with:

Content-Range: items 24-51/52
Content-Type: text/json; charset=ISO-8859-1

and data:

[{"c0":"MS", "c1":"Mississippi"},{"c0":"MO", "c1":"Missouri"},{"c0":"MT", "c1":"Montana"},{"c0":"NA", "c1":"Native American"},{"c0":"NB", "c1":"Nebraska"},{"c0":"NV", "c1":"Nevada"},{"c0":"NH", "c1":"New Hampshire"},{"c0":"NJ", "c1":"New Jersey"},{"c0":"NM", "c1":"New Mexico"},{"c0":"NY", "c1":"New York"},{"c0":"NC", "c1":"North Carolina"},{"c0":"ND", "c1":"North Dakota"},{"c0":"OH", "c1":"Ohio"},{"c0":"OK", "c1":"Oklahoma"},{"c0":"OR", "c1":"Oregon"},{"c0":"PA", "c1":"Pennsylvania"},{"c0":"RI", "c1":"Rhode Island"},{"c0":"SC", "c1":"South Carolina"},{"c0":"SD", "c1":"South Dakota"},{"c0":"TN", "c1":"Tennessee"},{"c0":"TX", "c1":"Texas"},{"c0":"UT", "c1":"Utah"},{"c0":"VT", "c1":"Vermont"},{"c0":"VA", "c1":"Virginia"},{"c0":"WA", "c1":"Washington"},{"c0":"WV", "c1":"West Virginia"},{"c0":"WI", "c1":"Wisconsin"},{"c0":"WY", "c1":"Wyoming"}]

note that there are 28 objects in the array

The grid then re-renders, but the visible rows reflect a huge gap in the data. Of the 16 visible rows, I am seeing "ID" through "MS", then "WV", "WI", and "WY". Effectively, of the 52 expected rows, those are rows (0 based) 12 through 24, then 49 through 51. In other words, the first 25 elements of the seconds request are missing.

After that point, if I move the scroll bar up and down it scrolls the existing incomplete data back to the beginning and end, and no more requests are made to the server.

Can anyone tell me something obvious that I've done wrong?

I have been trying to do debug references to the popupStore object to see if it sees the missing rows, but I can't figure out how to get anything meaningful out of it.

Help!

Just out of curiosity, can you try this again against most recent master? There was actually a regression a couple of days ago that was fixed yesterday, so I'm not sure whether your issue overlapped with that.

If the service is seriously operating as intended though, I would think this would be reproducible with other stores, too, even in-memory ones... would you be able to put together a test case?

[tonyabo]

I pulled a new dgrid and still see the same behavior. I am working on a portable test case now.

[tonyabo]

Here is a test case (apologies for my inelegant php coding. It is demonstrating the exact symptoms described above.

<?php if (!(isset($_GET['Action']) && $_GET['Action'] == 'getdata')) {
    header("Content-type: text/html");
?>
<html>
<head>
<!-- POP.UP -->

<title>test dgrid with states</title>

<!-- Libraries -->
<script type="text/javascript" src="/dojo/dojo/dojo.js" djConfig="parseOnLoad:true, isDebug:false"></script>

<style type="text/css">
   @import "/dojo/dijit/themes/tundra/tundra.css";
   @import "/dojo/dojo/resources/dojo.css";
</style>

<script type="text/javascript">
var thisApp = {
    sessionUrl: location.pathname,
    selectionMode: "single",
    columns: {c0: { label: "", className: "left"}, c1: { label: "", className: "left"} }
}

require(["dojo/_base/declare", "dgrid/OnDemandGrid", "dojo/store/JsonRest", "dojo/store/Observable", "dojo/store/Cache", "dojo/store/Memory", "dgrid/Selection", "dojo/domReady!"],
  function(declare, Grid, JsonRest, Observable, Cache, Memory, Selection){

      thisApp.popupStore = new JsonRest({
          target: thisApp.sessionUrl + "?Action=getdata",
          // idProperty: ??
      });

      thisApp.grid = declare([Grid, Selection])({
          columns: thisApp.columns,
          store: thisApp.popupStore,
          selectMode: thisApp.selectionMode,
          }, "popup");
  }
);

</script>

</head>
<body>
<div id="popup"></div>
</body>
</html>
<?php
} else {
    session_start();
    if (!isset($_SESSION['data'])) {
        $_SESSION['data'] = array(
            "AL" => "Alabama",
            "AK" => "Alaska",
            "AZ" => "Arizona",
            "AR" => "Arkansas",
            "CA" => "California",
            "CO" => "Colorado",
            "CT" => "Connecticut",
            "DE" => "Delaware",
            "DC" => "District of Columbia",
            "FL" => "Florida",
            "GA" => "Georgia",
            "HI" => "Hawaii",
            "ID" => "Idaho",
            "IL" => "Illinois",
            "IN" => "Indiana",
            "IA" => "Iowa",
            "KS" => "Kansas",
            "KY" => "Kentucky",
            "LA" => "Louisiana",
            "ME" => "Maine",
            "MD" => "Maryland",
            "MA" => "Massachusetts",
            "MI" => "Michigan",
            "MN" => "Minnesota",
            "MS" => "Mississippi",
            "MO" => "Missouri",
            "MT" => "Montana",
            "NA" => "Native American",
            "NB" => "Nebraska",
            "NV" => "Nevada",
            "NH" => "New Hampshire",
            "NJ" => "New Jersey",
            "NM" => "New Mexico",
            "NY" => "New York",
            "NC" => "North Carolina",
            "ND" => "North Dakota",
            "OH" => "Ohio",
            "OK" => "Oklahoma",
            "OR" => "Oregon",
            "PA" => "Pennsylvania",
            "RI" => "Rhode Island",
            "SC" => "South Carolina",
            "SD" => "South Dakota",
            "TN" => "Tennessee",
            "TX" => "Texas",
            "UT" => "Utah",
            "VT" => "Vermont",
            "VA" => "Virginia",
            "WA" => "Washington",
            "WV" => "West Virginia",
            "WI" => "Wisconsin",
            "WY" => "Wyoming"
        );
    }

    if(isset($_SERVER["HTTP_RANGE"])){
        $range = $_SERVER["HTTP_RANGE"];
    }else if(isset($_SERVER["HTTP_X_RANGE"])){
        $range = $_SERVER["HTTP_X_RANGE"];
    }
    if ($range) {
        preg_match('/(\d+)-(\d+)/', $range, $matches);
        $start = $matches[1];
        $end = $matches[2];
        if ($start > count($_SESSION['data']) - 1 ) $start = count($_SESSION['data']) - 1;
        if ( $end > count($_SESSION['data']) - 1 ) $end = count($_SESSION['data']) - 1;
    } else die("Received request with missing or invalid range: " . $range);

    header("Content-Type: text/json");
    header("Content-Range: items " . $start ."-". $end ."/". count($_SESSION['data']));

    $result = array();
    $i = 0;
    echo '[';
    foreach( $_SESSION['data'] as $key => $value ) {
        if ( $i > $end ) break;
        if ( $i >= $start ) {
            if  ( $i > $start ) echo ',';
            echo '{"c0":"' . $key . '", "c1":"' . $value . '"}';
        }
        ++$i;
    }
    echo ']';
}
?>

[tonyabo]

Ken, were you able to reproduce my results?

[bitpshr]

It looks like in the test page you provided, items are being returned without an id attribute. Each item in a JsonRest (and most other Dojo stores) is required to have a uniquely identifying attribute - this defaults to id, but can be explicitly specified via the idProperty on the store itself. If you return a unique id attr on each item in your test page, it works as expected. I hope this helps!

[tonyabo]

Thanks for that. Using the idProperty fixed the problem. I am new to dojo stores. Note that the description of idProperty in http://dojotoolkit.org/api/1.8/dojo/store/api/Store does not make that requirement clear:

"If the store has a single primary key, this indicates the property to use as the identity property. The values of this property should be unique."

Stores are generally free as to which APIs they choose to implement or not; however, dgrid generally expects stores to implement query, get, and getIdentity. I'll make a note to make sure we cover this in the wiki. Thanks.