Report v2

[Exilliar]

What is the issue with the old object?

The current issue with the way that we are storing reports is that they lead to longer load times than are neccessary.

What's the loading issue?

Loading of reports on act is currently a major issue. It can take a long time for reports to load, some of the recent changes have helped with this, however, as a general rule, the larger the model (and thus, the larger the report), the longer it will take to load the report. This is an inevitable issue for the main report viewer page, where we need to show all data from the report, however, on the project cards (the cards on the dashboard and the report branches page) this issue can be (mostly) solved.

What's causing the long load times?

The main thing causing the longer load times is getting the data to generate the donut charts on the project cards:

In order to generate this donut chart we need to pull in all objects for a report. There is roughly one object per each "element" in a model (so every door, wall, ceiling). There can quite easily be >10,000 objects in a model. We can pull down 1000 objects at a time from Speckle (and send off multiple requests of 1000 objects at the same time), but this loading can take a while, and even when pulling in 1000 objects at a time it can result in a large number of requests to Speckle. Then once we've got all the objects we then calculate some things using them. The calculations are simple, but we still have to loop through every object, which can take a while.

What are we calculating?

We currently calculate two things:

1. The A1-5 levels
2. How much carbon each material is producing

Full order of things that are done while loading each project card

1. Pull down the "parent" Speckle object for a report
2. Pull down all of the child objects for the report
3. Calculate the A1-5 values based off those child objects
4. Calculate the carbon emissions per material using the child objects

How is the current report structured/stored in Speckle

The report is stored by creating one "parent" object which contains general info about the report (the jn, name, some carbon totals). This object then links to many "child" objects which store the specific carbon info for each element on the model.

Parent object example:

{
  "data": {
    "stream": {
      "object": {
        "data": {
          "id": "92734982734982734-act",
          "volume": 493.5896691542059,
          "totalCO2": 193478.60256155572,
          "__closure": {},
          "projectData": {
            "cost": 10000,
            "name": "test",
            "notes": "notes that are probably different from what's currently there",
            "region": "UK",
            "floorArea": 10000,
            "jobNumber": "000000",
            "components": [
              {
                "name": "Superstructure",
                "color": "white",
                "backgroundColor": "#224a63"
              },
              {
                "name": "Mechanical Services",
                "color": "black",
                "backgroundColor": "#f0b4b4"
              }
            ]
          },
          "speckleType": "act-totals",
          "speckle_type": "act-totals",
          "transportCarbonA4": 29628.32608667148,
          "totalChildrenCount": 205,
          "constructionCarbonA5": {
            "site": 28700,
            "value": 36938.93012807775,
            "waste": 8238.930128077785
          },
          "productStageCarbonA1A3": 126911.34634680649
        }
      }
    }
  }
}

Child object example:

{
  "data": {
    "stream": {
      "object": {
        "data": {
          "id": "92034982374987234",
          "act": {
            "id": "029834098234809234",
            "formData": {
              "volume": 0.15582599999996738,
              "material": {
                "name": "Concrete - In situ, unreinforced c30/37 (0.1 kgCO2e/kg)",
                "color": "#CFDFD3",
                "units": "kgCO2e/kg",
                "source": "IStructE How to calculate Embodied Carbon (2021)",
                "density": 2400,
                "wastage": 0.05,
                "productStageCarbonA1A3": 0.103
              },
              "transport": {
                "name": "local",
                "color": "#53ac8b",
                "values": {
                  "sea": 0,
                  "rail": 0,
                  "road": 50
                }
              }
            },
            "reportData": {
              "transportCarbonA4": 2.1242200319995552,
              "constructionCarbonA5": {
                "site": 140,
                "value": 142.13917932799956,
                "waste": 2.1391793279995492
              },
              "productStageCarbonA1A3": 38.52018719999194
            },
            "speckle_type": "Objects.BuiltElements.Revit.FamilyInstance"
          },
          "speckleType": "act-object",
          "totalChildrenCount": 0
        }
      }
    }
  }
}

Proposed changes

The proposed change to the object is to store the data needed for the project card bar chart in the parent object. This would mean that only the parent object of a report needs to be pulled in on the dashboard and report branches page.

Example of new parent object:

{
  "data": {
    "stream": {
      "object": {
        "data": {
          "id": "92834908238409293480-act",
          "version": "2.0.0", // NEW
          ...,
          "productStageCarbonA1A3": 126911.34634680649,
          "materials": [{
            "label": "steel",
            "value": 10000,
            "color": "#f232"
          }, ...] // NEW
        }
      }
    }
  }
}

The new fields have been marked with the comments "NEW"

We would also remove the calculation of the A1-5 values on the dashboard and report branches page as they are not required and are likely a holdover from an older version of those pages.

How would we implement these changes

We would have a phased rollout of this new object version. We could continue to support both the old and new versions of this object by checking the new "version" property that is on the report. If the field is present and is set to "2.0.0" (or whatever we choose to make it) then we would just load in the data from the parent object. If that field does not exist, then we would use the old way of loading.

We would update reports to use this new report structure by changing the New Assessment page to save the reports in this new format. So as new reports are run on old models, the reports will be updated.

Other advantages of this new structure

1. Having the new "version" property would allow us to more easily make updates to the report object (which we are likely to want to do at some point). This is because it would allow act to know what report structure is being used by just checking this property. So we could continue to support old versions of report objects, while still makeing updates to the reports.