Add new `create_property` and `update_with_espm` endpoint #4012

kflemin · 2023-04-27T21:47:57Z

Create a new "Create property" endpoint which creates a property, propertystate, & propertyview via API.

Needed for 179d web application.

github-actions · 2023-04-27T21:48:11Z

Label error. Requires at least 1 of: Feature, Bug, Enhancement, Maintenance, Documentation, Performance, Do not publish. Found:

axelstudios

@kflemin I pushed up several fixes for unhandled errors, but there are TONS that remain:

There's no validation for the state object beyond the basic serializer (and I don't think we'd even want to use that serializer for validation since it requires 17 fields, many of which I wouldn't expect to be relevant like organization_id and total_ghg_emissions_intensity) - the user can pass in anything:

If the state's organization and organization_id reference a different org than the one used for the query parameter it will accept it regardless, and the state will be inserted for the wrong org
Should users be able to pass in an entire extra_data json object? This seems like a vulnerability since they could completely exhaust the database storage in no time with massive json objects
There are tons of state fields the user should not be able to input:
- id
- measures
- scenarios
- files
- import_file_id
- organization_id
- data_state
- merge_state
- normalized_address
- long_lat
- centroid
- bounding_box
- hash_object
- created
- updated
- import_file
- organization
- and certainly more

…orm/seed into 179d-endpoint-updates

* add single property download method * add viewset for downloading single espm report * fix BAE migration and update swagger schema * rename pm report single to download

…backend method finished

nllong · 2023-08-04T15:11:48Z

docs/source/translation.rst

@@ -3,7 +3,14 @@ Translating SEED

 1. Update translations on `lokalise`_.

-2. Copy lokalise.cfg.example to lokalise.cfg. Update API token.
+2. Copy lokalise.yml.example to lokalise.yml. Update API token.


our lokalise instructions were out of date

nllong · 2023-08-04T15:12:27Z

seed/building_sync/building_sync.py

        for item in assets:
-            seed_result[item['name']] = item['value']
-
+            seed_result[item.name] = item.value


new BAE tool typed the data.

nllong · 2023-08-04T15:15:24Z

seed/models/column_mapping_profiles.py

@@ -44,3 +50,56 @@ def get_profile_type(cls, profile_type):
        if profile_type in types_dict:
            return types_dict[profile_type]
        raise Exception(f'Invalid profile type "{profile_type}"')
+
+    @classmethod
+    def create_from_file(cls, filename: str, org: Organization, profile_name: str, profile_type: int = NORMAL, overwrite_if_exists: bool = False):


needed a way to import a column mapping profile from a file. We have a method to create column mappings but it wasn't a "named" profile.

nllong · 2023-08-04T15:15:52Z

seed/serializers/properties.py

@@ -212,6 +212,83 @@ def to_representation(self, data):
        return result


+class PropertyStatePromoteWritableSerializer(serializers.ModelSerializer):


this class handles the read_only fields so that we don't have to worry about writing to protected fields.

nllong · 2023-08-04T15:17:51Z

seed/tests/test_account_views.py

            'salesforce_enabled': False,
            'ubid_threshold': 1
        }

        org_payload = _dict_org(self.fake_request, [self.org])
+


if in your local_untracked you have set your own better and AT URLs, then this test fails locally. So let's just check to make sure that it matches a regex.

nllong · 2023-08-04T15:18:05Z

seed/tests/test_api_mixins.py

@@ -374,18 +374,18 @@ def test_get_profile_id(self):
        self.assertListEqual(columns['extra_data'], ['field_1'])

        # no extra data
-        columnlistprofile = self.column_list_factory.get_columnlistprofile(
+        column_list_profile = self.column_list_factory.get_columnlistprofile(


just spelling.

nllong · 2023-08-04T15:19:15Z

seed/tests/test_portfoliomanager.py

@@ -87,7 +88,7 @@ def test_template_list_interface_no_username(self):
        # status should be error
        # message should have "missing username"
        self.assertEqual(400, resp.status_code)
-        data = json.loads(resp.content)
+        data = resp.json()


I replace these when I see them.

nllong

@axelstudios -- this is ready for another review.

nllong · 2023-08-04T15:20:20Z

seed/views/v3/analyses.py

@@ -370,7 +370,7 @@ def get_counts(field_name):
        property_types = get_counts('extra_data__Largest Property Use Type')
        year_built = get_counts('year_built')
        energy = get_counts('site_eui')
-        sqftage = get_counts('gross_floor_area')
+        square_footage = get_counts('gross_floor_area')


just spelling

nllong · 2023-08-04T15:50:13Z

@kflemin I pushed up several fixes for unhandled errors, but there are TONS that remain:

There's no validation for the state object beyond the basic serializer (and I don't think we'd even want to use that serializer for validation since it requires 17 fields, many of which I wouldn't expect to be relevant like organization_id and total_ghg_emissions_intensity) - the user can pass in anything:

If the state's organization and organization_id reference a different org than the one used for the query parameter it will accept it regardless, and the state will be inserted for the wrong org

Should users be able to pass in an entire extra_data json object? This seems like a vulnerability since they could completely exhaust the database storage in no time with massive json objects

There are tons of state fields the user should not be able to input:

id

measures

scenarios

files

import_file_id

organization_id

data_state

merge_state

normalized_address

long_lat

centroid

bounding_box

hash_object

created

updated

import_file

organization

and certainly more

This should not be a problem. The org_id is validated on the endpoint.
We could enforce only columns that have been added to an org to be allowed in extra_data?
The Write Serializer protects all of the "Column.EXCLUDE_.*" objects.

nllong · 2023-08-12T15:48:50Z

seed/lib/xml_mapping/mapper.py

@@ -56,20 +56,24 @@ def get_bae_mappings():
    # units field name is the same with " Units" appended.

    bsync_assets = BAE.get_default_asset_defs()
-    for item in bsync_assets:
+    for asset in bsync_assets:
+        if isinstance(asset, dict):


This should never happen with the latest BAE code. @axelstudios - you probably just needed to run pip install -r requirements/local.txt

- Fixed extra_data columns to only ones that actually apply - Changed `merge_state` from 0 to 1 - Added ValueError handling

axelstudios · 2023-08-15T18:43:25Z

Here are the changes I've made:

Removed broken property_id option
Fixed extra_data columns to only ones that actually apply
Changed merge_state from 0 to 1 (new)
Added ValueError handling
Small cleanup

#4211) * Fixed bae object accessors * create two new methods for running save_raw and map to the foreground * fix type --------- Co-authored-by: Alex Swindler <Alex.Swindler@nrel.gov>

* 179d create_property endpoint * precommit * adding property id, state id, and view id to create response * Partial fixes for handling bad data * Verify property exists and belongs to org * start of new serializer and controlling extra_data fields for 179d * fix property create endpoint * Formatting * fix org_id on column views * add report_format to PM download so spreadsheet can be downloaded * updating BAE and small tweak since assets are now typed * update BAE processing in reader * updating BAE to v0.1.14 * remove uneeded view * Add ESPM download for single property (via API) (#4158) * add single property download method * add viewset for downloading single espm report * fix BAE migration and update swagger schema * rename pm report single to download * update lokalise notes and pull from latest from website * initial commit with a new update with ESPM function. need to get the backend method finished * enable udpate_with_espm * add test and column mapping profile helper to load in the data * fix example file that was downloaded from espm * fix comment formats * file paths * style espm modal and fix both upload methods * put tmp files into the projects media dir * add column mapping profile param to update_with_espm * put tmp files into the projects media dir * add status to dresponse for pyseed * do not save in media_root * precommit * allow different urls for better and at during testing * return the mapping profile object when creating from file * add tests to ensure read_only fields are protected * Update models.py * Update models.py * - Removed broken `property_id` option - Fixed extra_data columns to only ones that actually apply - Changed `merge_state` from 0 to 1 - Added ValueError handling * Small fixes * Fix 179d ESPM update endpoint to run in foreground with Celery enabled (#4211) * Fixed bae object accessors * create two new methods for running save_raw and map to the foreground * fix type --------- Co-authored-by: Alex Swindler <Alex.Swindler@nrel.gov> --------- Co-authored-by: Nicholas Long <1907354+nllong@users.noreply.github.com> Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> Co-authored-by: Nicholas Long <nicholas.long@nrel.gov>

* Remove Deprecated APIs (#4049) * Remove deprecated endpoints * Update tests * Remove additional code related to API v2 * Add ubid_threshold to expected org response * Lock `google-chrome` to v114 (#4165) Lock chrome version to webdriver version * Bump word-wrap from 1.2.3 to 1.2.4 (#4154) Bumps [word-wrap](https://github.com/jonschlinkert/word-wrap) from 1.2.3 to 1.2.4. - [Release notes](https://github.com/jonschlinkert/word-wrap/releases) - [Commits](jonschlinkert/word-wrap@1.2.3...1.2.4) --- updated-dependencies: - dependency-name: word-wrap dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> * Fix a few code-formatting issues (#4172) * run autopep to cleanup a few straggling issues * Run precommit against migrations * Cleanup unnecessary logging --------- Co-authored-by: Alex Swindler <Alex.Swindler@nrel.gov> * Add Open Street Map to CSP rules (#4169) * add error handling for sf edge case (#4159) add error handldling for sf edge case Co-authored-by: Hannah Eslinger <heslinge@nrel.gov> Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> * Remove ID from inventory document display name (#4125) remove id from inventory doc on inv det * Set the MEDIA_ROOT path back to original value during test (#4174) * set the mediaroot path back to where it started * naming * Optimize list analyses endpoint (#4206) * Optimize list analyses endpoint * Add outputfiles back in * Fix * Make it even faster * Move analyses endpoint (#4210) * Add new `create_property` and `update_with_espm` endpoint (#4012) * 179d create_property endpoint * precommit * adding property id, state id, and view id to create response * Partial fixes for handling bad data * Verify property exists and belongs to org * start of new serializer and controlling extra_data fields for 179d * fix property create endpoint * Formatting * fix org_id on column views * add report_format to PM download so spreadsheet can be downloaded * updating BAE and small tweak since assets are now typed * update BAE processing in reader * updating BAE to v0.1.14 * remove uneeded view * Add ESPM download for single property (via API) (#4158) * add single property download method * add viewset for downloading single espm report * fix BAE migration and update swagger schema * rename pm report single to download * update lokalise notes and pull from latest from website * initial commit with a new update with ESPM function. need to get the backend method finished * enable udpate_with_espm * add test and column mapping profile helper to load in the data * fix example file that was downloaded from espm * fix comment formats * file paths * style espm modal and fix both upload methods * put tmp files into the projects media dir * add column mapping profile param to update_with_espm * put tmp files into the projects media dir * add status to dresponse for pyseed * do not save in media_root * precommit * allow different urls for better and at during testing * return the mapping profile object when creating from file * add tests to ensure read_only fields are protected * Update models.py * Update models.py * - Removed broken `property_id` option - Fixed extra_data columns to only ones that actually apply - Changed `merge_state` from 0 to 1 - Added ValueError handling * Small fixes * Fix 179d ESPM update endpoint to run in foreground with Celery enabled (#4211) * Fixed bae object accessors * create two new methods for running save_raw and map to the foreground * fix type --------- Co-authored-by: Alex Swindler <Alex.Swindler@nrel.gov> --------- Co-authored-by: Nicholas Long <1907354+nllong@users.noreply.github.com> Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> Co-authored-by: Nicholas Long <nicholas.long@nrel.gov> * Fix display name (#4223) * Fix refresh_metadata (#4221) * Fix refresh_metadata * Rename * Bump uwsgi from 2.0.17.1 to 2.0.22 in /requirements (#4209) Bumps [uwsgi](https://github.com/unbit/uwsgi-docs) from 2.0.17.1 to 2.0.22. - [Commits](https://github.com/unbit/uwsgi-docs/commits) --- updated-dependencies: - dependency-name: uwsgi dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> * Remove Beta label from text of analysis button (#4225) remove beta from analysis Co-authored-by: Nicholas Long <1907354+nllong@users.noreply.github.com> * Fix Meters Without `source_id` (#4229) Fix showing meter readings for meters without `source_id` * Fix performance for fetching analyses and force user email to lowercase (#4213) * Set username to lowercase when updating user info * Removed unnecessary prefetch from analyses retrieval --------- Co-authored-by: Hannah Eslinger <heslinge@nrel.gov> Co-authored-by: Nicholas Long <1907354+nllong@users.noreply.github.com> * EEEJ Analysis Functionality (#4208) * eeej analysis * precommit * precommit * add EJScreen * update eeej analysis to link to EJScreen report * adding analysis description for eeej * Fix display name * fix for analysis reruns * cleanup * fix eeej tests * cleanup * fix searching for string in a NoneType * set manual geocoding * fix geocode tests and add census to geocode summary * fix typos * geocode with lat/lng over address if available * add copyright text * small doc formatting cleanup * cleanup * oops, missed this formatting --------- Co-authored-by: Nicholas Long <1907354+nllong@users.noreply.github.com> Co-authored-by: Hannah Eslinger <heslinge@nrel.gov> Co-authored-by: Nicholas Long <nicholas.long@nrel.gov> * Show Analysis `debug_message` If Available (#4198) Show analysis debug_message if available Co-authored-by: Katherine Fleming <2205659+kflemin@users.noreply.github.com> * Use display name in custom reports legend (#4060) * Use display name in custom reports legend * Fix docker compose * Fix Tests * Address comment * Clean --------- Co-authored-by: Katherine Fleming <2205659+kflemin@users.noreply.github.com> * Allow analysis property view related properties to be null (#4227) * update ondelete buildingfile > event > scenarios * update relationhsip to null when a property is deleted * add warning triangle * remove checkbox to delete analyses with properties * precommit * reorder migrations * reset * remove commit from outside branch * add warning to analyses page * styles * Update migrations --------- Co-authored-by: Hannah Eslinger <heslinge@nrel.gov> * Add more API documentation for meter and meter readings (#4237) * add more api doc info on meter and meter reading endpoints * Improved nested documentation, added swagger support for enums, sorted meter readings --------- Co-authored-by: Alex Swindler <Alex.Swindler@nrel.gov> * Update inventory selection to include current inventory type only (#3862) filter selection by current invetory type and return its length Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> * Update default meter range selection for BETTER analysis (#3819) * Show all navs in org * reverse the default better meter selection * Allow analysis to be by range or cycle * html updates to add cycle to BETTER * revert to allow merge * select cycle for better * update docker-dev better api endpoint * modify valid meter data to accept cycle start and end dates * precommit * formatting * remove console and updat better url * merge conflicts * order * default current cycle to eui and better on select_cycle * Simplified --------- Co-authored-by: Hannah Eslinger <heslinge@nrel.gov> Co-authored-by: Alex Chapin <a.n.chapin@gmail.com> Co-authored-by: Ryo <kyle.schulz@devetry.com> Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> * Delete BuildingFiles and Related Events on PropertyState delete (#4217) * update ondelete buildingfile > event > scenarios * precommit * delete scenario if event is deleted * cascade over nothing * Renumber migration --------- Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> * Fixed Issues with Scenario/Measure CRUD Operations (#4240) Fixed issues with scenario/measure CRUD operations, cherry-picked from a640926 * Add a cycle indicator to the analysis modal and filter analyses for current cycle (#3837) * expose current cycle on new analysis modal * display analysis associated with cycle * refactor help text * add cycle indicator to analysis_card * syntax * refactor for clarity * add cycle_name to analysis display tables * add cycle dropdown to inventory detail analyses page * find analyses relavent to current cycle * precommit * formatting * remove debug logs * clarifying note * consistency * Fixed ui-router navigation --------- Co-authored-by: Alex Swindler <Alex.Swindler@nrel.gov> * Disable ability to remove matching criteria fields once inventory is added (#4222) * update ondelete buildingfile > event > scenarios * base for locked matching criteria * refactor for clarity * remove matching preview from frontend * remove matching preview from backend * translations * precommit * remove unrelated migration * revert * update tests --------- Co-authored-by: Hannah Eslinger <heslinge@nrel.gov> Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> * Add smaller EEEJ data files for testing (#4243) add smaller EEEJ data files for testing * Add Uniformat Table (#4238) * Add uniformat * Minor fixes --------- Co-authored-by: Nicholas Long <1907354+nllong@users.noreply.github.com> * Fix Meter Overwrites (#4250) * add test to make sure meter readings do not overwrite * Fixed major bug where inserting a single meter reading would delete any existing readings for any meter with the same start time - Added the ability for inserting bulk meter readings to update existing data - Sanitized sql inputs in `copy_readings` * Simplification * EEEJ test fix * add test to test overwrite of bulk meter import --------- Co-authored-by: Nicholas Long <nicholas.long@nrel.gov> Co-authored-by: Nicholas Long <1907354+nllong@users.noreply.github.com> * Fix filter group dropdown bugs (#4249) Fix for filter group dropdown issues Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> * Add Import Export to Audit Template (#4215) * base for at get buildings * base batch at import functional from frontend button * frontend confirm before upating with at * frontend triggers backend batch update * functional but missing real time progress updates * functional with progress progress is moving, but cant get back to frontend properties updating, progress bar only 50% functional with progress * cleanup * precommit * summary and date formatting * bad credentials guards * test for full at import and property update * precommit * sample buildingsync xml file * swagger documentation * request validation * precommit * comment cleanup * comment cleanup * improve tests and disable update until selection * translate * lokalise translations * function description * update view permissions * clarify error message logic * precommit * add name to at building summary * add at_updated_at to pstate on at import, prevent duplicate uploads * create column on at import to display and compare * include translation * update tests * precommit * precommit * remove translation * build bsync xml from pstate * multiple iterations for xml export * base for batch export to at * add audit_template_report_type to org model * test cleanup * add audit_template_report_type to org model * add at report type to org settings * update batch test * toggle at password visibility * move batch export to celery * export properties to at from inv list actions * backfill org audit_template_report_type * mock tests for export to audit template * phrasing * refactor export results * build out export results table * precommit * revert * precommit * download current lokalise files * add default display field to failed exports * incorporate audit template report type in test * request body to be a dict not list * add translations * update ondelete buildingfile > event > scenarios * remove property name from inventory and expand modal * update states to reflect at_updated * add export to at from inv det actions * precommit * merge develop * migration conflict * Renumber migrations * Small fixes * Privilege enforcement --------- Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> * Fix EEEJ analysis errors when there are no lat/long or address (#4247) * fix eeej analysis errors * add eeej to tasks and fix for non-eager celery * Fix for dict key types changing based on ALWAYS_EAGER * fixes! * Simplification --------- Co-authored-by: Nicholas Long <1907354+nllong@users.noreply.github.com> Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> * Add envvar to specify loading test EEEJ dataset (#4254) * add envvar to specify loading test EEEJ dataset * slight mod to support strings as boolean values --------- Co-authored-by: Nicholas Long <1907354+nllong@users.noreply.github.com> * Default to excluding Address Line 1 as a matching criteria (#4256) * Default to excluding Address Line 1 for matching * [WIP] Fix some tests * update tests for address_line_1 default matching criteria removal --------- Co-authored-by: Nicholas Long <1907354+nllong@users.noreply.github.com> Co-authored-by: Hannah Eslinger <heslinge@nrel.gov> Co-authored-by: Nicholas Long <nicholas.long@nrel.gov> * Bump version (#4259) * bump versions * current change log * add remaining two PRs before release * Refactor map page for functionality and performance (#4260) * Major map/eeej/layout improvements * Added tract shading, database indices * Fix test failure --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Katherine Fleming <2205659+kflemin@users.noreply.github.com> Co-authored-by: Hannah Eslinger <heslinge@nrel.gov> Co-authored-by: Ross Perry <perryr16@gmail.com> Co-authored-by: ebeers-png <53191386+ebeers-png@users.noreply.github.com> Co-authored-by: Alex Chapin <a.n.chapin@gmail.com> Co-authored-by: Ryo <kyle.schulz@devetry.com>

* Remove Deprecated APIs (#4049) * Remove deprecated endpoints * Update tests * Remove additional code related to API v2 * Add ubid_threshold to expected org response * Lock `google-chrome` to v114 (#4165) Lock chrome version to webdriver version * Bump word-wrap from 1.2.3 to 1.2.4 (#4154) Bumps [word-wrap](https://github.com/jonschlinkert/word-wrap) from 1.2.3 to 1.2.4. - [Release notes](https://github.com/jonschlinkert/word-wrap/releases) - [Commits](jonschlinkert/word-wrap@1.2.3...1.2.4) --- updated-dependencies: - dependency-name: word-wrap dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> * Fix a few code-formatting issues (#4172) * run autopep to cleanup a few straggling issues * Run precommit against migrations * Cleanup unnecessary logging --------- Co-authored-by: Alex Swindler <Alex.Swindler@nrel.gov> * Add Open Street Map to CSP rules (#4169) * add error handling for sf edge case (#4159) add error handldling for sf edge case Co-authored-by: Hannah Eslinger <heslinge@nrel.gov> Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> * Remove ID from inventory document display name (#4125) remove id from inventory doc on inv det * Set the MEDIA_ROOT path back to original value during test (#4174) * set the mediaroot path back to where it started * naming * Optimize list analyses endpoint (#4206) * Optimize list analyses endpoint * Add outputfiles back in * Fix * Make it even faster * Move analyses endpoint (#4210) * Add new `create_property` and `update_with_espm` endpoint (#4012) * 179d create_property endpoint * precommit * adding property id, state id, and view id to create response * Partial fixes for handling bad data * Verify property exists and belongs to org * start of new serializer and controlling extra_data fields for 179d * fix property create endpoint * Formatting * fix org_id on column views * add report_format to PM download so spreadsheet can be downloaded * updating BAE and small tweak since assets are now typed * update BAE processing in reader * updating BAE to v0.1.14 * remove uneeded view * Add ESPM download for single property (via API) (#4158) * add single property download method * add viewset for downloading single espm report * fix BAE migration and update swagger schema * rename pm report single to download * update lokalise notes and pull from latest from website * initial commit with a new update with ESPM function. need to get the backend method finished * enable udpate_with_espm * add test and column mapping profile helper to load in the data * fix example file that was downloaded from espm * fix comment formats * file paths * style espm modal and fix both upload methods * put tmp files into the projects media dir * add column mapping profile param to update_with_espm * put tmp files into the projects media dir * add status to dresponse for pyseed * do not save in media_root * precommit * allow different urls for better and at during testing * return the mapping profile object when creating from file * add tests to ensure read_only fields are protected * Update models.py * Update models.py * - Removed broken `property_id` option - Fixed extra_data columns to only ones that actually apply - Changed `merge_state` from 0 to 1 - Added ValueError handling * Small fixes * Fix 179d ESPM update endpoint to run in foreground with Celery enabled (#4211) * Fixed bae object accessors * create two new methods for running save_raw and map to the foreground * fix type --------- Co-authored-by: Alex Swindler <Alex.Swindler@nrel.gov> --------- Co-authored-by: Nicholas Long <1907354+nllong@users.noreply.github.com> Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> Co-authored-by: Nicholas Long <nicholas.long@nrel.gov> * Fix display name (#4223) * Fix refresh_metadata (#4221) * Fix refresh_metadata * Rename * Bump uwsgi from 2.0.17.1 to 2.0.22 in /requirements (#4209) Bumps [uwsgi](https://github.com/unbit/uwsgi-docs) from 2.0.17.1 to 2.0.22. - [Commits](https://github.com/unbit/uwsgi-docs/commits) --- updated-dependencies: - dependency-name: uwsgi dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> * Remove Beta label from text of analysis button (#4225) remove beta from analysis Co-authored-by: Nicholas Long <1907354+nllong@users.noreply.github.com> * Fix Meters Without `source_id` (#4229) Fix showing meter readings for meters without `source_id` * Fix performance for fetching analyses and force user email to lowercase (#4213) * Set username to lowercase when updating user info * Removed unnecessary prefetch from analyses retrieval --------- Co-authored-by: Hannah Eslinger <heslinge@nrel.gov> Co-authored-by: Nicholas Long <1907354+nllong@users.noreply.github.com> * EEEJ Analysis Functionality (#4208) * eeej analysis * precommit * precommit * add EJScreen * update eeej analysis to link to EJScreen report * adding analysis description for eeej * Fix display name * fix for analysis reruns * cleanup * fix eeej tests * cleanup * fix searching for string in a NoneType * set manual geocoding * fix geocode tests and add census to geocode summary * fix typos * geocode with lat/lng over address if available * add copyright text * small doc formatting cleanup * cleanup * oops, missed this formatting --------- Co-authored-by: Nicholas Long <1907354+nllong@users.noreply.github.com> Co-authored-by: Hannah Eslinger <heslinge@nrel.gov> Co-authored-by: Nicholas Long <nicholas.long@nrel.gov> * Show Analysis `debug_message` If Available (#4198) Show analysis debug_message if available Co-authored-by: Katherine Fleming <2205659+kflemin@users.noreply.github.com> * Use display name in custom reports legend (#4060) * Use display name in custom reports legend * Fix docker compose * Fix Tests * Address comment * Clean --------- Co-authored-by: Katherine Fleming <2205659+kflemin@users.noreply.github.com> * Allow analysis property view related properties to be null (#4227) * update ondelete buildingfile > event > scenarios * update relationhsip to null when a property is deleted * add warning triangle * remove checkbox to delete analyses with properties * precommit * reorder migrations * reset * remove commit from outside branch * add warning to analyses page * styles * Update migrations --------- Co-authored-by: Hannah Eslinger <heslinge@nrel.gov> * Add more API documentation for meter and meter readings (#4237) * add more api doc info on meter and meter reading endpoints * Improved nested documentation, added swagger support for enums, sorted meter readings --------- Co-authored-by: Alex Swindler <Alex.Swindler@nrel.gov> * Update inventory selection to include current inventory type only (#3862) filter selection by current invetory type and return its length Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> * Update default meter range selection for BETTER analysis (#3819) * Show all navs in org * reverse the default better meter selection * Allow analysis to be by range or cycle * html updates to add cycle to BETTER * revert to allow merge * select cycle for better * update docker-dev better api endpoint * modify valid meter data to accept cycle start and end dates * precommit * formatting * remove console and updat better url * merge conflicts * order * default current cycle to eui and better on select_cycle * Simplified --------- Co-authored-by: Hannah Eslinger <heslinge@nrel.gov> Co-authored-by: Alex Chapin <a.n.chapin@gmail.com> Co-authored-by: Ryo <kyle.schulz@devetry.com> Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> * Delete BuildingFiles and Related Events on PropertyState delete (#4217) * update ondelete buildingfile > event > scenarios * precommit * delete scenario if event is deleted * cascade over nothing * Renumber migration --------- Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> * Fixed Issues with Scenario/Measure CRUD Operations (#4240) Fixed issues with scenario/measure CRUD operations, cherry-picked from a640926 * Add a cycle indicator to the analysis modal and filter analyses for current cycle (#3837) * expose current cycle on new analysis modal * display analysis associated with cycle * refactor help text * add cycle indicator to analysis_card * syntax * refactor for clarity * add cycle_name to analysis display tables * add cycle dropdown to inventory detail analyses page * find analyses relavent to current cycle * precommit * formatting * remove debug logs * clarifying note * consistency * Fixed ui-router navigation --------- Co-authored-by: Alex Swindler <Alex.Swindler@nrel.gov> * Disable ability to remove matching criteria fields once inventory is added (#4222) * update ondelete buildingfile > event > scenarios * base for locked matching criteria * refactor for clarity * remove matching preview from frontend * remove matching preview from backend * translations * precommit * remove unrelated migration * revert * update tests --------- Co-authored-by: Hannah Eslinger <heslinge@nrel.gov> Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> * Add smaller EEEJ data files for testing (#4243) add smaller EEEJ data files for testing * Add Uniformat Table (#4238) * Add uniformat * Minor fixes --------- Co-authored-by: Nicholas Long <1907354+nllong@users.noreply.github.com> * Fix Meter Overwrites (#4250) * add test to make sure meter readings do not overwrite * Fixed major bug where inserting a single meter reading would delete any existing readings for any meter with the same start time - Added the ability for inserting bulk meter readings to update existing data - Sanitized sql inputs in `copy_readings` * Simplification * EEEJ test fix * add test to test overwrite of bulk meter import --------- Co-authored-by: Nicholas Long <nicholas.long@nrel.gov> Co-authored-by: Nicholas Long <1907354+nllong@users.noreply.github.com> * Fix filter group dropdown bugs (#4249) Fix for filter group dropdown issues Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> * Add Import Export to Audit Template (#4215) * base for at get buildings * base batch at import functional from frontend button * frontend confirm before upating with at * frontend triggers backend batch update * functional but missing real time progress updates * functional with progress progress is moving, but cant get back to frontend properties updating, progress bar only 50% functional with progress * cleanup * precommit * summary and date formatting * bad credentials guards * test for full at import and property update * precommit * sample buildingsync xml file * swagger documentation * request validation * precommit * comment cleanup * comment cleanup * improve tests and disable update until selection * translate * lokalise translations * function description * update view permissions * clarify error message logic * precommit * add name to at building summary * add at_updated_at to pstate on at import, prevent duplicate uploads * create column on at import to display and compare * include translation * update tests * precommit * precommit * remove translation * build bsync xml from pstate * multiple iterations for xml export * base for batch export to at * add audit_template_report_type to org model * test cleanup * add audit_template_report_type to org model * add at report type to org settings * update batch test * toggle at password visibility * move batch export to celery * export properties to at from inv list actions * backfill org audit_template_report_type * mock tests for export to audit template * phrasing * refactor export results * build out export results table * precommit * revert * precommit * download current lokalise files * add default display field to failed exports * incorporate audit template report type in test * request body to be a dict not list * add translations * update ondelete buildingfile > event > scenarios * remove property name from inventory and expand modal * update states to reflect at_updated * add export to at from inv det actions * precommit * merge develop * migration conflict * Renumber migrations * Small fixes * Privilege enforcement --------- Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> * Fix EEEJ analysis errors when there are no lat/long or address (#4247) * fix eeej analysis errors * add eeej to tasks and fix for non-eager celery * Fix for dict key types changing based on ALWAYS_EAGER * fixes! * Simplification --------- Co-authored-by: Nicholas Long <1907354+nllong@users.noreply.github.com> Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> * Add envvar to specify loading test EEEJ dataset (#4254) * add envvar to specify loading test EEEJ dataset * slight mod to support strings as boolean values --------- Co-authored-by: Nicholas Long <1907354+nllong@users.noreply.github.com> * Default to excluding Address Line 1 as a matching criteria (#4256) * Default to excluding Address Line 1 for matching * [WIP] Fix some tests * update tests for address_line_1 default matching criteria removal --------- Co-authored-by: Nicholas Long <1907354+nllong@users.noreply.github.com> Co-authored-by: Hannah Eslinger <heslinge@nrel.gov> Co-authored-by: Nicholas Long <nicholas.long@nrel.gov> * Bump version (#4259) * bump versions * current change log * add remaining two PRs before release * Refactor map page for functionality and performance (#4260) * Major map/eeej/layout improvements * Added tract shading, database indices * Fix test failure * Release 2.20.0 (#4262) Merge main release commit into develop. * Fix the items in the 2.20.0 changelog (#4263) put the latest PRs in the right place * ArcGIS CSP rule (#4265) Add ArcGIS CSP rule * Add documentation for cleaning up conflicting column names during migration (#4264) * add more documentation around cleaning up columns * Update docs/source/migrations.rst Co-authored-by: Katherine Fleming <2205659+kflemin@users.noreply.github.com> * Update docs/source/migrations.rst Co-authored-by: Katherine Fleming <2205659+kflemin@users.noreply.github.com> --------- Co-authored-by: Katherine Fleming <2205659+kflemin@users.noreply.github.com> * Update changelog for 2.20.0 (#4268) add 2 remaining items for release * Fix edge case for map label filtering (#4270) Fix ids that are used for map filtering * Update changelog (#4272) one more entry --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Alex Swindler <alex.swindler@nrel.gov> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Katherine Fleming <2205659+kflemin@users.noreply.github.com> Co-authored-by: Hannah Eslinger <heslinge@nrel.gov> Co-authored-by: Ross Perry <perryr16@gmail.com> Co-authored-by: ebeers-png <53191386+ebeers-png@users.noreply.github.com> Co-authored-by: Alex Chapin <a.n.chapin@gmail.com> Co-authored-by: Ryo <kyle.schulz@devetry.com>

179d create_property endpoint

1bfc4a9

kflemin added the Feature Add this label to new features. This will be reflected in the change log when generated. label Apr 27, 2023

kflemin requested a review from nllong April 27, 2023 21:48

kflemin added 2 commits April 27, 2023 21:56

precommit

cd7f2dc

adding property id, state id, and view id to create response

81882a6

nllong changed the title ~~179d create_property endpoint~~ Add new create_property endpoint to support 179d May 4, 2023

nllong and others added 3 commits May 4, 2023 07:20

Merge branch 'develop' into 179d-endpoint-updates

3d114b6

Merge branch 'develop' into 179d-endpoint-updates

2bca047

Partial fixes for handling bad data

4dc8e3a

axelstudios requested changes May 22, 2023

View reviewed changes

axelstudios and others added 5 commits May 22, 2023 17:19

Verify property exists and belongs to org

1bb6028

start of new serializer and controlling extra_data fields for 179d

c0a8fc5

fix property create endpoint

0e77875

Formatting

d261f55

Merge branch 'develop' into 179d-endpoint-updates

de634a0

axelstudios self-requested a review June 23, 2023 22:28

kflemin and others added 11 commits June 26, 2023 13:41

fix org_id on column views

a9a6a25

Merge branch '179d-endpoint-updates' of https://github.com/SEED-platf…

1ccb8e1

…orm/seed into 179d-endpoint-updates

add report_format to PM download so spreadsheet can be downloaded

4a0a465

updating BAE and small tweak since assets are now typed

64831be

update BAE processing in reader

f9d3c5e

updating BAE to v0.1.14

884c157

remove uneeded view

a49c6df

Merge remote-tracking branch 'origin/develop' into 179d-endpoint-updates

83b94ce

Add ESPM download for single property (via API) (#4158)

1c9001e

* add single property download method * add viewset for downloading single espm report * fix BAE migration and update swagger schema * rename pm report single to download

update lokalise notes and pull from latest from website

9ec934a

initial commit with a new update with ESPM function. need to get the …

2f92c5e

…backend method finished

nllong force-pushed the 179d-endpoint-updates branch from b9fbfc4 to 2f92c5e Compare July 30, 2023 17:15

enable udpate_with_espm

d610b2e

return the mapping profile object when creating from file

b8b6464

nllong force-pushed the 179d-endpoint-updates branch from e519fe1 to b8b6464 Compare August 2, 2023 19:02

nllong added 2 commits August 4, 2023 07:41

Merge remote-tracking branch 'origin/develop' into 179d-endpoint-updates

59a90fe

add tests to ensure read_only fields are protected

ee8cd8b

nllong reviewed Aug 4, 2023

View reviewed changes

nllong added 2 commits August 4, 2023 09:13

Update models.py

12899a0

Update models.py

6c39998

nllong reviewed Aug 4, 2023

View reviewed changes

nllong approved these changes Aug 4, 2023

View reviewed changes

axelstudios force-pushed the 179d-endpoint-updates branch from 7cabaa3 to 96c00b0 Compare August 11, 2023 23:38

nllong reviewed Aug 12, 2023

View reviewed changes

- Removed broken property_id option

eaaba19

- Fixed extra_data columns to only ones that actually apply - Changed `merge_state` from 0 to 1 - Added ValueError handling

axelstudios force-pushed the 179d-endpoint-updates branch from 96c00b0 to eaaba19 Compare August 15, 2023 06:07

axelstudios approved these changes Aug 15, 2023

View reviewed changes

Small fixes

23dd5c0

nllong and others added 2 commits August 31, 2023 15:40

Fix 179d ESPM update endpoint to run in foreground with Celery enabled (

4b9079f

#4211) * Fixed bae object accessors * create two new methods for running save_raw and map to the foreground * fix type --------- Co-authored-by: Alex Swindler <Alex.Swindler@nrel.gov>

Merge branch 'develop' into 179d-endpoint-updates

68b3133

axelstudios merged commit 7925a85 into develop Sep 6, 2023
7 checks passed

axelstudios deleted the 179d-endpoint-updates branch September 6, 2023 21:13

nllong mentioned this pull request Oct 6, 2023

Release 2.20.0 (again) (#4273) #4275

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add new `create_property` and `update_with_espm` endpoint #4012

Add new `create_property` and `update_with_espm` endpoint #4012

kflemin commented Apr 27, 2023

github-actions bot commented Apr 27, 2023

axelstudios left a comment •

edited

nllong Aug 4, 2023

nllong Aug 4, 2023

nllong Aug 4, 2023

nllong Aug 4, 2023

nllong Aug 4, 2023

nllong Aug 4, 2023

nllong Aug 4, 2023

nllong left a comment

nllong Aug 4, 2023

nllong commented Aug 4, 2023

nllong Aug 12, 2023

axelstudios commented Aug 15, 2023

		@@ -212,6 +212,83 @@ def to_representation(self, data):
		return result


		class PropertyStatePromoteWritableSerializer(serializers.ModelSerializer):

Add new create_property and update_with_espm endpoint #4012

Add new create_property and update_with_espm endpoint #4012

Conversation

kflemin commented Apr 27, 2023

github-actions bot commented Apr 27, 2023

axelstudios left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nllong left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nllong commented Aug 4, 2023

Choose a reason for hiding this comment

axelstudios commented Aug 15, 2023

Add new `create_property` and `update_with_espm` endpoint #4012

Add new `create_property` and `update_with_espm` endpoint #4012

axelstudios left a comment •

edited