Developer model: new workflow

Approved new workflow for developer model

This is a high-level design proposal meant for discussion and review. I've tagged this and other such issues with the label Type: Proposal. These are not meant to be treated as user stories or handled in the GitHub project workflow. Everything in here needs to be broken down into a user story for development.

Stories

• Parcels should be able to be split into smaller developable entities (let's call them "development sites") based on rules or user input
• Parcels should be able to be aggregated into larger development sites based on rules or user input
• If a parcel is split or aggregated, AND one of the resulting sites is developed, that site and the other sites resulting from the split/aggregation should be "remembered" in the next simulation year so as not to attempt a new, incompatible split or aggregation.
• It should be possible to manage one or multiple development sites as a project (let's call it a "development project") that should be analyzed as a group
• It should be possible to perform feasibility analysis on a development project (group of development sites)
• It should be possible to make development decisions on a development project
• There should exist a "pipeline" of projects that are ready for development
• Projects in the pipeline should be able to be "built" - added to the list of buildings that exist in the model - in the current simulation year, or scheduled with an expected completion date in the future
• Projects in the pipeline should be able to make progress on construction in a given year, without completion
• Projects in the pipeline should be able to be delayed or canceled based on certain conditions
• Individual development sites within projects should get "built" from the pipeline.

Summary

See "proposed workflow" below for details.

• A new Project Creator step generates development projects (via splitting, aggregation, or most likely a one-to-one match) from parcels.
• A new Pipeline Adder step appends results of the developer model to the pipeline table, which persists across model years.
• A new Constructor step builds projects from the pipeline table, site-by-site.

Current workflow

Here I'll try to summarize the current workflow that utilizes the proforma and developer models. Feel free to skip to the proposed workflow below. I'm using the model names from the urbansim_parcels repo's models.py.

1: Preliminary models

• The residential and non-residential hedonic models estimate land prices in the region for different uses
• The household and job location choice models move a subset of agents around the region
• The transition models add households and jobs to the region based on control totals

2: Feasibility

Generates a table of parcels with some metrics about development feasibility, like profitability.

• Inputs: parcels table
• Outputs: Table with development metrics (e.g. maximum profit), indexed by parcel_id

In this model, each parcel represents the possible development (or redevelopment) of one building, and each parcel's feasibility is analyzed as such.

3: Developer

Decides which buildings get developed, based on the outputs from the feasibility step.

• Inputs: feasibility, some demand-side parameters to determine how many units to build
• Outputs: DataFrame of buildings that are built in this year

Proposed workflow

Here are all of the tables involved:

• Parcels
• Buildings
• Development sites: can be subset or superset of parcels. Empty in first year.
• Development projects: comprised of one or more development sites. Empty in first year.
• Pipeline: comprised of development projects that are scheduled to be built
• Feasibility, Project Feasibility: created and deleted each year between the Project Creator/Feasiblity and Developer steps

1: Preliminary models, including scheduled developments, which are added directly into the pipeline.

1.1: Project Creator

A new ProjectCreator model takes in the parcel table and pipeline (empty in year 1), and generates development projects and development sites.

• Inputs: parcels, pipeline, configs
• Outputs: development_projects, development_sites, project_feasibility

The project creator performs these steps:

• Read in parcels table into a parcel_candidates table
• Read in pipeline table; remove parcels involved in an active pipeline project from parcel_candidates
• For user-configured projects:
  • Read in user-configured projects: these would be rules for how to split/aggregate/pass specific parcels into development sites, then combine development sites into a development project.
  • Create development project (row in a table that includes reference to parcels and development sites)
  • Create development sites (rows that include reference to development project and parcels)
  • Append to the development_projects and development_sites tables
  • Run ProjectProForma model (below) and append results to project_feasibility
  • Remove parcels involved in user-configured projects from parcel_candidates
• For everything else:
  • Run the rest of parcel_candidates through the Splitter function
  • If parcel is split:
    • Create development project and development sites
    • Append to development_projects and development_sites
    • ProjectProForma has been run. Append results to project_feasibility
  • If parcel is aggregated:
    • Need to handle this use case with an Aggregator function.
  • If parcel is not split or aggregated:
    • Create one development project, comprised of one development site
    • Append to development_projects and development_sites

The Splitter function should take in parcels and if they qualify under some criteria (probably it's larger than a maximum parcel size), run an algorithm to determine the "best" split. One possible algorithm is:

• Based on certain settings, generate a list of possible splits. This is a combination of numbers (e.g. split the parcel into X development sites) and uses. Most common use case will be splitting a large agricultural parcel into many (potentially hundreds) of single family homes. Need to consider computation time here.
• For each possible combination, run ProjectProForma.
• Compare profitability for each combination and select the best one.
• For the best combination, save the results of ProjectProForma so it can be appended to project_feasibility

The ProjectProForma model takes a development project as an input, runs sqftproforma on every development site in the project, and aggregates results up to the project level. This requires some thoughtfulness about aggregation rules (should we just add up max_profit?).

2: Feasibility

Runs sqftproforma model for projects with only one development site, and combines results with the project_feasibility table from the previous step.

• Inputs: development_projects, project_feasibility, configs
• Outputs: feasibility

Unlike the existing workflow, this step performs analysis on development sites as units, rather than parcels. Steps are:

• Read in development_projects into projects_to_analyze
• Remove projects in project_feasilibility (precalculated from previous step) from projects_to_analyze
• Run sqftproforma as usual on projects_to_analyze, output feasibility
• Append project_feasibility to feasibility

3: Developer

Decide which development projects get pushed into the pipeline, based on the outputs from the feasibility step.

• Inputs: feasibility, some demand-side parameters to determine how many units to build, configs
• Outputs: Index of projects to send to pipeline

There is not much change in the Developer model itself, except use development sites instead of parcels. It does this:

• Read in feasibility
• Do some filtering out of unprofitable projects and such
• Assign every project a probability based on rules from specific implementation
• Based on target numbers, randomly select a certain number of projects based on probability and output, output index of projects to build

3.1: Append to pipeline

A new PipelineAdder step should append development sites to the pipeline table. This may be a function we simply want the developer model to handle, but there may end up being some more complicated rules regarding how we add to the pipeline.

• Inputs: Index of projects from Developer step, pipeline, development_projects, development_sites
• Outputs: pipeline

Steps:

• Read inputs
• For each new project to add to the pipeline, calculate the following information:
  • Construction time (how long does it take to build), expected completion year
  • Velocity (derivation of above)
  • Some probability of completion, or of continued construction in the current simulation year
• Output new pipeline

4: Build from the pipeline

A new Constructor model "constructs" development sites in development projects in the pipeline. These are then added to the buildings table.

• Inputs: pipeline, buildings
• Outputs: pipeline, buildings

At this point, pipeline is a list of development projects, and each row has information on the status of development sites within the project. Steps:

• Read inputs
• For each project in pipeline:
  • If completed this year:
    • Add any remaining development site(s) to buildings
    • Remove from pipeline
  • Else:
    • Based on velocity, figure out if one of the development sites should be built this year
    • Add those development sites to buildings
    • Mark those development sites as built in the pipeline

We can add in logic to delay or cancel projects as well. pipeline and buildings persist across simulation years. Everything else is reset.