-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CityScope Unification/Standardisation #15
Comments
Reiterating from the email chain: To allow for CityScope to be generic and reusable as possible, it's analysis modules must expect always the same values/data. Therefore mapping should occur as part of the onboarding process for new CS (like we're currently facing in Grasbrook, Aalto, and soon in Mexico). This is roughly how I'd imagine this process taking place: |
Hi, Can we should have Markus and Alex in this conversation. Maybe we can have a face to face meeting. |
Here's my proposal (which is similar to Ariel's idea above): Assuming we agree on a list of standard land uses, we also need a number of mapping files. I think we can solve the whole issue with the following text files (examples can be found for Detroit here: https://github.com/CityScope/CS_Mobility_Service/tree/master/scripts/cities/Detroit/mappings): lu_types.json lu_inputs.json lu_input_to_standard_lu.json base_lu_to_lu.json activities_to_lu.json The linked files are based on a “standard” list of land uses which I just made up. But the important point is that if we capture all of these relationships using simple mappings in json files, then it is very easy to update any of this information in the future without changing any line of code (eg. if and when we finally agree on a land use standard) so there is no reason to stall development while we wait for agreement to be reached. |
to me there are two things to build consensus.
1. Block DefinitionOne approach will be having a centralized hub that contains all of the block definition. The structure of that database is another discussion, but having a unique id(hash) for each definition may be handy for modules to lookup in O(1). Human users will also be able to search for block definitions that are close to their needs. Human Planners can make another pretty portal to curate the types they are specifically interested, more like the early Yahoo approach. The city science standard might by one of them. This is similar to separating the 2. Data formatI would As for the format itself, there will be major changes and minor changes. Minor changes are guaranteed to be compatible with the older version, but new modules consuming that data will have interpreted it differently. Adding fields to the JSON file will be a minor change. A Major change is a breaking change, where the modules need to update their parsing mechanism. Deleting a field is a major change(or a hardfork). |
OK, here is my migration from gmail: @ronan, the new profiles are aligned with the goals of the Urban Indicators (RADAR), I think that they emerged nicely from the set of data. Even when, probably a different place/country/city will have a different segmentation and behaviour, I think that this is a fantastic baseline. In my side I have some issues:
I think that the land use and the mapping are the first step for building this new impacts per person. I think that it is looking good and that will help us to correlate some impacts with the behaviour of people. |
@yasushisakai can you please provide an minimal example of each? I'm not 100% sure I get the difference |
Block Definitionblock definition is what kind of information that unit holds.
this can be anything and the JSON format is IMO very hard to standardize. Thus, hash it
this might be one endpoint, for the modules to refer:
this might be for someone looking for similar block, giving you two entries
this might be blessed block definitions that are curated by us.
It's not solving the mapping issue, but it gives the infrastructure to communicate and remix to make new definitions that are compatible with modules. A module maker can "inject" info to the block def. to have a table compatible with their module. You can, if needed link them in many ways. The server can provide statistics which block def is popular, and naturally converge. Data Formatthe Data Format is how it is presented in each table, the same information can be expressed in many different ways.
Note that it doesn't have to relate to the above block definition, where the table self explains what it is. That's why I said these are independent issues. this one is close to what we have now
or an extreme case might be
but as long as we have the version, the makers of the modules can refer to it. API formatThis is something the cityio currently dictates on how to get the data, I guess it's saner considering the above mess. |
It's kinda close (not same) to package managers such as |
Hello: Another conversation will could develop around CityScience Standards, which aggregate information into types we are looking to identify (ie, walkers, bikers, green buildings, and so on). Thus far we identified standards for: Land-Use and Activity - based on the LBCS Schema (see the following link for the directory): https://planning-org-uploaded-media.s3.amazonaws.com/document/LBCS.pdf Economic Activity - based on the NAICS classifications (see the following link for the definitions activity list): https://www.bls.gov/sae/additional-resources/what-is-naics.htm Schema Description: The following metics need further research to come up with a standard: Mobility - ? Energy Consumption - ? Waste Production - ? What do you think? |
@MarkusElKatsha I don't understand what you mean by "General, Type, Activity, Specific Activity". I would say that the number of digit is related to the granularity of the description or subcategory. Also by looking more in detail in this standard, it seems important to understand that LBCS is made of five dimensions: activity, function, structure, site, ownership. Do we want/need to keep this 5 dimensions? If we want to be compatible with the LBCS I would say yes. That means that any building should expose 5 attributes related to each dimension. It means that a potential module might be interested by a specific dimension (CRE might be more interested in the ownership dimension) whereas another module could deal with another dimension. |
Yes, we'll probably have minimal 20 characters (5*4 of type int xxxx) describing only this entity. This taps into what @yasushisakai described in the concept of hashing the unique types in a searchable dictionary. |
I've dug a bit about Germany Land Use for interoperability of the proposed LBCS Schema. I can personally relate to the German way of thinking since it is (not completely but) similar to planning in Japan. (I've been told they adopted circa WWI.) And I think the proposed LBCS schema can be abused to some extent to fit the German way of planning. turns out in Germany "Residence" can be translated to will the new standard let this happen or put this in consideration? |
Results of the 03/30/20 meetings something like this (with slight modification) can be used even if there is still some discussion especially for the physical attribute
|
Example 1, CS Type - Mixed Use: Ground Level Retail (store and bank), Upper Levels Residential
Example 2: CS Type - Singe Use: Housing
|
I agree with differentiating between levels but we can't be explicit about floor numbers (i.e. levels 1-4, 2-6 in the example above) because we don't know how many floors there will be. This is why we use percentages. I think the exception is defining ground floor vs upper floor uses. Below is my proposed edit to the example above where ground floor uses and upper floor uses are defined separately. {"CS_Mixed Use": This way, if I place this type somewhere on the table and set it to 13 floors using the UI, we will end up with a ground floor of LBCS 2140, 9 floors of LBCS 2210 and 3 floors of LBCS 1120. If we put this type in another location and set it to 45 floors, we would have a ground floor of LBCS 2140, 33 floors of LBCS 2210 and 11 floors of LBCS 1120. |
Hi - I see two issues, how we define or each grid and how the definition gets updated through user interaction. I also hope that the schema we pick is flexible enough to allow for course or more granular definition of each grid. In my mind, all attributes could be dynamic, depending on the interaction design. For more complex interactions, users may have the opportunity to define where upper level public spaces - things like rooftop parks/gardens or sky bridges and sky malls (as found in NYC, Tokyo, Singapore and Barcelona.) That being said, if complexity is not needed, the grid tile can be defined as single use (or two uses, Ground Level and Upper Levels as shown above: Example 2: CS Type - Singe Use: Housing "FAR": 3:1
} The second issue will be how we incorporate dynamic vs fixed attributes. Each table/setup/interactive... will have an "interaction area" that need dynamic attributes. Depending on what the users can change, we would build in var attributes. M |
After our last meeting and several conversations with most of you I am convinced that in terms of types, this is how we could operate (At least as a first iteration): 1) Type = Person ID: 2) Dynamic attributes = Exploration (what if scenarios): 2.1 - Basic types (mono use): At this point, I think that playing with "Density" and "color" in the basic types will be enough. This apply either for the meta-grid (Static part of the city) if any, as well as in the Dynamic grid (Users "playground) if any. 3) The modules decide: Then they will talk with the front end, and the front end will visualize it. Keeping that in mind, this are the schema that I think can work for the 2 types: Basic Type: "Basic_CS_type[out of 20]": { Mix use / Crazy Type: "crazy_type": { Methodology for building the types?: A - As a starting point: I suggest to have some 10 to 15 CS "pre-cooked types", both, basic and mix uses as a starting point for the open source tool, testing, and basic interaction. B - New CS Projects: In the kick off of the new projects, the first "workshop" will be focused on defining the "specific types" for that specific place an research question Exactly as we tried to do during the first workshops in Hamburg, in Shanghai, Madrid, Andorra, etc. Where the team spent several nights "building" in Grasshopper and other programs, new types from the students and stakeholders - Now will be muuuch easier! C - Try catch: All the above doesn't mean that we shouldn't think also about having a "try catch" simple CSV file, with some synthetic data, just in case everything crashes. |
I have a few questions/comments after conversations with Arnaud and Luis. The fundamental questions are: Initially, I had assumed that each cell, whether fixed or dynamic, would include all information about that cell (or tile). But after conversations, my understanding is that we need to decide what is the minimum information needed per cell and that all other data would be part of global file for the context. If the above is correct, what data is tile specific and what data is global? BASIC TYPE: Example 1: CS Type - Singe Use: Housing
Example 2, CS Type - Mixed Use: Ground Level Retail (store and bank), Upper Levels Residential
In the above scenarios the Social and Environmental attributes would be part fo the global data file. As Luis mentioned above, we could build a set of "pre cooked" types, then add as needed. M |
A few comments/updates I would make to Markus' last suggestion:
As for what is the minimum information, I would say NAICS, LBCS, color and people_per_floor or people_per_sq. After that, additional attributes could be added depending on what modules are being used and the input data they need. This will be case-by-case and require communication within the project team. As far as I can see, the only significant sticking point is about how we treat different uses on different floors/levels and the answer to this depends mainly on the front-end design. I don't think it's realistic (or desirable) to have a front-end that can edit multiple LU percentages on every floor of a cell. I would say that the FE should only be adjusting the number of street level floors and the number of total floors (the way the FE is now) and the mix of LUs on each level should be encoded in the type. See my last post for type example. @RELNO do you disagree with this? |
FAR is a tricky piece of data which cannot easily be adopted to many of our case studies. I suggest having FAR as optional indicator (like noise or water).
There are ways (though complex) of multi coloring elements. However, for brevity we should keep the color per main usage. So if the building/parcel is mostly housing, it will follow housing color. A click on this object will show the breakdown of the different usages. |
I created the examples for Corktown and Grasbrook so we can now work on the implementation: The contents of the types will be subject to change (especially GB as I just made very basic definitions) but we can still start updating the code and testing the modules etc. If the contents of the types change later, everything should still work without any code updates. |
City Science TypesLet me share the first draft of the City Science Types. They are builded by following the following Json structure, agreed in previous talks and issues: …Here are the proposal for the City Science Types | CS Amenities1 | Hotels 2 | Restaurants 3 | Night live 4 | Leisure and Wellness 4.1 – Pharmacies 4.2 –Dentist 4.3 –Ophthalmologists 4.4 –gyms 4.5 –Spas 4.6 –Clinics 4.7 –Daycare 4.8 –Hospital 4.9 –Nursery 5 | Culture 5.1 –Libraries 5.2 –Museums 5.3 –Cinemas 5.4 –Art-galleries 6 | Fashion Shops 7 | Luxury shops 7.1 –Jewelries 7.2 – Perfumeries 7.3 – Tobacco 8 | Shopping Centers 8.1 – Mall 8.2 – Shopping Centers 9 | Technology shops 9.1 – Vehicle/car sales 9.2 – Electronics 9.3 –Hardware 9.4 – Computer 10 | Super Markets 10.1 – Super Market 10.2 – Convenience 10.3 – Food /Markets 11 | Banks 12 | Educational 12.1 – Universities 12.2 – Schools 12.3 –Profesional training 13 | Post offices 14 | Working places 15 | Parks 16 | Public Transport 16.1 – Bus 16.2 – Taxi 16.3 – Air (Airports “+”) 16.4 – Rail 16.5 – Boat 17 | Police and security 18 | Healthy food shops RESIDENTIAL19 | Household activities (Residential activities) 20 | Transient living (Residential activities) 21 | Institutional living (Residential activities) 22 | Office activities 22.1 – Public Administration 22.2 – Finance 22.3 – Real Estate 22.4 – Scientific and Technical 22.5 – Management 22.6 –Health Care 23 | Industry 23.1 – Mining 23.2 –Manufacturing 23.3 –Wholesale 23.4 – Accommodation and Food 24 | Nature This can be the basic City Science Library. It is big, however, covers a big amount of the urban uses. Please, let me know your thoughts |
@LAAP thanks, two notes:
|
Good job @LAAP I also have some question/comment:
|
Thanks @LAAP, this is great.
|
Thank to @ALL, I totally agree. this was a first iteration trying to cover all the granularity of the NAICS, and maybe useful for @Cristian. Maybe his module can take the basic type and reed the granularity in his side. So, my next steps are:
Please, let me know your thoughts |
| CS Classification Amenities Dear @ALL, Here I share a second iteration. I have reduce the number and complexity of the list. However, we still have a lot of "symple" Classes (29): 3 Residential 29 Total It can be reduced into 20 Classes After talking to @MarkusElKatsha we have agreed on the following structure for the Class: A) Static attributes: "LBCS", "color", "NAICS" Example of the structure:
IMPORTANT NOTE: A type will include one or multiple CS Classifications depending on the context and project needs RESIDENTIAL 1 | Household activities
2 | Transient living
3 | Institutional living
AMENITIES 4 | Hotels
5 | Restaurants
6 | Night live
7 | Leisure and Wellness
8 | Culture
9 | Fashion Shops
10 | Luxury shops
11 | Shopping Centers
12 | Super Markets
13 | Banks
14 | Educational
PARK 15 | Parks
TRANSPORTATION 16 | Transportation
SAFETY AND SECURITY 17 | Public Safety
WELLBEING 18 | Healthy food shops
19 | Health Care
OFFICE 20 | Public Administration
21 | Finance
22 | Real Estate
23 | Scientific and Technical
24 | Management
INDUSTRY 25 | Mining
26 | Manufacturing
27 | Wholesale
28 | Accommodation and Food
NATURE 29 | Nature
Also, here is the document in Word 200501_City Science Types V01.docx Please, let me know your thoughts and ideas |
@LAAP great job! I would be in favor if it possible to reduce it to 20 (I think we can already do a lot) and it will be already challenging to integrate all of them and also easier to maintain with fewer |
Thank you @agrignard , Please, @RELNO, @doorleyr , @MarkusElKatsha , let me know if I should reduce the classes into 20 as @agrignard suggested or Keep it as it is now in 29 |
@LAAP I'm agnostic to how many, the goal is to have the minimal number of types to start a simple table, and a baseline to extend it later.
I might missed something, but what does |
@agrignard and @RELNO , Thanks for the comments. Ok, this is my plan:
Question to @RELNO : What action is required to answer this: "//dynamic attr is not part of a json, it was a leftover from past discussion"? Remove the text? Answer to @RELNO : Talking to Markus, we realize that economic is a missing parameter: High income, low income, etc.... The same LBSC+NICS + same m2 element is different if it is in a high income community or in a lower one.... Where are we going to get this info/data? No idea yet |
yes
I think we should be a bit more thorough here: adding economic indicator is something I'd want a module to do, not engrave it in the type itself. Also, If we go with economy, why not energy, or any other indicator? |
@RELNO , Ok. Thanks. I will Change that too. Regarding economics. Yes, a module can do the job. Same thing could happen with "density" of people and "height", however we have decided to put it in the dynamic attributes, isn't? I think that we can change the name of "economic" for other less "institutionalized" maybe "Worth" (?) |
Thanks @LAAP. A few comments below:
|
Thank you @doorleyr ,
3)_"Height and capacity are determined by physical design in a more direct way so it makes sense for the user to specify them directly"... So "Height" is also predetermined. "height_m" (3m is the normal height for indoor space. It can be the "fixed" number
|
Hi all, |
| CS Classes Dear @ALL, Here I share a new iteration with the latest changes, please, let me know your thoughts "areapp_sqm"= m2 per person. Source = https://www.engineeringtoolbox.com/number-persons-buildings-d_118.html "monetary$"_ = ranking from “0” to “1”. “0” low value asset “1” high value asset. Source = Educated guest "floor#"_ = Where the land use is likely to be placed. “1” most likely at ground level, “2” most likely above ground level, “3” indifferent, both options are open. Source = Educated guest HOUSING1 | Household activities (Residential activities)
2 | Transient living (Residential activities)
3 | Institutional living (Residential activities)
AMENITIES4 | Hotels
5 | Restaurants
6 | Night live
7 | Leisure and Wellness
8 | Culture
9 | Shopping Centers
10 | Banks
11 | Educational
PARK12 | Parks
TRANSPORTATION13 | Transportation
SAFETY AND SECURITY14 | Public Safety
WELLBEING15 | Health Care
OFFICE16 | Public Administration
17 | Finance
18 | Scientific and Technical
INDUSTRY19 | Manufacturing
NATURE20 | Nature
|
@LAAP thanks -- I'm confused as to why we went back to have arbitrary fields like "capacity_sqm", "monetary_$", "floor_#"? Especially after the significant effort put in finding citable, widely used standards like LBCS. (just as a side note, using ascii terms like "#" in a json is allowed, but has better chance to break when parsing given) |
@RELNO , This is justa a new iteration. I think that "capacity_sqm", "monetary_$", "floor_#" are fields that needs to be "Dynamic". However, If I am understanding correctly the needs of the team, I think that some people in the team, like @doorleyr et al. are asking about having "values" in this fields that provides information about where in the type this Class will be allocated (1st floor, 2nd floor...), How many m2 per person does the class has, and short of a "economic" element. With all that elements, I have made an iteration with "fixed weights" that a "future type-module" can just interpret and translate into his own understandable dynamic variables, but also, a more basic toy-model-table, can just use as a "placeholder" data for running. This placeholder data have a bit of sense, so this numbers are not totally "arbitrary", they are base in a lot of simplifications, average values, and assumptions, but I think that are ok for a toy model. I suggest a Zoom meeting between all of us |
yes, might worth having a call on this. There's a difference between dynamic props (that could be anything, and will be added to |
I totally agree with you @RELNO , and that was my first approach, and that is why I leave the However I understood that @doorleyr suggested, some how, to have a fixed numbers. Maybe it is a combination of both: we can have "capacity_sqm", "monetary_S", "floor_n" as dynamic props, and in parallel, we can have a "static-default library" with this "educated guest values" that, if the "type-module" doesn't work, or if the team implementing the models doesn't want to define their own type, they can use it. |
LBCS (SLUCM) to NAICS conversion@LAAP @MarkusElKatsha: |
Thank you @RELNO very much for sharing, Yes, we were aware of this correlation, and it has been used for defining the "simplified 20 classes". The mamut task was to "consolidate all this hundreds of "land uses" into only "20 classes". In any case, we will put this links in the documentation, so people doing a more "granular" correlation and research, can have all the correlations. Saying so, @MarkusElKatsha and I have a new proposal for "simplifying" the classes. I will explained above: |
After a eficient talk with @MarkusElKatsha , we have got in to the following conclusions:
So the class could look similar to:
Please, let us know your thoughts |
considering https://cityscope.media.mit.edu/docs/modules/CityScope%20Types%20System it seems this issue can be closed. Specific unification issues can be opened as needed. Thanks @MarkusElKatsha @LAAP for the effort! |
Great effort everyone, one of the biggest issue ever open (on Nov 2019 ;-)) and fixed The doc looks really great, good job! Next step will be to migrate existing projet or starts new one with this standardisation. Are grassbrook and corktown already using it? Happy to help for any specific part (for instance having a toy ABM structure running with this specification) |
Thank you @agrignard and @RELNO , I think that @MarkusElKatsha and I are ready to keep documenting guys... and I see some conversations going on with @crisjf regarding wikies' work... So let us know if you need us somewhere else. |
This is an attempt to have a common place to discuss on the unification task. Nothing is set in stone but at least let's try to converge on a common place to discuss.
What we have so far (not pretending to have an exhaustive view so feel free to complete/comment)
So here the current entry point that should be used to unify everything and that should be included in the wiki if everyone agree:
wiki https://github.com/CityScope/cityscope.github.io/wiki/CityScope-Data-Format-version-2.0 (it has been moved from the cityIO repo to here). It is pretty weak for now but this is what we have (this is more or less the RS, RM, RL OS,OM,OL, etc) and it will hopefully quickly evolve as a synthesise of this issue in the following days
Other non github reference (for non github user), is the paper an overleaf initiated by @LAAP that should be also used as a reference
https://www.overleaf.com/project/5dc19bd7180565000164bd25
Concerning cityIO
https://github.com/CityScope/CS_CityIO/wiki/cityIO-Data-Structure
Concerning ABM Standard:
https://github.com/CityScope/CS_Simulation_GAMA/wiki/GIS-Structure here the documentation of the input that a volpe like model is requiring to run
https://github.com/CityScope/CS_Simulation_GAMA/wiki/Game-IT for what we call the Lyon Model
Concerning Macro Mobility
https://github.com/CityScope/CS_Mobility_Service there is some standard here used to drive the ABM micro mobility model (both macro and micro) we can think on how to migrate it to this repo or at least make a link
Current github issues related to this topic:
MicroFromMacro: Agents to have sequence of activities from '/od' field on cityIO CS_Simulation_GAMA#66
Cell Attribute in Full Grid and Interactive grid CS_cityscopeJS#5
I am sure there is other entry point so please feel free to complete it.
The text was updated successfully, but these errors were encountered: