New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Visualising Local Autocorrelation #8

Merged
merged 44 commits into from Jun 8, 2018

Conversation

Projects
3 participants
@slumnitz
Copy link
Member

slumnitz commented May 25, 2018

creating a three plot visualisation for spatial autocorrelation

  • adding Matplotlib and Bokeh versions of functionality in .py format
  • adding utility functionality in .py format
  • adding 4 example notebooks one for each of two datasets (Guerry, Columbus) for each Bokeh and mpl functionality
  • updating mplot.py for implementation in three-plot visualisations
  • creating mpl.py and bokeh.py files for implementation of import splot.namespace

slumnitz added some commits May 16, 2018

add visualization option with significance to mplot
accept both moran and moran_local as inputs
and adapt test for both versions
added prototype for three plot moran, lisa, choropleth visualization
added example in notebook
updated connected functionality
include region masking in visualization
bug fix in three-plot-visualization
legend added to LISA cluster map (interactive)
started use of geoJSONDataSource for Bokeh plots
create static three-plot visualization in Bokeh
also initialize input as GeoJSONDataSource
rewrite utility functions
added coloring option for masking
move mask behind datapoint
clean notebooks, add documentation and tests
clean .py files with pyflakes
three-plot notebook cleanup
finalize notebooks for MPL and Bokeh three-plot implementation
add notebooks testing on Columbus dataset
clean naming of functions in .py files
add files working towards `splot.namespace` API
@slumnitz

This comment has been minimized.

Copy link
Member Author

slumnitz commented May 25, 2018

@sjsrey @darribas @ljwolf @TaylorOshan @jorisvandenbossche

Hi everyone, here the first official splot GSoC pull request!

Still To-Do:

  • call Guerry data from the libpysal examples
  • add interactivity in Bokeh (hover-tool)
  • autoscale figures in Bokeh
  • separate functionality (data handling and figure plotting) in Bokeh to allow for interactive connection between the three plots
  • add examples
  • legend positioning/ hover
  • color of underlay mixed up between two maps

slumnitz added some commits May 26, 2018

fixed bug in mpl.plot_local_autocorrelation
masking option of region is now visible together with quadrant masking
fixed coloration bug in bokeh.plot_local_autocorrelation
changed scaling of single visualizations in bokeh.plot_local_autocorrelation
Try to fix the geopandas install (dependencies were missing)
Also fix two imports in test files
the geopandas version labelled as dev on conda-forge is not recent en…
…ough

install directly from github instead with pip.
@slumnitz

This comment has been minimized.

Copy link
Member Author

slumnitz commented May 26, 2018

it was not so easy to make TravisCI run all the tests, but it's finally green:)

slumnitz added some commits May 28, 2018

add hover tool and connect datasource for three-plot visualization
change mplot to use ColumnDataSource
for three plot make everything come from the same geoJSONDataSource
therefore split functionality in two parts, data source creation and figure creation

@slumnitz slumnitz changed the title [WIP] Visualising Local Autocorrelation Visualising Local Autocorrelation May 29, 2018

slumnitz added some commits May 29, 2018

position legend next to map
therefore, change add_legend() in  _viz_utils.py
fix colours displayed when non-selected in tap tool
add outline around selected polygons in tap tool for clarity
add calc_data_aspect() to `_viz_utils.py` to scale data according to …
…different figure ratios

implement calc_data_aspect() in map visualizations and add x and y_ranges
place legend below maps for nicer visualization
change sizing_mode of three-plot to `scale-width`
and changed title of three-plot visualization to "Local Spatial Autocorrelation"
- pip install libpysal
# Pysal still needed for code not yet moved to libpysal
- pip install pysal
- pip install esda
- pip install mapclassify
- pip install -r requirements_dev.txt
- pip install .
# Now install splot (don't use 'pip install .', we'll run out of space)

This comment has been minimized.

@ljwolf

ljwolf May 30, 2018

Member

weird... that happens on travis?

This comment has been minimized.

@slumnitz

slumnitz May 30, 2018

Author Member

this was in /tmp/...; it seems pip copies everything in the folder, which includes the miniconda environment.

>>> show(fig)
'''
# We're adding columns, do that on a copy rather than on the users' input
df = df.copy()

This comment has been minimized.

@ljwolf

ljwolf May 30, 2018

Member

I thought from our last conversation, we agreed that df.assign(...) would make a copy already? In a few cases, you're making more than one new column, so keep this copying as is I guess.

But, if I recall correctly, @jorisvandenbossche was suggesting like... for the column name-style argument in geodataframe.plot('colname', **options), you could pass geodataframe.plot(numpy_vector, **options) to plot without assign, or copy?

This comment has been minimized.

@slumnitz

slumnitz May 30, 2018

Author Member

Ok, yep for the _viz_mpl.py file I am using the df.assign() already. My thoughts were, too, that df.assign might not work if I have to add multiple columns which is the case for the three-plot visualisations.

However, I am not using the .plot() functionality in _viz_bokeh.py at all and was wondering if you guys would have any suggestions how to handle adding columns here, besides using "df.copy()". I am happy to also just leave this as it is for now and add a note for future improvements.

@ljwolf

This comment has been minimized.

Copy link
Member

ljwolf commented May 30, 2018

@jorisvandenbossche
Copy link

jorisvandenbossche left a comment

Didn't look at everything in detail, but added a couple of comments

k=2, cmap=hmap, linewidth=0.1, ax=ax,
edgecolor='white', legend=legend, legend_kwds=legend_kwds)
ax.set_axis_off()
return fig

This comment has been minimized.

@jorisvandenbossche

jorisvandenbossche May 30, 2018

I personally think it can be more useful to return the ax, because often you need the ax to further modify the figure (although of course some things like saving the figure rather needs the fig object ..)
This is also what geopandas does (and I think eg also seaborn)

This comment has been minimized.

@slumnitz

slumnitz May 31, 2018

Author Member

I changed the returns for lisa_cluster() to fig, ax. I think this should be part of the discussion about the splot API, so I added this here: #9

----------
moran_loc : esda.moran.Moran_Local instance
Values of Moran's Local Autocorrelation Statistic
df : geopandas dataframe instance, optional

This comment has been minimized.

@jorisvandenbossche

jorisvandenbossche May 30, 2018

currently it is not "optional" because it is a positional argument in the function signature, so you actually need to pass one

This comment has been minimized.

@slumnitz

slumnitz May 31, 2018

Author Member

ok, this was changed df is not an "optional' input

angle=mask_angles[quadrant],
color='grey', zorder=-1, alpha=0.8))
# quadrant selection in maps
non_quadrant = ~(moran_loc.q==quadrant)

This comment has been minimized.

@jorisvandenbossche

jorisvandenbossche May 30, 2018

pep8: spaced around ==

This comment has been minimized.

@slumnitz

slumnitz May 31, 2018

Author Member

all pep8 issues are fixed

# quadrant selection in maps
non_quadrant = ~(moran_loc.q==quadrant)
mask_quadrant = df[non_quadrant]
df_quadrant = df.iloc[moran_loc.q == quadrant]

This comment has been minimized.

@jorisvandenbossche

jorisvandenbossche May 30, 2018

you can reuse the mask you created above?

This comment has been minimized.

@slumnitz

slumnitz May 31, 2018

Author Member

Yes, thanks that works!



__all__ = ['mplot']


def mplot(m, xlabel='', ylabel='', title='', custom=(7,7)):
def mplot(m, xlabel='', ylabel='', title='', figsize=(7,7), p=None, ax=None,

This comment has been minimized.

@jorisvandenbossche

jorisvandenbossche May 30, 2018

I don't know to what extent you are bound to historical names, but for clarity and for consistency with the other functions, I would personally rather call this something like plot_moran

This comment has been minimized.

@jorisvandenbossche

jorisvandenbossche May 30, 2018

Also, should this be moved to splot.mpl ? (or at least also exposed there, so people can import it from there)

This comment has been minimized.

@slumnitz

slumnitz May 31, 2018

Author Member

Good point, I also added this to the API discussion: #9

pep8 fixes for `_viz_mpl.py` and `_viz_utils.py`
changed parameter description for `.mpl` `lisa_cluster`, df is not an optional input
deleted `df['labels'] = labels in `_viz_utils.py` to not change the input df
added `df.assign()` wherever one column needed to be added and df.copy() where multiple columns were added
added `ax` as return for `.mpl` `lisa_cluster`

@slumnitz slumnitz added this to In progress in GSoC 2018 May 31, 2018

slumnitz added some commits May 31, 2018

integrate `plot.py` into `_viz_mpl.py`
and change `mplot` name to `moran_scatterplot` in mpl and bokeh version
add tests for `.mpl` `moran_scatterplot`
change default colors for bokeh and mpl plots
create `moran_hot_cold_spot()` as common utility function
 Please enter the commit message for your changes. Lines starting

@ljwolf ljwolf merged commit 8aa0cbf into pysal:master Jun 8, 2018

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

GSoC 2018 automation moved this from In progress to Done Jun 8, 2018

@slumnitz slumnitz deleted the slumnitz:three-plot branch Jun 20, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment