New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
working moran plot func #797
Conversation
fig1 = plt.figure(figsize=(5,5)) | ||
|
||
plt.scatter(y_std, yl_std, s=60, color='k', alpha=.6) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We might want to add a linear (and potentially non-linear) fit line. Also, maybe we could add a legend for that line that includes the coefficient for Moran's I (as it's its slope)?
At the very least, I'd add the linear fit and maybe introduce an option for a LOESS line, although that'd make us depend on statsmodels
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Definitely, the idea with this first pass is just to test the waters and
see what would work! It's still in progress -- you can see my notebook
kicking around some ideas and configs here
https://github.com/Makosak/PySalViz/blob/master/Untitled%20Folder/Scatter%20Plots%20-%20Moran.ipynb.
so:
- Module name: renaming to 'canned' as a master would be fine -- i'll
adjust that for the next round. (or 'cooked' ? ;P i guess there's time to
consider options.) - *Default style: *yup, the default style is not terribly pretty in
matplotlibs. I was trying to keep it to matplotlib for now following the
viz project specs, and to reduce dependencies, but would it make sense to
have a parallel style in seaborn? shouldn't be too difficult, and would add
some options.. (Also, check out the 4-color matplotlib quadrant figure in
the notebook above...) - *kwargs: *these I was reserving for title='', xlabel='', ylabel='',
and figsize() following matplotlib logic -- see comments in the notebook
link above. I may be totally misusing the term 'kwargs' here so apologies;
still green.. I haven't added them yet in this version but just to get
something out! These would be optional, but a way to enhance the visual if
you want it. Maybe we could connect seaborn through that, for a more
advanced user? like seaborn=false will give you a matplotlib default
object, but if seaborn=true then we assume seaborn has been added as a
dependency, and you get a nicer seaborn visual? - *m as input: *Serge had just recommended that too, as it also gets
around issues of weight transformations. working on this tonight for the
next iteration. - *adding lines: *Also working on that, for next iteration; probably
envelopes too? or envelopes=true as an optional input to add them?
On Sun, Apr 24, 2016 at 3:26 AM, Dani Arribas-Bel notifications@github.com
wrote:
In pysal/contrib/viz/plot.py
#797 (comment):
- var : array
values of variable
- w : array
values of spatial weight
- '''
- w.transform = 'r'
- slag = ps.lag_spatial(w, var)
Z-Score standardisation
- y_std = (var - var.mean())/var.std()
- yl_std = (slag - slag.mean())/slag.std()
- fig1 = plt.figure(figsize=(5,5))
- plt.scatter(y_std, yl_std, s=60, color='k', alpha=.6)
We might want to add a linear (and potentially non-linear) fit line. Also,
maybe we could add a legend for that line that includes the coefficient for
Moran's I (as it's its slope)?At the very least, I'd add the linear fit and maybe introduce an option
for a LOESS line, although that'd make us depend on statsmodels.—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/pysal/pysal/pull/797/files/2fb3d3b7eaa9d6bb2032f7393d54f04478937eb6#r60842378
This looks great @Makosak! I think it's a good example to kick the tires on the "canned views" project. I've made a few inline comments for us to think about before I merge. A couple of additional thoughts:
@Makosak @sjsrey what do you think? I'd like to use this example to streamline our approach to canned views so we create a more or less template that we can all follow later when adding new ones. |
Updated code with arguments for customizing and notes for what's needed next. Can't see Dani's comments inline -- feel free to add! |
@Makosak The inline comments are linked in this issue, they're comments on the diffs. |
Closing -- better version coming, will open a new pull. |
Updates made: added best fit line; matched zx and zy scores to match Moran function elsewhere in PySal. Long-term may still want to add more flexibility to plot your own Moran, instead of just a 'var' and 'w' input. For now, concerns that slightly different user-defined variable names of Moran would not work in a modular setting. (May want to add "m" as another input, that if entered, takes precedence?) |
I think this is a nice implementation that moves the idea of canned views forward. So I'm +1 on the merge. |
values of variable | ||
w : array | ||
values of spatial weight | ||
w : spatial weight | ||
''' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be nice to add a Returns
section explaining what the output is?
I'd vote for the function to take in a
See my inline comment on it above. |
Yes, I also like this concept but don't know how to accomplish it. Looking On Thu, May 19, 2016 at 6:59 AM, Dani Arribas-Bel notifications@github.com
|
With the plot, would you mean just plotting the x and y with a line of fit, On Thu, May 19, 2016 at 7:20 AM, Marynia Kolak marynia.kolak@gmail.com
|
Here's a quick and dirty example, modifying your original function: def plot_moran(m, xlabel='', ylabel='', title='', custom=(5,5)):
m.w.transform = 'r'
zx = m.z
zy = ps.lag_spatial(w, m.z)
fit = ps.spreg.OLS(zx[:, None], zy[:,None])
## Customize plot
fig1 = plt.figure(figsize=custom)
plt.xlabel(xlabel, fontsize=20)
plt.ylabel(ylabel, fontsize=20)
plt.suptitle(title, fontsize=30)
plt.scatter(zx, zy, s=60, color='k', alpha=.6)
plt.plot(zy, fit.predy, color='r')
plt.axvline(0, alpha=0.5)
plt.axhline(0, alpha=0.5)
plt.show()
return None
Quick setup: %matplotlib inline
import matplotlib.pyplot as plt
import pysal as ps
from pysal.contrib.pdutilities import file_utilities as psio
link = ps.examples.get_path('columbus.shp')
db = psio.read_files(link)
y = db['HOVAL'].values
w = ps.queen_from_shapefile(link)
w.transform = 'R'
m = ps.Moran(y, w) Then the implementation above allows you to just say: plot_moran(m) Does this make sense? Happy to elaborate. ]d[ |
Also, being biased by my work with Rob Pahle, I vote for separating viz and On Thu, May 19, 2016 at 7:24 AM, Marynia Kolak marynia.kolak@gmail.com
|
Aha! Got it -- that makes sense. I'll review and push... Thanks!! On Thu, May 19, 2016 at 7:49 AM, Dani Arribas-Bel notifications@github.com
|
Yes but, actually, if anything, your original implementation was coupling analytics and vis more than my proposed one because you were forced to standardize, etc. within the vis function. Effectively, we'd like as little of that as possible. The idea I think is that the vis methods would do as much as possible only styling and display and I think that can be attained by passing the output of the analytics bit, which is calculated in |
This is a great thread, fleshing out some key design issues. I'm thinking at some point we want to pull this info back into the design wiki so we don't loose these insights once the pr is merged. |
Totally agreed. I think some of these have actually become clearer in my mind while reading and writing responses. Definitely we should recycle some of this into a more visible part of the project! |
Just want to throw out a link to a similar implementation where I return an axis object (if one is not provided). If the axes object is passed / returned in, it is possible to leverage a lot of the existing MPL functionality in terms of subplots, etc. Since you have the axes object, you do not need to handle any of the MPL arguments ( The function also takes and passes all of the MPL args and kwargs. That means that you do not need to expose things like, This gets monkey patched onto the like object then, so |
Addendum: I should note that this does have a limitation - since the scatter object is not returned, doing a colorbar is proving painful. I might also return a tuple of all of the plot object. Here is the StackOverflow question where I am trying to determine how best to get access to the MPL data required for a colorbar. |
On a related thread, if we pass in analytic objects as arguments to these plot functions, it would be great if the objects themselves carried as much information as possible that could be accessed by the plot function. For example in this PR the idea of having a common approach to the repr for the analytic would provide the inspiration for things like variable labels. So for example, rather than 'x' we have something like 'income', if we extend the analytical signatures to take optional variable names ala spreg. |
That's a great point @jlaura. Something I do on my research code is to have Regarding the colorbar, that's right, I find it tricky in |
If we could pull more information from the analytic objects that are used On Thu, May 19, 2016 at 9:00 AM, Dani Arribas-Bel notifications@github.com
|
updates made: parameter and return descriptions; accessing m attributes directly instead of recalc; took out some hardcoded font sizes. For later: customize the plot graphics to look better, ala seaborn. termed "mplot" to make it easy to remember that only 1 value required (m), and may get moved elsewhere. feel free to change name! |
var : array | ||
values of variable | ||
w : spatial weight | ||
m : array |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not an array
, it's a ps.Moran
object, isn't it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oops, it is: m.class = <class pysal.esda.moran.Moran at 0x10d995a10>
Though while I was looking into this, and type(m) after getting an m, I got
class "instance," which after some reading is considered an old-style class
of type classobj,rather than inheriting from the object (see this
http://stackoverflow.com/questions/6666856/why-does-typemyfield-return-type-instance-and-not-type-field
discussion and this background
http://www.cafepy.com/article/python_types_and_objects/python_types_and_objects.html).
Is there a better way to inherit the right class for m in the applied side,
or is that a function of m?
On Sun, May 22, 2016 at 3:54 PM, Dani Arribas-Bel notifications@github.com
wrote:
In pysal/contrib/viz/plot.py
#797 (comment):---------
- var : array
values of variable
- w : spatial weight
- m : array
It's not an array, it's a ps.Moran object, isn't it?
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/pysal/pysal/pull/797/files/beace6676894fa61fd6a1328b8b0cc08e1fd8b49..6da47d169b5d14cf5d35592c41601653f4cc11b7#r64156092
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure I understand your view. Mine was really that we need to give the user some guidance as to what to pass in as m
for the function. Right now, it's asking for a (numpy) array
, but what it really takes is the output of ps.Moran
, hence my suggestion to use that in the documentation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No worries, I know -- it was my mistake! I can update it later today.
I was referencing a potential issue with how m is defined, and how the
class is being inherited (or not). I'm still getting familiar with under
the hood of PySal, and when I queried the structure of "m" :
type(m)
instance
The output was instance, which upon further reading, indicates that the
class might not be inheriting correctly. In other words, "type(m)" should
output the class object as a type, not "instance" (according to those
discussion and readings I linked earlier.
On Mon, May 23, 2016 at 10:30 AM, Dani Arribas-Bel <notifications@github.com
wrote:
In pysal/contrib/viz/plot.py
#797 (comment):---------
- var : array
values of variable
- w : spatial weight
- m : array
Not sure I understand your view. Mine was really that we need to give the
user some guidance as to what to pass in as m for the function. Right
now, it's asking for a (numpy) array, but what it really takes is the
output of ps.Moran, hence my suggestion to use that in the documentation.—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/pysal/pysal/pull/797/files/beace6676894fa61fd6a1328b8b0cc08e1fd8b49..6da47d169b5d14cf5d35592c41601653f4cc11b7#r64241013
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But, that doesn't seem to be causing any real issues -- sorry to confuse! :P
On Mon, May 23, 2016 at 10:39 AM, Marynia Kolak marynia.kolak@gmail.com
wrote:
No worries, I know -- it was my mistake! I can update it later today.
I was referencing a potential issue with how m is defined, and how the
class is being inherited (or not). I'm still getting familiar with under
the hood of PySal, and when I queried the structure of "m" :type(m)
instanceThe output was instance, which upon further reading, indicates that the
class might not be inheriting correctly. In other words, "type(m)" should
output the class object as a type, not "instance" (according to those
discussion and readings I linked earlier.On Mon, May 23, 2016 at 10:30 AM, Dani Arribas-Bel <
notifications@github.com> wrote:In pysal/contrib/viz/plot.py
#797 (comment):---------
- var : array
values of variable
- w : spatial weight
- m : array
Not sure I understand your view. Mine was really that we need to give the
user some guidance as to what to pass in as m for the function. Right
now, it's asking for a (numpy) array, but what it really takes is the
output of ps.Moran, hence my suggestion to use that in the documentation.—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/pysal/pysal/pull/797/files/beace6676894fa61fd6a1328b8b0cc08e1fd8b49..6da47d169b5d14cf5d35592c41601653f4cc11b7#r64241013
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do I indicate it like this? Want to make sure I have it correctly so the
REST piece works later:
m : pysal.Moran object
output of pysal.esda.moran.Moran
On Mon, May 23, 2016 at 10:48 AM, Marynia Kolak marynia.kolak@gmail.com
wrote:
But, that doesn't seem to be causing any real issues -- sorry to confuse!
:POn Mon, May 23, 2016 at 10:39 AM, Marynia Kolak marynia.kolak@gmail.com
wrote:No worries, I know -- it was my mistake! I can update it later today.
I was referencing a potential issue with how m is defined, and how the
class is being inherited (or not). I'm still getting familiar with under
the hood of PySal, and when I queried the structure of "m" :type(m)
instanceThe output was instance, which upon further reading, indicates that
the class might not be inheriting correctly. In other words, "type(m)"
should output the class object as a type, not "instance" (according to
those discussion and readings I linked earlier.On Mon, May 23, 2016 at 10:30 AM, Dani Arribas-Bel <
notifications@github.com> wrote:In pysal/contrib/viz/plot.py
#797 (comment):---------
- var : array
values of variable
- w : spatial weight
- m : array
Not sure I understand your view. Mine was really that we need to give
the user some guidance as to what to pass in as m for the function.
Right now, it's asking for a (numpy) array, but what it really takes is
the output of ps.Moran, hence my suggestion to use that in the
documentation.—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/pysal/pysal/pull/797/files/beace6676894fa61fd6a1328b8b0cc08e1fd8b49..6da47d169b5d14cf5d35592c41601653f4cc11b7#r64241013
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd remove object
(you wouldn't say ndarray object
). Rest looks good to me!
Hello! Please make sure to check all these boxes before submitting a Pull Request
(PR). Once you have checked the boxes, feel free to remove all text except the
justification in point 5.
nosetests
on your changes?pysal/dev
branch.docstrings and
unittests?
reviewer and added relevant labels [[ No option for assignee? Cant find it on my fork in issues or pull screen?? ]]
Adding Moran Plot view to contrib module. Will add additional functionality. In Progress