Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rspatial github organization repo #9

Open
tim-salabim opened this issue Jan 12, 2017 · 54 comments
Open

rspatial github organization repo #9

tim-salabim opened this issue Jan 12, 2017 · 54 comments

Comments

@tim-salabim
Copy link
Member

Dear all,
given that we need a repository for a new package for interactive feature editing and maybe other general packages in the future, I wonder if it would be a good idea to set up a "rspatial" organization github repository?

We could, as a first step, move this repo to the .io pages there and then start collaborative package development around the spatial universe from there. rOpenSci seems to be an extremely successful model for collaborative open source development. Something similar could also work for focused spatial package development.

Please let me know your thoughts.

Tim

@bhaskarvk
Copy link

Agree 100%. I can even move leaflet.extras there.

@edzer
Copy link
Member

edzer commented Jan 12, 2017

I've been thinking along similar lines to set up an organisation called r-spatial (rather than rspatial), but didn't as I felt short in answering the for what and for whom questions. I could move around 10 repos there, but to what effect?

@tim-salabim
Copy link
Member Author

It might help from a 'new to r spatial'-user perspective to get a concise overview of what is available. Especially when providing a structured overview in some r-spatial.github.io 'start page'. Also, I think that having a central repo might help identifying gaps and/or opportunities for all developers involved.

Most importantly, though, I feel that an r-spatial (I am more than happy with the dash :-) ) organisation would foster closer collaboration as there is more direct involvement of people with different skills (e.g. C++, JavaScript, ...) which would help overcome some 'invisible' issues (e.g. opening an issue would ideally trigger responses/solutions from a potentially larger group than with individual repositories).

I am curious how others feel about this?

@briatte
Copy link

briatte commented Jan 12, 2017

for what

Having just discovered leaflet.extras thanks to this thread, I would say: for initiating newbies like me to the wide array of spectacularly good R packages that can manipulate and visualise spatial data.

@edzer
Copy link
Member

edzer commented Jan 12, 2017

Well, here you are. Tim and I are now admin.

@tim-salabim
Copy link
Member Author

Thanks Edzer!

I believe that you initiating this is a good move. One thing to make sure is a widely visible link to CRAN Task View: Analysis of Spatial Data as this is still by far the best and most comprehensive collection of all that is available for spatial R.

@bhaskarvk
Copy link

Thanks @edzer! Could you or @tim-salabim add members of this repo to that org ? I didn't find a way to self add.

@edzer
Copy link
Member

edzer commented Jan 12, 2017

@bhaskarvk done; @tim-salabim yes, I suggest to do that the moment there is sth somewhat substantial growing here.

@tim-salabim
Copy link
Member Author

@edzer deal!

@edzer
Copy link
Member

edzer commented Jan 12, 2017

We even have a website, now!

@bhaskarvk
Copy link

I'm going to take a first stab at setting up that website using http://rmarkdown.rstudio.com/rmarkdown_websites.html, unless there are any objections.

@tim-salabim
Copy link
Member Author

tim-salabim commented Jan 12, 2017

no objections from me, I guess I will learn quite something if you do :-)

@bhaskarvk
Copy link

@edzer Could you make me admin for the website repo ?

@edzer
Copy link
Member

edzer commented Jan 12, 2017

You have write access now.

@bhaskarvk
Copy link

OK no worries, I just pushed the website, but make sure to change the settings to serve the website from 'docs' directory. I can't do that w/o admin rights. I chose to put the generated site under 'docs' to separate Rmds from HTMLs.

@edzer
Copy link
Member

edzer commented Jan 12, 2017

you can't do that for an organisations page.

@edzer
Copy link
Member

edzer commented Jan 12, 2017

We could auto-forward to docs.

@bhaskarvk
Copy link

OK I pushed everything under root then. It's still showing up old index.html, I suspect it takes some time to invalidate the cache. Or may be that has to be done manually.

@bhaskarvk
Copy link

I'm fine keeping Rmd and HTMLs together, for now that's what I've done. Just need the caches to invalidate.

@tim-salabim
Copy link
Member Author

it is updated now

@timelyportfolio
Copy link
Collaborator

Now that it is official, I think it is time for a logo - maybe a map + the R logo. Anyone creative out there?

@jhollist
Copy link
Collaborator

jhollist commented Jan 12, 2017

Just jumping on the bandwagon. This has potential to be a great resource as many of the R-spatial resources are widely dispersed. This r-spatial org will need to be careful not to be just another one of those many sites. Don't know that I have any brilliant ideas to prevent that.

And @tim-salabim in #9 (comment) you mentioned rOpenSci. Are you envisioning something similar to their group of "official" packages that have gone through a review process?

@mdsumner
Copy link
Member

@tim-salabim @edzer @bhaskarvk now we need a charter and some guidelines on what belongs here :) Is this a pre-CRAN staging platform, attracting oversight/discussion on best practices, advice on overlap with existing pkgs, and general visibility to r-spatial tragics? I do think there is a gap in resources for developers, so using this for putting tips, tricks, FAQ and in-progress problem solving would be good.

@mdsumner
Copy link
Member

@timelyportfolio I hacked the R logo here which might be useful for ideas.

https://github.com/r-gris/org-logo/blob/master/README.md

@timelyportfolio
Copy link
Collaborator

I love the triangulation. Perhaps we could add a map background or some graticules?

@mdsumner
Copy link
Member

mdsumner commented Jan 12, 2017 via email

@tim-salabim
Copy link
Member Author

I think the logo at https://github.com/r-spatial is just fine

@tim-salabim
Copy link
Member Author

@jhollist I am envisioning an open source collaboration for spatial analysis in R, not more, not less. How that will pan out I am not sure, but I don't wanna inflict any sort of rules. Also I am not thinking of any official packages. I hope that with a collaborative effort we can be smart enough to figure out a best practice (i.e. collection of packages) for spatial analysis needs.

@r-barnes
Copy link
Collaborator

I like the https://github.com/r-spatial logo, though I find it a bit complex. If a new logo were in order, I'd suggest something simpler, like so:

Possible simple R spatial logo

@tim-salabim
Copy link
Member Author

Ha! I do like the graticule in the R halo (or whatever that is called).

@r-barnes
Copy link
Collaborator

Looking at spatial stuff on R from the perspective of a recent outsider, I've encountered a few questions:

  • How do I do (blank)? (Draw a map, maybe with ggplot? Fill depressions in a raster? Filter vector data for shapes with particular properties?)
  • What packages are worth using/should be used? (R has a lot of packages.)
  • When I am developing which packages do I integrate with, and how do I do that? For instance, I'd like to build a raster analysis package around some C++ code I've written. It seems as though raster is a good target for integration, but it's unclear how to pass data from R to raster to my C++ and back.

Much of this relies on settling on a group of packages to handle standard geospatial data and other packages for manipulating that data, as I believe we've discussed elsewhere.

An ideal site in my mind would show a number of basic analysis workflows emphasizing how the different packages are used. (ggplot's docs are really superb this way)

But a more impactful approach to disseminating the information would be to, as a group, come up with some well-posed questions and answers, and then blitz StackOverflow with them. (Or add new answers to old questions.) This seems to be how I get all my R know-how anyway.

@mdsumner
Copy link
Member

Good points, I find the community is not well geared to development and sadly reliant on key authors. There aren't tools for developers really, but the recent move from old tools like svn and the R API to github and Rcpp have opened that right up.

As a practised spatial R user I can answer those how-to's and very happy to, but it's hard for me to see the right questions to be answered - so please pose them and we will have at it!

Fresh questions from someone with your expertise will be very valuable.

@edzer
Copy link
Member

edzer commented Jan 14, 2017

I've started r-spatial.org a bit less than a year ago, with the idea to provide an additional resource to r-sig-geo, the spatial task view, the SO tags, the journals (see here for links), and provide an overview of upcoming events as well as of spatial package on github. I don't complain about the attention the site got, but it has been an effort for over 99% by me and people in my group. The external pull requests I got were from people who wanted their event mentioned. In the meantime, Robert set up rspatial.org (no hyphen) for book-like documentation on the packages he wrote. I'd be happy to see r-spatial.org (or rspatial.org, I really don't care) become some sort of central hub for R spatial information, but that hasn't happened.

What can a github organisation such as http://github.com/r-spatial do, besides (collaboratively) working on a website? It's mostly used for working on a set of repositories as a group. A problem that I see here, is if we are not a legal or financial organisation, why should someone move ownership of her/his repo and put it in the r-spatial organisation where she/he is not owner? I wouldn't say that distrust is an issue, but the practical advantages still escape me. github gives all means to share push rights with others on private repos for those who want that. The moment we'd have more than 10 repos, they fall off the screen and nobody will click down past nr 20.

I agree with @mdsumner that we have too few active developers, or if this is not the case then too little interaction between them. github helps here tremendously: over the last 6 months, in which I spent a lot of time developing sf, sf received more than 10 times the amount of constructive contributions (including PRs) than sp did over 10 years, and it has triggered me to look much deeper into other projects too, and contribute there.

@r-barnes these are good questions, but should the goal be to list the questions together with answers? Or should we set up a discuss repo where questions can be discussed as issues? Should we focus on HowTo's for analysis, or for package development? This repo has helped, in any case, @tim-salabim , thanks for setting that up! The old r-sig-geo channel still works, and still has 3000 subscribers and an archive of 25K emails you can search. Regrettably, google will point you first to SO, where R spatial is scattered (with tags for packages or topics, and a separate gis.stackexchange.com), and many answers are wrong or substandard, and so are questions. I even heard there are youtube videos nowadays where people try to explain how R spatial works.

Anyway: to sum up, I see a bright future where here we should try to

  • benefit from the good things from git and github offer for collaboration
  • try to augment what we built in the past 15 years, by not starting from scratch, but doing what is missing, improve things, and eventually rebuild the things that need to be rebuild
  • actively take initiatives for more interaction: there are a lot of us, but if we don't interact nobody sees us; we need to draw people into this.

@rsbivand
Copy link
Member

@timelyportfolio
Copy link
Collaborator

The Wiki functionality in Github repos can also be very powerful. We could move the taskview content there and allow easy editing with review and history.

@tim-salabim
Copy link
Member Author

@timelyportfolio
Copy link
Collaborator

@tim-salabim yes I thought it would be interesting to see how it evolves with easier edits but profess to complete ignorance in much of what is there and how it got there.

@tim-salabim
Copy link
Member Author

I guess this is up to @rsbivand to ponder about

@timelyportfolio
Copy link
Collaborator

I'll work more on this, but here is a very ugly start https://github.com/tim-salabim/rspatial/wiki/CRAN-Task-View:-Analysis-of-Spatial-Data.

@tim-salabim
Copy link
Member Author

Ok, seems to be easy enough. Though I am still a bit reserved whether this is the best place for it. This repo should/will be moved to https://github.com/r-spatial soon and there we already have another repo called https://github.com/r-spatial/r-spatial.org. Maybe it would be better to have the task view as a special site in that blog?

@edzer
Copy link
Member

edzer commented Jan 18, 2017

CRAN task views are part of the cran web site and mirrors. It makes only sense to have them on github if we (i) make VERY clear that this is not THE CRAN task view,but only a mirror and (ii) if we have a mechanism in place that makes it easy for the maintainer to evaluate/accept changes in the CTV source code form, and upload those to CRAN. The CTV source code is here; CRAN receives regular updates from r-forge. Ctv's are written in xml. I guess for @rsbivand the easiest way of receiving change change proposals would be patches created by diff.

An independent effort was done here; I believe the authors here don't do the community a service by calling this document a CRAN Task View, since it isn't.

I still don't see exactly what the problem with the current spatial CTV is that would justify the effort of the round trip through github. If we want something better than the CTV, we might as well start from scratch, and not do it on CRAN (and not call it a CTV). The spatial CTV started 10 years ago, and grew incrementally.

@mdsumner
Copy link
Member

mdsumner commented Jan 18, 2017

I agree with @edzer here and it's a good summary. My tiny addtions to it are a hassle to do, and don't scale, and I'm way out of date with it. With modern tools and the vocal community attention we see "someone" could create an amazing new kind of CTV (or it would languish and no one would really mind because it didn't take much effort in the first place).

If we're to improve the Spatial CTV it should be by encouraging an overall modernization of the CTV system, which is out of scope here - or just do something new, recognizing the pretty amazing summary that exists on the CTV now and all the real work that went into it (not my miniscule additions).

What is in scope here is some feedback up from us about what's good, bad etc. about the current CTV and also statements like this, that despite its current form, it's still the current best single-point overview of R-spatial and that we collectively should refer to it and encourage contributions and improvements to it both incremental and more radical.

@mdsumner
Copy link
Member

mdsumner commented Jan 18, 2017

Also MetaCRAN https://www.r-pkg.org/ and R-hub https://builder.r-hub.io/ are brand new resources that show the kind of scope new approaches can have. CRAN has reverse dependencies which is the only "peer-attention" based metric CRAN has.

I get a huge amount of information from the star and follow system on Github, both for giving me an initial heads-up to a project's existence as well as the relative "quality/respect/coolness" vibes that these social platforms provide.

(It's making me realize that I see stackoverflow as being old-fashioned and passe now, I just see most of that as a dead-end - there are stellar questions and answers there, but the general "build an extended help system by community popularity vote" really didn't work).

@rsbivand
Copy link
Member

About the CRAN Task View - it as you say grew incrementally. The different task views are maintained depending on the choices and intentions of the maintainers, and their overview. The latter is an increasing problem, because of growth in the number of packages, changes in technology, etc.

It isn't impossible to see a jointly edited document in a different markup being rendered in CTV, and I'd suggest that this is a route that could be taken (might the SpatioTemporal Task View go the same way? might we split Spatial into Mapping and Analysis?). As of now, the CTV maintainer is alone in having commit rights, and that editorial responsibility might be worth keeping. So to get this going, someone should write for example an md to ctv renderer.

This is also linked to ongoing thinking about documentation (no, not the pushy website that just aggregates stuff). Could we for example insert links to R Journal articles into Task Views?

@rsbivand
Copy link
Member

A gaping hole that needs filling is to provide infrastructure for continuous testing of code examples that (once) solved problems. It would be sensible to see what is available and how to add a testing badge to code examples. Could code examples also be scraped from R-sig-geo, and would it help to provide text markup?

@mdsumner
Copy link
Member

Wow @rsbivand that is a great idea, it's got interesting aspects from the perspective of r-hub, text analysis tools, and the R documentation project. The idea of recording "code examples that ever worked" is a really good one, and has some tricky/interesting aspects.

Pinging

@gaborcsardi regarding r-hub as a platform for running R and package versions on all hardware

@juliasilge @dgrtwo regarding scanning online code community resources

@halpo regarding community-driven and community-resources for augmenting R doc and the ecosystems around it

@jhollist
Copy link
Collaborator

@edzer I am the maintainer of https://github.com/ropensci/maptools. Way back when it was intended to be a CTV focused on spatial data viz but we ended up with a bit more overlap with the spatial CTV and after some discussing with @rsbivand we agreed that it shouldn't go up as an official task view. I agree with you that calling it a task view is confusing. I'll get that edited this week.

I'll keep following this discussion. If it makes sense to eventually merge/sunset the rOpenSci repo with something else I am happy to do so.

@mdsumner
Copy link
Member

mdsumner commented Jan 18, 2017

Just to add to the mix https://github.com/rstudio/RStartHere "A guide to some of the most useful R Packages that we know about, organized by their role in data science.". I can see that being integrated somehow in the http://tidyverse.org eventually

(and hey @jhollist maptools looks really good!)

@timelyportfolio
Copy link
Collaborator

@rsbivand and @edzer, thanks so much for the much more thorough understanding of CTV. md->CTV would be very helpful, and I will explore that. I really like the ideas of including journal articles in the CTV or housing a summary list in MD or Wiki. Also, I love the idea of testing old code and modernizing it on a continual basis whether in a journal article, in a very popular answer to a question, or in a linked article.

In addition, it might be nice to have a summary list of geo videos, especially those from the prominent conferences.

@jhollist
Copy link
Collaborator

@timelyportfolio There is an R script and makefile that we used for maptools and @sckott used for the web services CTV (I think). It actually goes the other way (ctv to md) but might be a useful starting place.

I agree a bookdown kinda package ("ctvdown"?) would be nice.

@sckott
Copy link

sckott commented Jan 18, 2017

the web services task view changed via @leeper https://github.com/ropensci/webservices/blob/master/Makefile - i think they now start with md, and make ctv from that

@pat-s
Copy link
Member

pat-s commented Jan 19, 2017

Happy to join the discussion! After quickly browsing all repos and issues, here are some first thoughts:

  • Whats the difference between rspatial-spark and the discuss repo? Both are there for discussion and its unclear to me why there are 2 discussion repos

  • When using Github for discussion, issues with a lot of comments become very long (endless scrolling). Is there an option to page it or somehow organise it? (I know, Github isn´t indented to be a forum..)

  • In issues with a lot of comments, I think there is a need of a sticky-like post at the beginning of the issue which summarises the current discussion status - otherwise new users will need hours to get all information together. There´s already lots of information gathered here but it needs to be organised/summarised somehow. Otherwise comments will just stay comments. And yeah, this would need moderation + time..

@dankelley
Copy link
Collaborator

dankelley commented Jan 19, 2017

@pat-s As to your second and third points, you are quite right. The utility of GH issues for discussion is limited greatly by the lack of threading (or comment hierarchies, if that might be a clearer term). If there were no competition for a discussion interface, this wouldn't be as frustrating as it is. But there is a lot of competition; I imagine everyone reading this has spent time today reading other discussion forums in which threading makes it quite easy to grasp the main point quickly.

My impression (after having complained to GH about this numerous times over the years) is that things are not going to change. In my own projects, of which perhaps https://github.com/dankelley/oce/issues is germaine to the spatial project (since it provides ways to process and plot spatial geophysical data) I have come to the conclusion that the GH issues can be made a bit more useful if participants establish a convention that titles be narrowly focussed, and that tangents in the discussion are broken out into new issues. To some extent, the "labels" scheme can help a bit with this sort of thing, but not much, really.

My guess is that the wiki part of GH is going to be more helpful than the issue part, for discussion threads such as the one I just entered.

--D.

@pat-s
Copy link
Member

pat-s commented Jan 19, 2017

@dankelley Good summary!

I see the point using Github

  • most of the R-spatial people have an account and are active here
  • Possible hosting of different R-packages under the hat of an organisation
  • Syntax highlighting when talking about code

However it is not intended to be a forum (sadly) and if used as such, one needs to put in lots of moderation to keep it tidy and clear. Probably worth an extra issue..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests