-
Notifications
You must be signed in to change notification settings - Fork 39
/
rstudio-conf-2017.Rmd
109 lines (65 loc) · 14.6 KB
/
rstudio-conf-2017.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
---
output: github_document
---
```{r, echo = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "figs/",
fig.height = 3,
fig.width = 4,
fig.align = "center"
)
```
[@drsimonj](https://twitter.com/drsimonj) here to share my big takeaways from [rstudio::conf](https://www.rstudio.com/conference/) 2017. My aim here is to share the broad data science opinions and challenges that I feel bring together the R community right now, and perhaps offer some guidance to anyone wanting to get into the R community.
**DISCLAIMER**: this is based on my experience, my primary interests, the talks I attended, the people I met, etc. I'm also very jet lagged after flying back to Australia! If I've missed something important to you (which I'm sure I have), please comment in whichever medium (Twitter, Facebook, etc.) and get the discussion going!
## My overall experience
I'll start by saying that I had a great time. [RStudio](https://www.rstudio.com/) went all out and nailed everything from getting high-quality speakers, to booking a great venue and organizing a social event at Harry Potter world I won't forget. But if I do, [Hilary Parker](https://twitter.com/hspter) took some great shots!
<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">Here are all my photos of <a href="https://twitter.com/hashtag/WizardingWorldOfHarryPotter?src=hash">#WizardingWorldOfHarryPotter</a> for those at <a href="https://twitter.com/hashtag/rstudioconf?src=hash">#rstudioconf</a> <a href="https://t.co/xg4bUXJIyr">https://t.co/xg4bUXJIyr</a></p>— Hilary Parker (@hspter) <a href="https://twitter.com/hspter/status/820793512698544128">January 16, 2017</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
## Opinionated Analysis Development
I didn't see it's value immediately, but [Hilary Parker](https://twitter.com/hspter)'s talk on opinionated analysis development struck me as brilliant once I started writing this post.
<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">Here are my slides from <a href="https://twitter.com/hashtag/rstudioconf?src=hash">#rstudioconf</a>: "Opinionated Analysis Development" &/or "Trying to Make Fetch Happen" <a href="https://t.co/TLNrmUzW4M">https://t.co/TLNrmUzW4M</a></p>— Hilary Parker (@hspter) <a href="https://twitter.com/hspter/status/820354490553552897">January 14, 2017</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
To crudely (and probably inaccurately) summarize, Hilary was advocating that we focus on defining opinions about data science processes (like "we should be using version control") and then push for software that supports them. I liked this idea and will do my best to discuss the conference themes in this way.
In line with Hilary's talk, I think the broad goals driving discussion at the conference include **reducing wasted time, minimizing the chance of human error, and maximizing utility**. As a human factors researcher, this couldn't make me happier!
So, what were some of the broad opinions and challenges about data science processes that the R community discussed?
## Write code for humans
A strong opinion is that code should be written for humans. This doesn't mean that we need to communicate how every function works. Rather, if I share my code with someone, can they easily get a sense of what various chunks are doing? This means less time trying to interpret code and minimizing the chance of making errors along the way.
The R software of choice for writing human-friendly code most definitely includes the [tidyverse](http://tidyverse.org/). It's not just the functions, but it's the philosophy behind the tidyverse that makes it so apt for writing readable code. This was evident from the start when [Hadley Wickham](https://twitter.com/hadleywickham) gave an opening keynote on the topic.
<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">.<a href="https://twitter.com/hadleywickham">@hadleywickham</a> kicking off <a href="https://twitter.com/hashtag/rstudioconf?src=hash">#rstudioconf</a> by talking about the tidyverse: both the packages and the connections between them <a href="https://twitter.com/hashtag/rstats?src=hash">#rstats</a> <a href="https://t.co/r7WztkrFBO">pic.twitter.com/r7WztkrFBO</a></p>— David Robinson (@drob) <a href="https://twitter.com/drob/status/819908216339525632">January 13, 2017</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
The use of the [magrittr](https://github.com/tidyverse/magrittr) package's pipe operator `%>%` and packages like [dplyr](https://github.com/hadley/dplyr) and [ggplot2](https://github.com/tidyverse/ggplot2) was prolific. Whether you like these tools or not, understanding them is crucial for communicating with the R community right now. They're also quickly becoming the basis for many other packages like [Julia Silge](https://twitter.com/juliasilge) and [David Robinson](https://twitter.com/drob)'s tidytext.
<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">Slides from my <a href="https://twitter.com/hashtag/rstudioconf?src=hash">#rstudioconf</a> talk yesterday on Text Mining, the Tidy Way <a href="https://t.co/xgWzk0IfpA">https://t.co/xgWzk0IfpA</a></p>— Julia Silge (@juliasilge) <a href="https://twitter.com/juliasilge/status/820300676039208960">January 14, 2017</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
## Use Version Control
Another big opinion surrounding the optimization of time and reduction of errors is that we should be using proper version control. Accurately tracking how our code evolves and being able to share that evolution with others (or our future selves) is critical. Without version control, simple mistakes like mislabelling a file can have devastating consequences. It's also an essential process for working collaboratively.
The R community's current software of choice for version control and collaboratively developing code are Git and GitHub. You'll find that most of the top R packages are now hosted on GitHub, which enables good version control as well as an open environment that is easy for multiple people to work in at the same time. [Jenny Bryan](https://twitter.com/JennyBryan) gave a notable tutorial on this topic.
<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">I'm teaching Git/GitHub today at <a href="https://twitter.com/hashtag/rstudioconf?src=hash">#rstudioconf</a>. I take a lot of inspiration from <a href="https://twitter.com/alicebartlett">@alicebartlett</a>'s "Git for humans" <a href="https://t.co/hwjIvFZHBk">https://t.co/hwjIvFZHBk</a></p>— Jenny Bryan (@JennyBryan) <a href="https://twitter.com/JennyBryan/status/820255475098849281">January 14, 2017</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
## We need better modeling processes
<img src='https://cdn.meme.am/cache/instances/folder165/400x400/55074165.jpg'>
Data science is a hot topic and everyone agrees that we should model data in a way that is valid and useful. But the process by which we carry out modeling is still poorly defined and a major challenge when you consider human biases. This was really emphasized by data journalist, [Andrew Flowers](https://twitter.com/andrewflowers).
<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">Six types of data story + their dangers from <a href="https://twitter.com/andrewflowers">@andrewflowers</a> <a href="https://twitter.com/hashtag/rstudioconf?src=hash">#rstudioconf</a> <a href="https://t.co/NKmMT2kQO3">pic.twitter.com/NKmMT2kQO3</a></p>— Hadley Wickham (@hadleywickham) <a href="https://twitter.com/hadleywickham/status/820283681017364480">January 14, 2017</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
Modeling needs standardized processes, and tools to support them, to reduce error and maximize the validity of our work. We have a strong foundation. For example, in addition to the topics described above, we have great processes and tools for optimizing code such as [Dirk Eddelbuettel](https://twitter.com/eddelbuettel)'s [Rcpp](http://www.rcpp.org/) or [Winston Chang](https://twitter.com/winston_chang?lang=en)'s [Profvis](https://rstudio.github.io/profvis/). But what do we do when it comes to modeling data?
The community is gearing up to start tackling this problem. Before the conference, I publicized a new development package, [pipelearner](https://github.com/drsimonj/pipelearner), which is related to this problem. It was great to meet with people who were already testing it out (no doubt thanks to [Mara Averick](https://twitter.com/dataandme) tweeting about it).
<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">ICYMI, 'nother great <a href="https://twitter.com/drsimonj">@drsimonj</a> how-to: “Easy machine learning pipelines with pipelearner” <a href="https://t.co/SS3GDjEHZ1">https://t.co/SS3GDjEHZ1</a> <a href="https://twitter.com/hashtag/rstats?src=hash">#rstats</a> <a href="https://twitter.com/hashtag/machinelearning?src=hash">#machinelearning</a> <a href="https://t.co/5vnTR0kwpG">pic.twitter.com/5vnTR0kwpG</a></p>— Mara Averick (@dataandme) <a href="https://twitter.com/dataandme/status/819742991858405377">January 13, 2017</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
As an idea, pipelearner is a start, but it's a long way away from what we'll ultimately need. An exciting prospect is that [Max Kuhn](https://github.com/topepo), creator of the [caret](http://topepo.github.io/caret/index.html) package, has joined RStudio. We can only hope that he'll be focussing on this problem and we'll be seeing some proper opinions and tools soon.
## Present work in a visual format that is easy and enjoyable to digest
Another broadly held opinion is that we should present our work in a visual format that is easy and enjoyable to digest. This doesn't just mean creating good plots with ggplot2, but synthesising entire reports or documents that can be easily understood. It's not sufficient to do an analysis because you always have to present that analysis to someone! That someone could be a fellow data scientist or a manager who knows nothing about statistics or programming. To me, *enjoyable* is an important aspect. The details of data science work can be boring to most people. Presenting information so that people want to engage with it can make a big difference.
The software tools of choice here are [R Markdown](http://R Markdown.rstudio.com/) and [Shiny](https://shiny.rstudio.com/). Shiny is the go-to for interactivity. R Markdown is fast becoming the base for almost all top R visualization/communication formats, so I'd recommend starting to learn it before Shiny. With advances in tools like [R Notebooks](http://rmarkdown.rstudio.com/r_notebooks.html), many people are already creating their scripts in R Markdown-based documents that can be instantly converted to visually appealing formats. There were many great talks on these topics such as:
<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">My <a href="https://twitter.com/hashtag/rstudioconf?src=hash">#rstudioconf</a> slides: 1) Advanced Rmd <a href="https://t.co/etQlXVYgYI">https://t.co/etQlXVYgYI</a> 2) blogdown <a href="https://t.co/huitnIPx8Q">https://t.co/huitnIPx8Q</a> 3) Extending Rmd <a href="https://t.co/j1zVBU86CI">https://t.co/j1zVBU86CI</a></p>— Yihui Xie (@xieyihui) <a href="https://twitter.com/xieyihui/status/820477439050731521">January 15, 2017</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
## Be social
Being social is strongly valued by this community. This doesn't always get directly discussed, but it is something we all seem to share implicitly. The first meaning of "social" here is to be engaged with other community members. This can be sharing your content or, if you don't have content to share, liking, commenting, or otherwise engaging with the content that people do share. Keeping connected like this helps the R community to develop fast. When package errors are found, or challenging problems come up, these penetrate into the community and are often resolved incredibly quickly. The other meaning of "social" is to be friendly and supportive. More advanced community members are happy to help those in need, making a move into the R community a pleasant experience for newcomers (I can speak to this point directly). Expecting friendly dialect makes people more likely to share their work and problems. Overall, I think that some of the growing success of R can be attributed to the value that the R community places on being social.
In general, I feel that the best software for being a social R user is [Twitter](https://twitter.com). For example, my lightning talk on corrr got considerable attention outside the room as soon as [David Robinson](https://twitter.com/drob) tweeted about it.
<blockquote class="twitter-tweet tw-align-center" data-lang="en"><p lang="en" dir="ltr">.<a href="https://twitter.com/drsimonj">@drsimonj</a> talks about the corrr package- performing and exploring correlations in a tidy framework <a href="https://t.co/J6j4ybe9pq">https://t.co/J6j4ybe9pq</a> <a href="https://twitter.com/hashtag/rstats?src=hash">#rstats</a> <a href="https://t.co/jMwLdUzspZ">pic.twitter.com/jMwLdUzspZ</a></p>— David Robinson (@drob) <a href="https://twitter.com/drob/status/820035275040686082">January 13, 2017</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
Start to follow other R users, engage with them, and you can always look at other users' lists. E.g., you can check out my [daRk magic](https://twitter.com/drsimonj/lists/dark-magic) list that isolates some of my favorite R tweeters (though it needs some updating). [Stackoverflow](http://stackoverflow.com/) is another great place to get or give R help - but be sure to get a feel for it before diving in if you're new. Of course, emails are also good.
So speaking of being social, here comes my sign off and contact details...
## Sign off
Thanks for reading and I hope this was useful for you.
For updates of recent blog posts, follow [\@drsimonj](https://twitter.com/drsimonj) on Twitter or email me at <drsimonjackson@gmail.com> to get in touch.
If you'd like the code that produced this blog, check out the [blogR GitHub repository](https://github.com/drsimonj/blogR).