Open Science Utility Belt

[katieannemills]

Open Science 101

This is a session series introducing practical skills needed to get started in open science. A Mozilla Science Study Group can use this series to introduce open science over an academic semester.

Help Us Develop This Curriculum

We're trying to answer these questions:

• What skills are needed to practice open science?
• What's missing / What's unnecessary? (aiming for < 12 sessions over a semester)
• What's out there now? References! If you've seen related material, send it over.

Let us know your thoughts in the comments!

Sessions

1. Introduction: what & why
   • What skills will be part of this series?
     • working openly through the entire process (not just warehousing things on the web afterwards) in order to leverage collaboration
     • emphasizing legibility of research outputs for the sake of reuse & reproducibility
   • Why do these things matter?
     • lit review on citation benefits, efficacy benefits, retraction scandals & efficiency.
   • Sources: Working Open guide, TBD
2. Open Data I: Standards & Legibility
   • What is an ontology?
   • How to effectively use data standards and make data legible?
   • Sources: TBD
3. Open Data II: Clean Data
   • What is 'clean' vs 'dirty' data, and why do they matter?
     • how to keep data organized and easy to reuse at a later date (including in-house reuse); consider metadata, storage and formats.
   • Best practices for making a reusable dataset when no standard exists.
   • Sources: TBD
4. Collaboration I: Version Control
   • Basic git, with an emphasis on getting to GitHub as a platform for sharing & collaboration.
   • Source: TBD
5. Collaboration II: Roadmapping
   • How to lay out a project for effective collaboration.
   • Source: Working Open guide.
6. Collaboration III: Code Review
   • How to set expectations for good contributions that lead to easy-to-review code
   • How to make the code review process fast and efficient
   • Source: Working Open guide, Code Review Teaching Kit
7. Code Wrangling I: Sustainable Coding
   • Effective use of documentation.
   • Producing end-to-end analysis automation scripts (R, Python, Shell, or make); understanding of how a well-made automation script serves as 'living documentation'.
   • Sources: TBD
8. Code Wrangling II: Testing
   • Writing test suites to ensure code quality & build trust to support reuse.
   • Sources: this lesson in Python, TBD in R.
9. Code Wrangling III: Code Packaging
   • Making & distributing packages to support reuse & collaboration.
     • discussion of useful formalisms for organizing data & code in packages / repos
   • Sources: this lesson in Python, and this lesson in R
10. Publishing & Communication I: Citation & Discoverability
    • Software & data citation
      • DOIs
      • comments on how this addresses discoverability of code & data
    • Authoring for the Web
      • markdown / knittr
      • metadata
    • Sources: Working Open guide, TBD
11. Publishing & Communication II: The Research Cycle
    • Strategies for opening the entire research process:
      • Grant process
      • Online lab notebooks
      • blogging, twitter & social media
      • protocol publishing
      • study pre-registration
12. Publishing & Communication III: Licensing
    • open access publishing
      • comments on impact on science in the Global South / decoupling access from privilege
    • Why are licenses necessary?
    • What can they do? What can't they do?
    • Which ones are the most important and how do they work?
    • How to choose a license, and the intersection of licensing and copyright
    • The importance of agreeing on a license explicitly and early on a collaboration
    • sources: TBD
13. Change Making
    • how to champion change in real life?
    • what barriers are commonly encountered, and how to avoid them?
    • sources: https://speakerdeck.com/dsalo/changing-workflows

[kaythaney]

Thanks for this, Bill! Would be worth adding links to the Working Open guide here, and seeing if you could line up with some of the language and key categories there to strengthen / augment that work. That also may help with some of the verbiage issues (like "Programming" as a header - not sure that's the best term here, crisp up the language and minimize jargon).

Great start, and more comments to come!

[katieannemills]

yep, these lessons are going to pull heavily from the WOG, once we agree the curriculum. Changed 'programming' -> 'code wrangling'.

[abbycabs]

Suggested language for the beginning:

---

Open Science 101

This session series introduces practical skills needed to get started in open science. A Mozilla Science Study Group can use this series to introduce open science over an academic semester.

Help Us Develop This Curriculum

We're trying to answer these questions:

• What skills are needed to practice open science?
• What's missing / What's unnecessary? (aiming for < 12 sessions over a semester)
• What's out there now? References! If you've seen related material, send it over.

Let us know your thoughts in the comments!

Sessions

...

---

Going through actual sessions now :) Really excited for this work!

[katieannemills]

From the twitterverse:

@ttimbers suggests data storage & archiving - how to find data associated with a study, how to organize your own data for future reuse; also metadata, storage and formats.
@minisciencegirl suggests organizing data, with useful naming schemes, structure etc.

[taddallas]

+1 for @minisciencegirl 's suggestion about naming schemes

Is there any reason to have the Open Data sections after Collaboration sections? It might flow better (in my mind) from data wrangling into setting the wrangled data free (i.e. Open Data), then to dive into collaborations/workflows/code review/version control. This change could also make the transition into publishing easier, as collaborations may lead to publications ... 🙏

[abbycabs]

I'm thinking along the same lines as @taddallas re:flow. Teaching packages before git seems off to me.

[katieannemills]

The ordering of 'Code Wrangling', 'Collaboration', 'Open Data' and 'Publishing & Communication' are actually just in the order I thought of them in :)

So, how about the order:

• 'Open Data'
• 'Collaboration'
• 'Code Wrangling'
• 'Publishing & Communication'

[taddallas]

Looks good to me!

One tiny thing: I noticed that much of the material uses Python. Perhaps it would be worthwhile to also show some R examples, as some pretty solid tools for Open Science are built around R (e.g. reproducible analyses and manuscript writing with R Markdown, testing with testthat, etc.). This point is null if the course is designed to be Python-specific, or if you think there'd be too much overlap with the R utility belt you already have.

[katieannemills]

Nope, R-flavoured implementations of these techniques are definitely something we want! Which will get used depends on the audience, but we definitely want both options for the Code Wrangling section. The current examples are Python for no other reason than I speak Python. That said, I think there was a packages in R lesson from UBC recently I can dig up and link here - if you have a good lesson for testing in R, send it on by!

[blahah]

The only thing that is conspicuously missing to my mind is licenses - they are fundamental to open science and are relevant to all the sections above. I would think these are the most important aspects of licenses to cover:

• Why are licenses necessary?
• What can they do? What can't they do?
• Which ones are the most important and how do they work?
• How to choose a license
• The importance of agreeing on a license explicitly and early on a collaboration

[katieannemills]

@blahah - totally agree, added your points to an additional section under 'publishing & communication' - thanks! One thing that would be super helpful in that section, is ideas for hands-on activities, and engaging ways to introduce things like licenses as well as code and data citation; definitely A-list important stuff, but runs the risk of turning into a really dry lecture about DOIs and copyright.

[blahah]

I think a nice way to introduce licenses and citation is by doing a set of small hands-on data mining tasks. Introducing some frustrating scenarios that are solved by proper licensing and good data citation should be memorable. We just need a paper with great data but no license, and a paper that does something good with someone else's data but doesn't cite it properly.

[noamross]

This may be expanding the scope a bit, but some topics that would have been helpful for me early on, before I really did much coding or had a solid project together, would have been:

• Keeping and organizing a open, digital lab notebook
• Searching, collecting, reading and annotating content for re-usability and collaboration

[katieannemills]

@noamross could that first point fit with the social media unit?

I'd love to hear your ideas on your second point - to be honest, content aggregation is a pretty weak part of my own game, I've never found a method I really liked.

[noamross]

Yes, lab notebook could go in social media, but there's a fair amount of the topic that isn't explicitly social: metadata/tagging of notes, formats and organization for searching, plain-text for posterity, etc.

On collecting content, I'm similar. I have a semi-working system of Mendeley + a collection of tagged plain-text notes, but I'm not sure how well it works in terms of collaboration. @cboettig and I once wrote a review together where we built an annotated bibliography using markdown + bibtex, but it felt more like a one-use hack than a system. Ideas from others would be welcome.