Also interested in talking to you! (Psych-DS)

[mekline]

Hi folks - I wanted to introduce a project I'm a part of, including sharing some work a few members have been doing with dataspice! We'd love to think about coordinating efforts with you.

Psych-DS is an in-progress technical specification for datasets in psychology, which uses Schema.org compliant metadata. We began this project as a conference hackathon this summer, and since then have been hashing out the specification and simultaneously coming to learn who all else is working on related projects (including Frictionless Data, and now thanks to your issue #71, DataCrate!). Eventually we hope Psych-DS will support specific subfields to converge on more standard ways of representing particular kinds of data, but have realized that getting social scientists on a shared technical footing & ensuring discoverability is a big deal! The project is heavily inspired by BIDS for neuroimaging data.

We are currently in the stage of wrapping up the draft of the written technical specification and beginning to 'road test' it on some real datasets. Beyond creating dataset-level metadata we are attempting to enforce some basic folder organization, file formatting/naming, and well-structured documentation of variables. (Here's a direct link to the long-form specification doc.)

At the same time, some intrepid coders started working on some standalone R-Shiny apps designed to produce this kind of output. In particular, Erin Buchanan (@doomlab) forked dataspice a while ago to try and tinker with it, and has been working with undergrads to write a tutorial that's appropriate for people who haven't used R before to approach a tool like dataspice.

Here is Erin's update: "We took the structure of dataspice – a grouping of shiny apps that one interacted with using RStudio – and converted it into one Shiny app that is published to the web for anyone to use. The app auto builds the CSV structured files from an uploaded dataset and then allows the user to step through entering the information for the access, attributes, bibliography, and creators files. Here, we fixed a couple typos and other issues that was prohibiting dataspice from being fully functioning and added more detailed instructions. The app still allows users to “write spice” and create a schema.org compliant JSON file and HTML report with some CSS tweaks."

Erin will have more details on the structure of the version she's been working on, and we'd love to talk about ways to join forces or coordinate. In particular, would it make sense to open some issues to merge back Erin's work, or is there a better way to start out than that? If it makes sense, we can put out a call to the Psych-DS mailing list to see if we have some extra hands to help.

Also, if anyone has time to take a look, we'd love to know if you have any feedback on how Psych-DS could work more smoothly with apps like dataspice, or if you see anything we haven't considered that might cause conflicts.

Thanks all!

• Melissa & Erin

[annakrystalli]

Hello Melissa & Erin!

This sounds like a fantastic initiative that we'd love to discuss whether/how we might be able to join forces. I was wondering if there was online version of the shiny up somewhere that we could have a look at? Otherwise I think a really useful next step (not just for us but perhaps for early testers/adopters of your work) would be to add a little more detail to the README. Some instructions on how to use the code, a walk through of an example or even better, a few screenshots of the working shiny app would really help!

Once we get a better understanding maybe we can assess where we might be able to work together and where your project might make sense to stand alone for ease of use of your particular target group.

Thanks for reaching out! Look forward to learning more :)

PS as for a tutorial, you might want to have a look at this dataspice tutorial. I understand you are working on a more domain specific version but there might be something useful in there. Would be great to get a link to the materials you've been working on too. I feel practical data management tutorials seem few and far between so would be nice to collate available resources for different domains together!

[doomlab]

@annakrystalli

The code is located at: https://github.com/doomlab/data-dictionary/tree/master/dataschema for our update to dataspice. You can interact with the Shiny app at: https://doomlab.shinyapps.io/dataschema/

Otherwise I think a really useful next step (not just for us but perhaps for early testers/adopters of your work) would be to add a little more detail to the README. Some instructions on how to use the code, a walk through of an example or even better, a few screenshots of the working shiny app would really help!

A great idea - we've actually written a tutorial paper with step by step guides - it's here: https://osf.io/evnmf/ (linked in github but OSF will render it for you). The document isn't totally complete yet, we are still working but thought I would share it now. I will add it to the readme when we are done.

[amoeba]

This is amazing work, 👏 ! I love when open source software and awesome people come together.

Certainly other contributors here can comment to this, but I feel like joining forces is a great idea. I just ran through your linked Shiny app and it's very nicely done and worked pretty well. I haven't run through our Shiny app(s) in a while so I can't draw a comparison but yours seems like it works how ours would/should.

I think we could potentially merge efforts, round out dataspice's feature set (perhaps renaming it in the process?), go through ropensci onboarding, and finish with a release to CRAN. I have some free time to allocate to this so I could lend a hand with any integrative work we do.

[mekline]

Bryce & Anna - thanks so much for your kind responses, and I’m looking forward to working with you!

I’ll invite the two of you to the Psych-DS listserv (anyone else reading, please comment on this issue if you’d liked to be added as well) - we are just getting started on creating some guinea pig dataset conversions, so it’s possible that some of these will want to use dataspice (in some instantiation) - my guess is that even if dataspice is producing ‘vanilla’ schema.org dataset JSON, people will be able to manually modify from there.

In terms of deciding on the feature set, I suspect that there are people who will prefer both the integrated browser app and r script versions. What’s the right way (I don’t know the answer) to modularize so the logic itself is common between app and functions?

[annakrystalli]

Hello @doomlab and @mekline, and many apologies for the delayed reply!

Firstly, I love, love, LOVE the app, and know many researchers that were interested in dataspice but weren't R users so hesitant to try and use it. This will be an excellent resource for them!

I totally agree with @mekline that retaining the original R functionality is important (I definitely use it regularly) so the key would be to set things up so that improvements can be easily shared by the two approaches. Perhaps we should explore more formal modularisation of the shiny code using shiny modules: https://shiny.rstudio.com/articles/modules.html

Finally, I've also been thinking about accommodating different/more complex metadata standards and perhaps this collaboration could be a great time to consider this. I was thinking it would be cool to include functionality that allows folks to supply their own metadata standards. We could include functions to validate their schema and then create templates and even update the documentation in the shiny apps (so turn the helptext into a responsive table that can be overriden by user-supplied information). I think we might be talking about the same thing? so, @mekline, can I just ask, what you mean by:

even if dataspice is producing ‘vanilla’ schema.org dataset JSON, people will be able to manually modify from there.

In any case, this is more a wishlist item for the future that I thought I's just throw in in case it peaked your interest.

Thanks for inviting me to the Psych-Data-Standards group! I didn't realise you were running an OpenLeaders project too 😃, awesome! I didn't manage to make the demo call but I will look out for the archive version of it! Look forward to learning more about the whole project!

[doomlab]

@mekline @annakrystalli @amoeba

Hi everyone! :) I am back from the moving cross country - adjusting to a new job hole I was in. I think we are on the same page here about updates, and I like the ideas you guys have going. I'd be happy to talk about how to merge efforts and thinking about how one could adjust the output they get based on the standards they wanted - we were mostly going with schema.org format because it was what google was doing and seemed wise to map onto something that already existed (right Melissa?).

I have not tried Shiny modules yet, but does seem like a good way to allow for choice complexity that you guys are discussing. Either way, I just wanted to drop a line that I was still interested - this group has made some excellent tools, so I'd be glad to contribute to that.