Development workflow

[LydiaFrance]

A development workflow for working on tgvejs and running it via app.

• testing instructions
  • yarn test in tgvejs
    • Cannot find module 'react' from 'ReactSixteenAdapter.js' -- add to devDependencies
• experiments on master branch
  • build Docker image
  • yarn install/yarn start
• experiments on npm branch
  • build Docker image ❌ Error: Cannot find module 'webpack/lib/node/NodeTemplatePlugin'
  • yarn install/yarn start ❌ Could not find a required file. Name: index.html
  • npm install/npm start ❌ Webpack 4.42.0/4.46.0 dependency issue
• resolve Merge npm branch with master #92
• Hosted instance workflow app#4 (?)
• identify key users:
  • TGVE contributor
  • Node.js/React developer
  • R developer
  • Python developer (building Flask app)
• native Mac development
  • get to the point where we can run geoplumber::gp_plumb() in the R console
• Docker-based development
  • move Dockerfile from eAtlas/npm to app
  • locally modify package.json to point to file:../eAtlas (not link:!)
• identify key use cases
  • contribute to tgvejs
  • build custom app based on TGVE?
  • deploy my own copy of TGVE (e.g. locally to work with sensitive data)?
  • use the publicly available deployment of TGVE
• move change log to separate .md file
• Delete README.Rmd #100
• commit to yarn (vs. npm) to avoid warnings about package-lock.json
• locally modified package.json is a bit of a pain
  • npm link ../tgvejs seems to achieve same effect but without changing package.json
• dev workflow instructions in README.md for tgvejs contributors
  • clone tgvejs and app repos to local machine
  • locally modify @tgve/tgvejs dependency in app/package.json to point to link:../tgvejs
  • yarn watch to incrementally update dist dir of tgvejs?
    • app doesn't pick this up -- need to delete node_modules/@tgve/** and then yarn again

[layik]

Hopefully this becomes much clearer once we deal with #92 and #93.

[rolyp]

@layik In the proposed internal dev workflow (where we clone, rather than instantiate, eatlas-template), the developer (= me) has to have a locally modified package.json and yarn.lock. This is not ideal. Any ideas on how we might improve on this?

[layik]

OK here is part of the doc I am working on for developers which can replace this Wiki entry. I am happy to refer to how other npm package developers do it but would have to be similar. Here is part of the MD being written.

Local React App

To develop (make changes) or even better contribute to eAtlas, please set up your own development environment.

We propose two ways:

• using the eatlas-template OR
• using a clean CRA app

There are three steps in either of the two cases:

1. Clone the eAtlas template
2. Transpile it using the provided package.json script currently named dist. For instance:

git clone https://github.com/tgve/eAtlas
cd eAtlas
npm run dist

3. Make your dependency of eatlas a local one to your step (1)

That should set you up.

Clean CRA app

Using eatlas-template

[layik]

A Wiki entry like this: https://github.com/tgve/eAtlas/wiki/Developing-eAtlas

[layik]

Now, I am happy to say "We propose you create a CRA app from scratch" etc.

[layik]

Will do some more tests on using yarn and hopefully can commit to yarn as our other repos all use yarn. It is a very valid checklist item.

[layik]

RE checklist item: "get to the point where we can go gp_plumb if this Wiki entry is not fully reproducible, I am happy to make sure it stays fresh. It is one aspect of this work that I like to tell data scientists that our approach is fully reproducible.

https://github.com/tgve/eAtlas/wiki/Reproducible-TGVE

[layik]

And perhaps, I need to commit an .Rmd that can be run by users and reproduced using an R package like reprex::reprex().

[rolyp]

RE checklist item: "get to the point where we can go gp_plumb if this Wiki entry is not fully reproducible, I am happy to make sure it stays fresh. It is one aspect of this work that I like to tell data scientists that our approach is fully reproducible.

https://github.com/tgve/eAtlas/wiki/Reproducible-TGVE

Won't reproducibiity be exactly what's captured by the Dockerfile? Or do you have something else in mind?

[rolyp]

@layik Should we think of the development workflow (assuming a full stack setup that allows change to the R code, React code, etc) as belonging to full-template?

[layik]

RE checklist item: "get to the point where we can go gp_plumb if this Wiki entry is not fully reproducible, I am happy to make sure it stays fresh. It is one aspect of this work that I like to tell data scientists that our approach is fully reproducible.
https://github.com/tgve/eAtlas/wiki/Reproducible-TGVE

Won't reproducibiity be exactly what's captured by the Dockerfile? Or do you have something else in mind?

I agree a Dockerfile is a reproducible example. But more like a document in shape of a notebook or an Rmarkdown is what I have in mind for [data] scientists.

[layik]

@layik Should we think of the development workflow (assuming a full stack setup that allows change to the R code, React code, etc) as belonging to full-template?

For me this is what we discussed today and a "development workflow" would be what a developer would be doing to develop eAtlas npm package.

[rolyp]

@layik Ok. Perhaps this could build on what's currently in the jupyter repo?

[layik]

You mean the full-template sticking with R to make my life easier.

[rolyp]

Let's talk about this tomorrow morning.