create robots.txt for dev

[sformel]

@wardappeltans spotted that AI engines were ingesting the dev version of the manual. I think this is a fix, but I don't have time to test at the moment.

There is no manual.obis.org/robots.txt, which means there is no direction for agents to ignore the site.

I think if you create a robots.txt in the root directory of the dev branch like:

User-agent: *
Disallow: /

then add a step to the github workflow like:

- name: Render Book
  run: Rscript build.R
- name: Copy robots.txt
  run: cp robots.txt ./_book/robots.txt

then you can republish the dev manual. It should show up at https://manual-dev.obis.org/robots.txt

As extra protection, we can add a 'no index' to the html via the index.Rmd:

output:
  bookdown::pdf_book:
    template: null
  bookdown::gitbook:
    toc_depth: 3
    includes:
      in_header: [_noindex.html]

create _noindex.html in the root like:

html<meta name="robots" content="noindex, nofollow">

Also, notice that I got rid of google analytics from the dev version, which is being called via

includes:
   in_header: [_noindex.html]