Skip to content
Permalink
Branch: master
Find file Copy path
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
168 lines (141 sloc) 22.2 KB
## Building with Bookdown, GitHub, and Zotero {#bookdown}
*by [Jack Dougherty and Ilya Ilyankou](authors-and-contributors.html)*
We are building this open-access book with open-source tools---namely Bookdown, GitHub, and Zotero---and openly sharing the knowledge we acquire so that others may improve on the process. We welcome feedback from digitally-minded authors and software developers on [Twitter](https://twitter.com/doughertyjack) or directly on the GitHub repo for this page <https://github.com/OnTheLine/otl-bookdown/blob/master/09.3-bookdown.Rmd>.
- [Bookdown](http://bookdown.org), an open-source software package by Yihui Xie at RStudio, allows authors to compose in version of [Markdown](https://en.wikipedia.org/wiki/Markdown), an easy-to-read-and-write cross-platform syntax, and create one workflow that produces books in multiple formats, such as Web edition, PDF edition, ePUB ebook edition, and Microsoft Word. Furthermore, Bookdown generates the Web edition as a set of static-site HTML files, which display digital elements (such as interactive maps and videos) and load very quickly into readers' web browsers. Individual authors can install Bookdown on their Mac, Windows, or Linux computers, and publish their work through a free and simple GitHub Pages account, rather than a separate and complex web server. Interestingly, Bookdown requires installing the open-source [R Project](https://www.r-project.org/) and [RStudio console and editor](https://www.rstudio.com/), which typically are used for statistical data analysis. Behind the scenes, Bookdown converts the text of the book from R-flavored Markdown files into Web and eBook editions using [PanDoc](https://pandoc.org/), an open-source universal document converter, without requiring authors to learn complex Pandoc instructions. Although Bookdown may not be the best choice for novice computer users, installation steps are outlined below, along with a recommended workflow for authoring and publishing with GitHub and Zotero.^[For a detailed guide, see @xieBookdownAuthoringBooks2018]
- [GitHub](http://github.com) is an open-source repository that enables us to publicly share the text and code in this book, with a version-control system that allows collaborators to work on different sections at the same time. Also, the [GitHub Pages](http://pages.github.com) feature allows us to freely host online all of the book editions (Web, PDF, etc.), with a custom domain name purchased through another service, [Reclaim Hosting](http://reclaimhosting.com). As this is an open-access publication, we intentionally make its contents public---and also share instructions like this---while we work on the book, which is consistent with the contract signed with our open-access publisher. Note that GitHub also allows users the option to make repositories private.
- [Zotero](http://zotero.org) is an open-source bibliography manager to collect sources and organize citations, created by the Roy Rosenzweig Center for History and New Media at George Mason University. [Better BibTeX](https://github.com/retorquere/zotero-better-bibtex) is an extension by Emiliano Heynes that makes Zotero citation keys work better with Markdown text workflows. While Bookdown does not require using Zotero, scholars who need to manage multiple source materials and citations will benefit from the Bookdown---Zotero---Better BibTeX workflow described below.
Before leaping into Bookdown or any software tool, see [Alternative Publishing Platforms](alternative.html) in this book.
(ref:0-bookdown-desktop) Overview of the workflow on a Mac desktop: Compose and build the book in RStudio with Bookdown (top left), manage sources and insert citation keys with Zotero + BetterBibTex (bottom left), and push the built book files to GitHub Pages with settings shown via web browser to share online (right).
```{r 0-bookdown-desktop, echo=FALSE, fig.cap="(ref:0-bookdown-desktop)"}
knitr::include_graphics("images/bookdown/0-bookdown-desktop.png")
```
#### Install Bookdown with R, RStudio, and a LaTex Engine {-}
Below are steps we followed to setup the publishing platform for this book, using our Macintosh OS 10.14 computers. The same general principles also should apply to Windows computers. No special knowledge is required, but this may not be recommended for novice computer users. Installation steps---and inevitable problems that pop up---will be easier if users are willing to feel adventurous with their computers, or already have some familiarity with text editors, GitHub, or R Studio.
- Install R Project statistical programming language <https://www.r-project.org>, which Bookdown requires. [See screenshot](images/bookdown/1-r-download.png)
- Install the free RStudio Desktop to make R easier to use with a visual editor. [See screenshot](images/bookdown/2-download-rstudio.png)
- Inside RStudio, select the Packages tab, and select Install. [See screenshot](images/bookdown/3-packages-install.png)
- Inside RStudio, install the "bookdown" package to build your book, and select Install Dependencies. [See screenshot](images/bookdown/4-install-bookdown.png)
- Bookdown now should be successfully installed in RStudio. [See screenshot](images/bookdown/5-installed.png)
- For Bookdown to create a PDF edition of your book, you need to install a [LaTeX](https://en.wikipedia.org/wiki/LaTeX) engine to prepare your Markdown plain text, citations, and images into stylized pages. Since the full-sized [LaTeX project](https://www.latex-project.org/get/) is very large, Bookdown recommends the smaller TinyTeX package. Inside RStudio, select the Packages tab, select Install, and enter "tinytex" to find and upload the package. [See screenshot](images/bookdown/6-install-tinytex.png)
- To finish the installation, in the RStudio console, type `tinytex::install_tinytex()` and press return. [See screenshot](images/bookdown/7-finish-install-tinytex.png)
##### Sidenote: About the PDF edition, and TinyTeX vs XelaTex {-}
The PDF edition of this Bookdown-generated publication is still a work-in-progress, and the settings are not yet finalized. Read more about various options at <https://bookdown.org/yihui/rmarkdown/pdf-document.html>. To avoid some unicode error messages when generating the PDF edition for this book, we installed XelaTex instead of TinyTeX. If you do the same, be sure to modify this line in your Bookdown index.Rmd settings: `latex_engine: xelatex`
##### Sidenote: If TinyTeX installation error {-}
- Once upon a time, we received this error message when attempting to install tinytex in RStudio on our Mac: `/usr/local/bin not writeable`. We resolved by reading the software developer's GitHub issue <https://github.com/yihui/tinytex/issues/24> and following these steps:
- In the Mac Applications > Utilities folder, open the Terminal application.
- Carefully type `sudo chown -R `whoami`:admin /usr/local/bin` and press return.
- Carefully type (without "sudo") `~/Library/TinyTeX/bin/x86_64-darwin/tlmgr path add` and press return.
- Close the Terminal application.
- In the RStudio console, type `tinytex::install_tinytex(force = TRUE)` and press return.
#### Download and Build a Sample Bookdown Book {-}
- Create a free GitHub account <https://github.com> to simplify steps for the next two sections. While Bookdown does not require you to use GitHub, the workflow described below features GitHub to copy a Bookdown template and to host your own Bookdown editions online.
- In your web browser, log into your GitHub account, go to the Bookdown developer's `bookdown-minimal` repo <https://github.com/yihui/bookdown-minimal>, and fork a copy to your GitHub account. To learn about forking in GitHub, see this chapter <http://datavizforall.org/github.html> in the *Data Visualization for All* book.
- Install GitHub Desktop <https://desktop.github.com> to transfer files between your online GitHub repo and local computer. While software developers may prefer to access GitHub by typing commands in their terminal, GitHub Desktop provides easier point-and-click access for most users.
- In your web browser, go to your forked copy of `bookdown-minimal`, click the green `Clone or Download` button, and select `Open in Desktop`. This should automatically open the GitHub Desktop application, and you can navigate where you wish to store a copy of your code repo on a folder in your local computer.
- In RStudio in the upper-right corner, select Project > Open Project to open the `bookdown-minimal` folder on your local computer. [See screenshot](images/bookdown/8-project-open.png)
- In RStudio, open the `index.Rmd` file and make some simple edits to the text of this minimal book. For example, remove the hashtag `#` comment symbol in line 8 to "uncomment" and activate the PDF book option. Save your edits. [See screenshot](images/bookdown/9-edit-book.png)
- Optional: If you wish, you can modify your `bookdown-minimal` files outside of RStudio, by using your preferred text editor, such as Atom editor <https://atom.io>.
- In RStudio, upper-right corner, select the Build tab, select Build Book, and choose All Formats to build both the gitbook-style static web edition and PDF edition.
- If RStudio successfully builds both editions of your minimal book, the output will be saved into your `bookdown-minimal` folder, in a subfolder named `_book`, because that's how this sample is configured. The RStudio internal browser should automatically open your web edition (but it's not a very good browser, so we typically close it and manually open the `index.html` file with Firefox or Chrome browser.)
- Also, open the subfolder and inspect the PDF edition of your book. If RStudio found any errors, they will appear in the Build viewer. [See screenshot](images/bookdown/11-successful-build.png)
- Hint: In future sessions with RStudio, you may need to select Packages tab and click Update to keep bookdown and other software packages up to date. [See screenshot](images/bookdown/12-packages-update.png)
- Close RStudio.
#### Host your Book with GitHub Pages {-}
- Open GitHub Desktop and navigate to the `bookdown-minimal` folder on your local computer. Write a quick summary to commit (or save) the changes you made above to your master branch, and push this version to your online GitHub repo.
- In your web browser, go to your online GitHub repo, with a web address similar to `https://github.com/USERNAME/bookdown-minimal` (insert your own GitHub username).
- In your GitHub repo, select Settings, and scroll down to the GitHub Pages section, which is a free web hosting service to publish your code and book editions on the public web. Select Master Branch as your source, and Save.
- Scroll down to this section again, and the web address of your published site should appear. Copy this address.
- In a new browser tab, paste the web address from above, and at the end add `_book/index.html` because this sample is configured to store all web and PDF editions in your `_book` subfolder. Your web address should be similar to: `https://USERNAME.github.io/bookdown-minimal/_book/index.html`
- Reminder: You may need to wait up to one minute for edits to your GitHub online repo to appear live at your GitHub Pages web address. Also, after waiting for GitHub Pages to make changes, be sure to "force reload" or "hard refresh" your web browser to update directly from the GitHub Pages server, not the browser's internal cache. On Mac, press shift + command + R. On Windows, press ctrl + F5.
#### Customize Bookdown and GitHub Settings and Domain Name {-}
- To see the file structure, settings, and content for this *On The Line* book, view its online repository at <https://github.com/ontheline/ontheline-bookdown>.
- See details about configuration settings in Yihui Xie's [Bookdown technical guide](https://bookdown.org/yihui/bookdown).^[@xieBookdownAuthoringBooks2018]
- In this publication, the `_bookdown.yml` file is set to build all of the book outputs (Web edition, PDF edition, etc.) in directory folder named `docs` in the GitHub repo as shown below. (Proper indenting is important here.)
```
output_dir: "docs"
book_filename: "Dougherty-etal-OnTheLine"
edit: https://github.com/ontheline/otl-bookdown/edit/master/%s
language:
label:
fig: "Figure "
ui:
edit: "Edit"
chapter_name: "Chapter "
```
In this book, the Bookdown setting has been changed from the default `_books` folder to the `docs` folder to take advantage of the GitHub Pages setting that allows hosting online content from the `master/docs` folder, rather than the entire `master` branch. Choosing these Bookdown and GitHub Pages settings allows authors to publicly share their book source code and raw text on the `master` branch, **and** host online the Web and ebook editions from the `master/docs` subfolder of that branch. (Years earlier, GitHub previously required users to separate these tasks on two different branches, typically called
`master` and `gh-pages`, but now users can do both through one branch, which is simpler.) Also, following this `master/docs` method simplifies the address of your GitHub Pages published web edition to this format: `https://USERNAME.github.com/REPONAME`. (See further below about customizing the web address.)
In this publication, the `index.Rmd` file is set to produce the Web edition in the GitBook style, with additional settings explained further below. (Proper indenting is important here.)
```
output:
bookdown::gitbook:
dev: svglite
css: css/style.css
includes:
in_header: [custom-scripts.html, google-analytics-otl.html]
split_by: section
split_bib: true
number_sections: true
```
In the GitHub repo for this book, here is the root file structure:
- Preface of the book with non-numbered sections: `index.Rmd`
- Chapters with first-level headers, such as: `01-introduction.Rmd`
- Sections with second-level headers, which belong to chapters, such as `01.1-section.Rmd` and `01.2-section.Rmd`
- The `images` folder, where JPG, PNG, and PDFs to display in chapters are located.
- The `docs` folder, which contains the published book products, such as Web edition (`index.html`, `introduction.html`, etc.), the PDF edition (`Dougherty-etal-OnTheLine.pdf`), etc.
- Additional helper files described further below.
When you change the names of chapters/sections, Bookdown builds new HMTL pages based on those new names, but old HMTL pages based on old chapter/section names may still exist in the same subfolder. To avoid confusion, you may wish to carefully delete old HTML pages in `docs` whenever you significantly alter names and build a new version of the book.
In addition, this GitHub Pages repo is published with a custom domain name <https://OnTheLine.trincoll.edu>. Learn more about custom domain names at <https://help.github.com/articles/using-a-custom-domain-with-github-pages/>. Typically, a custom domain name is purchased from a web hosting service, such as [ReclaimHosting](http://reclaimhosting.com), and you may need customer support to access the Manage DNS (domain name system) to point your new URL to your GitHub Pages online site. For this book's custom domain name, <https://ontheline.trincoll.edu>, the IT staff at Trinity College created a subdomain on their .edu domain, which shows that my scholarship is affiliated with an academic institution. Adding a GitHub Pages custom domain name creates an additional `CNAME` file in the `docs` subfolder. Avoid accidentally erasing the `CNAME` file if you need to clear out any old HTML files in your subfolder.
The web edition of this book includes two custom files that are referenced in the `index.Rmd` file header, and are generated into the web edition when building the book:
```
output:
bookdown::gitbook:
dev: svglite
css: css/style.css
includes:
in_header: [custom-scripts.html, google-analytics-otl.html]
```
- the `custom-scripts.html` file includes specific instructions to set the height or width of iframes for specific interactive maps and charts in the web edition, which is explained further in the Style Guide section of this book. (Thanks to Michael Dorman for this tip.)
- the `google-analytics-otl.html` file records web traffic with Google Analytics, if you create a free account and insert your code.
- The web edition of this book also includes a custom `404.html` file that redirects any bad web addresses under the domain back to `index.html` page. A copy of this file must be manually transferred into the `docs` subfolder, because it is not automatically built by Bookdown.
In this book, the `index.Rmd` file also contains instructions to create footnotes and a bibliography from a Zotero-Better BibLaTeX exported file named `ontheline.bib`, in a specific Chicago-style format. Other `.csl` files are downloadable from the Zotero Styles Repository <https://www.zotero.org/styles>.
```
bibliography: ontheline.bib
citation-style: chicago-fullnote-bibliography.csl
```
Also in this book, the `index.Rmd` file is set to produce a PDF edition of the book using the xelatex engine (or see tinytex or latex alternatives described above). Furthermore, additional PanDoc settings are required to create the desired Chicago-style footnotes and bibliography in the PDF edition. See more details at <https://pandoc.org/>.
```
bookdown::pdf_book:
latex_engine: xelatex
citation_package: none
pandoc_args: [ "--csl", "chicago-fullnote-bibliography.csl" ]
```
#### Zotero and Better BibTex for Footnotes and Bibliography {-}
This workflow uses the open-source Zotero bibliography manager (http://zotero.org), with the Better BibTeX extension (https://retorque.re/zotero-better-bibtex/) to work more easily with Markdown and Bookdown. Rather than typing full references directly into the text, authors can insert a short citation key to each source, and the tools will automatically generate the desired references (such as Chicago-style footnotes, which historians love) and an alphabetized bibliography for the book. Furthermore, since authoring this *On The Line* book has been a multi-year process with multimedia sources whose web addresses have changed over time, this workflow ensures that the most up-to-date information in your Zotero database also appears in all editions of your book.
With these tools installed, here's an outline of the author's workflow:
- An entry for each source (book, journal article, document, etc.) must be created in the author's Zotero library (or a specific collection), either automatically from the web or manually.
- Better BibTex generates a unique citation key for each source (example: `^[@tyackOneBestSystem1974]`), which authors insert into the Markdown text to create a full reference (such as a footnote) and a bibliography.
- The author identifies their preferred citation style at <https://www.zotero.org/styles>, uploads the file (example: chicago-fullnote-bibliography.csl) to the local book repo, and lists it in the `index.Rmd` settings for both the Web edition and the PDF edition.
- Before each build in Bookdown, the author sets their Zotero preferences, and exports the Zotero library or specific collection to the root level of the local book repo (example: ontheline.bib).
##### Installation and Settings {-}
- Download and install [Zotero](http://zotero.org) for Mac, Windows, Linux, and add connectors to your browsers.
- Install the [Better BibTeX extension](https://retorque.re/zotero-better-bibtex/installation/) and follow all of the site's instructions for initial setup.
- At the top of each entry in Zotero, the extension will generate a unique citation code (example: tyackOneBestSystem1974).
- You can copy and paste the citation code into your Markdown text, and add a caret, square brackets, and the at symbol `^[@tyackOneBestSystem1974]`, as described in the Style Guide in this book.)
- Also, you can set Zotero preferences > Export > Better BibTeX Quick Copy to use Zotero's drag-and-drop quick copy feature.
- To export a bibliography (.bib) from Zotero library or collection into the local Bookdown repo:
- Select Library > Right-click to export the collection
- Select format > Better BibLaTeX (*IMPORTANT* I use this setting, rather than "Better BibTex", because "Better BibLaTex" includes full dates in newspaper citations, and urls). Also, I leave all of the checkboxes blank during the export, and do NOT select "keep updated". This means that if my Better BibTex citation codes suddenly change in Zotero because the author, title, or year changed, then I'm responsible for running find-and-replace to make these edits in the text of the book.
- Save the output as FILENAME.bib, save into your book repo, and be sure to match the settings in `index.Rmd`.
These Zotero types appear correctly with this Bookdown workflow:
- Book
- Book chapter
- Journal article
- Newspaper
- Thesis
- Report
- Blog post
- Web page
- Document -- Use this all-purpose entry in place of other types: Law case, Presentation, Interview, Video recording, Television broadcast, etc. Insert important details (such as the archival location information) in the Publisher field.
To help other researchers find items cited in this book, URLs should be included in Zotero entries whenever feasible, even if not required by convention. For example, some print-only books and documents are hard to locate, so include an OCLC WorldCat permalink to make them easier to find (example: http://www.worldcat.org/oclc/20683509). Also, if a print source has been digitized by HathiTrust, Google Books, or the Internet Archive, add one of these URLs to the Zotero entry.
Reminder: Chicago full-note works exactly as it was designed, meaning that the second instance of a citation currently appears as an abbreviated note (author, with title when appropriate). TODO: Find or create a modified .CSL file to display the full footnote entry in *every* instance in the web version, so that users do not need to scroll to find the complete source info.
![creative commons by-nc-sa icon](images/cc-by-nc-sa-88x31.png) *[On The Line](http://ontheline.trincoll.edu) is an open-access, born-digital, book-in-progress by [Jack Dougherty and contributors](authors-and-contributors.html) at Trinity College, Hartford CT, USA. This work is copyrighted by the authors and freely distributed under a [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License](http://creativecommons.org/licenses/by-nc-sa/4.0/). Learn about our [open-access policy and code repository](copyright-with-open-access.html) and [how to read and cite](how-to-read-and-cite.html) our work. This book-in-progress was last updated on: `r Sys.Date()`*
You can’t perform that action at this time.