Blog post about using Pandoc and doing everything in one branch for lessons #650

gvwilson · 2014-10-28T13:22:49Z

No description provided.

rgaiacs · 2014-10-28T14:48:46Z

Does pandoc conversion from Markdown to HTML support copying figures?

$ pwd
/tmp/prototype/src
$ ls 
01.jpg  sample.md
$ cat sample.md 
# Title

Body

![rio](01.jpg)

Body
$ pandoc -s -o ../sample.html sample.md
$ ls ..    
sample.html  src/
$ cat ../sample.html 
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
  <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  <meta http-equiv="Content-Style-Type" content="text/css" />
  <meta name="generator" content="pandoc" />
  <title></title>
  <style type="text/css">code{white-space: pre;}</style>
</head>
<body>
<h1 id="title">Title</h1>
<p>Body</p>
<div class="figure">
<img src="01.jpg" alt="rio" /><p class="caption">rio</p>
</div>
<p>Body</p>
</body>
</html>

rgaiacs · 2014-10-28T14:49:40Z

blog/2014/10/pandoc-and-gh-pages.html

+  Long-time readers of this blog and
+  <a href="{{site.mailing_lists}}/pipermail/discuss_lists.software-carpentry.org/">our discussion list</a>
+  will know that I'm unhappy with the choices we have for formatting our lessons.
+  Thanks to a tweet from <a href="http://kbroman.org/">Karl Broman</a>,


Could you add the link to the tweet?

wking · 2014-10-28T17:50:22Z

On Tue, Oct 28, 2014 at 06:22:50AM -0700, Greg Wilson wrote:

Fixing title casing

This doesn't seem to be related to this PR, and could probably be
pushed now without further review.

An idea for storing lessons

About:

“When an author makes a change, she must build locally, then commit
those files to the GitHub repository. Yes, this means that
generated files are stored in version control, which is normally
regarded as a bad idea.”

I'd prefer contributors make source-only changes to a lesson's master
branch. When they merge the PR, the lesson's maintainer merges the
updated master branch into the gh-pages branch (under src/ if you
want), runs the build, commits the resulting HTML, and pushes to
GitHub. That means:

No auto-generated files in version control :).
The burden of installing the build tools only falls on the
maintainers, and not on the contributor. Unless the contributor
wants to deploy their altered HTML for a workshop before the
maintainer gets around to merging, in which case there's no way
around them installing the build tools (or using a build-service
like gh-pages' Jekyll).
We really don't have to care if that build tool is Pandoc. So long
as a lesson's gh-pages branch will have HTML, we can assemble a
workshop website without needing to rebuild the lessons ourselves.

Of course, standardized build tools would make life easier for folks
who are pushing content that they need now to several lessons
simultaneously. But I think we can move towards something like a
Pandoc standard in an evolutionary manner, instead of throwing a
switch and moving everything over all at once.

cboettig · 2014-10-28T18:09:10Z

Certainly standardized and automated build tools would be more convenient.
It is simple enough with today's CI tools to automate the build process to
run on the CI machine and deploy to the gh-pages branch. This has the
advantage of being as seamless as the old Jekyll-based route while also
being independent of what build process tools are used (e.g. can be pandoc,
but can equally well use any other tools that we can run on the CI
machine). That does introduce a one-time overhead of setting up the CI
system along with the credentials to push back to github, but it's not so
bad.

On Tue, Oct 28, 2014 at 10:50 AM, W. Trevor King notifications@github.com
wrote:

On Tue, Oct 28, 2014 at 06:22:50AM -0700, Greg Wilson wrote:

Fixing title casing

This doesn't seem to be related to this PR, and could probably be
pushed now without further review.

An idea for storing lessons

About:

“When an author makes a change, she must build locally, then commit
those files to the GitHub repository. Yes, this means that
generated files are stored in version control, which is normally
regarded as a bad idea.”

I'd prefer contributors make source-only changes to a lesson's master
branch. When they merge the PR, the lesson's maintainer merges the
updated master branch into the gh-pages branch (under src/ if you
want), runs the build, commits the resulting HTML, and pushes to
GitHub. That means:

No auto-generated files in version control :).

The burden of installing the build tools only falls on the
maintainers, and not on the contributor. Unless the contributor
wants to deploy their altered HTML for a workshop before the
maintainer gets around to merging, in which case there's no way
around them installing the build tools (or using a build-service
like gh-pages' Jekyll).

We really don't have to care if that build tool is Pandoc. So long
as a lesson's gh-pages branch will have HTML, we can assemble a
workshop website without needing to rebuild the lessons ourselves.

Of course, standardized build tools would make life easier for folks
who are pushing content that they need now to several lessons
simultaneously. But I think we can move towards something like a
Pandoc standard in an evolutionary manner, instead of throwing a
switch and moving everything over all at once.

—
Reply to this email directly or view it on GitHub
#650 (comment).

Carl Boettiger
UC Santa Cruz
http://carlboettiger.info/

gvwilson · 2014-10-28T18:13:56Z

That does introduce a one-time overhead of setting up the CI
system along with the credentials to push back to github, but it's not so
bad.
I think it's challenging enough for many of our instructors that they
wouldn't get around to it, and therefore wouldn't get around to
contributing material...

jiffyclub · 2014-10-28T18:14:54Z

We want a workflow that allows people to preview their work. I have Travis set up to push docs to gh-pages when it does builds, but I still build the docs locally first to make sure there aren't problems.

cboettig · 2014-10-28T18:21:35Z

Right, as Matt points out, such a configuration would be purely 'bonus';
that is, everything could still be done manually as before. For a
maintainer getting lots of pull requests who didn't want to manually build
them locally each time, CI would merely be a potential time-saver (after
reviewing the edits to the source they could see already that the build run
was successful and merge without having to do extra edits).

Didn't mean to open a tangent, just wanted to observe that the proposal as
it stands is not necessarily incompatible with the ideals of automated and
common build chains.

On Tue, Oct 28, 2014 at 11:14 AM, Matt Davis notifications@github.com
wrote:

We want a workflow that allows people to preview their work. I have Travis
set up to push docs to gh-pages when it does builds, but I still build the
docs locally first to make sure there aren't problems.

—
Reply to this email directly or view it on GitHub
#650 (comment).

Carl Boettiger
UC Santa Cruz
http://carlboettiger.info/

Blog post about using Pandoc and doing everything in one branch for lessons

Greg Wilson added 2 commits October 27, 2014 15:13

Fixing title casing

64264ba

An idea for storing lessons

2bd85fb

rgaiacs reviewed Oct 28, 2014
View reviewed changes

gvwilson pushed a commit that referenced this pull request Oct 29, 2014

Merge pull request #650 from gvwilson/blog-pandoc-and-gh-pages

6137d1f

Blog post about using Pandoc and doing everything in one branch for lessons

gvwilson merged commit 6137d1f into swcarpentry:master Oct 29, 2014

gvwilson deleted the blog-pandoc-and-gh-pages branch October 29, 2014 15:40

Latest description of a Pandoc-based workflow

3a3fa5f

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Blog post about using Pandoc and doing everything in one branch for lessons #650

Blog post about using Pandoc and doing everything in one branch for lessons #650

gvwilson commented Oct 28, 2014

rgaiacs commented Oct 28, 2014

rgaiacs Oct 28, 2014

wking commented Oct 28, 2014

cboettig commented Oct 28, 2014

gvwilson commented Oct 28, 2014

jiffyclub commented Oct 28, 2014

cboettig commented Oct 28, 2014

Blog post about using Pandoc and doing everything in one branch for lessons #650

Blog post about using Pandoc and doing everything in one branch for lessons #650

Conversation

gvwilson commented Oct 28, 2014

rgaiacs commented Oct 28, 2014

rgaiacs Oct 28, 2014

Choose a reason for hiding this comment

wking commented Oct 28, 2014

cboettig commented Oct 28, 2014

gvwilson commented Oct 28, 2014

jiffyclub commented Oct 28, 2014

cboettig commented Oct 28, 2014