Why gatsby over metalsmith for the blog?

[bananatranada]

Could you guys go in depth about the choice of moving the blog over to gatsby?

What would you recommend and for what purposes?

[Ajedi32]

Could you clarify what site you're referring to? Pretty sure the regular metalsmith.io site is built with Metalsmith.

[bananatranada]

@Ajedi32 Here's the link that says the segment blog was built with gatsby. Metalsmith being perfectly suited for a blog and open sourced by segment, I was wondering why they went with gatsby.

[Ajedi32]

I'm not affiliated with Segment so I can't really comment on why they went with Gatsby. At first glance though it looks to me like Gatsby is far more opinionated than Metalsmith is. Metalsmith's core is tiny and relies almost entirely on plugins for its functionality, whereas Gatsby seems to try to include a lot more functionality out of the box, so maybe that has something to do with it.

I'll leave this issue open; maybe someone who works at Segment will be able to answer this better than I can.

[kylenstone]

Motion to close this issue...

[ghost]

Since Segment has chosen Gatsby over its own Metalsmith, there is a concern for longevity of this project, and what the future could hold. As a new user, I wouldn't choose Metalsmith if Segment wasn't dogfooding, i.e. using its own project, because there's a huge risk that this project can be shuttered, leaving many Metalsmith users out in the cold.

[Ajedi32]

@up0 Segment isn't the sole maintainer of Metalsmith. There are several maintainers such as myself who don't work for Segment.

Not to mention the Metalsmith core is pretty minimal. I'm certain that anyone with a solid grasp of Node and a little free time on their hands would have no trouble keeping Metalsmith afloat all on their own; for these reasons the longevity of the project isn't really much of a concern for me. See #243 for more discussion on this.

[leviwheatcroft]

Lets not read too much into it.. whoever was tasked with building the blog had a preference for Gatsby, no big deal.

[otterpro]

Awesome. Glad to hear from the supporters. Definitely moving forward with Metalsmith-based projects.

[kosirm]

I really like metalsmith for it's simplicity, but adding it all together: no new version since last year + segment blog built on gatsby + metalsmith.io down because they "forgot" to update heroku account... It all looks like there is no real interest in segment to lead this project further. Pretty sad conclusion for me as user. Hopefully other maintainers could move project forward, because it is so valuable project!

[Zearin]

@kosirm I don’t want to see this project die, either. That was part of my motivation for opening #303 .

@Ajedi32 and other maintainers must review my work first, but I’m hoping it will become the basis for Metalsmith 3. If that happens, my hope is that the community will notice and interest will be revived.

With enough interest and contributions, hopefully we can bring most or all of the Metalsmith ecosystem into modern JavaScript.

(And—hopefully—drop support for ancient versions of Node, too! Every time I run npm audit I smile a little. Just one small-but-fantastic part of the many leaps and bounds that Node.js and NPM have made since Metalsmith came onto the scene. ☻)

[Zearin]

(Also: “Metalsmith” is just so much better of a name than “Gatsby”. 👅 )

[itsMeBender]

Hmm, after reading this. And the project problems I have, with my first Metalsmith try-out. I'm not sure to continue. I'm very fond of NodeJS based solutions, but if it doesn't work...

[Zearin]

@itsMeBender What problems are you having? I’d like to help!

[itsMeBender]

@Zearin , Thank you!
I am trying to build a Metalsmith static site from the ground up (no experience at all). Looking for information on the web, I found this site: How to Create a Static Site with Metalsmith. But following these steps, my project didn't work.
But his project can be downloaded here Metalsmith demo. This demo of Metalsmith works! But relies on old Metalsmith node modules. As declared in packages.js.

When I copy those, old node modules, into my own project. My project works! So my project setup, etc. is good enough, to build a Metalsmith static site. Based on old module versions.

When I upgrade my project to the latest versions of the metalsmith modules. It fails again!

I got these error messages:
Error: no files to process. See https://www.npmjs.com/package/metalsmith-in-place#no-files-to-process
or
Error: no files to process. See https://www.npmjs.com/package/metalsmith-layouts#no-files-to-process

Solving this problem, with information from GitHub readme's does not help. Only, copying back the old modules, download by the demo. Solves the problem.

Working packages;
"handlebars": "^4.0.5",
"metalsmith": "^2.1.0",
"metalsmith-assets": "^0.1.0",
"metalsmith-browser-sync": "^1.1.0",
"metalsmith-collections": "^0.7.0",
"metalsmith-feed": "^0.2.0",
"metalsmith-html-minifier": "^2.0.0",
"metalsmith-in-place": "^1.4.3",
"metalsmith-layouts": "^1.6.4",
"metalsmith-mapsite": "^1.0.4",
"metalsmith-markdown": "^0.2.1",
"metalsmith-permalinks": "^0.5.0",
"metalsmith-publish": "^0.1.3",
"metalsmith-word-count": "0.0.4"

current metalsmith-in-place "^4.1.1"
current metalsmith-layouts "^2.1.0"

So what is changed in those major releases; API?, Configuration? ...

[Zearin]

Tell me if any of my statements are incorrect.

Here is what I understand so far…

Current Situation

Okay, so you say that the outdated—but working—demo, uses:

• metalsmith-in-place ^4.1.1
• metalsmith-layouts ^1.6.4

But things break when you update to:

• metalsmith-in-place ^1.4.3
• metalsmith-layouts ^2.1.0

Tracking Down the Problem

The first thing I did was go to metalsmith-in-place’s CHANGELOG.md.

In that changelog, under v2.0.0-beta.1, it says:

• switch to jstransformers for rendering (breaking change)

That sounded familiar to me, so I checked the link you provided. That page says the errors are always caused by one of the following:

• Your pattern does not match any files
• None of your files pass validation, validation fails for files that:
  • Have no extension
  • Are not utf-8
  • Need a jstransformer that hasn't been installed

Aha! So it looks like the error is JSTransformer related.

Possible Solution

In order for metalsmith-in-place to work, you need to install the appropriate JSTransformers. This section of the metalsmith-in-place README.md explains how it works.

I think you will need to at least do the following:

npm rm --save handlebars
npm install --save jstransformer-handlebars

You might need to do something similar for markdown as well, but it’s hard for me to say without access to your project.

---

UPDATE: Fixed a copy-paste error about metalsmith-in-place’s version

[itsMeBender]

Hi, thank you for your support.
I already had the the same line of thoughts. I also red the documentation about the jstransformer modules and tried it also. But no luck here.

my test setup:
"devDependencies": { "jstransformer-handlebars": "^1.1.0", "jstransformer-html": "^0.1.0", "jstransformer-markdown-it": "^2.1.0", "metalsmith": "^2.3.0", "metalsmith-autotoc": "^0.1.5", "metalsmith-in-place": "^4.1.1", "metalsmith-layouts": "^2.1.0", "metalsmith-markdown": "^0.2.2" }
Same error "Error: no files to process. See https://www.npmjs.com/package/metalsmith-in-place#no-files-to-process"

The documentation mentioned a JSON config file. But I'm using a build script. May be it's then the configuration;
`
var
consoleLog = true, // set true for metalsmith file and meta content logging
// modules
debug = require('./lib/metalsmith-debug'),
inplace = require('metalsmith-in-place'),
layouts = require('metalsmith-layouts'),
markdown = require('metalsmith-markdown'),
metalsmith = require('metalsmith');

var
templateConfig = {
engine: 'handlebars',
default: 'layout.html',
directory: './src/layout/',
engineOptions: {
"cache": false
},
partials: './src/partials/',
pattern: '**/*'
}

metalsmith(__dirname)
.clean(true)
.source('./src/html/')
.destination('./build/')
.use(markdown())
.use(inplace(templateConfig))
.use(layouts(templateConfig))
.build(function(err) {
if (err) throw err;
console.log('Build finished!');
});
`

[Zearin]

Can you put the project somewhere I can get a copy of it? I’ll see if I can figure out what’s going on here.

[itsMeBender]

@Zearin - https://github.com/itsMeBender/pr-metalsmith

[Zearin]

Initial Setup

I cloned your repo. I ran node ./build.js. I confirmed a successful build.

Then, I ran npm i --save metalsmith-in-place@latest metalsmith-layouts@latest. I ran node ./build.js. As expected, the build failed.

Debugging

As described in metalsmith-in-place’s “Errors and Debugging”.

export DEBUG=metalsmith-in-place,metalsmith-layouts

This will enable extra debug messages for both plugins.

Using these debug messages (and a lot of back-and-forth referencing README.md files), I got the build to work.

Solution

Do the following, and your build should now work with up-to-date plugins. :)

Step 0: Dependencies

(Step 0? Yes! For sanity.)

Ensure you've updated the plugins.

npm i --save metalsmith-in-place@latest metalsmith-layouts@latest

Part 1: metalsmith-in-place

Metalsmith-in-place and metalsmith-layouts use JSTransformers.

JSTransformers run automatically, based on file extension.

Since metalsmith-markdown automatically changes the
.md file extension to .html, it was tripping up metalsmith-in-place.

Out with the old, and in with the new:

npm rm --save metalsmith-markdown
npm install --save jstransformer-markdown jstransformer-handlebars

(Edit: Added jstransformer-handlebars. Thanks for pointing it out, @itsMeBender!)

Now, you will need to tweak your build.js.

Delete these lines from your imports:

• markdown = require('metalsmith-markdown'),
• handlebars = require('jstransformer')(require('jstransformer-handlebars')),

And delete this line from your metalsmith plugin chain:

• .use(markdown())

You don’t need to include any JSTransformer modules in your build script, because both metalsmith-in-place and metalsmith-layouts look up any installed JSTransformers automatically.

---

At this point, the build almost works. But metalsmith-layouts still gets tripped up.

Part 2: metalsmith-layouts

Can you guess why? Yup…file extensions!

Metalsmith-layouts was looking in your layouts directory and seeing index.html. Then, it thought, “I need to find a JSTransformer that works on *.html files!” … and it couldn’t find any.

• Rename file src/layouts/layout.html to src/layouts/layout.hbs
• In your templateConfig variable, change default: 'layout.html' to default: 'layout.hbs'

Now, metalsmith-layouts will see a file with the *.hbs extension. It will look up jstransformer-handlebars because of that file extension, and complete the transformation.

---

Parting Thoughts

I remember having a lot of trouble using metalsmith-layouts and metalsmith-in-place when I first started using Metalsmith. I also had a lot of trouble with the handlebars template language. It’s really popular, so it must be doing something right. But I just felt like I was fighting myself whenever I was using it.

As a result, I don’t just understand why this was a confounding issue for you—I empathize, because I had similar problems ages ago. ☻

In my case, I wasn’t familiar enough with all of these tools (or with Metalsmith itself) to figure out how to get this all to work. Eventually, I opted to use metalsmith-markdown for simple content, and metalsmith-pug to handle includes (or “partials”, as Handlebars calls them) and templates (or “layouts”). I had to spend a little bit of time to learn the Pug template language, but I was much happier as a result. I also have a lot more flexibility to separate my “layouts” into smaller pieces.

Better documentation from Metalsmith, and better error messages from metalsmith-in-place and metalsmith-layouts would save a lot of people from frustration, I suspect. I’d like to update the Metalsmith docs at some point.

If you agree with any of these thoughts, maybe you could open an Issue at metalsmith-in-place and metalsmith-layouts and ask for more helpful error messages.

Finally, if anything still isn’t working after you did the above, let me know and I’ll dive back in and try to figure out what’s wrong.

Thanks!

[itsMeBender]

Thank you @Zearin, with your step through problem solving approach. I finally solved my problems.
In your description above, you're forgot to mention the installation of jstransformer-handlebars

npm i --save jstransformer-handlebars

I copied your approach of problem solving into my project as WIKI. See GitHub. It will be useful for other people, I think.

[reubano]

Golden! I had a similar problem. Differences were my project isn't using metalsmith-in-place, and instead of metalsmith-markdown it uses metalsmith-markdownit. So because of those differences, I left the following lines in...

import markdown from 'metalsmith-markdownit'

and

.use(markdown({html: true}))

[borisovg]

If anyone is interested, I made this blog site example: https://github.com/borisovg/metalsmith-blog-example, which is a simplified version of what I use for building my own site.

Demo: https://borisovg.github.io/metalsmith-blog-example-demo/

[woodyrew]

I'm unsure why Segment decided to use Gatsby for their blog. As most of the maintainers are not employee's or no longer employees of Segment it is hard to know the answer.

As Ajedi32 said (#288 (comment)) Gatsby and Metalsmith have very different approaches with different pros and cons. As Segment haven't answered the question thus-far I suspect they're not likely to so I'll close this issue.

[burritoIand]

@itsMeBender @Zearin just landed here and ran into same no file to process error, which was finally resolved after finding this thread and adding npm i --save jstransformer-handlebars. Perhaps it would be useful to add that to the main metalsmith doc site for other people new to the project and using handlebars?