feat(theme-docs): support a way to ignore some contents

[nozomuikuta]

Is your feature request related to a problem? Please describe.

I'm creating a static doc site on the top of content-theme-docs, which will have many page eventually.
For now, some pages are completed and ready to be published, while other markdown files are being written.

I decided to publish the doc site with some pages at first, and looked for a way to exclude or hide work-in-progress contents.
But, I don't want to move such files to other directory because then I can't access them in the doc site with yarn run dev.

I just want work-in-progress pages to be excluded from production build and generate.

At first, I tried ignore option and generate.exclude option like below but I found that they cannot prevent contents from being added to database and rendered in the sidebar.

{
  ignore: process.env.NODE_ENV === 'production' ? [ /^\/sample\/.*/ ] : []
}

{
  generate: {
    exclude: process.env.NODE_ENV === 'production' ? [ /^\/sample\/.*/ ] : []
  }
}

Secondly, I tried to do something on database in content:file:beforeInsert hook, but I couldn't do anything because the files that I want to be excluded namely had not yet inserted.

FYI: As I described in #576, theme-docs replaces user's content:file:beforeInsert hook. So I modified the installed @nuxt/content's file directly to test it.

As far as I read parts of code related to database, it ignores node_modules and hidden files whose name begin with . by default. So, I guess it's possible to add a logic here to ignore some contents.

Describe the solution you'd like

Adding either content.ignore or content.exclude which is an array of regular expressions will be so nice, in that they align with ignore and generate.exclude options.

{
  content: {
    ignore: process.env.NODE_ENV === 'production' ? [ '**/wip.md' ] : [ ]
  }
}

Describe alternatives you've considered

Adding content.dirs which is an array of paths would be also possible. In this case, we can dynamically include or exclude directory for work-in-progress pages, for example, based on process.env.NODE_ENV.

{
  content: {
    dirs: process.env.NODE_ENV === 'production' ? [ 'content' ] : [ 'content', 'content-wip' ]
  }
}

But, we have to resolve conflict between content.dir and content.dirs.

Additional context

Although I'm not so familiar with the database-related parts of code, I will give it a try to implement this feature and submit a draft PR, so that we can discuss this feature in more detail. 🙋

Thank you for reading so long message. 🙇

[nozomuikuta]

I came up with another idea: setting a flag like publish: true/false in the front matter. This would be more blog-like DX rather than documentation, but the way is much easier than defining patterns to match.

[Twinki14]

I have a large amount of markdown files in my project I'd like to hide only in the sidebar, and only link to from specific markdown content. I'd like to see another optional flag on top of publish such as sidebar: true/false

[atinux]

Thanks for the idea @nozomuikuta

Indeed for DX, it will be better to have a published or draft attribute directly.

[atinux]

Also, why not creating a branch for your draft content instead?

[atinux]

I have a large amount of markdown files in my project I'd like to hide only in the sidebar, and only link to from specific markdown content. I'd like to see another optional flag on top of publish such as sidebar: true/false

@Twinki14 Can you open another issue for this please?

[nozomuikuta]

@atinux

Thank you for creating PR! 👏
All I want to achieve seems to become possible. 😀

In addition, combination with #619 will enable us to have a full control on contents.

---

Also, why not creating a branch for your draft content instead?

Indeed, this approach is an option.

For your information, my team's use case was like the following:

I was in charge of design structure of documentation. I defined what pages should be included as well as in what order they should be set. After defining the structure, I created "empty" pages which have only front matters, where title and position are set.

The reason I created all the pages first was that writers are not frontend developers and not familiar with frontend tool chains.
I wanted them to focus on writing actual content without caring about business logics. Another concern was that position might be out of control when many people set it to pages at the same time in many branches.

While writing content, we found that there are some new pages to add (it's natural, isn't it?). So, I created new pages with position in the central branch and let writers pull and create branch to fill out the contents.

Due to schedule management, we decided to publish some pages and to start incremental release, so that users of our product can see documentation, with only essential pages at first.

By the way, if draft flag is available, writers could merge pages to the central branch, and the editor could check all the pages at once and comprehensively, instead of looking through PRs from branches.

I'm not sure if there is a standard workflow to complete documentation (yes, I don't work for publishing company). It may be common to publish documentation when everything is ready. However, I believe providing draft flag can cover much more use cases.

[johanvergeer]

I ran into the same issue (wanted to filter out draft posts in production) and solved it using the builder design pattern. This way the posts will still be generated, but they won't show up in the overview pages.

I first created a WhereFilterBuilder, which contains a couple of functions to create a where filter. I'm using TypeScript, but the same can be done with JavaScript.

interface WhereFilter {
  status?: StatusFilter
  tags?: Object
}

/**
 * Creates a filter to be used in the `where` method of a {@QueryBuilder}
 */
export default class WhereFilterBuilder {
  private whereStatement: WhereFilter = {}
  private readonly vue: Vue

  constructor(vue: Vue) {
    this.vue = vue

    this._addStatusIfNotDevMode()
  }

  /**
   * posts with status draft should only be displayed when running in dev mode
   */
  _addStatusIfNotDevMode(): void {
    if (!this.vue.$nuxt.context.isDev) {
      this.whereStatement.status = { $ne: 'draft' }
    }
  }

  /**
   * onlu include posts that contain the given tag
   * @param tag name of the tag that are used by posts that should be included
   */
  addTagsContain(tag: string): WhereFilterBuilder {
    this.whereStatement.tags = { $contains: tag }
    return this
  }

  /**
   * Build the filter
   */
  build(): WhereFilter {
    return this.whereStatement
  }
}

Now this builder can be used in the QueryBuilder:

<script lang="ts">
import { IContentDocument } from '@nuxt/content/types/content'
import { Component, Vue } from 'nuxt-property-decorator'
import WhereFilterBuilder from '~/utils/WhereFilterBuilder'

@Component
export default class Tag extends Vue {
  private articles: IContentDocument | IContentDocument[] = []

  async fetch() {
    this.articles = await this.$nuxt
      .$content('blog')
      .where(
        new WhereFilterBuilder(this)
          .addTagsContain(this.$route.params.tag)
          .build()
      )
      .only(['title', 'slug', 'description', 'createdAt', 'body'])
      .sortBy('createdAt', 'asc')
      .fetch()

    if (this.articles.length < 1) return this.$nuxt.context.redirect('/404')
  }
}
</script>

I decided to include the filter on dev status by default (in the constructor) so it cannot be forgotten, (and saves some duplicate code), but you could also choose to call it each time you need it.

[leviwheatcroft]

There's a bunch of different use cases for excluding various files from being inserted into content.

In my own case I have a folder repo with a bunch of markdown files, but only some of which are relevant to the project I'm building with nuxt.

I don't particularly like @nozomuikuta 's suggestion of including globs in nuxt.config.js > content, because this only allows you to exclude files by path.

I don't particularly like the suggestion to exclude based on a published or draft property, because it doesn't allow filtering by any other metric.

@johanvergeer 's suggestion of building where clauses to exclude content is a workaround, but it doesn't address the problem of reading unnecessary content into the db.

this is the code that calls the beforeInsert hook and then inserts the document. You could add a try / catch like this.

    for (const item of items) {
      try {
        await this.callHook('file:beforeInsert', item)
        this.items.insert(item)
      } catch (err) {
        if (err.message.length)
          logger.warn(`Skipped file: ${err.message}`)
      }
    }

Then in nuxt.config.js you could set up filtering like this, to exclude files where published is falsy for example:

  hooks: {
    'content:file:beforeInsert': async (document) => {
      if (!document.published)
        throw new Error('is not published')
    }
  }

... or you could throw new Error() to squelch the warnings.

I think this fits well with the current architecture, doesn't require additional features or functions, and serves all possible use cases.

Edit: actually I just tried this and it won't work. this.callHook provided by hookable package catches errors internally so you can't throw from inside a hook. That being the case, I think adding the ability to include an ignore array or function in the config might be the best approach.

[hacknug]

I believe this has been addressed in v2 where you can mark a document as: draft, partial or ignore it.

[farnabaz]

This is something you can in achieve using partials in nuxt/content@v2.

I'm closing this one because nuxt/content@v1 will not receive any update for improvements.