Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement: New field type for 'list' or array of sub-collections. #176

Closed
bwklein opened this issue Nov 12, 2021 · 93 comments
Closed

Enhancement: New field type for 'list' or array of sub-collections. #176

bwklein opened this issue Nov 12, 2021 · 93 comments
Labels
enhancement New feature or request
Projects

Comments

@bwklein
Copy link

bwklein commented Nov 12, 2021

With 'Page Sections' or 'Blocks' in front matter data, there needs to be a way to define section/block templates and then manage a list/array of them in front matter.

To add, remove, change order of elements in the list and edit the front matter data in the block elements.

For an example see: https://forestry.io/blog/sawmill-layout-composer-for-hugo-and-forestry/#hugo-example

For a working example of this system in action see:
Website - https://psrfcu.org/
Repo - https://gitlab.com/psrfcu/psrfcu-2019/-/blob/master/content/_index.md

@estruyf estruyf added the enhancement New feature or request label Nov 12, 2021
@estruyf
Copy link
Owner

estruyf commented Nov 12, 2021

Thanks @bwklein for the suggestion! The repo link you shared ends up on a 404 page. Would you be able to check the URL again? Once I can see how you are using it, I can understand the logic a bit better.

@bwklein
Copy link
Author

bwklein commented Nov 12, 2021

Note, that with content files like this, there usually isn't any {{.Content}} in the body of the markdown file. Everything is defined in front matter variables to populate variables in the partials used to render the data into the page block.

It makes the site more like a traditional CMS where the information for the design components (blocks) is stored in the front matter data model and not all in markdown/html within the body of the content file.

@estruyf
Copy link
Owner

estruyf commented Nov 12, 2021

Thank you for providing a sample! This makes it clear to me. As of what I can see this is specific to Forestry. I like the idea, but also think it's a bit abusing the front matter. Shortcodes would be a cleaner way and probably makes it better suitable for other SSGs.

Just to understand a couple of things. These page sections, where do you define these? I assume these are custom things you need to configure before they will be able to be used. Is that true?

@bwklein
Copy link
Author

bwklein commented Nov 12, 2021

Yes, essentially you make design elements or blocks in your partials directory. Then you have a layout type that reads through the front matter and for each 'page_section' it calls the appropriate 'block' and passes the data from that page_section array element into the partial. All of that would need to be setup first, to define what inputs are required for that block partial.

{{- define "main" -}}
  {{ $count := 0}}
  {{- range .Params.page_sections -}}
    {{ partial (printf "blocks/block-%s.html" .block) (dict "Section" . "Page" $ "section_count" $count ) }}
    {{ $count = (add $count 1) }}
  {{- end -}}
{{- end -}}

You also need to define the set of inputs for a particular block and use that for editing the data in the array of page_sections/blocks. In Forestry this is done with their Front Matter Templates, in Cloud Cannon this is done with array structures, see the attached file starting at line 80 for the Cloud Cannon array structures that match with the file I pasted above.

cloudcannon.toml.zip

This approach works great in the situation where you want to reuse design elements on different pages and the editor can move them around and fill in the required information for each block as they 'assemble' the page from top to bottom.

In the site I posted the file for above, there are 19 different blocks that someone could build a page out of.

You are correct shortcodes could work too, but would make it difficult to reopen the editor for the block to make changes to an existing shortcode, unless you had a parser that would read the shortcode string and populate the snippet insertion UI with the values from the shortcode string. It seems like it would be a bit easier to define an array field type and a mini 'content type' for each 'block' that could be added into the array.

@zivbk1
Copy link

zivbk1 commented Nov 12, 2021

I think of shortcodes as a simple way to insert a smaller design element (figure, table, video, tweet) into the body of a larger Markdown file. Where this 'blocks' system is a way of composing entire pages from top to bottom, where each layer in the stack has a specific set of inputs to that specific layer.

@zivbk1
Copy link

zivbk1 commented Nov 12, 2021

One thing to note is that shortcodes will NOT work within the content of a layer field. So you will see in my example file code that there are 'content_markdown' fields for some blocks. If you put a shortcode in there it won't render the shortcode result into the page, there will just be a shortcode string in the content of that block. You can 'markdownify' the content_markdown into HTML, but the shortcodes are ignored. There is an existing Hugo project issue to resolve this and allow for shortcodes in frontmatter text to be rendered.

@zivbk1
Copy link

zivbk1 commented Jan 10, 2022

@estruyf this has become a blocker for me. I can't use it on sites where I have arrays of data in Front Matter. Do you have any suggestions for a workaround, or ideas on how to implement this in FM?

@estruyf estruyf added this to To do in v6.0.0 Jan 10, 2022
@estruyf
Copy link
Owner

estruyf commented Jan 10, 2022

@zivbk1 added it to the board for the next release. At the moment there is no workaround for it. Think it is best to start with #197

@estruyf
Copy link
Owner

estruyf commented Jan 12, 2022

@zivbk1 does #197 get this supported as well? Or are there missing pieces still?

@bwklein
Copy link
Author

bwklein commented Jan 14, 2022

@zivbk1 does #197 get this supported as well? Or are there missing pieces still?

I think it is closer, but the missing piece is a field type in a template where you can choose from a set of defined data structures to add them into an array in the front matter. Do the examples above help or should I setup a more simple example/test for this feature?

@estruyf estruyf removed this from To do in v6.0.0 Jan 21, 2022
@apowell656
Copy link
Sponsor

My two cents - I use Forestry Blocks for a two client sites (Hugo) and one personal(Pelican) to allow the users a familiar front end to contribute content and build pages. Their "Blocks" feature is a game changer for changing pages. I thought about asking for this feature also, but I could not even think of how it would take advantage of the existing FM UI without a modal/dialog box and my Pelican sites don't use shortcodes, so I did not propose them.

If it can happen that would be cool if not I might just make project level snippets.

@estruyf
Copy link
Owner

estruyf commented Jan 22, 2022

Project level snippets is going to be the next major improvement for Front Matter, this is certain

There needs to be a benefit of using these over what VS Code provides by default. We're thinking about:

  • allowing you to add snippets
  • visualising all snippets on a dashboard
  • dialog to insert placeholders, which include inputs/choices/...

One thing that would make it great, is to have a way to get easily update the snippets that are already defined in the content.

About the list field, I've been thinking of using the same data field schema (JSON schema) which is used in the new data view.

@wemasoe
Copy link

wemasoe commented Jan 22, 2022

@estruyf yes that are nice features.

About the list field, I've been thinking of using the same data field schema (JSON schema) which is used in the new data view.

i am still not familiar with json and the scheme technics. Hopefully this project may clarify some aspects of using json and json schemes.

i am thinking of the Specifications that are announced on the Specification documents on the JSON Schema org.

Versioning and looking forward is nice, but if you are out of date it may be getting complicated to be up to date. telling it from a perspective of an unemployed who stucks learning after some issues.
Which schema is the right one? Should this be hardcoded somehow, or would/should be let over to the developer?

i am getting overhelmed trying to find ressources for a clear explanation for json schemes, it looks realy fluent. but, it is a nice solution, but for me not clear enough right now.

also yaml is explaining itself as a clear solution, binding both together with a clean solution might be helpfull.
yaml to json schema and json data, and also json schema and json data to yaml? decide :-) well i need some vacation :-)

@estruyf
Copy link
Owner

estruyf commented Jan 23, 2022

@wemasoe the version of the JSON schema does only matter when you are going to create really complicated data types. If you only need a couple of data fields, the schema is "pretty" easy to understand. There is always a trade-off with each road picked or decision.

The same goes for YAML, it is easy to understand, but this is just the data representation. When you would create a YAML schema to define the data structure, it would also mean you need to think it through a bit and do some trial and error.

Another reason why JSON Schema has been taken is that VS Code its settings are JSON driven and this keeps it all aligned.

Future

If we get more contributors on board, or if I manage to make more time available, one of the ideas that are on the backlog is to create a content type/settings dashboard (#129). This could become a drag and drop kind functionality to create data types and content types. Which eventually might make it easier for any that is not familiar with it.

@wemasoe
Copy link

wemasoe commented Jan 23, 2022

@wemasoe the version of the JSON schema does only matter when you are going to create really complicated data types. If you only need a couple of data fields, the schema is "pretty" easy to understand. There is always a trade-off with each road picked or decision.

The same goes for YAML, it is easy to understand, but this is just the data representation. When you would create a YAML schema to define the data structure, it would also mean you need to think it through a bit and do some trial and error.

Another reason why JSON Schema has been taken is that VS Code its settings are JSON driven and this keeps it all aligned.

Future

If we get more contributors on board, or if I manage to make more time available, one of the ideas that are on the backlog is to create a content type/settings dashboard (#129). This could become a drag and drop kind functionality to create data types and content types. Which eventually might make it easier for any that is not familiar with it.

step by step, everything is going to be good

i have to say the vscode extension Front Matter is a nice project, again, again, again.

@zivbk1
Copy link

zivbk1 commented Feb 6, 2022

@zivbk1 does #197 get this supported as well? Or are there missing pieces still?

I think it is closer, but the missing piece is a field type in a template where you can choose from a set of defined data structures to add them into an array in the front matter. Do the examples above help or should I setup a more simple example/test for this feature?

@estruyf does this help clarify the need or should I provide a better example?

I have begun setting up a Hugo starter that uses this method of page building. https://gitlab.com/zivbk1/hugostarter

You can see in that project that there are already 2 'blocks' that a person can use to build a page.
https://gitlab.com/zivbk1/hugostarter/-/tree/main/layouts/partials/blocks

You can see here the code that is used to take the frontmatter of a page and call the correct block partial into place from top to bottom of the page. https://gitlab.com/zivbk1/hugostarter/-/blob/main/layouts/_default/page.html

Finally, you can see in this file where both blocks are called into the page from the page_sections array.
https://gitlab.com/zivbk1/hugostarter/-/blob/main/content/_index.md

Here is an example of the definition of that 'page' content type. Where there are two choices (for brevity) that can be 'added' into the page_sections front matter array of a 'page' content type.

"frontMatter.taxonomy.contentTypes": [
  {
    "name": "page",
    "fileType": "md",
    "fields": [
      {
        "title": "Title",
        "name": "title",
        "type": "string"
      },
      {
        "title": "Publishing date",
        "name": "date",
        "type": "datetime"
      },
      {
        "title": "Article preview",
        "name": "featured_image",
        "type": "image",
        "isPreviewImage": true
      },
      {
        "title": "Is in draft",
        "name": "draft",
        "type": "draft"
      },
      {
        "title": "Page Sections",
        "name": "page_sections",
        "type": "choice",
        "multiple": true,
        "choices": [
          {
            "icon": "hail",
            "title": "Hero Banner",
            "fields": [
              {
                "title": "Page Block",
                "name": "block",
                "type": "string",
                "hidden": true,
                "default": "hero"
              },
              {
                "title": "Hero Text",
                "name": "heading",
                "type": "string"
              },
              {
                "title": "Background Color",
                "name": "background_color",
                "type": "choice",
                "choices": [
                  {
                    "id": "white",
                    "title": "White"
                  },
                  {
                    "id": "green",
                    "title": "Green"
                  },
                  {
                    "id": "black",
                    "title": "Black"
                  }
                ]
              },
              {
                "title": "Text Color",
                "name": "text_color",
                "type": "choice",
                "choices": [
                  {
                    "id": "white",
                    "title": "White"
                  },
                  {
                    "id": "green",
                    "title": "Green"
                  },
                  {
                    "id": "black",
                    "title": "Black"
                  }
                ]
              }
            ]
          },
          {
            "icon": "newspaper",
            "title": "Markdown Content",
            "fields": [
              {
                "title": "Page Block",
                "name": "block",
                "type": "string",
                "hidden": true,
                "default": "content"
              }
            ]
          }
        ]
      }
    ]
  }
]

In this context, any 'icon' defined would be pulled from something like https://fonts.google.com/icons?selected=Material+Icons

After building that data structure, it seems like it would be nice to define choice arrays somewhere else in the config file that can be reused by referring to them by name. Like a background color and text color set of options that can be used in different choice fields to keep things DRY.

@zivbk1
Copy link

zivbk1 commented Feb 6, 2022

To add to my comment above. It would be nice to see the list of page sections in the UI in an expandable accordion or something like that, where the order of the array can be changed and then edited in the expanded accordion space below the heading or maybe an editing modal that opens for that specific data item in the array. It would also be good to identify what field in the block you want to show in the stacked list of page_sections, similar to "labelField" in the data file definition. This is where it would be nice to just show the icon and then the text for the "labelField".

A lot of what I am talking about seems very similar to how the data file editor works now. Maybe use a similar UI with the stack of sections similar to the array of data items in a data file, and a very similar order and editing UI for those sections.

@estruyf
Copy link
Owner

estruyf commented Feb 10, 2022

Still work in progress, but this is what the new collection data field looks like:

Screenshot 2022-02-10 at 20 16 52

estruyf added a commit that referenced this issue Feb 10, 2022
@estruyf estruyf added this to In progress in 6.1.0 Feb 11, 2022
@estruyf
Copy link
Owner

estruyf commented Feb 11, 2022

Style changes + some fixes have been added. Just committed the first version of the new control.

In order to use it, you'll need to define the field in your content type as follows:

{
  "title": "Page sections",
  "name": "page_sections",
  "type": "data-collection",
  "dataType": "page_sections2"
}

In the frontMatter.data.types, you can define your data type to use for the collection.

Screenshot 2022-02-11 at 11 24 07

Screenshot 2022-02-11 at 11 27 48

@zivbk1
Copy link

zivbk1 commented Feb 11, 2022

Can "dataType": "page_sections2" be set up as a list of data types that can be added into the Record set?

In most use cases there would be different 'blocks' or data structures that would be added to the list of records. Where in your example 'Record 1' would be one data type, 'Record 2' would be another, and so on.

Also, would there be an ability to reorder the list, to put Record 3 above Record 1?

This is some example frontmatter that I would want to make work.

title: Hello World
date: 2022-02-04T21:48:46.000Z
draft: true
layout: page
page_sections:
  - block: hero
    heading_markdown: Hello World
    background_choice: gray
    text_choice: white
  - block: content
type: page
lastmod: '2022-02-04T23:52:45.963Z'

Where the first 'block' or record in the array has a data structure that includes the fields, block, heading_markdown, background_choice, text_choice and a second data type that only contains the field block set to the default value of 'content'.

@estruyf
Copy link
Owner

estruyf commented Feb 11, 2022 via email

@zivbk1
Copy link

zivbk1 commented Feb 15, 2022

I tried the structure that you mentioned and I am getting an unexpected output from that in front matter data.

authors:
      - authors:
          name: Bryan
          social:
            - authors: {}
              social:
                name: Twitter
              fieldGroup: social
        fieldGroup: author

Authors field of type 'block' linked to 'author' field group:
https://gitlab.com/zivbk1/hugostarter/-/blob/main/frontmatter.json#L230

Author field group specification, with a 'social' field of type 'block' with a link to 'social' field group.
https://gitlab.com/zivbk1/hugostarter/-/blob/main/frontmatter.json#L291

Social field group with a single 'service name' field.
https://gitlab.com/zivbk1/hugostarter/-/blob/main/frontmatter.json#L309

@zivbk1
Copy link

zivbk1 commented Feb 15, 2022

Fix for this is coming up.

Image preview is working now!!! 🎉

@estruyf
Copy link
Owner

estruyf commented Feb 15, 2022

I tried the structure that you mentioned and I am getting an unexpected output from that in front matter data.

authors:
      - authors:
          name: Bryan
          social:
            - authors: {}
              social:
                name: Twitter
              fieldGroup: social
        fieldGroup: author

Authors field of type 'block' linked to 'author' field group: gitlab.com/zivbk1/hugostarter/-/blob/main/frontmatter.json#L230

Author field group specification, with a 'social' field of type 'block' with a link to 'social' field group. gitlab.com/zivbk1/hugostarter/-/blob/main/frontmatter.json#L291

Social field group with a single 'service name' field. gitlab.com/zivbk1/hugostarter/-/blob/main/frontmatter.json#L309

Trying it as well, the output looks ok, but the experience is not great. This will require some enhancements.

@zivbk1
Copy link

zivbk1 commented Feb 15, 2022

the output looks ok

How are you getting the output to look right?

I have empty objects from higher level entities down in the lower level.

@estruyf
Copy link
Owner

estruyf commented Feb 15, 2022

Took your example:

Content Type

"frontMatter.taxonomy.contentTypes": [
    {
      "name": "page",
      "fields": [
        {
          "title": "Authors",
          "name": "authors",
          "type": "block",
          "fieldGroup": [
            "author"
          ]
        },

Field Groups

"frontMatter.taxonomy.fieldGroups": [
  {
    "id": "author",
    "labelField": "name",
    "fields": [
      {
        "title": "Author Name",
        "name": "name",
        "type": "string",
        "single": true
      },
      {
        "title": "Social",
        "name": "social",
        "type": "block",
        "fieldGroup": [
          "social"
        ]
      }
    ]
  },
  {
    "id": "social",
    "labelField": "name",
    "fields": [
      {
        "title": "Service Name",
        "name": "name",
        "type": "string"
      }
    ]
  }
]

As mentioned the UX needs to have some love in order to make it usable.

@apowell656
Copy link
Sponsor

@estruyf ahh, so field groups will get me closer to what I need along with the block field. I will hopefully have time to test it tomorrow.
@bwklein the author information I use appears on the homepage only once at the bottom.

@estruyf
Copy link
Owner

estruyf commented Feb 15, 2022

Added a couple of fixes + you can now save/cancel the data.

Save: closes the form
Cancel: resets the data back before you started editing

Screenshot 2022-02-15 at 07 40 03

@zivbk1
Copy link

zivbk1 commented Feb 15, 2022

Added a couple of fixes + you can now save/cancel the data.

Save: closes the form Cancel: resets the data back before you started editing

@estruyf it is working perfectly!!!! Thank you!

image

@bwklein
Copy link
Author

bwklein commented Feb 15, 2022

Can labelField be used for the text in the records list, with the name of the record type (author & social) being used as a fallback if labelField is not used?

@estruyf
Copy link
Owner

estruyf commented Feb 15, 2022

@zivbk1 can be done for sure!

@zivbk1
Copy link

zivbk1 commented Feb 15, 2022

At this point, I don't think this is a blocker anymore, I can start work on the 2 projects requiring Field Groups in frontmatter. Thank you for getting this done so quickly.

@estruyf
Copy link
Owner

estruyf commented Feb 15, 2022

Great to hear! Let me know if there are things to optimize or if you find additional issues.

@zivbk1
Copy link

zivbk1 commented Feb 15, 2022

I've updated my demo project with support for this. https://gitlab.com/zivbk1/hugostarter

I am thinking of adding this into your examples for a Hugo site.

@estruyf
Copy link
Owner

estruyf commented Feb 15, 2022

Label field support added 🚀

image

@apowell656
Copy link
Sponsor

apowell656 commented Feb 17, 2022

@estruyf , one word amazing! I have been able to rebuild my troublesome about-section. A couple of things, is it possible to add an image field to fieldGroup and add a default string to a field?
When I select an image the dialog closes or when I look at the current about-section, the image is not found (that could be because it is not in the media folder).
A workaround for the default string is to create a string with one choice and I tried to use a placeholder without success.

@estruyf
Copy link
Owner

estruyf commented Feb 17, 2022

@apowell656 thanks! 🙏

Don't think I completely follow what you are asking for. Could you share some more details or screenshots?

About the default value, this is something the image field is not yet supporting. I'll create an issue for it to get that in as well.

@estruyf
Copy link
Owner

estruyf commented Feb 17, 2022

The default values enhancement #255 is now included.

@apowell656
Copy link
Sponsor

@estruyf my image issue may be related to an earlier comment dealing with images, but I have 'the image cannot be loaded' error for images referenced in the fieldGroup.
I believe my issue may is because they are in the asset folder of the theme and not in the media folder, but when I try to use an image from the media folder, the dialog box closes and just shows the rest of the block section. So I cannot test to see if an image works in the fieldGroup.
I am stuck in meetings until later tonight, so I won't be able to test until then, and thanks for adding the default value support.

estruyf added a commit that referenced this issue Feb 18, 2022
estruyf added a commit that referenced this issue Feb 18, 2022
@zivbk1
Copy link

zivbk1 commented Feb 25, 2022

This is all working well for me now.

@estruyf
Copy link
Owner

estruyf commented Feb 25, 2022

Thank you @zivbk1 to verify this.

@estruyf estruyf moved this from In progress to Done in 6.1.0 Feb 25, 2022
@estruyf estruyf closed this as completed Feb 28, 2022
@estruyf estruyf mentioned this issue Feb 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
No open projects
6.1.0
  
Done
Development

No branches or pull requests

5 participants