Skip to content

Enhancement: Editor for data files in the Hugo 'data' folder. #193

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
zivbk1 opened this issue Nov 25, 2021 · 20 comments
Closed

Enhancement: Editor for data files in the Hugo 'data' folder. #193

zivbk1 opened this issue Nov 25, 2021 · 20 comments
Milestone

Comments

@zivbk1
Copy link

zivbk1 commented Nov 25, 2021

Is your feature request related to a problem? Please describe.

I have a site where there are data files (JSON) used to drive the content in tables.
https://gohugo.io/templates/data-templates/

Describe the solution you'd like

I would like to describe the data model in the files like I do now with content types, and then have a UI to edit the information in the files with a GUI.

Describe alternatives you've considered

I can edit the plain text files by hand, but what's the fun in that? Also, asking my editors to write JSON, YAML, TOML, etc. by hand is a bit much.

Additional context

Here is an example of a UI for the JSON content below.

image

{
  "columns": [
    "Rating",
    "Credit Score"
  ],
  "rows": [
    {
      "cells": [
        "A++",
        "750+"
      ]
    },
    {
      "cells": [
        "A+",
        "710 - 749"
      ]
    },
    {
      "cells": [
        "A",
        "680 - 709"
      ]
    },
    {
      "cells": [
        "B",
        "650 - 679 up to 80% LTV"
      ]
    },
    {
      "cells": [
        "C",
        "620 - 649 up to 70% LTV"
      ]
    }
  ]
}
@estruyf
Copy link
Owner

estruyf commented Jan 12, 2022

Thinking about this one, what if you can define data types via the settings like:

{
  "frontMatter.data.types": [
    {
      "name": "sponsors",
      "file": "[workspace]/data/sponsors.json",
      "schema": {
        "title": "Sponsors",
        "type": "object",
        "required": [
          "url",
          "name",
          "avatar"
        ],
        "properties": {
          "url": {
            "type": "string",
            "title": "URL"
          },
          "name": {
            "type": "string",
            "title": "Name"
          },
          "avatar": {
            "type": "string",
            "title": "Avatar"
          }
        }
      }
    }
  ]
}

If there is data defined like this, the extension would render a new data tab and allow you to start editing the content.

@estruyf
Copy link
Owner

estruyf commented Jan 13, 2022

@zivbk1 been working on a new data dashboard.

What do you think?

image

Settings

The settings look as follows:

{
	"frontMatter.data.files": [
    {
      "id": "sponsors",
      "title": "Sponsors",
      "file": "[[workspace]]/data/sponsors.json",
      "labelField": "name",
      "schema": {
        "title": "Sponsors",
        "type": "object",
        "required": [
          "url",
          "name",
          "avatar"
        ],
        "properties": {
          "url": {
            "type": "string",
            "title": "URL"
          },
          "name": {
            "type": "string",
            "title": "Name"
          },
          "avatar": {
            "type": "string",
            "title": "Avatar"
          }
        }
      }
    },
    {
      "id": "projects",
      "title": "Projects",
      "file": "[[workspace]]/themes/eliostruyf-v2/data/projects.json",
      "labelField": "title",
      "schema": {
        "title": "Projects",
        "type": "object",
        "required": [
          "title",
          "href"
        ],
        "properties": {
          "title": {
            "type": "string",
            "title": "Title"
          },
          "href": {
            "type": "string",
            "title": "Link"
          }
        }
      }
    }
  ]
}

@estruyf estruyf added this to the 6.0.0 milestone Jan 13, 2022
estruyf added a commit that referenced this issue Jan 13, 2022
@zivbk1
Copy link
Author

zivbk1 commented Jan 14, 2022

@estruyf can you use wildcards to match all .json files in a particular folder?
Ex. [[workspace]]/data/tables/*.json

Matches all of these files in the tables folder...

image

@zivbk1
Copy link
Author

zivbk1 commented Jan 14, 2022

@estruyf how would I represent this data structure in frontmatter.data.files?

{
  "name": "Credit-HELOC",
  "columns": [
    "Rating",
    "Credit Score"
  ],
  "rows": [
    {
      "cells": [
        "A++",
        "750+"
      ]
    },
    {
      "cells": [
        "A+",
        "710 - 749"
      ]
    },
    {
      "cells": [
        "A",
        "680 - 709"
      ]
    },
    {
      "cells": [
        "B",
        "650 - 679 up to 80% LTV"
      ]
    },
    {
      "cells": [
        "C",
        "620 - 649 up to 70% LTV"
      ]
    }
  ]
}

@estruyf
Copy link
Owner

estruyf commented Jan 14, 2022

@estruyf can you use wildcards to match all .json files in a particular folder? Ex. [[workspace]]/data/tables/*.json

Matches all of these files in the tables folder...

For now, the data files would have to be defined in the settings. The definition is required to correctly render the form.

@estruyf
Copy link
Owner

estruyf commented Jan 14, 2022

@estruyf how would I represent this data structure in frontmatter.data.files?

...

More testing is required and for sure some styling is missing for lists. The schema for your data would looks as follows:

{
  "id": "test",
  "file": "[[workspace]]/data/tests.json",
  "labelField": "title",
  "title": "Test",
  "schema": {
    "type": "object",
    "properties": {
      "name": {
        "type": "string"
      },
      "columns": {
        "type": "array",
        "items": { "type": "string" }
      },
      "rows": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "cells": {
              "type": "array",
              "items": { "type": "string" }
            }
          }
        }
      }
    }
  }
}

@bwklein
Copy link

bwklein commented Jan 14, 2022

@estruyf can you use wildcards to match all .json files in a particular folder? Ex. [[workspace]]/data/tables/*.json
Matches all of these files in the tables folder...

For now, the data files would have to be defined in the settings. The definition is required to correctly render the form.

Is it possible to define a structure for a data file once, then for each file in the folder indicate that the same structure is used. Repeating the whole structure in the configuration file for every file doesn't seem practical. I'm thinking of a way to say "Everything in this folder has this data structure."

@estruyf
Copy link
Owner

estruyf commented Jan 14, 2022

That would mean files and types need to be split up. So you can have data types, to define the schema, and data files to specify the location and defined type ID.

@bwklein
Copy link

bwklein commented Jan 14, 2022

That would mean files and types need to be split up. So you can have data types, to define the schema, and data files to specify the location and defined type ID.

I like that idea. I could also see something like this working with a new Frontmatter field type that lets you define a set of defined data structures to pick from as a template, and add into an array. This concept would resolve the other issue I have for "blocks" (page_sections) stored in Front matter data. #176

@estruyf
Copy link
Owner

estruyf commented Jan 14, 2022

Support for arrays has been added as well:

image

This allows you to at least use the configuration from above. I'll see what I can do for the type and file splitting.

estruyf added a commit that referenced this issue Jan 14, 2022
estruyf added a commit that referenced this issue Jan 14, 2022
@estruyf
Copy link
Owner

estruyf commented Jan 14, 2022

@zivbk1 data type support has been added. You can specify the schema on file level or on the type level.

On file level is easy when you'll use it a single time, when defining it on type level, you can reuse it.

"frontMatter.data.files": [
	  {
	    "id": "test",
	    "file": "[[workspace]]/data/tests.json",
	    "labelField": "name",
	    "title": "Test",
	    "type": "genericContent"
	  }
],
"frontMatter.data.types": [
	  {
	    "id": "genericContent",
	    "schema": {
	      "type": "object",
	      "properties": {
	        "name": {
	          "type": "string"
	        },
	        "columns": {
	          "type": "array",
	          "items": { "type": "string" }
	        },
	        "rows": {
	          "type": "array",
	          "items": {
	            "type": "object",
	            "properties": {
	              "cells": {
	                "type": "array",
	                "items": { "type": "string" }
	              }
	            }
	          }
	        }
	      }
	    }
	  }
]

Hope this will make it easier for you.

@estruyf
Copy link
Owner

estruyf commented Jan 14, 2022

As discussed, I will add another setting frontMatter.data.folders. This setting will allow the extension to load all files from the folder and use the same data type

estruyf added a commit that referenced this issue Jan 15, 2022
@wemasoe
Copy link

wemasoe commented Jan 16, 2022

@estruyf realy nice project. i have tested the datafolder and can confrim it works.

Can you give me a hint for a good yaml to json scheme editor from an example yaml file?

i am thinking somtehing like a yaml file and to generate a schema from this example file?

entries:
- title: sidebar
  product: Jekyll Doc Theme
  version: 6.0
  folders:

  - title:
    output: pdf
    type: frontmatter
    folderitems:
    - title:
      url: /titlepage.html
      output: pdf
      type: frontmatter
    - title:
      url: /tocpage.html
      output: pdf
      type: frontmatter

  - title: Overview
 ...

or from a json example file

{
  "entries": [
    {
      "title": "sidebar",
      "product": "Jekyll Doc Theme",
      "version": 6,
      "folders": [
        {
          "title": null,
          "output": "pdf",
          "type": "frontmatter",
          "folderitems": [
            {
              "title": null,
              "url": "/titlepage.html",
              "output": "pdf",
              "type": "frontmatter"
            },
            {
              "title": null,
              "url": "/tocpage.html",
              "output": "pdf",
              "type": "frontmatter"
            }
          ]
        },
        {
          "title": "Overview",
          "output": "web, pdf",

...

i have tried to generate a schema for this example file, and did not get it work.

entries:
  - title: other
    folders:
      - title: Other Links
        folderitems:
          - title: Automated links bookmark
            url: /bookmark_one.html
          - title: Bookmark links
            url: /bookmarks.html
          - title: Some link bookmark
            url: /other.html
 "frontMatter.data.types": [
...
   {
      "id": "genericContent1",
      "schema": {
        "type": "object",
        "additionalProperties": false,
        "properties": {
          "entries": {
            "type": "array",
            "items": {
              "$ref": "#/definitions/Entry"
            }
          }
        },
        "required": [
          "entries"
        ]

      },
      "Entry": {
        "type": "object",
        "additionalProperties": false,
        "properties": {
          "title": {
            "type": "string"
          },
          "folders": {
            "type": "array",
            "items": {
              "$ref": "#/definitions/Folder"
            }
          }
        },
        "required": [
          "folders",
          "title"
        ],
        "title": "Entry"
      },
      "Folder": {
        "type": "object",
        "additionalProperties": false,
        "properties": {
          "title": {
            "type": "string"
          },
          "folderitems": {
            "type": "array",
            "items": {
              "$ref": "#/definitions/Folderitem"
            }
          }
        },
        "required": [
          "folderitems",
          "title"
        ],
        "title": "Folder"
      },
      "Folderitem": {
        "type": "object",
        "additionalProperties": false,
        "properties": {
          "title": {
            "type": "string"
          },
          "url": {
            "type": "string"
          }
        },
        "required": [
          "title",
          "url"
        ],
        "title": "Folderitem"
      }
    }
  ]

@estruyf
Copy link
Owner

estruyf commented Jan 16, 2022

@wemasoe easiest is to use a JSON to YAML converter.

I converted the JSON object, to the following schema:

{
  "type": "object",
  "properties": {
    "entries": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "title": {
            "type": "string"
          },
          "output": {
            "type": "string"
          },
          "type": {
            "type": "string"
          },
          "folderitems": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "title": {
                  "type": "string"
                },
                "url": {
                  "type": "string"
                },
                "output": {
                  "type": "string"
                },
                "type": {
                  "type": "string"
                }
              }
            }
          }
        }
      }
    }
  },
  "additionalProperties": false,
  "required": [
    "entries"
  ]
}

@wemasoe
Copy link

wemasoe commented Jan 17, 2022

@wemasoe easiest is to use a JSON to YAML converter.

I converted the JSON object, to the following schema:

{
  "type": "object",
  "properties": {
    "entries": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "title": {
            "type": "string"
          },
          "output": {
            "type": "string"
          },
          "type": {
            "type": "string"
          },
          "folderitems": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "title": {
                  "type": "string"
                },
                "url": {
                  "type": "string"
                },
                "output": {
                  "type": "string"
                },
                "type": {
                  "type": "string"
                }
              }
            }
          }
        }
      }
    }
  },
  "additionalProperties": false,
  "required": [
    "entries"
  ]
}

sorry to say the schema did not work.

i dont get any data entry in the data tab for this file.

Your other.yaml data items
No other.yaml data entries found

Create or modify your other.yaml data
Add new data
Entries

source file:

entries:
  - title: other
    folders:
      - title: Other Links
        folderitems:
          - title: Automated links bookmark
            url: /bookmark_one.html
          - title: Bookmark links
            url: /bookmarks.html
          - title: Some link bookmark
            url: /other.html

when adding an entry old entries are gone.

- entries:
    - title: new title
      output: output
      type: type
      folderitems:
        - title: folderitem title
          url: folder item url
          output: folder item outpur
          type: folderitem type

i just want to reply to your help and give a feedback.

@estruyf
Copy link
Owner

estruyf commented Jan 18, 2022

Recreated it with the new sample you gave, and the following schema can create your data structure:

{
  "id": "data-entries",
  "title": "Data Entries",
  "file": "[[workspace]]/data/data-entries.yml",
  "fileType": "yaml",
  "schema": {
    "type": "object",
    "properties": {
      "entries": {
        "type": "object",
        "properties": {
          "title": {
            "type": "string"
          },
          "folders": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "title": {
                  "type": "string"
                },
                "output": {
                  "type": "string"
                },
                "type": {
                  "type": "string"
                },
                "folderitems": {
                  "type": "array",
                  "items": {
                    "type": "object",
                    "properties": {
                      "title": {
                        "type": "string"
                      },
                      "url": {
                        "type": "string"
                      },
                      "output": {
                        "type": "string"
                      },
                      "type": {
                        "type": "string"
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    },
    "additionalProperties": false,
    "required": [
      "entries"
    ]
  }
}

image

@wemasoe
Copy link

wemasoe commented Jan 19, 2022

@estruyf thx a lot :-)
json schema is not as easy as it seems. hope i will understand it sometime.

@wemasoe
Copy link

wemasoe commented Jan 19, 2022

i have set it for a folder and can confirm it works. Although in "frontMatter.data.types": [ a file is set in the json schema "file": "[[workspace]]/data/data-entries.yml",

 "frontMatter.data.types": [
    {
...
{
  "id": "data-entries",
  "title": "Data Entries",
  "file": "[[workspace]]/data/data-entries.yml",
  "fileType": "yaml",
  "schema": {
    "type": "object",
    "properties": {
      "entries": {
        "type": "object",
        "properties": {
          "title": {
            "type": "string"
          },
          "folders": {
...
  "frontMatter.data.folders": [
    {
      "id": "test1folder",
      "path": "[[workspace]]/data/",
      "labelField": "name",
      "type": "data-entries"
    }
  ],

@estruyf
Copy link
Owner

estruyf commented Jan 20, 2022

@estruyf thx a lot :-) JSON schema is not as easy as it seems. hope I will understand it sometime.

I understand, there are PROs and CONs on each schema/approach to use. I also validated Simple Schema, which seems "simple", but complicates objects IMO. That is why I picked JSON schema.

@estruyf
Copy link
Owner

estruyf commented Jan 20, 2022

i have set it for a folder and can confirm it works. Although in "frontMatter.data.types": [ a file is set in the json schema "file": "[[workspace]]/data/data-entries.yml",

Documentation for these new settings is on its way (still needs to be written before releasing v6).

  • frontMatter.data.types: This only defines the object and its fields. There is no reference to any file or folder. It has two properties: id and schema
  • frontMatter.data.files: Defines how a single file needs to be handled. It can use a schema or an ID of the data type.
  • frontMatter.data.folders: Defines that all files from a folder need to be handled the same. It can use a schema or an ID of the data type.

In your case, you better define the type via the frontMatter.data.types setting and reuse it in the file and folders setting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants