WIP: discuss and cleanup date handling #4340

vassudanagunta · 2018-01-28T04:30:25Z

I opened this PR to expose and support discussion about peculiarities in the default date metadata logic.

The PR initial commit includes improved test code that:

makes it easier to examine current behavior
adds more coverage both to expose more of the current behavior as well as to help Hugo stability (avoid unintentional behavioral changes).
lays the groundwork for testing new options for default date values

date peculiarities

I believe as Hugo heads toward v1.0 it needs to get tighter with respect to both coherence and stability of behavior. This cases this issue covers is a good example of where Hugo could use some tightening up.

Take a look at the test case table, which includes instructions on how to read it. Each of the test cases pass for the current Hugo release and thus represent current Hugo behavior.

The default value for four dates is ''Jan 1, 0001", and for the fifth its nil. I'd think nil makes most sense, but in any case it should be consistent. (See test cases N1 and N2)
If the frontmatter specifies modified , it fills both LastMod and Modified page values. But if the frontmatter specifies lastmod, it fills only to the LastMod page value. (1L, 1M, 4M and 4E)
The useModTimeAsFallback config setting affects Date and LastMod, but not Modified. Do we really want Date to be set by something as volatile as the file timestamp? Git, for example, doesn't preserve them. Doesn't it make more sense for user templates to make such determination? As long as useModTimeAsFallback sets LastMod, templates have the option to use it when Date is not specified. (M0)
Normally when Date has a value but PublishDate doesn't, PublishDate defaults to Date's value. But when Date is set by useModTimeAsFallback, this doesn't happen. I think this makes sense because PublishDate shouldn't be set by something as volatile as the file timestamp. But in the same vein why does useModTimeAsFallback beat out a lastmod explicitly set in the front matter as a default value for Date? (ML) The right thing happens in the other direction: frontmattter date overrides useModTimeAsFallback to fill out lastMod. (MD)
Maybe my Spidey Sense is wrong, but the way either one of date and publishdate front matter will fill three page date values feels haphazard. (1D and 1P, 21-23) I can see that it is helpful for some people, for lazy templates, but should this be hardcoded Hugo behavior? There's a reason we have three different fields, right? Wouldn't it be better for templates to be explicit about such things? Wouldn't it be better to support something like {{ .LastMod|.Date|.PublishDate }} in templates to support the exact same precedence logic or custom one rather than hardcoding it in Hugo? My observation is that someone opens an issue requesting that such and such date default to some other date, it sounds reasonable, and in the logic goes. But over time, these changes that feel right on their own don't make for a clear bigger picture.

solutions

We could update the logic right here in this PR. But my sense is that there will be no way to please everybody. But there are at least a couple of other options to consider:

Something like what I propose above: a way for templates to say: here is a list of variables in precedence order. Give me the value of the first non-nil one.
Design Proposal: defaultDate & defaultTitle. It addresses Extract date and permalink from filename #285 and Use filename as title when it’s not defined in front matter #3805 and provides a way out of the above wonkiness.

But if the decision is to leave things as they are, I'd still like this PR merged because the tests are much better. Not now though. Let's have a test code review-based discussion.

bep · 2018-01-28T09:27:33Z

Hugo is in some sense legacy software. Code with a long history will have parts that when looking at it now, have room for improvement. But don't use the word "wonky" and other negative loaded words to describe other people's work if you expect to start a serious discussion about improving it.

bep · 2018-01-29T00:36:34Z

As to the PR itself:

The way of testing this, with front matter variations and then create pages from it, was probably convenient at some point.
But if you want full test coverage for all of these front matter variations, this gets wordy very fast
And assuming that we have plenty of front matter unmarshal tests in other places, we should try to get proper unit tests for this

But that requires refactoring of the code and not just adding more tests.

If you want to clean up the code/tests, that would be appreciated. I have looked at your numbered list, and in general, I don't agree.

bep

See my comment in the main thread.

bep · 2018-01-29T08:40:46Z

I have looked at your numbered list, and in general, I don't agree.

Some details on the above:

I'm a little confused about your comments about values for Page date fields that do not exist, e.g. Modified, but I guess you refer to how we pass on values to the .Params object.
Dates are really important to Hugo, so we have date adapters for people coming from Jekyll etc. We need to use those dates during the build (for sorting, expiry date, publish date), so we cannot push that logic to the templates. Besides, that template logic would be ... not pretty.
You could probably argue that we should ask people to get their data correct and just use those values literally (date, lastMod etc.), but after enough of the same questions on "why my site doesn't work as expected", you kind of give in to some compromises.
I agree about the not-so-usefulness of useModTimeAsFallback, but it is there, and some people are using it, so removing it would create some noise. Not sure it's worth it. If useModTimeAsFallback beats lastMod in front matter, that is an obvious bug and should be fixed.

bep · 2018-01-29T12:47:35Z

But thinking about it, useModTimeAsFallback does work as originally designed. But see #4351

vassudanagunta · 2018-01-31T02:01:59Z

I apologize if I offended by use of the word "wonky". But in America at least it simply means "strange" or "feels off". I am not criticizing the project or anyone's work, just giving my honest observation of the current state of something. I don't have all the knowledge that you do about the history of Hugo and how various behavior evolved. But in my experience feedback from the fresh eyes of a newcomer can be valuable. And easy to discard if wrong.

The funny thing is that my recent PRs and Issues are really just offshoots of the real work I'm doing. What I'm after is for Hugo to properly support content without front matter.

That in turn is an offshoot of a standards proposal I'm working on, which I plan to demo with a version of Hugo I've been implementing in a fork. Whenever I come across something that seems like it has universal benefit to Hugo, I think to offer it upstream. Hence PR #4322, which I preceded with PR #4321 to be a good citizen. Producing those tests revealed two bugs which I fixed in #4322, but also triggered my observations in this PR.

With regards to my observations, I personaly don't have a dog in this race. I simply am offering them up for consideration. I believe in simple, coherent and intuitive data models and software interfaces. But I also know I am ignorant of Hugo's history and use cases, and I know current logic is an artifact of compromise, that maybe these compromises are now locked in, and maybe they are not even a big deal. In that case ignore my observations or file them away, but please merge this pull request because the tests provide better coverage and are easier to understand, and they provide a form of documentation of these compromises. They guards against unintentional changes.

vassudanagunta · 2018-01-31T02:03:18Z

Deprecate useModTimeAsFallback #4351

Are you sure you want to do this? See #2239.

vassudanagunta · 2018-02-07T16:07:52Z

See also #3977, which just fell of the map. @bep your last comment there (3 months ago) implied a logic different from what we have today.

This commit adds a new config option which, when enabled and no date is set in front matter, will make Hugo try to parse the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] defaultDate = ["filename"] ``` Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762 Closes gohugoio#4340

This commit tightens up and extends the recently added "combinatorial" page metadata/frontmatter dates test: * Updated the test case representation to make them easier to understand. This will aid discussion on current behavior, which I believe needs examination. * Test code now guards against changes to Hugo's default configuration that unintentionally changes "default default date" behavior. * The tests sometimes didn't distinguish between two types of zero date values: nil and "year one" dates. Fixed. Aids discussion about inconsistencies. * Added support for anticipated new "filename" option for default date, e.g. #285, #3310 and #3762. * `page_test.go` was getting unwieldy. Metadata/frontmatter handling is complicated and important enough that makes sense to give it special treatment. Moved these tests to a new `page_frontmatter_test.go` file. We can incrementally move other frontmatter related tests over too.

This commit adds a new config option which, when enabled and no date is set in front matter, will make Hugo try to parse the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] defaultDate = ["filename"] ``` Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762 Closes gohugoio#4340

This commit adds a new config option which, when enabled and no date is set in front matter, will make Hugo try to parse the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] date = [":filename", "date"] ``` This commit also creates a testable unit from the date front matter handling. We should try to get `:git` as a keyword in the same config map, but that will have to wait for a later time. Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762 Closes gohugoio#4340

This commit adds a new config option which, when enabled and no date is set in front matter, will make Hugo try to parse the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] date = [":filename", "date"] ``` This commit also creates a testable unit from the date front matter handling. We should try to get `:git` as a keyword in the same config map, but that will have to wait for a later time. So, if you want to use the `file modification time`, this can be a good configuration: ```toml [frontmatter] date = [ "date",":fileModTime"] lastMod = ["lastMod" ,":fileModTime", "date"] ``` The current config (what you get when doing nothing): ```toml [frontmatter] date = ["date","publishDate", "lastmod"] lastmod = ["lastmod", "date","publishDate"] publishDate = ["publishDate", "date"] expiryDate = ["expiryDate"] ``` Note: * We have some built-in aliases to the above: lastmod => modified, publishDate => pubdate, published and expiryDate => unpublishdate. * If you want a new configuration for, say, `date`, you can provide only that line, and the rest will be preserved. * All the keywords to the right that does not start with a ":" maps to front matter parameters, and can be any date param (e.g. `myCustomDateParam`).. Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762 Closes gohugoio#4340

This commit adds a new config option which, when enabled and no date is set in front matter, will make Hugo try to parse the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] date = [":filename", "date"] ``` This commit also creates a testable unit from the date front matter handling. We should try to get `:git` as a keyword in the same config map, but that will have to wait for a later time. So, if you want to use the `file modification time`, this can be a good configuration: ```toml [frontmatter] date = [ "date",":fileModTime"] lastMod = ["lastMod" ,":fileModTime", "date"] ``` The current config (what you get when doing nothing): ```toml [frontmatter] date = ["date","publishDate", "lastmod"] lastmod = ["lastmod", "date","publishDate"] publishDate = ["publishDate", "date"] expiryDate = ["expiryDate"] ``` Note: * We have some built-in aliases to the above: lastmod => modified, publishDate => pubdate, published and expiryDate => unpublishdate. * If you want a new configuration for, say, `date`, you can provide only that line, and the rest will be preserved. * All the keywords to the right that does not start with a ":" maps to front matter parameters, and can be any date param (e.g. `myCustomDateParam`). Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762 Closes gohugoio#4340

This commit adds a new config option which, when enabled and no date is set in front matter, will make Hugo try to parse the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] date = [":filename", "date"] ``` This commit also creates a testable unit from the date front matter handling. We should try to get `:git` as a keyword in the same config map, but that will have to wait for a later time. So, if you want to use the `file modification time`, this can be a good configuration: ```toml [frontmatter] date = [ "date",":fileModTime"] lastMod = ["lastMod" ,":fileModTime", "date"] ``` The current config (what you get when doing nothing): ```toml [frontmatter] date = ["date","publishDate", "lastmod"] lastmod = ["lastmod", "date","publishDate"] publishDate = ["publishDate", "date"] expiryDate = ["expiryDate"] ``` Note: * We have some built-in aliases to the above: lastmod => modified, publishDate => pubdate, published and expiryDate => unpublishdate. * If you want a new configuration for, say, `date`, you can provide only that line, and the rest will be preserved. * All the keywords to the right that does not start with a ":" maps to front matter parameters, and can be any date param (e.g. `myCustomDateParam`). * The keywords to the left are the **4 predefined dates in Hugo**, i.e. they are constant values. Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762 Closes gohugoio#4340

This commit adds a new config option which, when enabled and no date is set in front matter, will make Hugo try to parse the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] date = [":filename", ":default"] ``` This commit is also a spring cleaning of how the different dates are configured in Hugo. So, if you want to use the `file modification time`, this can be a good configuration: ```toml [frontmatter] date = [ "date",":fileModTime", ":default"] lastMod = ["lastMod" ,":fileModTime", ":default"] ``` The current `:default` values for the different dates are ```toml [frontmatter] date = ["date","publishDate", "lastmod"] lastmod = ["lastmod", "date","publishDate"] publishDate = ["publishDate", "date"] expiryDate = ["expiryDate"] ``` The above will now be the same as: ```toml [frontmatter] date = [":default"] lastmod = [":default"] publishDate = [":default"] expiryDate = [":default"] ``` Note: * We have some built-in aliases to the above: lastmod => modified, publishDate => pubdate, published and expiryDate => unpublishdate. * If you want a new configuration for, say, `date`, you can provide only that line, and the rest will be preserved. * All the keywords to the right that does not start with a ":" maps to front matter parameters, and can be any date param (e.g. `myCustomDateParam`). * The keywords to the left are the **4 predefined dates in Hugo**, i.e. they are constant values. * The current "special date handlers" are `:fileModTime` and `:filename`. We will soon add `:git` to that list. Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762 Closes gohugoio#4340

This commit makes it possible to extract the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] date = [":filename", ":default"] ``` This commit is also a spring cleaning of how the different dates are configured in Hugo. So, if you want to use the `file modification time`, this can be a good configuration: ```toml [frontmatter] date = [ "date",":fileModTime", ":default"] lastMod = ["lastMod" ,":fileModTime", ":default"] ``` The current `:default` values for the different dates are ```toml [frontmatter] date = ["date","publishDate", "lastmod"] lastmod = ["lastmod", "date","publishDate"] publishDate = ["publishDate", "date"] expiryDate = ["expiryDate"] ``` The above will now be the same as: ```toml [frontmatter] date = [":default"] lastmod = [":default"] publishDate = [":default"] expiryDate = [":default"] ``` Note: * We have some built-in aliases to the above: lastmod => modified, publishDate => pubdate, published and expiryDate => unpublishdate. * If you want a new configuration for, say, `date`, you can provide only that line, and the rest will be preserved. * All the keywords to the right that does not start with a ":" maps to front matter parameters, and can be any date param (e.g. `myCustomDateParam`). * The keywords to the left are the **4 predefined dates in Hugo**, i.e. they are constant values. * The current "special date handlers" are `:fileModTime` and `:filename`. We will soon add `:git` to that list. Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762 Closes gohugoio#4340

This commit makes it possible to extract the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] date = [":filename", ":default"] ``` This commit is also a spring cleaning of how the different dates are configured in Hugo. Hugo will check for dates following the configuration from left to right, starting with `:filename` etc. So, if you want to use the `file modification time`, this can be a good configuration: ```toml [frontmatter] date = [ "date",":fileModTime", ":default"] lastMod = ["lastMod" ,":fileModTime", ":default"] ``` The current `:default` values for the different dates are ```toml [frontmatter] date = ["date","publishDate", "lastmod"] lastmod = ["lastmod", "date","publishDate"] publishDate = ["publishDate", "date"] expiryDate = ["expiryDate"] ``` The above will now be the same as: ```toml [frontmatter] date = [":default"] lastmod = [":default"] publishDate = [":default"] expiryDate = [":default"] ``` Note: * We have some built-in aliases to the above: lastmod => modified, publishDate => pubdate, published and expiryDate => unpublishdate. * If you want a new configuration for, say, `date`, you can provide only that line, and the rest will be preserved. * All the keywords to the right that does not start with a ":" maps to front matter parameters, and can be any date param (e.g. `myCustomDateParam`). * The keywords to the left are the **4 predefined dates in Hugo**, i.e. they are constant values. * The current "special date handlers" are `:fileModTime` and `:filename`. We will soon add `:git` to that list. Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762 Closes gohugoio#4340

This commit makes it possible to extract the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter. This should make it easier to move content from Jekyll to Hugo. To enable, put this in your `config.toml`: ```toml [frontmatter] date = [":filename", ":default"] ``` This commit is also a spring cleaning of how the different dates are configured in Hugo. Hugo will check for dates following the configuration from left to right, starting with `:filename` etc. So, if you want to use the `file modification time`, this can be a good configuration: ```toml [frontmatter] date = [ "date",":fileModTime", ":default"] lastmod = ["lastmod" ,":fileModTime", ":default"] ``` The current `:default` values for the different dates are ```toml [frontmatter] date = ["date","publishDate", "lastmod"] lastmod = ["lastmod", "date","publishDate"] publishDate = ["publishDate", "date"] expiryDate = ["expiryDate"] ``` The above will now be the same as: ```toml [frontmatter] date = [":default"] lastmod = [":default"] publishDate = [":default"] expiryDate = [":default"] ``` Note: * We have some built-in aliases to the above: lastmod => modified, publishDate => pubdate, published and expiryDate => unpublishdate. * If you want a new configuration for, say, `date`, you can provide only that line, and the rest will be preserved. * All the keywords to the right that does not start with a ":" maps to front matter parameters, and can be any date param (e.g. `myCustomDateParam`). * The keywords to the left are the **4 predefined dates in Hugo**, i.e. they are constant values. * The current "special date handlers" are `:fileModTime` and `:filename`. We will soon add `:git` to that list. Fixes gohugoio#285 Closes gohugoio#3310 Closes gohugoio#3762 Closes gohugoio#4340

github-actions · 2022-02-06T02:03:23Z

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

vassudanagunta mentioned this pull request Jan 28, 2018

Design Proposal: defaultDate & defaultTitle #4341

Closed

bep changed the title ~~WIP: discuss and cleanup wonky(?) date handling~~ WIP: discuss and cleanup date handling Jan 28, 2018

bep self-requested a review January 29, 2018 00:26

bep requested changes Jan 29, 2018

View reviewed changes

bep mentioned this pull request Jan 29, 2018

Deprecate useModTimeAsFallback #4351

Closed

vassudanagunta mentioned this pull request Feb 5, 2018

WIP: refining/clarifying data dir functionality #4379

Closed

vassudanagunta mentioned this pull request Feb 22, 2018

Work In Progress: hugolib: Extract date and slug from filename #4436

Closed

bep mentioned this pull request Mar 11, 2018

hugolib: Extract date and slug from filename #4494

Merged

bep closed this in #4494 Mar 11, 2018

dependencies bot mentioned this pull request Apr 2, 2018

Update https://github.com/gohugoio/hugo in repo_dependencies.yml from v0.37 to v0.38 matrixik/dobek.pro#14

Closed

github-actions bot locked as resolved and limited conversation to collaborators Feb 6, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: discuss and cleanup date handling #4340

WIP: discuss and cleanup date handling #4340

vassudanagunta commented Jan 28, 2018 •

edited

Loading

bep commented Jan 28, 2018

bep commented Jan 29, 2018

bep left a comment

bep commented Jan 29, 2018

bep commented Jan 29, 2018

vassudanagunta commented Jan 31, 2018

vassudanagunta commented Jan 31, 2018

vassudanagunta commented Feb 7, 2018

github-actions bot commented Feb 6, 2022

WIP: discuss and cleanup date handling #4340

WIP: discuss and cleanup date handling #4340

Conversation

vassudanagunta commented Jan 28, 2018 • edited Loading

date peculiarities

solutions

bep commented Jan 28, 2018

bep commented Jan 29, 2018

bep left a comment

Choose a reason for hiding this comment

bep commented Jan 29, 2018

bep commented Jan 29, 2018

vassudanagunta commented Jan 31, 2018

vassudanagunta commented Jan 31, 2018

vassudanagunta commented Feb 7, 2018

github-actions bot commented Feb 6, 2022

vassudanagunta commented Jan 28, 2018 •

edited

Loading