Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create TranslationBaseName from filename only #9343

Closed
bep opened this issue Jan 3, 2022 · 5 comments · Fixed by #11894
Closed

Create TranslationBaseName from filename only #9343

bep opened this issue Jan 3, 2022 · 5 comments · Fixed by #11894
Assignees
Milestone

Comments

@bep
Copy link
Member

bep commented Jan 3, 2022

I'm in the last leg of some improvements in Hugo which requires some simplification in the content path logic . This is also looking a little to the future where content will not just be files, and the task of finding the cannonical path of a piece of content should be a simple task.

Given this content:

content
├── foo.en.md
├── foo.jp.md
└── foo.no.md

Currently, the value of .Page.File.TranslationBaseName, depends on the language configuration. If only en is configured, then TranslationBaseName for foo.jp.md will be foo.jp.

In the new setup, every part of the base filename after the first "." separator will be considered identifiers:

 foo.no.md =>  foo + IDENTIFIER_LANGUAGE + IDENTIFIER_EXTENSION

Note that the Page backed by a foo.unknownlanguage.md will still be assigned the defaultContentLanguage.

I'm sure there will be some subtle site breakage in the above, but I'm also sure that we need to do this.

Shout if you disagree.

@bep bep added the Proposal label Jan 3, 2022
@bep bep added this to the v0.92.0 milestone Jan 3, 2022
@bep bep self-assigned this Jan 3, 2022
@jmooring
Copy link
Member

jmooring commented Jan 3, 2022

Currently, with hugo new content/foo.bar.baz.de.md, .File.TranslationBaseName is foo.bar.baz. Will this change? I have seen several sites where the "." separator is used... abundantly.

@bep
Copy link
Member Author

bep commented Jan 3, 2022

Will this change? I have seen several sites where the "." separator is used... abundantly.

No, your example will still be foo.bar.baz (if that is how it is today). But, we will treat de as a language identifier even if it's not defined as a language in config.toml.

That said, I would not recommend using the "." ... abundantly.

@bep
Copy link
Member Author

bep commented Jan 3, 2022

That said, I would not recommend using the "." ... abundantly.

That said, I don't see us ever adding more identifiers to the filename itself. I have at least one more idea for a "partition" type of identifier -- role based partition (RBAC), but that fits much better in a mount configuration.

bep added a commit to bep/hugo that referenced this issue Jan 5, 2022
TODO(bep) improve commit message.

Hugo has always been a active user of in-memory caches, but before this commit we did nothing to control the memory usage.

One failing example would be loading lots of big JSON data files and unmarshal them via `transform.Unmarshal`.

This commit consolidates all these caches into one single LRU cache with an eviction strategy that also considers used vs. available memory.

Hugo will try to limit its memory usage to 1/4 or total system memory, but this can be controlled with the `HUGO_MEMORYLIMIT` environment variable (a float value representing Gigabytes).

A natural next step after this would be to use this cache for `.Content`.

Fixes gohugoio#8307
Fixes gohugoio#8498
Fixes gohugoio#8927
Fixes gohugoio#9192
Fixes gohugoio#9189
Fixes gohugoio#7425
Fixes gohugoio#7437
Fixes gohugoio#7436
Fixes gohugoio#7882
Updates gohugoio#7544
Fixes gohugoio#9224
Fixes gohugoio#9324
Fixes gohugoio#9352
Fixes gohugoio#9355
Fixes gohugoio#9343
bep added a commit to bep/hugo that referenced this issue Jan 9, 2022
TODO(bep) improve commit message.

Hugo has always been a active user of in-memory caches, but before this commit we did nothing to control the memory usage.

One failing example would be loading lots of big JSON data files and unmarshal them via `transform.Unmarshal`.

This commit consolidates all these caches into one single LRU cache with an eviction strategy that also considers used vs. available memory.

Hugo will try to limit its memory usage to 1/4 or total system memory, but this can be controlled with the `HUGO_MEMORYLIMIT` environment variable (a float value representing Gigabytes).

A natural next step after this would be to use this cache for `.Content`.

Fixes gohugoio#8307
Fixes gohugoio#8498
Fixes gohugoio#8927
Fixes gohugoio#9192
Fixes gohugoio#9189
Fixes gohugoio#7425
Fixes gohugoio#7437
Fixes gohugoio#7436
Fixes gohugoio#7882
Updates gohugoio#7544
Fixes gohugoio#9224
Fixes gohugoio#9324
Fixes gohugoio#9352
Fixes gohugoio#9355
Fixes gohugoio#9343
@bep bep modified the milestones: v0.92.0, v0.93.0 Jan 12, 2022
bep added a commit to bep/hugo that referenced this issue Jan 13, 2022
TODO(bep) improve commit message.

Hugo has always been a active user of in-memory caches, but before this commit we did nothing to control the memory usage.

One failing example would be loading lots of big JSON data files and unmarshal them via `transform.Unmarshal`.

This commit consolidates all these caches into one single LRU cache with an eviction strategy that also considers used vs. available memory.

Hugo will try to limit its memory usage to 1/4 or total system memory, but this can be controlled with the `HUGO_MEMORYLIMIT` environment variable (a float value representing Gigabytes).

A natural next step after this would be to use this cache for `.Content`.

Fixes gohugoio#8307
Fixes gohugoio#8498
Fixes gohugoio#8927
Fixes gohugoio#9192
Fixes gohugoio#9189
Fixes gohugoio#7425
Fixes gohugoio#7437
Fixes gohugoio#7436
Fixes gohugoio#7882
Updates gohugoio#7544
Fixes gohugoio#9224
Fixes gohugoio#9324
Fixes gohugoio#9352
Fixes gohugoio#9343
bep added a commit to bep/hugo that referenced this issue Jan 13, 2022
TODO(bep) improve commit message.

Hugo has always been a active user of in-memory caches, but before this commit we did nothing to control the memory usage.

One failing example would be loading lots of big JSON data files and unmarshal them via `transform.Unmarshal`.

This commit consolidates all these caches into one single LRU cache with an eviction strategy that also considers used vs. available memory.

Hugo will try to limit its memory usage to 1/4 or total system memory, but this can be controlled with the `HUGO_MEMORYLIMIT` environment variable (a float value representing Gigabytes).

A natural next step after this would be to use this cache for `.Content`.

Fixes gohugoio#8307
Fixes gohugoio#8498
Fixes gohugoio#8927
Fixes gohugoio#9192
Fixes gohugoio#9189
Fixes gohugoio#7425
Fixes gohugoio#7437
Fixes gohugoio#7436
Fixes gohugoio#7882
Updates gohugoio#7544
Fixes gohugoio#9224
Fixes gohugoio#9324
Fixes gohugoio#9352
Fixes gohugoio#9343
bep added a commit to bep/hugo that referenced this issue Jan 16, 2022
TODO(bep) improve commit message.

Hugo has always been a active user of in-memory caches, but before this commit we did nothing to control the memory usage.

One failing example would be loading lots of big JSON data files and unmarshal them via `transform.Unmarshal`.

This commit consolidates all these caches into one single LRU cache with an eviction strategy that also considers used vs. available memory.

Hugo will try to limit its memory usage to 1/4 or total system memory, but this can be controlled with the `HUGO_MEMORYLIMIT` environment variable (a float value representing Gigabytes).

A natural next step after this would be to use this cache for `.Content`.

Fixes gohugoio#8307
Fixes gohugoio#8498
Fixes gohugoio#8927
Fixes gohugoio#9192
Fixes gohugoio#9189
Fixes gohugoio#7425
Fixes gohugoio#7437
Fixes gohugoio#7436
Fixes gohugoio#7882
Updates gohugoio#7544
Fixes gohugoio#9224
Fixes gohugoio#9324
Fixes gohugoio#9352
Fixes gohugoio#9343
bep added a commit to bep/hugo that referenced this issue Jan 16, 2022
TODO(bep) improve commit message.

Hugo has always been a active user of in-memory caches, but before this commit we did nothing to control the memory usage.

One failing example would be loading lots of big JSON data files and unmarshal them via `transform.Unmarshal`.

This commit consolidates all these caches into one single LRU cache with an eviction strategy that also considers used vs. available memory.

Hugo will try to limit its memory usage to 1/4 or total system memory, but this can be controlled with the `HUGO_MEMORYLIMIT` environment variable (a float value representing Gigabytes).

A natural next step after this would be to use this cache for `.Content`.

Fixes gohugoio#8307
Fixes gohugoio#8498
Fixes gohugoio#8927
Fixes gohugoio#9192
Fixes gohugoio#9189
Fixes gohugoio#7425
Fixes gohugoio#7437
Fixes gohugoio#7436
Fixes gohugoio#7882
Updates gohugoio#7544
Fixes gohugoio#9224
Fixes gohugoio#9324
Fixes gohugoio#9352
Fixes gohugoio#9343
bep added a commit to bep/hugo that referenced this issue Jan 22, 2022
TODO(bep) improve commit message.

Hugo has always been a active user of in-memory caches, but before this commit we did nothing to control the memory usage.

One failing example would be loading lots of big JSON data files and unmarshal them via `transform.Unmarshal`.

This commit consolidates all these caches into one single LRU cache with an eviction strategy that also considers used vs. available memory.

Hugo will try to limit its memory usage to 1/4 or total system memory, but this can be controlled with the `HUGO_MEMORYLIMIT` environment variable (a float value representing Gigabytes).

A natural next step after this would be to use this cache for `.Content`.

Fixes gohugoio#8307
Fixes gohugoio#8498
Fixes gohugoio#8927
Fixes gohugoio#9192
Fixes gohugoio#9189
Fixes gohugoio#7425
Fixes gohugoio#7437
Fixes gohugoio#7436
Fixes gohugoio#7882
Updates gohugoio#7544
Fixes gohugoio#9224
Fixes gohugoio#9324
Fixes gohugoio#9352
Fixes gohugoio#9343
bep added a commit to bep/hugo that referenced this issue Jan 22, 2022
TODO(bep) improve commit message.

Hugo has always been a active user of in-memory caches, but before this commit we did nothing to control the memory usage.

One failing example would be loading lots of big JSON data files and unmarshal them via `transform.Unmarshal`.

This commit consolidates all these caches into one single LRU cache with an eviction strategy that also considers used vs. available memory.

Hugo will try to limit its memory usage to 1/4 or total system memory, but this can be controlled with the `HUGO_MEMORYLIMIT` environment variable (a float value representing Gigabytes).

A natural next step after this would be to use this cache for `.Content`.

Fixes gohugoio#8307
Fixes gohugoio#8498
Fixes gohugoio#8927
Fixes gohugoio#9192
Fixes gohugoio#9189
Fixes gohugoio#7425
Fixes gohugoio#7437
Fixes gohugoio#7436
Fixes gohugoio#7882
Updates gohugoio#7544
Fixes gohugoio#9224
Fixes gohugoio#9324
Fixes gohugoio#9352
Fixes gohugoio#9343
Fixes gohugoio#9171
@bep bep modified the milestones: v0.93.0, v0.94.0 Mar 1, 2022
@bep bep modified the milestones: v0.94.0, v0.95.0, v0.96.0 Mar 9, 2022
@bep bep modified the milestones: v0.96.0, v0.97.0 Mar 24, 2022
@bep bep modified the milestones: v0.97.0, v0.98.0 Apr 13, 2022
@bep bep modified the milestones: v0.98.0, v0.99.0 Apr 28, 2022
@bep bep removed this from the v0.99.0 milestone May 24, 2022
@bep bep added this to the v0.121.0 milestone Oct 31, 2023
@bep
Copy link
Member Author

bep commented Nov 2, 2023

No, this is not happening.

@bep bep closed this as completed Nov 2, 2023
Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 24, 2023
bep added a commit to bep/hugo that referenced this issue Dec 29, 2023
bep added a commit to bep/hugo that referenced this issue Dec 30, 2023
bep added a commit to bep/hugo that referenced this issue Jan 5, 2024
bep added a commit to bep/hugo that referenced this issue Jan 6, 2024
bep added a commit to bep/hugo that referenced this issue Jan 12, 2024
bep added a commit to bep/hugo that referenced this issue Jan 12, 2024
bep added a commit to bep/hugo that referenced this issue Jan 12, 2024
bep added a commit to bep/hugo that referenced this issue Jan 13, 2024
bep added a commit to bep/hugo that referenced this issue Jan 14, 2024
bep added a commit to bep/hugo that referenced this issue Jan 14, 2024
bep added a commit to bep/hugo that referenced this issue Jan 14, 2024
bep added a commit to bep/hugo that referenced this issue Jan 14, 2024
bep added a commit to bep/hugo that referenced this issue Jan 15, 2024
bep added a commit to bep/hugo that referenced this issue Jan 15, 2024
bep added a commit to bep/hugo that referenced this issue Jan 15, 2024
bep added a commit to bep/hugo that referenced this issue Jan 15, 2024
bep added a commit to bep/hugo that referenced this issue Jan 16, 2024
bep added a commit to bep/hugo that referenced this issue Jan 16, 2024
bep added a commit to bep/hugo that referenced this issue Jan 17, 2024
bep added a commit that referenced this issue Jan 19, 2024
…, and some general spring cleaning

There are some breaking changes in this commit, see #11455.

Closes #11455
Closes #11549

Fixes #10104
Fixes #10380
Fixes #10694
Fixes #11439
Fixes #11453
Fixes #11457
Fixes #11466
Fixes #11540
Fixes #11551
Fixes #11556
Fixes #11654
Fixes #11661
Fixes #11663
Fixes #11664
Fixes #11669
Fixes #11671
Fixes #11807
Fixes #11808
Fixes #11809
Fixes #11815
Fixes #11840
Fixes #11860
Fixes #11883
Fixes #7425
Fixes #7436
Fixes #7437
Fixes #7544
Fixes #7882
Fixes #8307
Fixes #8498
Fixes #8927
Fixes #9192
Fixes #9324
Fixes #9343
bep added a commit that referenced this issue Jan 24, 2024
…, and some general spring cleaning

There are some breaking changes in this commit, see #11455.

Closes #11455
Closes #11549

Fixes #10104
Fixes #10380
Fixes #10694
Fixes #11439
Fixes #11453
Fixes #11457
Fixes #11466
Fixes #11540
Fixes #11551
Fixes #11556
Fixes #11654
Fixes #11661
Fixes #11663
Fixes #11664
Fixes #11669
Fixes #11671
Fixes #11807
Fixes #11808
Fixes #11809
Fixes #11815
Fixes #11840
Fixes #11860
Fixes #11883
Fixes #7425
Fixes #7436
Fixes #7437
Fixes #7544
Fixes #7882
Fixes #8307
Fixes #8498
Fixes #8927
Fixes #9192
Fixes #9324
Fixes #9343
bep added a commit that referenced this issue Jan 24, 2024
…, and some general spring cleaning

There are some breaking changes in this commit, see #11455.

Closes #11455
Closes #11549

Fixes #10104
Fixes #10380
Fixes #10694
Fixes #11439
Fixes #11453
Fixes #11457
Fixes #11466
Fixes #11540
Fixes #11551
Fixes #11556
Fixes #11654
Fixes #11661
Fixes #11663
Fixes #11664
Fixes #11669
Fixes #11671
Fixes #11807
Fixes #11808
Fixes #11809
Fixes #11815
Fixes #11840
Fixes #11860
Fixes #11883
Fixes #7425
Fixes #7436
Fixes #7437
Fixes #7544
Fixes #7882
Fixes #8307
Fixes #8498
Fixes #8927
Fixes #9192
Fixes #9324
Fixes #9343
bep added a commit that referenced this issue Jan 24, 2024
…d some general spring cleaning

There are some breaking changes in this commit, see #11455.

Closes #11455
Closes #11549

Fixes #10104
Fixes #10380
Fixes #10694
Fixes #11439
Fixes #11453
Fixes #11457
Fixes #11466
Fixes #11540
Fixes #11551
Fixes #11556
Fixes #11654
Fixes #11661
Fixes #11663
Fixes #11664
Fixes #11669
Fixes #11671
Fixes #11807
Fixes #11808
Fixes #11809
Fixes #11815
Fixes #11840
Fixes #11860
Fixes #11883
Fixes #7425
Fixes #7436
Fixes #7437
Fixes #7544
Fixes #7882
Fixes #8307
Fixes #8498
Fixes #8927
Fixes #9192
Fixes #9324
Fixes #9343
bep added a commit that referenced this issue Jan 25, 2024
…d some general spring cleaning

There are some breaking changes in this commit, see #11455.

Closes #11455
Closes #11549

Fixes #10104
Fixes #10380
Fixes #10694
Fixes #11439
Fixes #11453
Fixes #11457
Fixes #11466
Fixes #11540
Fixes #11551
Fixes #11556
Fixes #11654
Fixes #11661
Fixes #11663
Fixes #11664
Fixes #11669
Fixes #11671
Fixes #11807
Fixes #11808
Fixes #11809
Fixes #11815
Fixes #11840
Fixes #11860
Fixes #11883
Fixes #7425
Fixes #7436
Fixes #7437
Fixes #7544
Fixes #7882
Fixes #8307
Fixes #8498
Fixes #8927
Fixes #9192
Fixes #9324
Fixes #9343
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.