Enhancement: Perform the Great Fork Merge

[lcn2]

The Great Fork Merge

The Great Fork Merge will occur when the multi-thousand commits that the temp-test-ioccc repo is ahead of the Official IOCCC winner repo are bright back to main Official www.ioccc.org web site.

TODOs

In order to perform the Great Fork Merge, the following tasks need to be completed:

• Revise status.json format.

• Write a new tool to generate the new top level status.html from status.json

NOTE: This may require a rewrite of the bin/ioccc-status.sh tool as well change the format of the from status.json file.

• Fix name www under RHEL 9

Port the bin tools tools to run under RHEL 9 version of Linux. With this port, the IOCCC judges should be able to use these tools on a wide enough variety of systems for their purposes.

• Complete issue Enhancement: Address XXX lines in top level files #2008

• Complete issue Enhancement: improve the consistency of winning directories #3

• Verify that the format of markdown files is good

See also comment 1993767060.

• Complete issue Enhancement: add templates for issue forms #858

• Complete issue Enhancement: Improve the IOCCC manifest #1933

• Integrate manifest.numbers into the build process

This step requires the completion of issue #1933 before it can be fully designed.

• Complete issue Enhancement: improve the look and feel of the constructed HTML files #2006

• Add an FAQ about how to handle so-called spoiler content.

See comment 919283.

• Add a fun "Fixing the judges" menu item for fun

See discussion 6542769.

• Move all FAQ content into their own menu section

See comment 9192931.

• Verify that the top level README.md contains suitable content for a GitHub repo README file.

The exception, of course, is the initial text down to the XXX that indicates this is a test repo.

• Verify that the installed location(1) tool is up to date

Be sure the repo is up to date and perform a make install.

• Remove all trailing whitespace from markdown files

Use this to look for any whitespace from markdown files

find . -name '*.md' -print0 | xargs -0 grep -E -l '[[:space:]][[:space:]]*$'

• Verify that the Nu Html Checker finds no errors, warnings nor info messages for all generated HTML

After making sure that the temp-test-ioccc repo is up to date and related GitHub pages have been rendered, use the ✓ on the navbar to check all generated HTML pages.

Fix any errors, warnings and info messages reported. Update the temp-test-ioccc repo and recheck those pages.

• Verify the markdown code blocks are properly indented

Use:

find . -name '*.md' ! -name markdown.md -print0 | xargs -0 pcregrep -M '^``.*\n[^|`\n \t]'

• Determine that we are about ready to perform the Great Fork Merge

NOTE: Once the above TODOs have been completed, the following last minute TODO actions must be completed in relatively short order before the Great Fork Merge takes place.

Be sure that this exits without error and does not make any changes to files:

make www timestamp

Near Final TODOs

• Purge repo of previous ioccc.tar.bz2 files

Using the method outlined in GitHub notes on Removing sensitive data from a repository, remove ioccc.tar.bz2 from the few previous commits, in order to reduce the size of this repo somewhat.

• Consider reducing Git repository size

See stack overflow replies.

See also How to clean up the git repo and reduce its disk size.

Test these idea first in a clone of this cloned repo.

• Perform make update on macOS and push out any changes

• Verify that running make update again on macOS changes nothing

If not, reset all TODOs under the Near Final TODOs section and fix the cause.

• Verify make www tar on RHEL 9 changes nothing

If not, reset all TODOs under the Near Final TODOs section and fix the cause.

• update README.md

Remove from the top line down to an including the line "## What follows is the actual README.md file .." when the great fork merge is ready to be performed.

• Change references from temp-test-ioccc to winners in faq.md

When the great fork merge is performed, references to temp-test-ioccc should be changed to winners and then the XXX note should be removed.

• Update FAQ 1.2 with the same of the date of the great fork merge

When the great fork merge is performed, the XXX marked date should be added and the XXX note should be removed.

• Change bin tool to use www.ioccc.org

Change default URLs and REPOs to refer to the https://www.ioccc.org web site and winner repo.

• Change references to use www.ioccc.org

In the cases where a absolute URL is required, change that URL to be under https://www.ioccc.org instead of under https://ioccc-src.github.io/temp-test-ioccc/.

This includes markdown files, HTML files, var.mk, Makefiles, text files, etc.

Try:

make www
find . \( -name .git -o -name NOTES \) -prune -o -type f -print0 | xargs -0 grep -l -F temp-test-ioccc

• Rebuild web tree via make www

• Scan repo to verify that references to temp-test-ioccc are gone

If not, reset all TODOs under the Near Final TODOs section and fix the cause.

• Add news event announcing the new www.ioccc.org web site.

Edit news.md as needed.

• Verify that the local temp-test-ioccc repo is clean

Verify that:

git status

reports:

On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

• Verify that the local temp-test-ioccc repo is up to date

Fetch any last minute changes:

cd docroot/temp-test-iocc

git fetch && git rebase

git clean -f

git status

The last command should report:

On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

• Verify manifest.numbers agrees with the local temp-test-ioccc repo

Perform the 12 steps as noted in comment 2089758002.

• Remove tmp directory.

• Update all web pages on macOS

make clobber www

• Build all compressed tarballs on macOS

make clobber tar

• Update all timestamp related files on macOS

make timestamp

• Commit all changes and push the result to the temp-test-ioccc repo

git commit -m'final pre-Great Fork Merge'

git push

• Re-Verify that the local temp-test-ioccc repo is up to date

Fetch any last minute changes:

cd docroot/temp-test-iocc

git fetch && git rebase

git clean -f

git status

The last command should report:

On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

• Re-Verify make update on macOS changes nothing

If not, reset all TODOs under the Near Final TODOs section and fix the cause.

• Re-Verify make www tar on RHEL 9 changes nothing

If not, reset all TODOs under the Near Final TODOs section and fix the cause.

• Perform the Great Fork Merge

Click Create Pull Request from the temp-test-ioccc repo.

From the Official IOCCC winner repo aspect the pull request.

Post Great Fork Merge TODOs

Once the Great Fork Merge occurs and the official IOCCC winner repo and related Official www.ioccc.org web site has been updated, these TODOs need to be performed on the official IOCCC winner repo:

• Close issue Question: Issues that aren’t really major issues but are still issues #5

• Transfer any remaining issues to the official IOCCC winner repo

• Mark the temp-test-ioccc repo read-only

From temp-test-ioccc repo settings click on the ((Archive this repository)) button.

• Verify site links work

Process TBD.

Fix any problems found as needed.

[xexyl]

Please assign this to me.

And as I noted before: if needed I would like to update my script that updates the status.json file.

Thanks.

I will have to look at this issue later on today as it seems rather important!

[xexyl]

Also I am not sure if I can finish the year README files before the 10th but I should be able to finish them sometime soon anyway.

But for now I must go do other things again. Back in a while.

[xexyl]

In order to perform the Great Fork Merge, the following tasks need to be completed:

• Revise status.json format.
• Write a new tool to generate the new top level status.html from status.json

NOTE: This may require a rewrite of the bin/ioccc-status.sh tool as well change the format of the from status.json file.

It's funny you should say that .. the generating the file. Do you mean creating the file rather than modifying it? I actually had the thought of adding an option to the script to do that and I seem to recall that I almost did do it. It would be easy to do. But since the format is not necessarily the final format I will hold off on that.

• Complete issue Enhancement: improve the consistency of winning directories #3

The anticipated completion date for this issue is 2024 March 10 2:30 (2:30AM) Pacific.

Like I said over there mostly it is done: except for the YYYY/README.md files. But I can do those after 10 March if necessary so I have no problem with it being closed. You were going to take care of the silencing the warnings and right now would be fine as I don't anticipate modifying any Makefiles or if I'm honest for now anything at all (except maybe the FAQ which you asked me to look at). That probably will hold for a day or two at least but even if I do make other changes it'll likely only be the YYYY/README.md files. I say I might not finish it before 10 March but I might yet: I just have to have the energy and time and the former seems to be a problem lately I'm afraid. Even so I might be able to do some of it today: that remains to be seen however.

• Complete issue Enhancement: add templates for issue forms #858

Okay in order to do this I think I even created a new repo to test how this works. I think the other issues should come first though. Is that correct?

• Complete issue Enhancement: Improve the IOCCC manifest #1933

Ah yes ... I forgot. I have to update descriptions and also check the html files! It's been very hectic and I totally forgot about this. I was also holding off as you were planning many changes that would conflict. Those changes are now done so I can look at this sometime soon too. If I don't please remind me.

• Complete issue Enhancement: improve the look and feel of the constructed index.html files #2006

Right ... well this one might be good to look at when I look at the manifest too. After all the result of the manifest is in the index.html files.

• Complete issue Enhancement: Address XXX lines in top level files #2008

Well some of this cannot be done until the submit server is ready, right?

Near Final TODOs

Once the above TODOs have been completed, the following last minute TODO actions must be completed in relatively short order before the Great Fork Merge takes place:

• Change HTML references to use www.ioccc.org

In the (few?) cases where a absolute URL is required, change that URL to be under https://www.ioccc.org and NOT this repo's top level URL.

sgit(1) should be useful for this one! :)

• Add news event announcing the new www.ioccc.org web site.

Edit news.md as needed.

That's a great idea .. looking forward to see what you have in mind! Almost eager enough to try and get myself to work on YYYY/README.md files today but I think that might be a mistake with how tired I feel :(

• Verify that the installed location(1) tool is up to date

How do you mean verify that it's up to date? Do you mean locally installed? But what version should it be then? Did you have other updates in mind? (I in any case have the most recent version at the other repo.) Then again perhaps this is something you have to do so I might not have to worry about it. Or maybe a tool here uses it and I do have to worry about it. But as noted I do have them installed so it's fine.

• Verify manifest.numbers agrees with the local temp-test-ioccc repo

How do you mean verify that it agrees with it when it's in the repo? Though again I guess this might have to be something you do so I don't need to worry about it: updating the manifest I do have to worry about but perhaps not this step?

Perform the 12 steps as noted in comment 2089758002.

Don't forget the script I wrote! It's very useful. By default it does not commit automatically but there is an option to allow it to do so. It even asks if you're on the right branch.

• Remove tmp directory.

Ever notice what tmp spells backwards? :-)) Well okay it is a British English abbreviation but still :-)

Post Great Fork Merge TODOs

Once the Great Fork Merge occurs and the official IOCCC winner repo and related Official www.ioccc.org web site has been updated, these TODOs need to be performed on the official IOCCC winner repo:

• Close issue Question: Issues that aren’t really major issues but are still issues #5

I've enjoyed this one though I must say! A lot of good things, a lot of fun, a lot of laughs and stories and other things ... thanks for the privilege and honour!

• Transfer any remaining issues to the official IOCCC winner repo

Good idea.

• Mark the temp-test-ioccc repo read-only

From temp-test-ioccc repo settings click on the ((Archive this repository)) button.

THANK YOU for going for that (as I suggested) rather than deleting it.

• Verify site links work
  Ah yes .. that'll be fun! :-)

Process TBD.

Fix any problems found as needed.

Of course.

At the other repo I left a comment regarding an issue I have (about finishing something we talked about the other day) but I think that and this comment is probably all I'll manage to do today for the contest. But who knows. I might be able to take a look at some things at this repo too. For now I'll be afk (or I intend to be afk soon). I think today though I will mostly be reading when not doing the things I have to do.

[lcn2]

Incomplete TODO

NOTICE: This TODO list will be built over a few days. Until this notice is removed, consider this list to be incomplete.

:-)

=-=

• Revise status.json format.
• Write a new tool to generate the new top level status.html from status.json

NOTE: This may require a rewrite of the bin/ioccc-status.sh tool as well change the format of the from status.json file.
It's funny you should say that .. the generating the file. Do you mean creating the file rather than modifying it?

The content of status.json is still TBD, and so is the tool that will mange it.

=-=

The anticipate completion date for this issue is 2024 March 10 2:30 (2:30AM) Pacific.

Like I said over there mostly it is done: except for the YYYY/README.md files. But I can do those after 10 March if necessary so I have no problem with it being closed.

Sure.

BTW: issue #2006 does indicate that it covers such files, and the title has been changed to say "constructed HTML files".

[lcn2]

• Complete issue Enhancement: add templates for issue forms #858

Okay in order to do this I think I even created a new repo to test how this works. I think the other issues should come first though. Is that correct?

It is a ((*top priority)) issue and has been for a while. It pertains to something that is much needed after the Great Fork Merge when people start considering opening up issues on the revised web site.

=-=

• Complete issue Enhancement: Address XXX lines in top level files #2008

Well some of this cannot be done until the submit server is ready, right?

We will need to put in some placeholder text that indicates the exact process has not been created, but will be before the IOCCCMOCK is held.

=-=

• Verify that the installed location(1) tool is up to date

How do you mean verify that it's up to date? Do you mean locally installed? But what version should it be then? Did you have other updates in mind? (I in any case have the most recent version at the other repo.) Then again perhaps this is something you have to do so I might not have to worry about it. Or maybe a tool here uses it and I do have to worry about it. But as noted I do have them installed so it's fine.

As we need to be the one be performing these last steps, you @xexyl do not need to worry about the details.

=-=

• Remove tmp directory.

Ever notice what tmp spells backwards? :-)) Well okay it is a British English abbreviation but still :-)

:-)

[xexyl]

• Complete issue Enhancement: add templates for issue forms #858

Okay in order to do this I think I even created a new repo to test how this works. I think the other issues should come first though. Is that correct?

It is a ((*top priority)) issue and has been for a while. It pertains to something that is much needed after the Great Fork Merge when people start considering opening up issues on the revised web site.

Let me reword that. Besides #3 (and I hope I can look at some of it today in a while but we shall see) do you have an order of priority for the remaining issues or should it be 'whatever I have the time and energy and inclination for'?

That idea of whatever I can work on would be useful but if you have an order I could try and work with it.

=-=

• Complete issue Enhancement: Address XXX lines in top level files #2008

Well some of this cannot be done until the submit server is ready, right?

We will need to put in some placeholder text that indicates the exact process has not been created, but will be before the IOCCCMOCK is held.

That all makes sense to me yes. Thanks.

=-=

• Verify that the installed location(1) tool is up to date

How do you mean verify that it's up to date? Do you mean locally installed? But what version should it be then? Did you have other updates in mind? (I in any case have the most recent version at the other repo.) Then again perhaps this is something you have to do so I might not have to worry about it. Or maybe a tool here uses it and I do have to worry about it. But as noted I do have them installed so it's fine.

As we need to be the one be performing these last steps, you @xexyl do not need to worry about the details.

I guessed that but I wanted to be sure.

=-=

• Remove tmp directory.

Ever notice what tmp spells backwards? :-)) Well okay it is a British English abbreviation but still :-)

:-)

:-) :-)

[lcn2]

Let me reword that. Besides #3 (and I hope I can look at some of it today in a while but we shall see) do you have an order of priority for the remaining issues or should it be 'whatever I have the time and energy and inclination for'?

After closing issue #3 , the next priorities will be issue #1933, then issue #2006 , then issue #858 .

We home this clarifies @xexyl .

We will be mostly offline for about 2-3 days working on issue #2008 as well as prep actions for this issue.

[xexyl]

Let me reword that. Besides #3 (and I hope I can look at some of it today in a while but we shall see) do you have an order of priority for the remaining issues or should it be 'whatever I have the time and energy and inclination for'?

After closing issue #3 , the next priorities will be issue #1933, then issue #2006 , then issue #858 .

I presume that this is in order and not equal priority - given that you say 'then' and 'then'?

We home this clarifies @xexyl .

Hopefully :-) it does .. yes it does - thanks!

We will be mostly offline for about 2-3 days working on issue #2008 as well as prep actions for this issue.

Best wishes! Today I'm not doing anything for the contest I'm afraid. Having a very off day. Tomorrow i'm sure will be better.

[lcn2]

After closing issue #3 , the next priorities will be issue #1933, then issue #2006 , then issue #858 .
I presume that this is in order and not equal priority - given that you say 'then' and 'then'?

Correct, @xexyl

[xexyl]

After closing issue #3 , the next priorities will be issue #1933, then issue #2006 , then issue #858 .
I presume that this is in order and not equal priority - given that you say 'then' and 'then'?

Correct, @xexyl

Thanks. I want to be sure of something before I finish my part of #3 which I'll ask you there. My part left in that is minimal except of course for checking typos etc. in the FAQ and other like files so that's good. But I won't be doing anything today. Depending on when you get back to me on the concern and the answer too I hopefully can start tomorrow.

[lcn2]

After closing issue #3 , the next priorities will be issue #1933, then issue #2006 , then issue #858 .
I presume that this is in order and not equal priority - given that you say 'then' and 'then'?

Correct, @xexyl

Thanks. I want to be sure of something before I finish my part of #3 which I'll ask you there. My part left in that is minimal except of course for checking typos etc. in the FAQ and other like files so that's good. But I won't be doing anything today. Depending on when you get back to me on the concern and the answer too I hopefully can start tomorrow.

Best wishes, @xexyl

[xexyl]

After closing issue #3 , the next priorities will be issue #1933, then issue #2006 , then issue #858 .
I presume that this is in order and not equal priority - given that you say 'then' and 'then'?

Correct, @xexyl

Thanks. I want to be sure of something before I finish my part of #3 which I'll ask you there. My part left in that is minimal except of course for checking typos etc. in the FAQ and other like files so that's good. But I won't be doing anything today. Depending on when you get back to me on the concern and the answer too I hopefully can start tomorrow.

Best wishes, @xexyl

Should be good: I'm going to soon get back to LR! And you too!

[lcn2]

With commit 449205e the new status.json model is complete.

The first 2 TODO items of this issue #2239 have been completed.

[lcn2]

We will next focus on issue #2008 as that issue is somewhat independent of the other ((Top
Priority)) issues.

[lcn2]

We will next focus on issue #2008 as that issue is somewhat independent of the other ((Top Priority)) issues.

Issue #2008, as per comment 1987654095 and commit f482380 has been completed.

[lcn2]

We made more tweaks to the TODO list for this issue.

We think the TODO list is nearly ready.

UPDATE 0

Comments and corrections welcome.

[lcn2]

We believe we have addressed all of the current questions that still need answering at this time. If we've missed something or something else needs to be clarified, please ask again.

[xexyl]

We believe we have addressed all of the current questions that still need answering at this time. If we've missed something or something else needs to be clarified, please ask again.

I will have to get back to this tomorrow and determine that. I'm sure you did though.

[lcn2]

With commit 4471875 the bin tools have been ported to RHEL 9.

The make quick_www and make www should work well under Linux, assuming that up to date critical tools such as location(1), jparse(1) are installed and assuming that pandoc(1) has ben installed with a minimum 3.1.12.2 version, and when bash(1) found by #!/usr/bin/env bash is version 5.1.8 or later.

UPDATE 0

The TODO list has been updated accordingly.

[xexyl]

With commit 4471875 the bin tools have been ported to RHEL 9.

The make quick_www and make www should work well under Linux, assuming that up to date critical tools such as location(1), jparse(1) are installed and assuming that pandoc(1) has ben installed with a minimum 3.1.12.2 version, and when bash(1) found by #!/usr/bin/env bash is version 5.1.8 or later.

UPDATE 0

The TODO list has been updated accordingly.

Well now I ran into a new problem. Don't you use pandoc from Homebrew? If so: I updated it yesterday and it's not a high enough version for running the make www. Where do you get it from then? Thanks.

[xexyl]

With commit 4471875 the bin tools have been ported to RHEL 9.
The make quick_www and make www should work well under Linux, assuming that up to date critical tools such as location(1), jparse(1) are installed and assuming that pandoc(1) has ben installed with a minimum 3.1.12.2 version, and when bash(1) found by #!/usr/bin/env bash is version 5.1.8 or later.

UPDATE 0

The TODO list has been updated accordingly.

Well now I ran into a new problem. Don't you use pandoc from Homebrew? If so: I updated it yesterday and it's not a high enough version for running the make www. Where do you get it from then? Thanks.

Oh I know why. I have it from MacPorts too. My guess is it'll be a problem to remove it so I have to maybe put the Homebrew path before the MacPorts path.

UPDATE 0

All good it seems. Thanks. I can now look at the comments in the other issue and hopefully get them done in the next few days. Right now though I must go again. It is unclear to me yet if I can do much today or anything at all but at least last night was better than the previous three nights! Hopefully that continues.

[lcn2]

Priorities

Regarding the top priory open issues:

• top: issue Enhancement: Improve the IOCCC manifest #1933
• next to top: issue Enhancement: improve the look and feel of the constructed HTML files #2006

While going thru the IOCCC manifest, reviewing the data fields from the manifest.numers spreadsheet:

• inventory_order
• OK_to_edit
• display_as
• display_via_github
• entry_text

Look at each entry's inventory, as found in index.html#inventory to see how the contents are being expressed. Look at the sorted lines in the manifest.numers spreadsheet that relate to the entry in question.

Because issue #1933 and issue #2006 are somewhat intertwined @xexyl, we suggest you process both them at the same time. Like you did for issue #3, you can announce your progress on an IOCCC year by YEAR basis.

Regarding issue #2006

And since one is looking a an entry's inventory, give a glance over the rest of the index.html page.

Here are the types of questions, @xexyl, to ask yourself, does the index.html page contain:

• Some formatting that looks obnoxious?
• Information displayed in way that is difficult to read?
• Have too much to too little whitespace that makes the web page awkward to read?

We are NOT focusing on the accuracy of the index.html page per se (although if you see something glaring, feel free to fix it under the guide of something that slipped passed the issue #3 effort). We are focusing on HOW it LOOKS with an emphasis on correcting dysfunctional presentations.

In nearly all cases, corrections can be addressed by editing the entry's README.md file. Please modify the entry's README.md file and re-build the entry's index.html in a pull request.

If, however, you see an issue that may be more a matter of changing ioccc.css or the related bin tool, please raise the matter as a comment in this issue #2239 so we can discuss the matter before editing the CSS and/or bin tool. Again, this is less likely to happen, but if it does, we recommend a comment related discussion first. Modification to the entry's README.md file can be done directly via pull requests.

Regarding issue #1933

Here are the types of questions, @xexyl, to ask yourself:

• Are the brief (59 characters or less) descriptions accurate or at least somewhat useful?

See the entry_text field in the manifest.numers spreadsheet.

See also the Manifest fields section of the top comment of issue 1933.

• Is the order listed reasonable?

See the inventory_order field in the manifest.numers spreadsheet.

See also the Manifest fields section of the top comment of issue 1933.

• The the more useful files to look at in the Primary files section? (see inventory_order)

See the inventory_order field in the manifest.numers spreadsheet.

See also the Manifest fields section of the top comment of issue 1933.

• Are files that are built by bin tools marked as false for _ OK_to_edit_?

See the OK_to_edit field in the manifest.numers spreadsheet.

See also the Manifest fields section of the top comment of issue 1933.

• Are files that marked as true or _ OK_to_edit_ files not created by bin tools nor Makefile rules?

See the OK_to_edit field in the manifest.numers spreadsheet.

See also the Manifest fields section of the top comment of issue 1933.

• Is the general type of file consistent across similar classes of file types?

See the display_as field in the manifest.numers spreadsheet.

See also the Manifest fields section of the top comment of issue 1933.

• Do files that need to be dowloaded (instead of viewed), downloading in a browser?

See the display_via_github field in the manifest.numers spreadsheet.

See also the Manifest fields section of the top comment of issue 1933.

• Do files that need to be viewed (such as C code, Makefiles, etc.) via GitHub going to the GitHub site?

See the display_via_github field in the manifest.numers spreadsheet.

See also the Manifest fields section of the top comment of issue 1933.

• Are files that we OK to be viewed directly in a browser (such as images, photos, etc.) workin in the browser?

See the display_via_github field in the manifest.numers spreadsheet.

See also the Manifest fields section of the top comment of issue 1933.

See the Manifest fields section of the top TODO comment for issue #1933 for more information.

Comments, Suggestions as Questions welcome

Any Comments, Suggestions as Questions you may have about this comment are welcome.

[lcn2]

As per comment 2010767087, this TODO item has been marked complete:

• Verify that the format of markdown files is good

[lcn2]

Also how did you find out that sed -i '' is not portable? If so what versions does it not work? I ask of course because if there can be a workaround in sgit(1) that would be good though I have tested it with macOS sed and also GNU sed, so...

Thanks.

Linux wants:

sed -i -e 'sed stuff' foo.txt

macOS wants:

sed -i '' -e 'sed stuff' foo.txt

[xexyl]

Also how did you find out that sed -i '' is not portable? If so what versions does it not work? I ask of course because if there can be a workaround in sgit(1) that would be good though I have tested it with macOS sed and also GNU sed, so...
Thanks.

Linux wants:

sed -i -e 'sed stuff' foo.txt

macOS wants:

sed -i '' -e 'sed stuff' foo.txt

With linux you can also specify an extension so that shouldn't matter.

$ echo foo > foo
$ sed -i'' -e 's/foo/&d/g' foo
$ cat foo 
food

(Actually I believe I did that explicitly in sgit(1) but I added an option to specify an extension.)

[lcn2]

Comments, Suggestions as Questions welcome

Any Comments, Suggestions as Questions you may have about this comment are welcome.

I have one that will be important before I start making changes. Important in case you have to make changes to the manifest as I don't want to have to figure out what I added and how to merge a Numbers spreadsheet.

Previously there was a three part commit for new files or file name changes. How do you wish commits to be processed (which commits, what stages etc.) for fixing the manifest in text and content of the spreadsheet? That I deem is important before I can really start working on it. That being said I can hopefully look at the index.html pages and determine if there are problems like you suggested (or others I notice) that can be fixed at the same time as the manifest.

We recommend that, we hold off on editing the manifest.numbers spreadsheet to make it easier for you, @xexyl . So you have the "feel free to edit token" the manifest.numbers spreadsheet and related friends until the TODOs in the "Initial TODO list" of issue #1933 are finished.

Once the TODOs in the "initial TODO list" of issue #1933 are finished, we will be happy to take back the "feel free to edit token" and complete the rest of issue #1933.

We recommend you use macOS to process stuff after spreadsheet updates, not Linux.

There are porting issues to Linux for tools under tmp that are NOT worth fixing because the tools under tmp are going away sometime after issue #1933 closes (there is a TODO in this issue to that effect).

Only process spreadsheet manifest stuff under macOS please.

Feel free "batch" changes to the manifest.numbers spreadsheet if that will help you. There is NO NEED to update things for every single change.

When it does come time to push out a "batch" changes to the manifest.numbers spreadsheet, we suggest you:

((sort the spreadsheet in numbers))
((export the CSV file in numbers))
((save the spreadsheet in numbers))

and:

cd tmp
./fix_csv.sh
./gen_path_list.found.sh

then check that all files are accounted for in the manifest and that there are no extra files not in the manifest:

./check_path_list.sh

If all is well, then update and .entry.json files:

./run_all.sh ./gen_entry_json.sh

Once that is done, update the web pages:

cd ..
make www

.. or make quick_www if you wish and trust your file modification dates.

At that point, you will be able to look at the impact on the generated HTML files on your local macOS machine.

Look on your local macOS machine, at the type of changes you have made to web pages by using the configured Apple pre-installed apache. We suggest you review the recently updated comment 1893176110 for how configure the Apple pre-installed apache to run on your macOS AND for how to open HTML files to look at them locally.

If all seems well, then form a pull request.

In the pull request comments, you can write general and high level remarks about the kind of changes that were made.
There is no need to go into explicit detail on the comments.

[lcn2]

BTW: a tip for not adding spaces at the end of markdown files in vim. Add in your .vimrc the following:

autocmd! BufWritePre *.md %s/\s\+$//e

That will remove spaces of *.md files at the end of lines when you save the file. You can of course specify other globs as well.

Thanks!

[lcn2]

QUESTION for make www, make quick_www and related rules

Since it takes quite some time to run each time even if one file only is modified, is it a problem that you can think of to do:

make YEARS="2020 2019" quick_www

for example ?

Of course it'll still run the other rules that don't use the YEARS variable but it would speed things up a fair bit. On that note is it a problem if the YYYY/Makefiles have those specific rules (the ones that act on the YYYY)? I can look at adding this if you think it's worth it. I mean it might not be useful later on but right now it might be.

No, this is not possible nor what the Makefile rules were designed to pick and choose things like years. Moreover due to the way the site is integrated, such a feature is not desired in the usual cases.

...
Would that be possible?
...

Please do not add such complexity of specifying YEARS. It is an integrated site, and the tools are designed to support that. The use case for the bon tools is to manage the entire site.

The tools were written for other IOCCC judges to be able to see the impact of someone's pull request before pushing out changes. They are designed to see the potential impact of a pull request before some IOCCC judge commits it to the official site.

We do understand that for testing purposes, one might wish to regenerate a given entry's index.html file. We did not write the bin tools as a single massive script. You can pick and choose for testing purposes, to process a given entry, and say up the verbosity level:

bin/readme2index.sh -v 3 2020/ferguson1

You can even see the "about to run" command at level 3 for things line the bin/md2html.sh command line if you want to go that deep and manually execute those low level commands.

Similar stuff can be done for the year level index.html files:

bin/gen-year-index.sh -v 3 2020

etc.

UPDATE 0

When were were doing early testing of building entry index.html files, for example, we would:

bin/readme2index.sh -v 3 2020/ferguson1
open 2020/ferguson1/index.html

over and over again until things "failed to suck" as they say. 🤓

[lcn2]

With commit e4a7687 we have been successful in testing bin tools that form compressed tarballs for entries and IOCCC years and untar the same compressed tarballs. This worked well under both macOS (macOS 14.4.1) and Linux (RHEL9.3).

Does this address the problem you asked about yesterday? If not: I think it'd be good if we could have an ioccc.tar.bz2 file that is the entire contest history but if this is not possible with say, LFS, then it might be good to have both the YYYY.tar.bz2 and also maybe a decade. I think it'd be good to have a YYYY.tar.bz2 for each year but to have more than one year is ultimately a good thing if possible.

Please understand that we WILL continue to release compressed tarballs of individual entries AND we WILL continue to release IOCCC year level compressed tarballs.

What is in question is tarballs that cover multiple years,

OPTION 0 - use Git-LFS

0. Use the Git Large File Storage (Git-LFS) to manage the top level ioccc.tar.bz2 compressed tarball

and unless the Git-LFS is free of undesirable side effects, we will not have a single large top level ioccc.tar.bz2 compressed tarball.

Such a test would need to be done using the large data file on some other repo to see what happens. It would require looking at how others use it, understanding the side effects they encounter, etc.

We did see enough FAQs about Git-LFS to suggest that this option is not a simple change to test.

Given time pressures, if this option were to be considers, it may be best to wait until after the Great Fork Merge AND to release the updated IOCCC without an single large compressed top level tarball. The testing of Git-LFS could happen in this repo AFTER the web site update, and changes ported over only after testing was complete and the result was satisfactory.

OPTION 1 - release sets of years

Alternatives to (0) above are:

1. Release mulit-year compress tarball sets: group by decade or group by sets of every N-contents into several top level compressed tarballs

We break up the years into several sets:

• ioccc.1984-1989.tar.bz2
• ioccc.1990-1998.tar.bz2
• ioccc.2000-2006.tar.bz2
• ioccc.2011-2019.tar.bz2
• ioccc.2020-2029.tar.bz2

These sets of years are smaller and don't run into GitHub size limitations. And if a single entry needs modifying, we don't have to replace the single large top level ioccc.tar.bz2 compressed tarball with a new single large top level ioccc.tar.bz2 compressed tarball.

We can also break up years into sets of, say, 5 IOCCC contents each.

OPTION 2 - do not release multi-year sets

2. Only release year level and entry level compressed tarballs (no single large top level ioccc.tar.bz2 compressed tarball).

[lcn2]

We believe we have addressed all of the current questions that still need answering at this time. If we've missed something or something else needs to be clarified, please ask again.

[xexyl]

QUESTION for make www, make quick_www and related rules

Since it takes quite some time to run each time even if one file only is modified, is it a problem that you can think of to do:

make YEARS="2020 2019" quick_www

for example ?
Of course it'll still run the other rules that don't use the YEARS variable but it would speed things up a fair bit. On that note is it a problem if the YYYY/Makefiles have those specific rules (the ones that act on the YYYY)? I can look at adding this if you think it's worth it. I mean it might not be useful later on but right now it might be.

No, this is not possible nor what the Makefile rules were designed to pick and choose things like years. Moreover due to the way the site is integrated, such a feature is not desired in the usual cases.

I noticed that too. It's not a big deal but thanks for confirming.

...
Would that be possible?
...

Please do not add such complexity of specifying YEARS. It is an integrated site, and the tools are designed to support that. The use case for the bon tools is to manage the entire site.

Oh I am not going to. No worries about that.

The tools were written for other IOCCC judges to be able to see the impact of someone's pull request before pushing out changes. They are designed to see the potential impact of a pull request before some IOCCC judge commits it to the official site.

That makes sense.

We do understand that for testing purposes, one might wish to regenerate a given entry's index.html file. We did not write the bin tools as a single massive script. You can pick and choose for testing purposes, to process a given entry, and say up the verbosity level:

bin/readme2index.sh -v 3 2020/ferguson1

Indeed. But I'm more likely to not even go that far. Probably just run the rule. It will take a bit longer but that's okay.

You can even see the "about to run" command at level 3 for things line the bin/md2html.sh command line if you want to go that deep and manually execute those low level commands.

Yes I've noticed that but thanks for the reminder.

Similar stuff can be done for the year level index.html files:

bin/gen-year-index.sh -v 3 2020

etc.

True.

UPDATE 0

When were were doing early testing of building entry index.html files, for example, we would:

bin/readme2index.sh -v 3 2020/ferguson1
open 2020/ferguson1/index.html

over and over again until things "failed to suck" as they say. 🤓

Oh I've done similar to that in the past too but before there was a tool to do it. I just used discount or fork of discount and then indeed 'open'. I did that when working on the README.md files quite a few times!

[xexyl]

With commit e4a7687 we have been successful in testing bin tools that form compressed tarballs for entries and IOCCC years and untar the same compressed tarballs. This worked well under both macOS (macOS 14.4.1) and Linux (RHEL9.3).

Does this address the problem you asked about yesterday? If not: I think it'd be good if we could have an ioccc.tar.bz2 file that is the entire contest history but if this is not possible with say, LFS, then it might be good to have both the YYYY.tar.bz2 and also maybe a decade. I think it'd be good to have a YYYY.tar.bz2 for each year but to have more than one year is ultimately a good thing if possible.

Please understand that we WILL continue to release compressed tarballs of individual entries AND we WILL continue to release IOCCC year level compressed tarballs.

What is in question is tarballs that cover multiple years,

OPTION 0 - use Git-LFS

0. Use the Git Large File Storage (LFS) to manage the top level ioccc.tar.bz2 compressed tarball

and unless the Git-LFS is free of undesirable side effects, we will not have a single large top level ioccc.tar.bz2 compressed tarball.

Such a test would need to be done using the large data file on some other repo to see what happens. It would require looking at how others use it, understanding the side effects they encounter, etc.

We did see enough FAQs about Git-LFS to suggest that this option is not a simple change to test.

Given time pressures, if this option were to be considers, it may be best to wait until after the Great Fork Merge AND to release the updated IOCCC without an single large compressed top level tarball. The testing of Git-LFS could happen in this repo AFTER the web site update, and changes ported over only after testing was complete and the result was satisfactory.

That sounds like a good compromise to me. It doesn't have to be done now. That seems even more important if it's not easy to test the LFS feature.

OPTION 1 - release sets of years

Alternatives to (0) above are:

1. Release mulit-year compress tarball sets: group by decade or group by sets of every N-contents into several top level compressed tarballs

I like the idea of decades but I guess if ever they become too big or too unwieldy five years could work too. I say 5 only because it can be evenly divided in half.

We break up the years into several sets:

• ioccc.1984-1989.tar.bz2
• ioccc.1990-1998.tar.bz2
• ioccc.2000-2006.tar.bz2
• ioccc.2011-2019.tar.bz2
• ioccc.2020-2029.tar.bz2

These sets of years are smaller and don't run into GitHub size limitations. And if a single entry needs modifying, we don't have to replace the single large top level ioccc.tar.bz2 compressed tarball with a new single large top level ioccc.tar.bz2 compressed tarball.

True.

We can also break up years into sets of, say, 5 IOCCC contents each.

I really did not see this when I noted that above!

OPTION 2 - do not release multi-year sets

2. Only release year level and entry level compressed tarballs (no single large top level ioccc.tar.bz2 compressed tarball).

I think it's good to have both, personally, but that's me. Maybe part of that is because in the past you had the full history as a single tarball. That's how I always downloaded the entries in fact.

[xexyl]

Comments, Suggestions as Questions welcome

Any Comments, Suggestions as Questions you may have about this comment are welcome.

I have one that will be important before I start making changes. Important in case you have to make changes to the manifest as I don't want to have to figure out what I added and how to merge a Numbers spreadsheet.
Previously there was a three part commit for new files or file name changes. How do you wish commits to be processed (which commits, what stages etc.) for fixing the manifest in text and content of the spreadsheet? That I deem is important before I can really start working on it. That being said I can hopefully look at the index.html pages and determine if there are problems like you suggested (or others I notice) that can be fixed at the same time as the manifest.

We recommend that, we hold off on editing the manifest.numbers spreadsheet to make it easier for you, @xexyl . So you have the "feel free to edit token" the manifest.numbers spreadsheet and related friends until the TODOs in the "Initial TODO list" of issue #1933 are finished.

That sounds great! Thanks.

Once the TODOs in the "initial TODO list" of issue #1933 are finished, we will be happy to take back the "feel free to edit token" and complete the rest of issue #1933.

Sure.

We recommend you use macOS to process stuff after spreadsheet updates, not Linux.

I only work with GitHub with macOS but thanks for the note!

There are porting issues to Linux for tools under tmp that are NOT worth fixing because the tools under tmp are going away sometime after issue #1933 closes (there is a TODO in this issue to that effect).

That makes sense.

Only process spreadsheet manifest stuff under macOS please.

Of course. I'm not even sure if it's possible in linux. Certainly not the spreadsheet file itself.

Feel free "batch" changes to the manifest.numbers spreadsheet if that will help you. There is NO NEED to update things for every single change.

Thanks! That's helpful indeed (it's what I had hoped to do).

When it does come time to push out a "batch" changes to the manifest.numbers spreadsheet, we suggest you:

((sort the spreadsheet in numbers))
((export the CSV file in numbers))
((save the spreadsheet in numbers))

I do indeed sort and export and save.

and:

cd tmp
./fix_csv.sh
./gen_path_list.found.sh

Okay. I have to look at the script I wrote to see what it does too. Might not have to do the commands manually.

then check that all files are accounted for in the manifest and that there are no extra files not in the manifest:

./check_path_list.sh

If all is well, then update and .entry.json files:

./run_all.sh ./gen_entry_json.sh

Once that is done, update the web pages:

cd ..
make www

.. or make quick_www if you wish and trust your file modification dates.

I think I do that yes though with the script I wrote.

At that point, you will be able to look at the impact on the generated HTML files on your local macOS machine.

Look on your local macOS machine, at the type of changes you have made to web pages by using the configured Apple pre-installed apache. We suggest you review the recently updated comment 1893176110 for how configure the Apple pre-installed apache to run on your macOS AND for how to open HTML files to look at them locally.

That might or might not be a problem but I will worry about that at the time. Thanks for the link again.

If all seems well, then form a pull request.

In the pull request comments, you can write general and high level remarks about the kind of changes that were made. There is no need to go into explicit detail on the comments.

Thanks.

Now as for the commits though. I'm not sure what commit messages to use for each phase. Before it was because of file name changes or new files / deleted files but now it's not that. Does it matter that much?

[xexyl]

We believe we have addressed all of the current questions that still need answering at this time. If we've missed something or something else needs to be clarified, please ask again.

I'll look more tomorrow but I think you answered everything yes. Off to do other things for the rest of the day.

[lcn2]

Now as for the commits though. I'm not sure what commit messages to use for each phase. Before it was because of file name changes or new files / deleted files but now it's not that. Does it matter that much?

It might not matter that much.

[lcn2]

At that point, you will be able to look at the impact on the generated HTML files on your local macOS machine.

Look on your local macOS machine, at the type of changes you have made to web pages by using the configured Apple pre-installed apache. We suggest you review the recently updated #4 (comment) for how configure the Apple pre-installed apache to run on your macOS AND for how to open HTML files to look at them locally.

That might or might not be a problem but I will worry about that at the time. Thanks for the link again.

You REALLY do want to look at HTML files on your local machine, via a web browser as served from a local web server.

[xexyl]

At that point, you will be able to look at the impact on the generated HTML files on your local macOS machine.
Look on your local macOS machine, at the type of changes you have made to web pages by using the configured Apple pre-installed apache. We suggest you review the recently updated #4 (comment) for how configure the Apple pre-installed apache to run on your macOS AND for how to open HTML files to look at them locally.

That might or might not be a problem but I will worry about that at the time. Thanks for the link again.

You REALLY do want to look at HTML files on your local machine, via a web browser as served from a local web server.

I understand it. But I'll have to worry about that when I am looking at the html files. I have another way that might work too that would be more natural for me (less of a burden and also quicker to set up). Since I have bind configured with views and since I already have a subdomain I might do it on the server. Or I might set up a new vhost and do it that way. But I'll consider the options (testing of course) when the time comes.

[lcn2]

So if we batch multiple IOCCC years together: do we span a sets of

• 5 years
• 10 years
• N year (for some other value of N)
• 5 IOCCC contents
• 10 IOCCC contests
• N IOCCC contents (for some other value of N)

[xexyl]

So if we batch multiple IOCCC years together: do we span a sets of

• 5 years
• 10 years
• N year (for some other value of N)
• 5 IOCCC contents
• 10 IOCCC contests
• N IOCCC contents (for some other value of N)

For me ten years seems most natural but if that's too many why not five? Ah I see .. so contests instead of years. Interesting thought. I still think ten years in the sense of decades might be better. So the first tarball would be not ten years but all the others would (though the last one of course wouldn't be ten at first). Thus 1984-1989 and the next ones 1990-1999, 2000-2009 and so on.

[lcn2]

So if we batch multiple IOCCC years together: do we span a sets of

• 5 years
• 10 years
• N year (for some other value of N)
• 5 IOCCC contents
• 10 IOCCC contests
• N IOCCC contents (for some other value of N)

For me ten years seems most natural but if that's too many why not five? Ah I see .. so contests instead of years. Interesting thought. I still think ten years in the sense of decades might be better. So the first tarball would be not ten years but all the others would (though the last one of course wouldn't be ten at first). Thus 1984-1989 and the next ones 1990-1999, 2000-2009 and so on.

We would have tarballs that look something like this:

• ioccc.1984-1989.tar.bz2
• ioccc.1990-1998.tar.bz2
• ioccc.2000-2006.tar.bz2
• ioccc.2011-2019.tar.bz2
• ioccc.2020-2029.tar.bz2

They will be placed somewhere under the archive directory: location TBD.

We will now go about building the bin apps to tar and untag those 10 year tarballs, removing the top level ioccc.tar.bz2, and editing the web pages such as years.html accordingly.

These actions won't impact your work on the manifest / your work on issue #1933.

UPDATE 0

There is a problem with the decade approach. If we sum the sizes of the year level "YYYY/YYYY.tyar.bz2" compressed tarballs as a reasonable approximation for what the "combined" total would look line, the "ioccc.2011-2019.tar.bz2" would be >52 Mbytes. GitHub begins to gripe / warn about files > 50 Mbytes. Worse still, as we are considering expanding entry sizes to a few megabytes, this "decade" problem is only going to get worse over time.

Even if things stay under the "50 Mbyte GitHub warning" limit, editing entries and reforming the compressed tarballs would run into "big blob file editing many times" pain.

A 5-year span would keep under the 50 Mbyte limit so long as the average contribution of a year is < 10 Mbytes. In the last IOCCC decade, we had 3 IOCCC years that were 17.9 Mbytes. 17.4 Mbytes, and 10.0 Mbytes. An "ioccc.2011-2015.tar.bz2" would be 38.0 Mbytes.

The trend of allowing for entries with larger data sizes continues. Recall that "MAX_SUM_FILELEN" as defined in soup/limit_ioccc.h is "27651*1024" or 27Mbytes. Not that we expect all winners will be pushing the maximum size. The compressed tarball "MAX_TARBALL_LEN" < 3.81 Mbytes.

Now 15 winners for a given year remains common peak (there is no real limit on the number of winners for given year, it just turns out that 6 years had 15 winners, 7 years had 14 winners. If the of a year is < 10 Mbytes and we go with limiting to 5 years or 5 contests, then to avoid crossing the 50 Mbyte limit, the average entry has to be < 2/3 Mbytes.

Adjusting the limits to force an entry to be that small might be too constraining on creativity.

So: The decade grouping approach will not work. The 5 year or 5 content set starts to gets close to the problem area and would likely exceed the "50 Mbyte GitHub warning" limit.

AGAIN: While the past will work for 5 years or 5 contents, future trends put such as grouping of 5 into problem territory.

What about groups of 4 years or 4 contests?

With 27 IOCCC years, a 4 year or 4 content would require 7 multi-year tarballs. The average for 4 years would need to be < 12.5 Mbytes. With 15 winners per year, that an average of < 5/6 Mbytes per winner.

What about groups of 3 years or 3 contests? Well now with 9 multi-year tarballs to cover 27 contests, the point of having multi-year tarballs gets a bit silly.

UPDATE 1

So it seems the only reasonable "multiple IOCCC years in one compressed tarball" is to go for an option using Git Large File System OR no have any multi-year compressed tarballs.

We will ponder this some more.

[lcn2]

There might have been problem with bin/gen-year-index.sh -v 5 2020, where the debug level was >= 5. However that turned out to be a debugging "mis-feature" and not a bug. With commit 420c7ae those debugging "mis-features" have been resolved in favor of "less sucky" / "less misleading" debug messages. :-)

[xexyl]

So if we batch multiple IOCCC years together: do we span a sets of

• 5 years
• 10 years
• N year (for some other value of N)
• 5 IOCCC contents
• 10 IOCCC contests
• N IOCCC contents (for some other value of N)

For me ten years seems most natural but if that's too many why not five? Ah I see .. so contests instead of years. Interesting thought. I still think ten years in the sense of decades might be better. So the first tarball would be not ten years but all the others would (though the last one of course wouldn't be ten at first). Thus 1984-1989 and the next ones 1990-1999, 2000-2009 and so on.

We would have tarballs that look something like this:

• ioccc.1984-1989.tar.bz2
• ioccc.1990-1998.tar.bz2
• ioccc.2000-2006.tar.bz2
• ioccc.2011-2019.tar.bz2
• ioccc.2020-2029.tar.bz2

They will be placed somewhere under the archive directory: location TBD.

We will now go about building the bin apps to tar and untag those 10 year tarballs, removing the top level ioccc.tar.bz2, and editing the web pages such as years.html accordingly.

These actions won't impact your work on the manifest / your work on issue #1933.

UPDATE 0

There is a problem with the decade approach. If we sum the sizes of the year level "YYYY/YYYY.tyar.bz2" compressed tarballs as a reasonable approximation for what the "combined" total would look line, the "ioccc.2011-2019.tar.bz2" would be >52 Mbytes. GitHub begins to gripe / warn about files > 50 Mbytes. Worse still, as we are considering expanding entry sizes to a few megabytes, this "decade" problem is only going to get worse over time.

Even if things stay under the "50 Mbyte GitHub warning" limit, editing entries and reforming the compressed tarballs would run into "big blob file editing many times" pain.

A 5-year span would keep under the 50 Mbyte limit so long as the average contribution of a year is < 10 Mbytes. In the last IOCCC decade, we had 3 IOCCC years that were 17.9 Mbytes. 17.4 Mbytes, and 10.0 Mbytes. An "ioccc.2011-2015.tar.bz2" would be 38.0 Mbytes.

The trend of allowing for entries with larger data sizes continues. Recall that "MAX_SUM_FILELEN" as defined in soup/limit_ioccc.h is "27651*1024" or 27Mbytes. Not that we expect all winners will be pushing the maximum size. The compressed tarball "MAX_TARBALL_LEN" < 3.81 Mbytes.

Now 15 winners for a given year remains common peak (there is no real limit on the number of winners for given year, it just turns out that 6 years had 15 winners, 7 years had 14 winners. If the of a year is < 10 Mbytes and we go with limiting to 5 years or 5 contests, then to avoid crossing the 50 Mbyte limit, the average entry has to be < 2/3 Mbytes.

Adjusting the limits to force an entry to be that small might be too constraining on creativity.

So: The decade grouping approach will not work. The 5 year or 5 content set starts to gets close to the problem area and would likely exceed the "50 Mbyte GitHub warning" limit.

AGAIN: While the past will work for 5 years or 5 contents, future trends put such as grouping of 5 into problem territory.

What about groups of 4 years or 4 contests?

With 27 IOCCC years, a 4 year or 4 content would require 7 multi-year tarballs. The average for 4 years would need to be < 12.5 Mbytes. With 15 winners per year, that an average of < 5/6 Mbytes per winner.

What about groups of 3 years or 3 contests? Well now with 9 multi-year tarballs to cover 27 contests, the point of having multi-year tarballs gets a bit silly.

UPDATE 1

So it seems the only reasonable "multiple IOCCC years in one compressed tarball" is to go for an option using Git Large File System OR no have any multi-year compressed tarballs.

We will ponder this some more.

Well I like the idea of allowing larger entry sizes in the future though of course us contestants don't really know the size until the rules have been finalised. But if you are indeed going to increase the size some it might be that 4 years is good. I agree with 3 it would be kind of silly. But then I think we have to ask if 5 is pushing the limits is 4 not also silly? Maybe it would be better to have single year only and then anyone who wants the full contest can clone the repo?

I admit I like the idea of tarballs with more than one year but that's a 'used to it' reason. I don't need that now as I have the repo cloned. So now I don't know if it's necessary or not. I wonder.

[xexyl]

There might have been problem with bin/gen-year-index.sh -v 5 2020, where the debug level was >= 5. However that turned out to be a debugging "mis-feature" and not a bug. With commit 420c7ae those debugging "mis-features" have been resolved in favor of "less sucky" / "less misleading" debug messages. :-)

Thanks for the note!

[xexyl]

At that point, you will be able to look at the impact on the generated HTML files on your local macOS machine.
Look on your local macOS machine, at the type of changes you have made to web pages by using the configured Apple pre-installed apache. We suggest you review the recently updated #4 (comment) for how configure the Apple pre-installed apache to run on your macOS AND for how to open HTML files to look at them locally.

That might or might not be a problem but I will worry about that at the time. Thanks for the link again.

You REALLY do want to look at HTML files on your local machine, via a web browser as served from a local web server.

I understand it. But I'll have to worry about that when I am looking at the html files. I have another way that might work too that would be more natural for me (less of a burden and also quicker to set up). Since I have bind configured with views and since I already have a subdomain I might do it on the server. Or I might set up a new vhost and do it that way. But I'll consider the options (testing of course) when the time comes.

I set up so that my web server will take care of the problem so now I can look at the rendered html files before committing the changes. All it'll take is when files change I can scp or rsync them to the server (which tool depending on how many files) and if all is okay I can then commit. If not all okay I will make fixes and try again.

This was very simple to do and much easier and quicker than setting it up on in macOS as apache is already configured there with a subdomain/vhost anyway (just in a subdirectory of it).

[lcn2]

With commit 1979ea0 we have selected option 2.

In particular:

     Removed top level `ioccc.tar.bz2` compressed tarball.  NOTE: Sorry
    (tm Canada 🇨🇦) to remove the top level `ioccc.tar.bz2` file.

We will look into [Git Large File Storage Git-LFS. If Git-LFS works well and does not have undesirable side effects, we will consider restoring ioccc.tar.bz2.

A number of other things were fixed and improved as well. See the commit comment for other details.

[xexyl]

With commit 1979ea0 we have selected option 2.

In particular:

     Removed top level `ioccc.tar.bz2` compressed tarball.  NOTE: Sorry
    (tm Canada 🇨🇦) to remove the top level `ioccc.tar.bz2` file.

We will look into [Git Large File Storage Git-LFS. If Git-LFS works well and does not have undesirable side effects, we will consider restoring ioccc.tar.bz2.

A number of other things were fixed and improved as well. See the commit comment for other details.

Thanks for the update! I'll look in more detail tomorrow. Enjoy the rest of your day!

[lcn2]

We believe we have addressed all of the current questions that still need answering at this time. If we've missed something or something else needs to be clarified, please ask again.

[lcn2]

Current activity

We have been working "heads down" as they say, on the IOCCC submit server and the IOCCC registration process.

FYI: This is the reason why we have been away from this repo for a bit.

IOCCC submit server progress report

The IOCCC submit server is a critical chunk of the IOCCC infrastructure: one that will accept compressed tarballs produced by mkiocccentry(1) and checked by txzchk(1) and verified by the chkentry(1) tools (See mkiocccentry repo. The IOCCC submit server is now on the critical path of holding the next IOCCC.

Without an IOCCC submit server, there will be no IOCCC. So it is rather important. 😉

The IOCCC submit server is being co-developed with an engineer who lives in Switzerland. We have a good concept design that we believe will work well. Nevertheless we have been "heads down" on this project.

The IOCCC submit server is currently in a private repo at the moment, in part because it is in a very pre-alpha testing phase. We do plan to make the repo public in perhaps a month or so.

The IOCCC submit server development environment

The IOCCC submit server has been written in Python and runs under a Docker container.

We are trying to, once again, to be "at peace with the python programming environment", so pardon us if we do not express our opinion about Python as a programming language. <<<-- go ahead, click that link 🐇

Docker is another interesting provisional system. Think of it like "chroot(2) done much better". We can recommend docker for a wide variety of solutions. That solution set seems to include what the IOCCC will need in a submit server. For example, the docker container is allowing us to test this Linux-based solution under macOS.

Once the IOCCC submit server is in beta test, we will make the repo public and invite comments.

IOCCC registration process progress report

We are also working on the IOCCC registration process, a smaller bit of IOCCC infrastructure that will tie into the IOCCC submit server. Folks will use the IOCCC registration process to be given access to the IOCCC submit server where, using the mkiocccentry tool, they will upload their submissions to the IOCCC for judging.

As the IOCCC registration process and the IOCCC submit server are strongly tied together, we are working on both at the same time. Nevertheless we need to get farther along on the IOCCC submit server before we can make reasonable progress on the IOCCC registration process.

IOCCC registration process development environment

We are considering using the TopicBox service as a key part of the IOCCC registration process infrastructure.

We believe the work needed to finish the IOCCC registration process is not nearly as large as the work needed to finish the IOCCC submit server .. we believe/hope. We need to push the IOCCC submit server farther along before we can say that for certain.

The tools needed for a complete IOCCC registration process implementation are likely some private shell scripts that will be used by the IOCCC judges in conjunction with the TopicBox service.

IOCCC MOCK replacement

We are considering to NOT hold the IOCCC MOCK* as we previously envisioned. Instead we plan to hold a IOCCC submit server public test and IOCCC registration process public test. And as part of this test, folks will use the mkiocccentry tool to upload simple "Hello, world!"-like entries whose content will NOT be judged.

We are going to use this approach in order to speed up the date when the 28th IOCCC will eventually start.

Great Fork Merge date

Once the IOCCC submit server and the IOCCC registration process are design stable and in alpha testing phase, we plan to document their process on the temp-test-ioccc web site.

Once that documentation is ready, the ((top priority)) issues that are required to close before we can begin work on the Near Final TODOs (i.e., issue #858 and issue #1933 and issue #2006) ... those issues will be rapidly brought to a close (somewhat regardless of their state).

When this will happen will depend on how fast the the IOCCC submit server and the IOCCC registration process can move into their beta testing phase. This will be done in order to NOT push the start date of the 28th IOCCC way way far into the future.

Stay tuned for these exciting developments.

P.S.

We DO APPRECIATE the efforts being made on the ((top priority)) issue #858 and issue #1933 and issue #2006 and DO UNDERSTAND that those helping us with these issues have their own matters to attend to.

We plan to give a bit of warning (but not too much, maybe 2 weeks) of when we will force closed, issue #858 and issue #1933 and issue #2006. That won't happen until the IOCCC submit server and the IOCCC registration process are able to move into their testing phase such that we can document how to use them on the web site. So there is more time left to close those issues .. hopefully not a lot more time left as we want to get the 28th IOCCC started .. sometime sooner than later 🗓️‼️

[lcn2]

Current activity

As mentioned in comment-2095164921 we continue to work on tthe ioccc-reg tool and the ioccc-submit tool.

Tasks related to the ioccc-reg tool and the ioccc-submit tool TODO include (but may not be limited to):

• Revise the ioccc-submit tool tasks below and add them as an (private) repo issue
• Revise the ioccc-reg tool tasks below and add them as an (private) repo issue
• Make use of the GutHub machoism for handling keys within a repo and modify source code accordingly
• Approach the topicbiox service about of ioccc-reg tool mailing list design
• Isolate submit server uploads into separate directories
• Add a mechanism to allow IOCCC judges to collect submissions and change the submission status accordingly
• Add a mechanism to allow IOCCC judges further change the submission status to include an arbitarary string
• Implement a lock / unlock mechanism to allow upload status updates by IOCCC judges while submission are being uploaded
• Add a mechanism to have new ioccc-submit tool users change their initial password
• Build the submit.ioccc.org server that will only be up when needed by an open IOCCC
• Use letsencrypt.org to generate a cert for HTTPS connections on submit.ioccc.org
• Map the internal port for ioccc-submit tool into tcp/443 for HTTPS use
• Test install the current ioccc-submit tool on the the submit.ioccc.org server
• Obtain screenshots from the ioccc-reg tool and add them to the FAQ
• Obtain screenshots from the ioccc-submit tool and add them to the FAQ

Relevance to this issue

While the the ioccc-reg tool and the ioccc-submit tool are beyond the scope of this repo, those projects ARE GATING FACTORS for the Great Fork Merge,

Both the ioccc-reg tool and the ioccc-submit tool need to be far enough along that useful screenshots can be added to the faq.md file. And in particular, improve / add FAQs about how to enter an IOCCC.

Once we reach the stage where the the faq.md file has been updated with useful FAQ information about how to register for the IOCCC and how to submit to the IOCCC, WE PLAN TO SET A DEADLINE for performing the Great Fork Merge. This will include setting a deadline to bring to a close the ((top priority)) issue #858 and issue #1933 and issue #2006. Those issues will need to be brought to a close (somewhat regardless of their state).

P.S.

We DO APPRECIATE the efforts being made on the ((top priority)) issue #858 and issue #1933 and issue #2006 and DO UNDERSTAND that those helping us with these issues have their own matters to attend to.

We plan to give a bit of warning (but not too much, maybe 2 weeks) of when we will force closed, issue #858 and issue #1933 and issue #2006. That won't happen until the IOCCC submit server and the IOCCC registration process are able to move into their testing phase such that we can document how to use them on the web site. So there is more time left to close those issues .. hopefully not a lot more time left as we want to get the 28th IOCCC started .. sometime sooner than later 🗓️‼️

[lcn2]

We believe we have addressed all of the current questions that still need answering at this time. If we've missed something or something else needs to be clarified, please ask again.

[xexyl]

Current activity

As mentioned in comment-2095164921 we continue to work on tthe ioccc-reg tool and the ioccc-submit tool.

Tasks related to the ioccc-reg tool and the ioccc-submit tool TODO include (but may not be limited to):

• Revise the ioccc-submit tool tasks below and add them as an (private) repo issue

• Revise the ioccc-reg tool tasks below and add them as an (private) repo issue

• Make use of the GutHub machoism for handling keys within a repo and modify source code accordingly

• Approach the topicbiox service about of ioccc-reg tool mailing list design

• Isolate submit server uploads into separate directories

• Add a mechanism to allow IOCCC judges to collect submissions and change the submission status accordingly

• Add a mechanism to allow IOCCC judges further change the submission status to include an arbitarary string

• Implement a lock / unlock mechanism to allow upload status updates by IOCCC judges while submission are being uploaded

• Add a mechanism to have new ioccc-submit tool users change their initial password

• Build the submit.ioccc.org server that will only be up when needed by an open IOCCC

• Use letsencrypt.org to generate a cert for HTTPS connections on submit.ioccc.org

• Map the internal port for ioccc-submit tool into tcp/443 for HTTPS use

• Test install the current ioccc-submit tool on the the submit.ioccc.org server

• Obtain screenshots from the ioccc-reg tool and add them to the FAQ

• Obtain screenshots from the ioccc-submit tool and add them to the FAQ

Relevance to this issue

While the the ioccc-reg tool and the ioccc-submit tool are beyond the scope of this repo, those projects ARE GATING FACTORS for the Great Fork Merge,

Both the ioccc-reg tool and the ioccc-submit tool need to be far enough along that useful screenshots can be added to the faq.md file. And in particular, improve / add FAQs about how to enter an IOCCC.

Once we reach the stage where the the faq.md file has been updated with useful FAQ information about how to register for the IOCCC and how to submit to the IOCCC, WE PLAN TO SET A DEADLINE for performing the Great Fork Merge. This will include setting a deadline to bring to a close the ((top priority)) issue #858 and issue #1933 and issue #2006. Those issues will need to be brought to a close (somewhat regardless of their state).

P.S.

We DO APPRECIATE the efforts being made on the ((top priority)) issue #858 and issue #1933 and issue #2006 and DO UNDERSTAND that those helping us with these issues have their own matters to attend to.

We plan to give a bit of warning (but not too much, maybe 2 weeks) of when we will force closed, issue #858 and issue #1933 and issue #2006. That won't happen until the IOCCC submit server and the IOCCC registration process are able to move into their testing phase such that we can document how to use them on the web site. So there is more time left to close those issues .. hopefully not a lot more time left as we want to get the 28th IOCCC started .. sometime sooner than later 🗓️‼️

Thank you for the warning. Bad timing for me but I will do my best. The html issue is probably mostly good and if necessary there can always be future fixes. But that would greatly speed things up.

The manifest issue I will work on next. It might take more than a few commits but I would think it shouldn't take that much effort.

The others I will have to look at to see what has to be done.

I will say that the html issue is probably the one that takes the most effort and time so if you think that could be closed soon or even now then it might be that the other issues could be finished sooner.

But I left some comments there.

I was able to look at these messages with the phone but I will be away for the rest of the day now most likely so I will reply tomorrow.