Redundancy / error-correcting algorithm and benchmarks.

[nohehf]

As you say in the readme, encoding bytes into colors is tricky as very prone to error due to youtube's compression. You also mentioned that you use 2*2 "blocks" to reduce compression issues.

Have you measured the loss (1*1 black and white, 2*2, RGB...) ?

I think we could implement some redundancy algorithm such as the Reed-Solomon algorithm (used in QR codes) to prevent data loss (I'm almost sure it would be better / more reliable than the 2*2 technique). We could then measure the impact in the different modes mentioned above, maybe with this addition RGB would be more profitable.

Also I think it would be interesting to figure out what is the impact of the video resolution, (is it better to have a long low-res video or a shorter high-res one to encode one file, in terms of data loss, read/write speeds ect.

This is an early reflection / public mind dump / start of a thread and I'd be happy to discuss about this with anyone susceptible of reading this :) I would then be happy to implement some of those features/tests.

[vanakema]

Not sure if the sound channel would provide enough bandwidth for this in its compressed form, but it'd be cool to use the audio for ECC bits, since it's a different modality and is subject to different treatment at the various quality levels in YouTube. They'll also be subject to compression artifacts at different times temporally than the video stream. Youtube gives you up to 5.1 (aka 6) channels of audio that you could use.

I don't know what playback settings will result in 5.1 vs stereo, but I'm thinking you can encode a minimal amount of ECC bits in the left channel (the default for mono), and then provide a checksum (either for the files itself, or the ECC bits to know when you can use the ECC bits or not) of some sort in the right channel, and then use the remaining channels for higher redundancy. You'd probably be able to stuff more small pixels into each frame this way too

I'm not an expert in any of this stuff, so this is more of a braindump and speculation

[nohehf]

That's a great idea! However, this means introducing a lot of new logic in the package.
In my opinion, the way to go here should be:
Get a baseline:

• Implement benchmarks to monitor loss and performance.
• Implement ECC / some redundancy to the system, only through black and white video.
• Benchmark this with different settings (resolution, ECC algorithm, etc.)

Then in separate MRs, introduce RGB, the sound channels you mentioned, etc., to compare it with the above baseline.
I'm swamped right now and have shallow rust knowledge, but I'd like to start working on the first point if I can find the time.

About using sound channels, my wild guess is that ECC and actual data should live in the same space (audio/video) as it will make it hard and probably inefficient to decode using two different channels at once, tho using audio to store more data in a single video can be interesting. (This is a wild guess, too haha).