updated regularly while attending Recurse Center in Spring 2 '18
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
README.md

README.md

My Recurse Center Syllabus

            o                   o
             \               __/
              \___          /
                  \__    __/
                     \  /
 ____________________\/_________________
/   _________________________________   \
|  /__/  \__  _____________ \__/  \__\  |
| |    __   \____My_________    __   \| |
| |\__/  \_________________ \__/  \__ | |
| |    __   \___Recurse_____    __   \| |
| |\__/  \__  _____________ \__/  \__ | |
| |    __   \_______Center__    __   \| |
| |\__/  \__  _____________ \__/  \__ | |
| |    __   \__Syllabus____     __   \| |
| |\__/  \__  _____________ \__/  \__ | |
|  \________\________________________/  |
|                                       |
|                          _   _   _    |
|  Ashley Blewer!         (=) (=) (=)   |
\______________________________________/
     "--"                      "--"

Creating a syllabus for myself

I will be attending the Recurse Center from April 2 until June 28. For the uninundated (non-inundated?), I have been describing RC succinctly as "like a writer's retreat for programmers." although maybe that's not quite sufficient. But in general, it is a place to focus intensely on my craft and be surrounded by and inspired by brilliant people.

I am verrrrrry excited!

Because the time you spend at RC and what you choose to work on is self-initiated and self-motivated, I am setting up this repository to create a sort of "syllabus" for myself and what I want to accomplish. BUT! I do not expect to follow it, just give myself some rough structure to make sure I hit my personal productivity goals while leaving enough room for unanticipated discovery.

This will also be a living syllabus,
updated with links to blog posts,
works I produced,
and things I used to tackle these problems
or build things for each week.

Some things will get bumped and other things will get shuffled around as part of the normal un-learning process.

Here is how I answered the application question "What would you like to work on at the Recurse Center?":

I really love video and I understand it (relatively) well, I think, but would really like to better understand compression algorithms and how things work. I think trying to make my own video codec would be a good way to do that, and RC seems like a great place to attempt a task that sounds difficult and academic. I think this kind of broad, open-ended task would allow me to have one major goal to accomplish, and would allow me to break down the problem and potentially tackle mini-problems along the way, accepting productive distraction (like committing to an open source video framework as a mini-goal towards the larger educational goal).

General suggestions from RC to help think about projects:

  • What are the main 2-3 things I'd like to learn? (i.e. why am I doing this?)
  • Which features could I build to learn them?
  • Are there other things I can ignore because they won't help me achieve my learning goals?
  • Does my project divide naturally into certain chunks?
  • How will I know when my project is finished?

Thinking about how to think about this

I want to create goals for myself that are concrete enough to be able to tackle but open enough that I can move in different directions as I grow and learn more.

I also think it's important for me to think about this work as at most 80% of a week, leaving 20% to spread out and learn about or engage in other projects unrelated to this one. But I hope that for each week, I can formulate an idea of readings/tasks to complete, and ensure a solid balance of learning-time and active-programming time.

Week 0 (Now until April 2)

Create this syllabus.

Here are some things inspiring me (and related to this topic):

Popular algorithm names:

  • Huffman
  • Adaptive Huffman!
  • Shannon-Fano
  • Schwartz-Kallick

(Pre-RC Ideas Zone, this will move)

  • Video container/codec review, how do significant properties relate to each other when established by each format?
  • containers versus codecs, how do they work together (or not together?)
  • Begin manipulation of raw video data
  • "signal processing"
  • Muxing, demuxing, compiling code
  • ffmpeg filters
  • Visualizing compression: Is there a way to extract only-the-compression from audio or video for visualization purposes?
  • Build something in C -> I did build something in Rust with much help, so maybe this has been satisfied
  • Colorspaces
  • Subtitles
  • Codec manipulation
  • Web Assembly, put C in the browser
  • Electron, put JS on the desktop
  • Thinking about open source contributions here

Week 1 (April 2-6): "Containers/codecs"

Code written

  • Hmm, nothing that would merit a check-in...

Blogging

Exploring Matroska attachments

  • file format polyglot-ing (put mkv file inside mkv file)
  • This was very easy thanks to MKVToolNix!

Understand how codecs are made

  • read up on codecs

Open or non-open codecs, standards:

ffmpeg!

  • FFmpeg has been coming up a lot in conversation, with a lot of people working on different things. Someone told me about this book which is like the best thing ever. I'm ready to start a graduate level program and teach only this book. So many example commands and a visual representation of what they do.
  • I need to follow advice that Misty De Meo was given and passed along via Mastodon, which is "hacking around some stuff in FFmpeg's encoder to see what happens"
  • Misty's Sega FILM demuxer is in ffmpeg now github and I'm #fangirling

Codec exploration: ./lenticular

  • investigation of lenticular, a codec-in-progress for lenticular film
  • Running lenticular partially successfully
  • Ran into segmentation faults and I'd like to explore these more, but moving on for now

File formats, methods of implementation

  • deep read: the secret life of nan. This blog post casually mentions standardization drama and I clicked on the link to read more SO fast. It's here
  • deep read: google docs, not a file format I like these two quotes in a succinct blog post: "Proprietary formats are always risky. When the format isn’t a file format at all, you can’t even reverse-engineer it in principle." and "Don’t be a cheapskate. Pay for the storage you need."
  • deep read: How can we preserve Google Documents?

gdb / debugging in C

  • brew install gdb
  • extra stuff for macOS
  • Is the problem: macOS, gcc, gdb, Homebrew, XCode, the code, or something else?
  • "Reading symbols from lenticular...(no debugging symbols found)...done."
  • "Undefined symbols for architecture x86_64" - sometimes this is macOS being weird, but sometimes this is just me not remembering to call an external lib in C
  • "duplicate symbol _main in:"" - added -c to the two .o rules to tell gcc to stop at the compilation stage. But also noting that this could be main being defined twice in the program.
  • My brain today: "How do you debug a debugger?"
  • Valgrind
  • brew install valgrind

Rust

Manipulating data

gpg/pgp

  • Intro from Ernest
  • pki, public key infrastructure
  • "web of trust" / "strong set"
  • expiration and revokation
  • "security convenience balance"
  • yubikey is closed source, nitrokey is open software/hardware
  • Ring (end to end encrypted Skype) ring.cx

Cultural heritage and copyleft opinions

Lots of talks and inspiration from batchmate Marianne:

Preservation of complex media

Feelings

  • Something obvious to me when working in the context of the humanities is that we each have specific knowledge about a subject derived from our own lenses and cumulative individual experience (hello Black feminist critical theory), and the sooner I acknowledge this applies to technical subjects too, the better off I’ll be. RC is a great place for this because of the diverse experience and backgrounds collaborating in one space, but harder in a small monoculture like digital preservation.
  • adding increasingly dangerous amounts of coffee, trying to reign it in

Misc

  • first added a xml feed to my blog, next day I actually showcase that endpoint so people can use it: http://bits.ashleyblewer.com/feed.xml if you want to subscribe
  • TIL zulip offers free hosting for OSS projects https://zulipchat.com/for/open-source/
  • more zulip love: you can view source, you write in markdown
  • Looking at http://www.vapoursynth.com/about/
  • my review was requested on audiorecorder
  • "are you upstate or something? I picture you in some sort of compound like the end of mad men"
  • YCbCr: "But how do you make yellow?" Because colorspaces are cubes, and the absence of blue produces yellow, and the absence of red produces green.
  • fell into the same Jekyll time zone trap as Marianne did this week: A few of my posts was originally written at something like "23:16:00 -0400" so when the time changed for daylight savings, it reverted to thinking it "should" be the next day, deleting the original page and replacing it with a date that was one day later. Tricky! My hack to resolve it was to not get so granular with time-of-posts and kept it at the day-level for these late-ish night additions.
  • Lydia recommends: https://www.goodreads.com/book/show/44882.Code

Week 2 (April 9-13): R U S T

Code written

Blogging

Minimum Viable Codec

  • What does this look like?

Rust vs multimedia

  • Kostya's Boring Codec World
  • but especially: NihAV — Concept and Principles
  • The above blog post helps me break down my own problems-I-will-be-creating-for-myself with this statement: "I hope the domain for NihAV is clear: it will take ByteIO input, demux data using it (packets or elementary stream chunks—if you want them in packet format then use a parser), optionally fill timestamp information, decode frames, reorder them in display order if requested, similar approach for writing data."
  • Huffman coding implementation in Rust

Rust in general

rust-av

Codecs, etc

Video, in general

Linear Algebra

I think I may have to overcome my utter lack of maths knowledge soon...

Feelings

  • Learning can feel painful and that's okay!
  • How to make mistakes, acknowledge them, correct them, and move forward.
  • The difficulties of committing (and wanting to commit) to 100% focus on something (this) but still having to manage winding down paused or ending freelance projects, maintaining pre-existing relationships, and trying not to worry about housing and employment after RC.

Misc

Week 3 (April 16-20) "Minimum Viable Codec" (Encoding)

Code written

Blogging

Analog Video!

Encoding

Rust

Misc

Week 4 (April 23-27) "Minimum Viable Codec" (Decoding)

Goals

Goals at the start of this week:

  • Writing h264 decoder / mini-player
  • Getting Rust to the browser with wasm - done!
  • Writing encoder for video streams using huffman algo
  • Not have a cold anymore - done!

Code written

  • Hmmm...

Blogging

WASM

Video

Deep video

Misc

YUV2RGB shader math

vec3 yuv = vec3(yColor.r - 16.0/256, uColor.r - 0.5, vColor.r - 0.5);
oColor.x = dot(yuv, vec3(1.164,  0.000,  1.596));
oColor.y = dot(yuv, vec3(1.164, -0.391, -0.813));
oColor.z = dot(yuv, vec3(1.164,  2.018,  0.000));

Week 5 (April 30 - May 4) (Floating in space)

Code written

Blogging

Calling C in Rust

A return to Rust+WASM+JS

Video

Video players

Misc

Week 6 (May 7-11) (Planets in alignment)

Code written

Blogging

Malaysian General Election 2018

Video

Misc

BONUS WEEK (May 14-18)

!!Con May 12-13 - I got a ticket!

This is when Never Graduate Week happens. SPRING BREAKKKKKKKK!

Code written

  • I-Ching gets a save-state upgrade and makeover

Week 7 (May 21-25) (Reboot)

Code written

  • X

Blogging

  • X

Back 2 video

Manipulating video in the browser

Deepfakes again

MATH

Misc

Week 8 (May 28 - June 1)

Code written

Blogging

Encoding

Rust

Misc

YUV - RGB Conversion

R = Y + 1.4075 * (V - 128)
G = Y - 0.3455 * (U - 128) - (0.7169 * (V - 128))
B = Y + 1.7790 * (U - 128)

Y = R *  .299000 + G *  .587000 + B *  .114000
U = R * -.168736 + G * -.331264 + B *  .500000 + 128
V = R *  .500000 + G * -.418688 + B * -.081312 + 128

Week 9 (June 4-8)

Code written

Blogging

Real Life Hour

Rust

Video

Misc

Week 10 (June 11-15) 🎊🎂🆎

Video Pixel Formats

Misc

Week 11 (June 18-22) ⿒

Blogging

Python

Misc

Week 12 (June 25-28) 🙀

Python

Back on my bullshit

AV1

scanlime video content

Misc

THE END! 🎉

There is no end! Never graduate!

Thanks for reading!