No description, website, or topics provided.
Clone or download

README.md

Nopelepsy

About 1 in 4,000 people in the world are affected by photosensitive epilepsy, and general photosensitivity affects even more than that. While the UK has guidelines to prevent bright flashing in television broadcasts, this isn't true of the rest of the world.

A Harding test is a type of program that tries to automatically detect when a video can cause seizures from flashing lights and patterns. Nopelepsy is a type of Harding test, but with a twist: Nopelepsy also has the ability to automatically edit video frames to remove the flashes it detects.

Nopelepsy isn't complete, and isn't perfect, but it's a start in the right direction. It's also experimental, so use at your own risk.

Installation

Building from source requires OpenCV, and OpenCL for the (buggy) OpenCL version.

There's no installation at the moment. On Windows, just download the latest release and unzip. For OSX or Linux, you'll have to build from source.

One prerequisite to running the software is installing FFmpeg. This is required at the very end of every conversion to add back the original audio to the cleaned file.

Before running Nopelepsy on any videos, run

nopelepsy -test

and observe the output. If there are any failures, then something has gone horribly wrong, and you shouldn't try to use it.

Usage

To invoke the cli tool, make sure the Nopelepsy executable is in your PATH and do

nopelepsy [path/to/video]

to run with the current settings. To adjust the settings, you'll need to edit the settings.json file. To find where this is located, invoke

nopelepsy -settings

Here's an explanation of the following fields:

  • percentage_thresh (float): The percentage of a 10 degree field of view that needs to be flashing concurrently in order to trigger the detector. Default is 0.25, and should probably never exceed 0.5.
  • luminance_thresh (float): The change in brightness of a flash that needs to occur in order to trigger the detector. Default is 0.1.
  • reduction_iters (int): The maximum amount of iterations to spend minimizing the amount of screen space to change. This number does not effect the safety of the program, but does effect the performance. Higher numbers result in higher quality, but are less performant, and -1 stands for no upper bound.
  • stack_size (float): The size, in seconds, of the maximum detectable flash. Lower numbers increase performance, but also increase risk. See below for more details. Default is 1.0.
  • block_size (int): Size of local buckets to process together. If you want to increase performance of Nopelepsy, increase this number first. This should not effect the safety of the program.
  • display_frames (bool): Whether or not to display the output frames. This actually decreases the efficiency of the program by a significant amount.

other fields are either optional, experimental, development, or are unused.

The fields percentage_thresh, luminance_thresh, and stack_size are the only ones above that should affect the safety of the software. The default settings for the first two come from this W3C spec. The luminance_thresh value may have to change, depending on the brightness of your TV/monitor.

The default for the stack_size is by far the most personal field. The default value comes from me playing around with the number and consulting with someone who is very sensitive to flashes. For most situations, the default setting of 1.0 is overkill, and 0.4 works well and has a higher-quality output. However other situations, like this harry potter scene seem to require a setting of at least 0.75 to get rid of all flashes.

However, there are other people for which flashes aren't as painful, and so a much lower number like 0.2 (max 5-frame flash at 24fps) might be acceptable. Keep in mind that videos have framerates, and that the detector is limited by the Nyquist frequency, and so the minimum effective parameter is 2/framerate, which is usually either 0.09 or 0.03 for 24fps and 60fps video respectively.

A standalone C++ API will be coming later.

Questions I haven't been asked yet, but I forsee being asked

You're not using the GPU, what's up with that?!

I don't have a really good reason why I'm not. I wanted a "ground truth" version of Nopelepsy as a baseline before I went all in, and it was simple enough to everything so far using OpenCV. In all fairness, OpenCV is very well optimized, and using it means that I can multithread the a large portion of the program, so it's not like it's that bad. I mean it is, but not that that bad.

I'll get to it when I get to it. There are more important fish to fry right now.

Why are all flashes getting removed, and not just strobes?

For one thing, it's much easier this way. It's a whole other can of worms to try to find out what flashes constitute "near each other" in order to remove them.

Also, many people who are photosensitve and/or epileptic feel physical pain when seeing a flash of light, even if it's not enough to cause a seizure. If it's easier to fix both the seizures and the pain than it is to fix just the seizures, then why not do both?

I plan on adding functionality at some point to be able to switch to only multiple flash detection. But again, bigger fish to fry.

I used Nopelepsy to remove the flashes from a Star Wars movie, and there were still painful red flashes! What gives?!

Yeah, so.... That's a known issue, but I don't know how to fix that.

The W3C spec defines flashes to happen in the presence of general light, and separately in the presence of saturated reds. The problem is that certain types of motion are definitely flashes, but aren't bright enough or red enough to qualify. It's possible that a future motion detector might do the trick, but I haven't been able to get it to work.

Does this software detect and remove saturated red flashes? What about harmful patters, like moving stripes of polka dots?

Not yet. Haven't gotten around to implementing either of those.

Why are you using the W3C spec instead of the Ofcom guidelines?

The W3C spec is more useful. It gives the parameters in terms of steradians and percentages, instead of areas of the screen. This extends well to different sized TVs and monitors.

One annoying thing about the W3C spec though is that it references a paper by Harding that supposedly lays out these guidelines, but I can't for the life of me find a copy of it anywhere.

TODO

Where do I begin? In no particular order:

  • System
    • GPU port
    • OSX and Linux builds. (if there's any otool wranglers out there willing to help with the OSX build, please get in touch)
  • Detection
    • Red/other color flash detection (this will require more research)
  • Cleaning algorithm
    • Dynamic block sizes
    • Dual brightness modification
    • Or better yet, correction using motion and object tracking instead
  • Different modes
    • Only multiple flashes
    • Reactive/predictive mode