serpentify

a safeguard for ophidiophobic cinematophiles

films are great! we all love watching films we haven't seen before. films contain many new and exciting things, like hot-air balloons and mountains and sometimes snakes.

some people are however not quite as keen on snakes as others, and would like to know if the film is going to have them turn up unexpectedly, without having to scrupulously read the plot summary in advance or look on IMDB to see if they credited a snake wrangler.

hence, serpentify: a very rough-and-ready bash script to scan a video file and identify if it contains any snakes.

1. It opens a single video file (of any size/format) and samples one frame every few seconds using ffmpeg (the default is once per four seconds)
2. Each frame is processed using the python cameratrapai package, which uses MegaDetector to identify things that might be animals and SpeciesNet to classify them by taxa
3. The resulting file is scanned for any frames that seem to have a snake, even at low probability, and these are flagged and reported with a timestamp.

The overall reliability is not (at this point) immensely high.

• False negatives: Very brief appearances by snakes might be missed due to the once-per-X-seconds timing, and extended sampling times may miss even a longer appearance
• False negatives: Not every recognisable snake is identified by the image processor (in some cases not at all, in some cases just as an unspecified reptile), but it always seems to pick up at least one frame if the appearance is substantial. The underlying problem here is probably that this tool is designed for wildlife camera traps, and the snakes we see in films are often in quite artificial contexts.
• False positives: Some shots of the sea apparently can be misidentified, and even accurate identification of non-snake reptiles will trigger it (the search is picking up any matches for "squamata").
• Efficiency: without any optimisation, it currently takes about half as long as the actual film to do the scanning and it chews up a lot of processing power. This is not great.

But it does, approximately, work. And isn't that the whole point of weekend projects?

Testing results

• Casablanca (1942): the control film. There are no snakes in this film, and it returned no snakes.
• Robin Hood (1973): it struggles with cartoon snakes, unsurprisingly. It does pick up some appearances of the snake, but buried in a lot of false positives
• Death on the Nile (1978): correctly identifies appearance of a (real) cobra at around 1h22m; triggered on three of the four frames showing a full snake but none of the four showing only the head.
• Blade Runner (1982): it identified only one frame with the (real) python, at 52m - albeit identified as a yellow-spotted monitor.
• Master and Commander (2003): another snakeless control. This one did return some false positives; seven frames of the sea at the very start and end, and one of a bit of rope 50m in. But it also returned two more bona fide errors; the Galapagos iguanas are identified as iguanas and so returned by the script as they include "squamata".
• The Jungle Book (2016): it identified two of the frames containing the (CGI) python, at 30m and 33m.

Blade Runner and The Jungle Book are not compelling results, but they would suffice to warn about broadly where the snake scene is going to be. In the case of Blade Runner it may in part be unlucky timing; the snake scene is a bit longer but the frames it had to work with were often ones where it wasn't obvious what it was. Rerunning with a different offset might have been luckier.

FAQ

Can it detect other things?

If "other things" is an animal, very probably. Try running the script, look at the predictions.json file, and then change the squamata part of the search to the appropriate term. It seems very keen to identify zebras and white-tailed deer, however; it found deer in a blank screen at one point.

Can it run more efficiently? I don't want to wait an hour to find out if I'm about to get snaked.

In theory cameratrapai should pick up your computer's GPU and run much more quickly. For reasons I haven't worked out yet it hasn't for me. I felt it best to not go down that snake hole of optimisation or I'd never get anything done.

Can you guarantee this will stop snakes being in my films?

No - please speak to your casting director and ask them to tighten up their interviewing.

Why was there a cobra in Death on the Nile, anyway?

You've got me. Biggest plot point in the whole film. Where did [blank] get it from, on a boat? It just makes no sense.