Permalink
Browse files

Initial commit

  • Loading branch information...
0 parents commit e243da24610832d868bdd3b33aaca2a53cfe970a @iafan committed Jan 20, 2013
@@ -0,0 +1,2 @@
+~*
+~*.*
@@ -0,0 +1,20 @@
+Copyright (c) 2013 Igor Afanasyev, https://github.com/iafan/Hacksby
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
196 README.md
@@ -0,0 +1,196 @@
+About Hacksby
+=============
+
+### Hacksby = Hack-a-Furby project
+
+Hasbro's Furby toy (year 2012 model) uses audio protocol to communicate with other
+nearby Furbys and with the official 'Furby' applications for iOS and Android.
+
+This project is an educational attempt to analyze and re-create the audio protocol
+and communicate with Furby using computer in a search for some easter eggs or
+otherwise undocumented features.
+
+At its current state, the project includes:
+
+ 1. Description on the audio protocol (see below)
+ 2. [An incomplete] description of various Furby commands
+ 3. An Perl library and scripts that can generate and play WAV files
+ with arbitrary commands (to talk to Furby)
+ 4. A script that can decode commands from a provided audio stream
+ (to interpret Furby's responses).
+
+
+Disclaimer
+==========
+
+### This information is provided for personal educational purposes only.
+### The author does not guarantee the accuracy of this information.
+### By using the provided information, libraries or software, you solely take the risks of damaging your hardware or your ears.
+
+See `MIT-LICENSE.txt` for more information.
+
+
+Audio Protocol
+==============
+
+Furby audio protocol uses high-pitch frequencies to encode special commands (command
+is an integer number in [0..1023] range). Furby decodes such commands using its built-in
+microphone and may respond to some of them. When an event occurs (Furby pronounces
+some phrase or performs an action), he can also emit such a command in addition to
+an audible sound. This feature is used in the official application 'Translator'
+mode, when it recognizes what Furby said and provides an instant translation.
+
+Each command is divided into two packages with 0.5 sec gap in between.
+The first package carries the higher 5 bits of the command number, and the second one
+carries the lower 5 bits.
+
+If you record the responce from Furby or from iOS application and view its spectrum,
+You will see that each packet looks like this:
+
+ 2 --------##--------------------------------------##----
+ 3 ----##----------------------##--##----------##--------
+ X --##--##--##--##--##--##--##--##--##--##--##--##--##--
+ 1 --------------------##--------------##----------------
+ 0 ------------##--##------##--------------##------------
+
+The total length of the packet is 0.5 second. Here `X` is a central frequency (17500 Hz),
+and `0`, `1`, `2` and `3` are the data frequencies. The distance between each adjacent frequency
+is approx. 557 Hz.
+
+The central frequency carries no data and was likely introduce to aid in packet decoding.
+The other four frequencies carry data. If one writes down the packet depicted above as a number in the
+quaternary numeral system using 0, 1, 2, 3 digits after the name of each frequency, he would
+get the following number: `3200 1033 1032` (for clarity, the number is separated into three quadruplets
+each representing a byte).
+
+The first byte, `3200`, if written in the binary form, will look like this: `11 1 00000`, where the first
+two bits are always `11`, the second bit will be `0` for the first packet and `1` for the second, and
+the remainder `00000` represents the 5 data bits themselves.
+
+The second byte, `1033`, depends on the data bits and is used as a checksum (original algorithm is unknown).
+The `lib/Furby/Packet.pm` file lists all 64 checksums (32 for the first packet and 32 for the second) needed
+to reconstruct any arbitrary command in [0..1023] range.
+
+The last byte, `1032`, is always the same.
+
+To eliminate ticks when playing back such audio packets, the original waveform uses smooth changes in
+frequency between each data tone. Also, such frequencies are not noticable if combined with pretty loud
+audible responses Furby generates at the same time.
+
+
+Commands
+========
+
+Based on initial research, a list of known commands (or events) and their descriptions is provided in
+`lib/Furby/Commands.pm` and `lib/Furby/Dictionary.pm`. The list is incomplete and the already existing
+descriptions may be inaccurate.
+
+The task of interpreting the meaning of the commands is complicated further by the fact that Furby
+can be in one of 6 personalities, and depending on its current personality, may respond to commands
+differently and produce different events on its own.
+
+### Furby Personality
+
+When Furby understands a command, it will respond back with his current personality id.
+Known personalities (as listed in official Android application) are:
+
+ 1. Princess (command id `901`) — a lovable one
+ 2. Diva (command id `902`) — a musical one
+ 3. Warrior (command id `903`) — also known as 'evil'
+ 4. Joker (command id = `904`) — also known ad 'mad' or 'freaky'
+ 5. Gossip Queen (command id `905`) — a chatty one
+
+There is no known way to instantly change Furby's personality via some special command. Sending a command
+with the personality id back to Furby seems to produce no effect.
+
+### Communication Mode
+
+When Furby chats or performs any action, it will ignore any commands sent to him.
+So one needs to ensure Furby is listenting before sending any command. Luckily, there's command `820`
+which will put him into such listening mode for one minute (during this period, Furby just stay awake
+and listen for other commands). This comamnd is used in the official applications and is sent every 40
+seconds or so to keep Furby listen and stop doing silly things.
+
+Unfortunately, even if one sends this command periodically, Furby will go into deep sleep mode after
+10 minutes of inactivity. The only way to prevent it from sleep is turning or flippng Furby periodically
+so that its orientation sensor detects the movement.
+
+Tools
+=====
+
+Tools are located in `bin` directory.
+
+### Send Commands to Furby
+
+**CAUTION: setting the volume too high while playing back Furby commands may damage your ears!**
+
+Put your Furby near the speaker connected to your computer (or connect an earbud to a headphone jack
+and put it in front of Furby). Turn the volume all the way down. Wake up Furby and wait till it listens quietly.
+Run the command below:
+
+ perl furby-send.pl 350
+
+Turn the volume up a little and run the command again to see if Furby recognizes it
+(he should chew and then say something like "mmm, yum!"). If it doesn't, turn the volume up a bit and repeat
+the procedure until it does.
+
+#### Troubleshooting
+
+If you can hear a discomforting high-pitch noise when the command plays but Furby doesn't repsond,
+then something is wrong. Try putting Furby closer to the speaker or the earbud and make sure Furby doesn't
+do anything on his own while you play back the command. Normally, Furby should pick up the command even if
+you are not hearing it yet.
+
+#### Interactive Mode
+
+You can also run the tool in interactive mode:
+
+ perl furby-send.pl 350 --interactive
+
+After playing back the first command (`350`), the script will wait for your input. You can just press Enter to
+play the next command in the range, or input a command number to play, or input anything else that doesn't
+evaluate to a number (for example, `r`) to repeat the last command. This mode will allow you to explore the
+Furby reactions to different commands.
+
+The generated WAV file is saved as `out.wav` in the current directory. Under Windows, the playback is performed
+using the provided command-line utility (`bin/win32/dsplay.exe`). Under Unix / Mac, an attempt to use
+already existing console players is used (but not tested).
+
+
+### Listen and Decode Furby Commands
+
+The provided `furby-decode.pl` decodes the RAW PCM data from STDIN and displays any commands it decodes.
+The only supported format is 44.1KHz mono 16bit signed PCM format.
+
+You can pre-record a WAV file using your microphone and then decode it using this tool like this:
+
+ perl furby-decode.pl < record.wav
+
+or pipe in PCM data from a streaming application or virtual device. Under Windows, a binary tool is included
+(`bin/win32/rec_stdout.exe`) which records data from a default input source and constantly streams it to STDOUT.
+You can use it like this:
+
+ win32\rec_stdout.exe | perl furby-decode.pl
+
+or just run
+
+ furby-listen.bat
+
+Before you run this command, make sure you have a microphone plugged in, that it is selected in Sound settings
+as a default capturing device and put your microphone close to Furby.
+
+The sound capture hasn't been tested on platforms other than Windows.
+
+#### Troubleshooting
+
+Some microphones (especially the ones integrated into web cameras) use a low-pass filter that would essentially
+wipe out all frequencies that are used to transmit commands. If your decoding doesn't work, try recording
+the audio sample into the 44.1KHz mono 16bit signed PCM WAV file (touch Furby's head or tum to make sure it responds
+to your interaction and thus emits an event command), then open the file in a sound editor like Audacity,
+make sure the audio format is correct, switch to a spectrum view and see if there are any frequencies recorded
+around the 17.5KHz range (you will clearly see if there are command packets there).
+
+
+### Feedback
+
+Feedback and any further research results are always welcome.
@@ -0,0 +1 @@
+out.*
@@ -0,0 +1,152 @@
+#!/usr/bin/perl
+
+# Copyright (C) 2013 Igor Afanasyev, https://github.com/iafan/Hacksby
+
+use strict;
+
+# only raw 16bit signed PCM data is supported
+
+use GD;
+use Math::FFT;
+
+my $filename = $ARGV[0];
+
+if ($filename eq '') {
+ print "Usage: perl $0 filename.raw\n";
+ exit(1);
+}
+
+if (!-f $filename) {
+ print "File '$filename' doesn't exist\n";
+ exit(1);
+}
+
+my $base_freq = {
+ '0' => 16386,
+ '1' => 16943,
+ 'X' => 17500, # base frequency, used to construct raw commands (delta is approx 557 Hz) // was 17498 with 556 Hz delta
+ '3' => 18057,
+ '2' => 18614,
+};
+
+my $allowed_deviation = abs($base_freq->{'X'} - $base_freq->{'1'}) * 0.2;
+
+my $buffer;
+my $read_block_size = 10000;
+my $sample_rate = 44100;
+my $size_in_samples = 256;
+my $expand_index = 5;
+my $max_image_width = 10000;
+
+my $fft_min_freq = 0;
+my $fft_max_freq = $sample_rate / 2;
+my $fft_spectrum_size = int($size_in_samples / 2) + 1;
+my $fft_freq_delta = ($fft_max_freq - $fft_min_freq) / ($fft_spectrum_size - 1);
+
+my $spectrum_first_idx = int($base_freq->{'0'} / $fft_freq_delta) - $expand_index;
+$spectrum_first_idx = 0 if $spectrum_first_idx < 0;
+
+my $spectrum_last_idx = int(($base_freq->{'2'} + $fft_freq_delta / 2) / $fft_freq_delta) + $expand_index;
+$spectrum_last_idx = $fft_spectrum_size - 1 if $spectrum_first_idx > $fft_spectrum_size - 1;
+
+my $image_height = $spectrum_last_idx - $spectrum_first_idx + 1;
+
+my @spectrum_frequencies;
+for my $y ($spectrum_first_idx..$spectrum_last_idx) {
+ $spectrum_frequencies[$y] = $fft_min_freq + $y * $fft_freq_delta;
+}
+
+my $image_width = int((-s $filename) / ($size_in_samples * 2));
+if ($image_width > $max_image_width) {
+ $image_width = $max_image_width;
+ print "Image would be too big; it will be trimmed to first $max_image_width spectrum pxiels\n";
+}
+
+my $image = new GD::Image($image_width, $image_height, 1);
+
+my $base_freq_colors = {
+ '0' => $image->colorResolve(255, 0, 255),
+ '1' => $image->colorResolve( 0, 204, 255),
+ 'X' => $image->colorResolve(255, 0, 0),
+ '3' => $image->colorResolve( 0, 255, 0),
+ '2' => $image->colorResolve(255, 255, 0),
+};
+
+open(IN, $filename);
+#sysread(IN, $_, 56); # skip RIFF header
+read_from_handle(*IN);
+close(IN);
+
+my $x = 0;
+
+my @samples;
+my $buffer_leftover;
+sub read_from_handle {
+ my $handle = shift;
+ while ((my $n = sysread($handle, $buffer, $read_block_size)) > 0) {
+ #print "Read $n bytes\n";
+ $buffer = $buffer_leftover.$buffer if $buffer_leftover; # prepend leftover byte, if any
+ while (length($buffer) >= 2) {
+ my $sample = unpack('s<', substr($buffer, 0, 2, ''));
+ push(@samples, $sample);
+ if ($size_in_samples == @samples) {
+ analyze_samples();
+ undef @samples;
+
+ $x++;
+ last if $x > $max_image_width - 1;
+ }
+ }
+ $buffer_leftover = $buffer; # potentially there can be a single byte that we need to preserve for next iteration
+ }
+}
+
+sub analyze_samples {
+ my $fft = new Math::FFT(\@samples);
+ my $spectrum = $fft->spctrm(window => 'hann');
+
+ my $avg = 0;
+ my $max = 0;
+ my $max_y = -1;
+ my $freq = 0;
+ for my $y ($spectrum_first_idx..$spectrum_last_idx) {
+ $avg += $spectrum->[$y];
+ if ($spectrum->[$y] > $max) {
+ $max = $spectrum->[$y];
+ $max_y = $y;
+ $freq = $spectrum_frequencies[$y];
+ }
+ }
+ $avg = $avg / $fft_spectrum_size;
+ my $contrast = ($avg == 0) ? $fft_spectrum_size : $max / $avg;
+
+ my $known_frequency_key;
+ foreach my $key (keys %$base_freq) {
+ if (abs($base_freq->{$key} - $freq) < $allowed_deviation) {
+ $known_frequency_key = $key;
+ last;
+ }
+ }
+
+ for my $y ($spectrum_first_idx..$spectrum_last_idx) {
+ my $color = $max > 0 ? int($spectrum->[$y] / $max * 255) : 0;
+ $color = 255 if $color > 255;
+ #$color = int($color * 0.5);
+ if ($contrast < ($fft_spectrum_size / 3.5)) {
+ $color = $color * 0.2;
+ $color = $image->colorResolve($color, $color, $color)
+ } else {
+ $color = ($y == $max_y) ? $image->colorResolve(255, 255, 255) : $image->colorResolve($color, $color, $color);
+ if (defined $known_frequency_key && ($y == $max_y)) {
+ $color = $base_freq_colors->{$known_frequency_key};
+ }
+ }
+ $image->setPixel($x, $image_height - 1 - ($y - $spectrum_first_idx), $color);
+ }
+}
+
+print "Saving out.png...\n";
+open(OUT, ">out.png");
+binmode OUT;
+print OUT $image->png;
+close OUT;
Oops, something went wrong.

0 comments on commit e243da2

Please sign in to comment.