Program: jcblock

A program to block telemarketing (junk) calls.
_________________________________________________
RECENT ACTIVITY (December 2014)

Additional extensive optimization of jcblock for Raspberry Pi - Model B, running
Raspbian, a TRENDnet TFM-561U USB modem, Microcenter Class 10 USB card, EasyAcc
Micro USB Power Supply, in the US was done.

The option of logging information to a jcblock.log was added for informational
purposes and to assist in debugging and optimizing modem command timing.

The timing of file I/O command and modem commands was studied and optimized with the
intent of reducing the time to detect end blacklist calls. Using a phone that has the
ability to silence the first ring has resulted in totally silent call blocking.

This version can handle caller ID strings with the NAME and NMBR strings in either
order.

This pi setup has been running successfully for eighteen months.

________________________________________________________________________________
RECENT ACTIVITY (March 2014):

This version of jcblock is a modification of the program written in 2008 by 
Walter S. Heath and subsequently updated on SourceForge (see below).

This version is simplified and optimized for Raspberry Pi - Model B, running 
Raspbian, and a TRENDnet TFM-561U USB modem. It was not tested with other 
modems.

This pi setup has been running successfully for eight months.

Specific modifications from latest WSH SourceForge or github/rajraj/jclbock
versions are:
- Deleted truncate.c, tones.c, common.h and all references to these files 
  (makejcblock).
- Because the tones function was eliminated, no special packages need to be 
  installed on the pi (ALSA).
- Set /dev/ttyACM0 as the default serial port.
- The timestamp for messages is obtained from the OS intead of the modem 
  (assumes that the raspberry pi has internet access - to get time)
- The format of entries in callerID.dat, whitelist.dat, and blacklist.dat has 
  been modified and unified to use a W3C Profile of ISO 8601 (YYYY-MM-DDThh:mm)
  as the timestamp and to allow easy use of standard Linux commands, such as
  sed, sort, awk, grep, cut, paste, etc. to report on junk call status and
  "clean up" the data files. 
- The whitelist.dat and blacklist.dat files can be easily sorted, 
  chronologically by using the Linux sort command. For example:
      cat blacklist.dat | sort -r -t"|" -k2.1,2.10 > blacklist.dat.new
  This sorting will place commented (#) entries at the end of the list. Then old
  entries can be easily deleted and/or archived using Linux commands of an 
  editor.
- In addition to using blacklist entries to check for junk calls, the length of  
  the caller ID entry is used to drop calls that have single character phone 
  numbers and single character caller ID's.
- Function write_blacklist has been deleted since it was not used
- The space insertion code was eliminated from the wait_for_response function.
- Shell script ConvertCID.sh is included to convert whitelist.dat, 
  blacklist.dat, and callerID.dat from the WSH jcblock old format to the new 
  "condensed" format (check the result with an editor).
- Future enhancements will convert checked entries to upper case to compare 
  against upper case blacklist entries since some callers resort to changing 
  case of their callerID.
  
Tips:
- Copy the whitelist.dat.example file to whitelist.dat and make your own 
  changes.
- Copy the blacklist.dat.example file to blacklist.dat and make your own 
  changes.
- Some phone answering machines allow silencing of the first ring. This 
  potentially (it does work) allows silent call blocking if jcblock can detect 
  and block the call before the second ring. Make sure the most recent or most 
  received blacklist.dat entries are at the top of the list.
- Use Linux commands to sort the blacklist.dat and whitelist.dat files. Put 
  most recent entries first (and delete or archive very old entries) to improve 
  performance
- Periodically archive the data files to reduce size to improve performance
- Run the raspberry pi headless and use ssh to access the junk call blocking pi 
  to edit the files, make changes, and produce reports
- Use crontab to start the jcblock upon reboot of pi 
              @reboot /home/pi/jcblock/jcblock

The following is a description of the new format of callerID.dat, blacklist.dat, 
and whitelist.dat files.

*******************  DO NOT USE TABS IN ANY DATA FILE ENTRY!   ****************

callerID.dat - written by jcblock
Each record contains the callerID data received after the first ring of the call
YYYY-MM-DDThh:mm|PhoneNumber|CallerIdString|
2014-03-19T08:01|12345678901|CallerIDString|

blacklist.dat and whitelist.dat - read and written by jcblock. Both files use
the same format.

The first field is the one tested. It can be the telephone number or some other
field in the caller ID string. It must be terminated with a '?'character and
have a | character in column 20.
The second field is the timestamp of the call. It must be 13 characters long and
be terminated with a | character.
The third field is a comment. It must not go beyond column 80 (for the |) and it
must be terminated with a | character. Use a comment to describe the entry.
Test field?        |YYYY-MM-DDThh:mm|Comment string                               |
TOLL FREE CALL?    |2014-03-02T12:00|18556006059, 18664715508|
________________________________________________________________________________


RECENT ACTIVITY (04/2013):

Several people have been successful running the program on
a Raspberry Pi processor using a USB modem. The modems that
have been used successfully are: US Robotics USR805637 and
TRENDnet TFM-561U. There is a good writeup on the details at:
http://weeklytechforum.com/2013/03/28/
                  block-unwanted-calls-with-a-raspberry-pi/

MODEM SELECTION:

I have been using the USRobotics model 5686G modem since
October 20, 2010 and have seen no errors. I have therefore
decided to recommend using it instead of the Zoom 3048 modem.

I used the Zoom modem during most of the development work on
this project. However, late in the testing I discovered that
occasionally it did not deliver caller ID data between the first
and second rings. I tried two 3048 units and saw the same result.
I also upgraded the firmware in one of them with no effect. I
sent email to Zoom and got no reply.

If you have a Zoom 3048 modem, definitely use it for evaluation
of the jcblock program. It will demonstrate all functionality of
the program and you can start building your blacklist. The symptom
you will see when the modem fails to deliver caller ID is that a
call that would normally be blocked by an entry in the blacklist
will be passed through. To get error free performance, you will
need to purchase(~$90) a USRobotics 5686G modem.

If you test the program with other modem brands, please report
your results (good or bad) using this site's mailing list or by
contacting me directly from this project's website. I will post
any useful reports I receive. Thank you...WSH


PROGRAM DESCRIPTION:

This program connects to a serial port modem and listens for
the caller ID string that is sent between the first and second
rings. It records the string in file callerID.dat. It then
reads strings from file whitelist.dat and scans them against
the caller ID string for a match. If it finds a match it accepts
the call. If a match is not found, it reads strings from file
blacklist.dat and scans them against the caller ID string for a
match. If it finds a match to a string in the blacklist, it sends
an off-hook command (ATH1) to the modem, followed by an on-hook
command (ATH0). This terminates the junk call.

The program also updates the date field of a matching whitelist or
blacklist entry. Entries that are old may then be identified so
that they may be removed. Note that the program will operate with
only a blacklist.dat file defined. A whitelist is not required.

Functions to manage the truncation (removal) of records from the
blacklist.dat and callerID.dat files are present in file truncate.c.
Records in the blacklist.dat file that have not been used to
terminate a call in the last year are removed. Records in the
callerID.dat file that are older than a year are removed. The
operations are performed every thirty days. Alternatively, entries
in the .dat files may be edited manually.

An additional feature is supported by functions in file tones.c.
The program will add a record to the blacklist.dat file for the
current call if the operator presses the star (*, asterick) key on
a touch tone telephone handset during an allotted time period. The
program detects the tone via a microphone placed near the modem
speaker.  Functions in file tones.c detect the presence of tones
(941 Hz and 1209 Hz) produced by pressing the star (*) key. If an
answering machine is installed on the same line this feature must be
restricted to be active only if a call is answered after the third
ring (the answering machine *must* also be set to answer on the
fourth or later ring). If an answering machine is not installed
on the line the star key feature may be made available for calls
answered after the third or later ring. This option is controlled
by a conditional compile flag (see the 09/24/10 UPDATES file entry
for more details). Also see the NOTE: in the file concerning
an ALSA version conditional compile selection.

The program requires a serial modem that can deliver caller
ID and contains a speaker. The recommended unit is the USRobotics
model 5686G. It will return caller ID if it is sent command:
AT+VCID=1. Note that the modem is used just to detect a call's
caller ID and the star key tones; the modem's normal communication
function is not used. The program may be terminated by sending
it a SIGINT (Ctrl-C) signal or a SIGKILL signal.

The program runs on a standard PC (it was written and tested
on a Dell Dimension B110 running Ubuntu). The features
supported by functions in files truncate.c and tones.c may be
removed by disabling conditional compile flags DO_TRUNCATE and
DO_TONES, respectively, in file jcblock.c. The default serial port
specifier is: /dev/ttyS0. The program will accept an option to
select a different port. The syntax of the call is:

        jcblock -p /dev/portID

The entire program may be compiled with the following command: 

gcc -o jcblock jcblock.c tones.c truncate.c -lasound -ldl -lm

Linux installations may or may not install the libasound library.
It is usually installed in /usr/lib. Also, the tones.c file
includes file asoundlib.h, which is usually located in directory
/usr/include/alsa. If not installed, they can be by using
the distribution's package manager. For example, for an Ubuntu
distribution, the Synaptic Package Manager may be used to
install alsa-base, to install the library, and alsa-source, to
install the source files. If /usr/include/alsa is not yet present
install libasound2-dev. It is also a good idea to install
alsa-utils in case testing/debugging of the sound interface is
needed. Here is a full list of the alsa packages I have installed
on my Ubuntu 10.04 machine (you can show the ones installed on
your machine by entering 'alsa' in the Synaptic Package Manager's
Quick Search: window):
alsa-utils		alsa-base		alsa-source
libasound2		libasound2-doc		libasound2-dev
libasound2-plugins	libsdl1.2debian-alsa	linux-sound-base
gstreamer0.10-alsa	bluez-alsa		libao2

No doubt some of these are not needed for the jcblock program,
but if you install all of them, jcblock should run.

If jcblock is run on a desktop or laptop, a microphone must be
placed near the modem's speaker and turned on. The volume level
must then be adjusted so that when the star key is pressed, the
audio generated causes the program's tone filter thresholds to be
exceeded.  Mic audio volume may usually be set by a Preferences
option. For Ubuntu the path is: System|Preferences|Sound. Then make
sure the proper sound device is selected under Hardware. The mic
volume may then be set under Input. The setting depends on the
position of the mic relative to the modem's speaker. The slider is
quite sensitive. Another way is to use the alsamixer program: choose
the sound interface (F6) and the Mic slider (arrow keys). Using
this program you have a numeric setting displayed that can be
reused. Remember to keep the phone you answer with away from the
mic, since the mic will respond to the tones received by the
handset!

For continuous use, the program should be run on a low-power single
board computer so that it can be left on all the time.

More on the whitelist:

As a result of operational experience it became evident that
the addition of a whitelist capability would be helpful. The whitelist
file (whitelist.dat) holds entries containing the telephone numbers
of calls the user definitely wants to accept. The program checks for
caller ID matches of these entries first. If a match is found, the
blacklist check is bypassed and the call is accepted.

The issue came up when it was discovered that some telemarketers use
cell phones! For cell phone calls, the 'NAME =' field in the caller ID
string is usually: 'Cell Phone   XX', where XX is the originating state
designator (e.g., CA for California). So using the 'NAME =' field as the
search string in a blacklist entry would cause all cell phone calls
from the indicated state to be blocked! In addition, the telemarketers
use several cell phones (probably as employees come and go), so using
the 'NMBR =' field in the caller ID as the search string does not catch
new cell phone callers. Adding a whitelist partially solves this
problem.

With a whitelist, the user can include entries that contain the cell
phone numbers of calls they definitely want to receive from a specific
state (e.g., CA). Then the cell phone 'NAME =' field for the state (e.g.,
'Cell Phone   CA') can be used as the search string in a blacklist
entry. With this arrangement all cell phone calls from the state will
be blocked *except* the ones with numbers present in the whitelist.

As another example, a person might want to receive calls from just
certain numbers and no others (an elderly person, for example). In that
case the records for the numbers of interest would be placed in the
whitelist and a single record placed in the blacklist that contains a
field that is present in all caller IDs (e.g., 'NMBR'). Then all
calls not matched by records in the whitelist would be terminated.

Country codes:

The USRobotics (and probably other modems) must be sent a country
code command if the country of use is other than USA (the default).
The command syntax is: "AT+GCI=XX\r", where XX is the country code.
The codes for the USRobotics modem are as follows (check your modem
'Command Set' documentation for confirmation):

Austria		0A	Hungary		51	Norway		82
Belgium		0F	Iceland		52	Poland		8A
Cyprus		2D	Ireland		57	Portugal	8B
Czech and
Slovak Federal	2E	Israel		58	Spain		A0
Republic
Denmark		31	Italy		59	Slovakia	FB
Estonia		F9	Latvia		F8	Slovenia	FC
Finland		3C	Liechtenstein	68	Sweden		A5
France		3D	Lituania	F7	Switzerland	A6
Germany		42	Luxembourg	69	Turkey		AE
Greece		46	Malta		B4	United Kingdom	B4
			Netherlands	7B

The command should be sent in the init_modem() function after the modem
reset command (ATZ) is sent (the location is indicated in the listing).
Other program changes may be needed to conform to non-USA country telephone
systems.

Real-time prototype system:

The photo shows a prototype jcblock system consisting of a USRobotics
5686G modem and an eBox-3300 Server-in-a-box (SIB) purchased from EMAC,
Inc. The SIB contains a Vortex86DX System-on-a-Chip that implements
most of the functions of a standard x86 PC. The system shown draws about
6 watts, so the cost of running it 24/7 is minimal. The eBox came with
a version of Linux installed that did not have the ALSA audio interface.
Since this application requires a microphone to implement the star (*)
key feature, I chose to boot a stripped down version of Knoppix from a
USB memory stick.

I started with the version included on a DVD with the book:
"KNOPPIX HACKS, 2nd Edition", O'Reilly, ISBN:978-0-596-51493-8
and "remastered" it (trimmed its size) using the procedures
described in hacks #109 and #110 (in addition, I removed all
documentation and Linux source code, etc.). The result was a
version that would fit on a CD, for backup. The procedure results
in a knoppix.iso file which I then used to make USB boot version
using a procedure I found at:
	http://www.knoppix.net/wiki/USB_Based_FAQ
Note that if you don't need the star key feature, the program will
run just fine on the version of Linux that comes with the eBox.
The Knoppix version has the additional advantage that the gcc
compiler, libraries and include files needed to compile the program
are present on the target.

The knoppix system did not contain an Ethernet module that was
compatible with the Vortex86DX chip. The needed module is
r6040.ko. It may be downloaded from a site reached via:
  http://dir.filewatcher.com/d/Debian/all/net/r6040-source
The module depends on module mii.ko, which was included with
Knoppix.

The photo shows the Knoppix USB stick installed in a front USB
socket. All application files are contained on the USB stick
installed in the back. This provides non-volatile storage for
them, which is needed for the callerID, blacklist and whitelist
files. The eBox BIOS may be instructed to boot from the USB stick.

During the process of constructing the knoppix.iso file, as
described above, I installed a script called startApp in /etc/init.d
and link S99startApp to it in /etc/rc2.d to provide for automatic
startup of jcblock (the grub loader menu.lst file instructs grub to
boot to level 2). Script startApp mounts the application USB stick
on /mnt and then runs a setPorts script on it to perform various
system administration functions not included in the knoppix.iso
build (some that were forgotten!) before starting the application
program jcblock. As a result, the entire system starts up and runs
when the power is turned on without the need for manual operations
via a terminal and keyboard. Since sshd and ftpd daemons are started,
all maintenance may be performed via the Ethernet interface. It is
especially convenient to maintain data files using a Firefox plugin
called FireFTP. So the system may be maintained through a GUI from
any computer that can run Firefox (e.g., Windows, Mac, etc.).

There is the issue of interrupting a write to flash memory by a
power failure (or turn-off), which could cause damage to the memory.
The application program is normally blocked in a serial port read
operation, waiting for a call. The only time writes occur is for a
short time immediately after a call is received. So, if the unit is
turned off at other times, no damage will occur to the USB memory.
On the outside chance that memory is damaged, the application USB
memory stick may be easily replaced. Since Knoppix loads into a
RAM disk, no damage can occur to it. 

Wrap-up:

That about summarizes my experience with this project. The
system runs continuously at our home and terminates junk calls
indicated in the blacklist file. After about six months the number
of junk calls we received decreased to only a few a week as the
telemarketers removed us from their call lists.

The easiest way to get started with this project is to just compile
the program and run it on a standard PC. It works fine that way,
but you have to leave your PC on all the time to be useful in
terminating calls. If you do that anyway then you are all set!
Otherwise you should eventually consider installing it on a dedicated
low power embedded computer system similar to the one I described
above. If your PC does not have a serial port, a PCI serial port card
or USB serial port adapter can be installed at minimal expense.