Skip to content

Commit

Permalink
copy edits
Browse files Browse the repository at this point in the history
  • Loading branch information
Marc DaCosta authored and Marc DaCosta committed Sep 25, 2017
1 parent 68d8db6 commit 637cb05
Show file tree
Hide file tree
Showing 2 changed files with 17 additions and 18 deletions.
33 changes: 16 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,37 +1,36 @@
# Ambient Shipping

Utilities for capturing AIS messages from passing ships and joining them with public data on import records to understand what the ships are carrying.
This repo contains utilities for capturing AIS messages broadcast by passing ships and then joining them with public data sets that reveal what the ships are carrying.

### Overview:
Ambient Shipping enables you to look inside cargo ships. It has two main components:

- A simple set of functions that allows for joining between an arbitrary MMSI or vessel name and querying the Enigma API in order to determine what cargo that ship was carrying most recently and to examine what else has been transported in the shipping containers that it has unloaded.
- `ship-utils.py`: A simple set of functions that enables (1) the translation between an MMSI and a Vessel's Name and (2) wrappers around the [Enigma Public API](https://public.enigma.com) to query bill of lading import records for that Vessel to reveal what is inside it.

- A data relay server for recording AIS transmissions and which is meant to be run on a small linux machine with a SDR and in proximity to waterways frequented by cargo ships.
- `ais-server.py`: A data relay server for relaying decoded AIS transmissions to a remote database server. It is intended to be run on a small linux machine with a SDR (e.g. a raspberry pi) and in proximity to waterways frequented by cargo ships.

![NY Harbor](https://s3.amazonaws.com/marcdacosta.com/storage/nyharbor.jpg)


### Context:

The Automatic Identification System (AIS) is a broadcast radio signal that reports a ship's position, speed, heading and other metadata about its movements and is useful for keeping track of where ships are. The signals are broadcast on two VHF channels (161.975 MHz, 162.025 MHz) and generally have a range of 10-20 miles at sea, although building interference on land degrades the signals and can make them difficult to receive without line of sight for the antenna.
The [Automatic Identification System (AIS)](https://en.wikipedia.org/wiki/Automatic_identification_system) is a broadcast radio signal that reports a ship's position, speed, heading and other metadata about its movements. It is primarily used for safety and for ships to have situational awareness. The signals are broadcast on two VHF channels (161.975 MHz, 162.025 MHz) and generally have a range of 10-20 miles at sea, although, on land, buildings interfere with the signal and it can be difficult to receive without a clear line of sight for the antenna.

![AIS Transciever](https://s3.amazonaws.com/marcdacosta.com/storage/Matsutec-boat-GPS-navigation-equipment-5-6-Color-LCD-Marine-GPS-SBAS-Navigator-w-High-Sensitivity.jpg_640x640.jpg)
![AIS Transceiver](https://s3.amazonaws.com/marcdacosta.com/storage/Matsutec-boat-GPS-navigation-equipment-5-6-Color-LCD-Marine-GPS-SBAS-Navigator-w-High-Sensitivity.jpg_640x640.jpg)


There are many different types of AIS messages that are broadcast ([detailed here] (https://en.wikipedia.org/wiki/Automatic_identification_system#AIS_messages)) but, most commonly, an AIS transceiver sends basic positional data every 2 to 10 seconds depending on a vessel's speed while underway, and every 3 minutes while a vessel is at anchor.

There are many different types of AIS messages that are broadcast (detailed here https://en.wikipedia.org/wiki/Automatic_identification_system) but, most comonly, an AIS transceiver sends the following data every 2 to 10 seconds depending on a vessel's speed while underway, and every 3 minutes while a vessel is at anchor.

Using a [RTL-SDR](https://www.amazon.com/NooElec-NESDR-Mini-Compatible-Packages/dp/B009U7WZCA) dongle, these transmissions can be picked up and decoded. The [RTL-AIS](https://github.com/dgiardini/rtl-ais) project is quite useful as a local server which will receive AIS messages and output them on a local host port.
Using a [RTL-SDR](https://www.amazon.com/NooElec-NESDR-Mini-Compatible-Packages/dp/B009U7WZCA) dongle, these transmissions can be picked up and decoded. The [RTL-AIS](https://github.com/dgiardini/rtl-ais) project is quite useful as a local server which will receive AIS messages and output them on a local host port. It must be run as a background process for `ais-server.py` to function properly.

Example AIS data:
```
!AIVDM,1,1,,B,15MwDp0P0hJe>pLGBUq;q?wN2@KI,0*27
!AIVDM,1,1,,B,15N85>PP00re>T8GBVmf4?v:2<4R,0*6E
!AIVDM,1,1,,B,15N`GB0P01re?>nGBaPdCwvt26`T,0*45
```

The [spec](http://catb.org/gpsd/AIVDM.html#_aivdm_aivdo_sentence_layer) for AIS encoding is quite complicated, but convenient tools like Python's [libais](https://pypi.python.org/pypi/libais) make it straightforward to decode. Using this in conjunction with the `ais-server.py` script will enable recieved messages to be uploaded to a remote database for later analysis.
`!AIVDM,1,1,,B,15MwDp0P0hJe>pLGBUq;q?wN2@KI,0*27`

`!AIVDM,1,1,,B,15N85>PP00re>T8GBVmf4?v:2<4R,0*6E`


The [spec](http://catb.org/gpsd/AIVDM.html#_aivdm_aivdo_sentence_layer) for AIS encoding is quite complicated, but convenient tools like Python's [libais](https://pypi.python.org/pypi/libais) make it straightforward to decode. Using this in conjunction with the `ais-server.py` script will enable received messages to be uploaded to a remote database for later analysis.

For instance, a decoded AIS message will look something like this:

Expand Down Expand Up @@ -61,11 +60,11 @@ For instance, a decoded AIS message will look something like this:
}
```

The MMSI field contains the ship's Maritime Mobile Service Idenity number. The MMSI is a unique identifier for the ship's radio that can be used to communicate directly with the ship. MMSIs are regulated and managed internationally by the International Telecommunications Union in Geneva, Switzerland, just as radio call signs are regulated.
The MMSI field contains the ship's Maritime Mobile Service Identity number. The MMSI is a unique identifier for the ship's radio that can be used to communicate directly with the ship. MMSIs are regulated and managed internationally by the International Telecommunications Union in Geneva, Switzerland, just as radio call signs are regulated.

The `ship-utils.py` containes a function for looking up the MMSI with the ITU in order to find out the Vessel Name (you can also get the Vessel's satellite phone number if you're interested in giving them a call). If the ship is transporting containers or other commodities, the Vessel Name can be used to query the [Enigma API](https://public.enigma.com) to discover what the ship contains.

![Bill of Lading](https://s3.amazonaws.com/marcdacosta.com/storage/bill+of+lading.jpg)

Bills of lading is a document that describes a shipment: who sent it, where its going, what it contains, &c. It is filed with governmental tax authorities so that they can assess the relvant levies on the imported goods. Enigma regularly aggregates this data for the United States and its API can be used to query the most recent bill of lading filed by the observed container ship.
The `ship-utils.py` contains a function for looking up the MMSI with the ITU in order to find out the Vessel Name (you can also get the Vessel's satellite phone number if you're interested in giving them a call). If the ship is transporting containers or other commodities, the Vessel Name can be used to query the [Enigma Public API](https://public.enigma.com) to discover what the ship contains.

The script relies upon data published in the Automated Manifest System by the US Customs and Border Protection. It contains structured copies of the bills of lading of everything that is imported into the United States. A bill of lading is a document that describes a shipment: who sent it, where its going, what it contains, &c.
2 changes: 1 addition & 1 deletion ship-utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
def find_most_recent_snapshotid(dataset_id):
# for a particular enigma dataset, find the most recent update of the table.
url = "https://public.enigma.com/api/datasets/" + dataset_id
r = requests.get(url)
r = requests.get(url)
most_recent_snapshot_payload = r.json()
most_recent_snapshot = most_recent_snapshot_payload['current_snapshot']['id'] #most recent Enigma data snapshot for given dataset_id
return most_recent_snapshot
Expand Down

0 comments on commit 637cb05

Please sign in to comment.