feat: updated dataset (2022-06-28) (#88)

* chore: update DATA_HASH to 20220628
* feat: b-tree for GeoLite2-City-CSV_20220628

This was generated using 'npm run generate' from ancient version 5.0.1
but with source data being GeoLite2-City-CSV_20220628

README.md

@@ -11,6 +11,26 @@
 
 > geoip lookup over ipfs
 
+
+# Table of Contents
+
+- [IPFS GeoIP](#ipfs-geoip)
+- [Table of Contents](#table-of-contents)
+  - [Install](#install)
+    - [NPM](#npm)
+    - [CDN](#cdn)
+  - [Usage](#usage)
+  - [API](#api)
+    - [`lookup(ipfs, ip)`](#lookupipfs-ip)
+    - [`lookupPretty(ipfs, multiaddrs)`](#lookupprettyipfs-multiaddrs)
+  - [Maintenance](#maintenance)
+    - [CIDs of the lookup dataset](#cids-of-the-lookup-dataset)
+    - [Updating GeoLite2 dataset](#updating-geolite2-dataset)
+  - [Testing in CLI](#testing-in-cli)
+  - [Contribute](#contribute)
+    - [Want to hack on IPFS?](#want-to-hack-on-ipfs)
+  - [License](#license)
+
 ## Install
 
 ### NPM
@@ -79,14 +99,15 @@ Returns a promise that resolves to an object of the form
 Provides the same results as `lookup` with the addition of
 a `formatted` property that looks like this: `Mountain View, CA, United States, Earth`.
 
-## b-tree
+## Maintenance
 
-The utility geoip-gen reads csv files provided from GeoLite, and turns them into a 32-way branching b-tree, which is stored as ipfs json objects.
+### CIDs of the lookup dataset
 
-> 👉  **Note:** this library uses old type of ipfs json objects for legacy reasons,
-be mindful of that and do not use its code as an example.  Modern code should
-use [`dag-cbor`](https://github.com/ipld/specs/blob/master/block-layer/codecs/dag-cbor.md)
-and [`ipfs.dag`](https://github.com/ipfs/js-ipfs/blob/master/docs/core-api/DAG.md) or [`ipfs.block`](https://github.com/ipfs/js-ipfs/blob/master/docs/core-api/BLOCK.md) APIs.
+The current root hash for lookups is defined under `GEOIP_ROOT` in `src/lookup.js`.
+
+It is a proprietary b-tree generated from source files provided defined under `DATA_HASH` in `src/generate/index.js`.
+
+### Updating GeoLite2 dataset
 
 There is a generator included, that can be run with
 
@@ -96,7 +117,20 @@ $ npm run generate
 
 This takes quite a long time to import, but you only need to do it once when updating the global index used by the lookup feature.
 
-## Example
+It reads original GeoLite CSV files provided from `DATA_HASH` directory defined
+in `src/generate/index.js`, and turns them into a 32-way branching b-tree,
+which is stored as ipfs json objects.
+
+The produced CID should then be pinned and stored as the new `GEOIP_ROOT` in
+`src/lookup.js`
+
+> 👉  **Note:** this library uses old type of ipfs json objects for legacy reasons,
+be mindful of that and do not use its code as an example.  Modern code should
+use [`dag-cbor`](https://github.com/ipld/specs/blob/master/block-layer/codecs/dag-cbor.md)
+and [`ipfs.dag`](https://github.com/ipfs/js-ipfs/blob/master/docs/core-api/DAG.md) or [`ipfs.block`](https://github.com/ipfs/js-ipfs/blob/master/docs/core-api/BLOCK.md) APIs.
+
+
+## Testing in CLI
 
 You can find an example of how to use this in [`example/lookup.js`](example/lookup.js), which you can use like this:
 
@@ -115,9 +149,6 @@ Result: {
 Pretty result: New York, NY, USA, Earth
 ```
 
-## Root hash
-
-The current root hash for lookups is defined under `GEOIP_ROOT` in `src/lookup.js`
 
 ## Contribute
 

src/generate/index.js

@@ -19,7 +19,7 @@ const CHILDREN = 32
 //     DATA_HASH
 //     |- locationsCsv
 //     |- blocksCsv
-const DATA_HASH = 'bafybeid3munsqqt36qhoumn3kvgwmft6dsswzgl3wiohsanlyqemczcsvi' // GeoLite2-City-CSV_20201013
+const DATA_HASH = 'bafybeiacaphdy5t63opus5mzeerco7ckpbyqathghnxnf732xdwvrk73qm' // GeoLite2-City-CSV_20220628
 const locationsCsv = 'GeoLite2-City-Locations-en.csv'
 const blocksCsv = 'GeoLite2-City-Blocks-IPv4.csv'
 

src/lookup.js

@@ -8,7 +8,7 @@ const utf8Decoder = new TextDecoder('utf8')
 
 const formatData = require('./format')
 
-const GEOIP_ROOT = new CID('Qmbt1YbZAhMoqo7r1t6Y5EJrYGVRgcaisNAZhLeJY6ehfg') // GeoLite2-City-CSV_20201013
+const GEOIP_ROOT = new CID('QmQe6m4QRoKk4Q7gxGMkUfanmtC4zgw1cS4nAux75iZqG4') // GeoLite2-City-CSV_20220628
 
 /**
  * @param {Object} ipfs

test/lookup.spec.js

@@ -34,11 +34,11 @@ describe('lookup', function () {
     ).to.be.eql({
       country_name: 'USA',
       country_code: 'US',
-      region_code: 'NY',
-      city: 'New York',
-      postal_code: '10004',
-      latitude: 40.7126,
-      longitude: -74.0066,
+      region_code: 'VA',
+      city: 'Ashburn',
+      postal_code: '20103',
+      latitude: 39.0019,
+      longitude: -77.4556,
       planet: 'Earth'
     })
   })
@@ -56,7 +56,7 @@ describe('lookup', function () {
       const result = await geoip.lookupPretty(ipfs, '/ip4/66.6.44.4')
       expect(
         result.formatted
-      ).to.be.eql('New York, NY, USA, Earth')
+      ).to.be.eql('Ashburn, VA, USA, Earth')
     })
   })