Dev v0.2 (#5)

* Added Notepad++ *.session file pattern to .gitignore

* Added some URL shorteners to whitelist

* Add git status

* Prepare direct usage of Gravity DB to import list of domain block lists

* Blocklist file list_of_blocklists.txt is now also imported into pihole Gravity.db via sqlite3

* Simplify to one SQL statement

* Make refresh_all.sh a bit more generic

* Added versioning and parameter parsing; added a domain to whitelist_exact.txt

* Finalize v0.2-RC1: all blocklists will now be imported automatically, depending on mode specified

* Edit of list of Microsoft telemetry hosts, based on latest published information from BSI Bund Germany

* Added watson.* to MSFT telemetry block list; added some documentation about import modes to README.md

* Formatting of README.md

README.md

@@ -23,18 +23,36 @@ Usage:
 
 Install a crontab like this with crontab -e
 
-    40 0 * * * sh /etc/pihole/pihole-bl-msft-telemetry-bsi/refresh_all.sh
-
-
-**TODO**
-
-Currently, there is no way of importing the ad domain block lists programmatically via CLI. So unfortunately, you have to add
-the contents of the file `list_of_blocklists.txt` manually using the web interface. **Sorry!**  
+    40 0 * * * sh /etc/pihole/pihole-bl-msft-telemetry-bsi/refresh_all.sh merge
 
 
 The blacklists and whitelists are loaded with the supplied scripts. Also, if you have entered the blocklists manually, one entry of them
 is a pointer to the file `msft_telemetry_bsi.txt` here in this GitHub repo, so this will be included in the refresh processing when it has changed here.  
 
+The import process can be run in four different modes: ADD, MERGE, DELETE and FULL. Please call the script import_lists.sh with the parameter HELP to learn more:
+
+
+    $ sh import_lists.sh HELP
+    import_lists.sh v0.2 (c) 2022 Peter Schneider, provided under MIT License
+
+    Synopsis: import_lists.sh [MODE]
+
+    This script will import the contents of the supplied file "list_of_blocklists.txt" into your Pi-Hole Gravity DB,
+    where MODE may be one of:
+
+    - HELP:    display this help info
+    - ADD:     only add new lists, don't do anything to existing lists. This is the recommended mode of operation
+                when you have other sources for your block lists, too, other than my repo.
+                It is also the default when no MODE is specified.
+    - MERGE:   add new lists, disable missing ones, re-enable disabled existing lists if they are in the import file.
+                This retains group assignments on existing list entries. This is the recommended mode of operation
+                when my repo is the ONLY source of block lists for your Pi-Hole installation.
+    - DELETE:  add new lists, delete missing ones, re-enable disabled existing lists if they are in the import file.
+                Group assignments on deleted groups are of course lost, and they cannot just be re-enabled again,
+                but will be newly imported when they happen to be in the next import file again.
+    - FULL:    fully replace all existing list entries in Gravity DB with the imported ones. Group assignments are thus lost.
+                That means that before inserting anything from the import file, everything is deleted in the Gravity DB.
+
 
 I will try to check for updates and new lists on a regular basis, but I can't promise anything.
 As of today (Dec 19, 2022) the total number of blocked unique domains from all these lists is 5838819.

blacklist_regex.txt

@@ -0,0 +1 @@
+watson.*.microsoft.com