Offline local-only wifi hotspot (#716)

* refactor oref0-online to use functions

* start_hotspot if no connectivity; stop_hotspot if online via BT

* refactor one more check_ip

* pass MACs to function

* pass MACs to function

* pass MACs to function

* stop/start backwards

* check if hostapd is running before stop/start

* stop_hotspot if running any time we have a public IP

* logic

* grep -q  instead of checking pid

* syntax

* make variables global

* ifup before hostapd; systemctl daemon-reload

* ifdown/ifup wlan0 in stop_hotspot

* stop_hotspot when we get BT

* try ifdown later

* stop udhcpd before starting dnsmasq

* echo

* no more need for sleep

* install prerequisites and configs for local-only hotspot

* only do local-only hotspot if BT configured

* offline webapp

* check if wlan0 is still in AP mode

* pass MACs to start_hotspot

* make sure hostapd and dnsmasq services are running if hotspot is up

* only stop hotspot if we have internet

* get ns-temptargets directly, then merge with any local-temptargets

* print temptarget start time

* untested set-local-temptarget.js

* use current time if unspecified

* oref0-set-local-temptarget

* oref0-append-local-temptarget

* oref0-append-local-temptarget

* regenerate profile after refreshing temptargets

* regenerate profile after merging temptargets

* allow oref0-append-local-temptarget.sh to run oref0-set-local-temptarget

* parse inputs as integers

* remove spurious output

* delete any local-temptarget files last modified more than 24h ago

* make offline_hotspot a preference, false by default

* per #716 it'd be better for xDrip to keep BT connected when offline

* if no wifi and no BT, try cycling wlan0

* make sure all apt-get packages are up to date

* Optimize oref0-ns-loop.sh (#739)

* don't get_ns_bg if fresh; only run every 5m if highload

* fix glucose fresh check

* json is slow: just check file age

* use oref0_glucose_since directly; only get 24h of glucose every hour

* syntax

* use nightscout carb_history directly

* set mtime of ns-glucose.json based on its last reading

* write profile.json.new and validate before mv'ing

* don't trust awk exit status

* cd to ~/myopenaps for oref0-online to read preferences.json

* Create linux-bootup.sh

Instructions for /etc/rc.local startup

* Update oref0-setup.sh

Stop automatic startup of hostapd & dnsmasq

* Update oref0-setup.sh

add linux-bootup.sh to /etc/rc.local

* Update linux-bootup.sh

check to see if .conf actually exists before overwriting with an empty file.

* Update oref0-setup.sh

Edit /etc/hostapd/hostapd.conf for Hostname and wpa

* Update package.json

add linux-bootup.sh

* Delete linux-bootup.sh

bad idea

* Update package.json

removed bad idea

* Update oref0-setup.sh

removed bad idea and writing to rc.local directly

* Update oref0-setup.sh

change hotspot name

* Update oref0-setup.sh

add wpa security for Hotspot with #OpenAPS

* Update oref0-setup.sh

add logic to rc.local

* Update hostapd.conf

add security to hotspot

* Update oref0-setup.sh

Changes for Hotspot setup

* Update oref0-setup.sh

change to setup for local-only hotspot

* Update oref0-setup.sh

changes for the different type of wpa-roam or wpa-conf

* reverse logic so failure = not enabled

* remove HostAPDIP if still configured after stop_hotspot

* Update oref0-online.sh

Changes to improve oref0-online

* Update oref0-setup.sh

changes to cron

* syntax

* if online via BT, cycle wlan0

* check for interfaces.ap; stop_hotspot if not enabled

* reorder to always stop_hotspot if not enabled

* don't release hotspot IP; reorder to stop_hotspot w/o BT MAC

* add /tmp/disable_hotspot support

* grammar capitalization etc.

* logic

* don't wait for 47 years if clock is mismatched

* mv offline folder to www so it doesn't conflict with branch name

* only release the wifi IP/route if it's from DHCP

* whitepace

* print Connected to Bluetooth with IP:

* consistency

* display wifi is not connected if no SSID

* whitepace

* consistency

* whitepace

* consistency

* whitepace

* print unassigned if no

* reorder and print local wifi/BT IPs separately

* syntax

* print name of phone we're connected to with Bluetooth

* only run check_ip once

* print start/finish times

* don't try to dhcp_renew when running hostapd

* print when PUBLIC_IP not found

* fix return code

* don't renew wlan0 IP/route if it's failing

* logging

* syntax

* don't wifi_dhcp_renew if bad_wifi

* f we can't connect via BT, might as well try previously bad wifi networks again

* remove extra -HotSpot from SSID

* disconnect bnep0 even w/o an IP

* temporarily disable hotspot for 1m every hour to allow it to try to connect via wifi again

* download carbhistory to carbhistory.json.new and mv it over if valid

* egrep quietly

* download ns-temptargets to .new and mv it over if valid

* escape $directory for crontab

* let's play spot that syntax error

* only retry SMB w/o waiting if it was <1 m wait

* change default curve to rapid-acting

bin/oref0-append-local-temptarget.sh

@@ -0,0 +1,15 @@
+#!/bin/bash
+
+if [[ ! -z "$2" ]]; then
+    input=$(oref0-set-local-temptarget $@)
+elif [[ ! -z "$1" ]]; then
+    input=$(cat $1)
+else
+    input=$(cat /dev/stdin)
+fi
+#cat "${1:-/dev/stdin}" \
+echo $input \
+    | tee /tmp/temptarget.json \
+    && jq -s '[.[0]] + .[1]' /tmp/temptarget.json settings/local-temptargets.json \
+    | tee settings/local-temptargets.json.new \
+    && mv settings/local-temptargets.json.new settings/local-temptargets.json

bin/oref0-dex-wait-until-expected.sh

@@ -10,7 +10,9 @@ if (( $(bc <<< "$TIME_SINCE >= $OLD") )); then
   echo "CGM Data $TIME_SINCE mins ago is old (>=$OLD), not waiting"
 else
   WAIT_MINS=$(bc <<< "$OLD - $TIME_SINCE")
-  if (( $(bc <<< "$WAIT_MINS >= $MAX_WAIT") )); then
+  if (( $(bc <<< "$WAIT_MINS > 6") )); then
+    echo "Clock mismatch ($WAIT_MINS > 6); not waiting"
+  elif (( $(bc <<< "$WAIT_MINS >= $MAX_WAIT") )); then
     echo "CGM Data $TIME_SINCE mins ago is fresh (< $OLD), $WAIT_MINS mins > max wait ($MAX_WAIT mins) waiting for next attempt"
     exit 1
   else

bin/oref0-ns-loop.sh

@@ -35,7 +35,7 @@ function overtemp {
 
 function highload {
     # check whether system load average is high
-    uptime | awk '$NF > 2'
+    uptime | awk '$NF > 2' | grep load
 }
 
 
@@ -81,17 +81,33 @@ function find_valid_ns_glucose {
 }
 
 function ns_temptargets {
-    openaps report invoke settings/temptargets.json settings/profile.json >/dev/null
+    #openaps report invoke settings/temptargets.json settings/profile.json >/dev/null
+    nightscout ns $NIGHTSCOUT_HOST $API_SECRET temp_targets > settings/ns-temptargets.json.new
+    cat settings/ns-temptargets.json.new | jq .[0].duration | egrep -q [0-9] && mv settings/ns-temptargets.json.new settings/ns-temptargets.json
     # TODO: merge local-temptargets.json with ns-temptargets.json
     #openaps report invoke settings/ns-temptargets.json settings/profile.json
-    echo -n "Refreshed temptargets: "
-    cat settings/temptargets.json | jq -c -C '.[0] | { target: .targetBottom, duration: .duration }'
+    echo -n "Refreshed NS temptargets: "
+    cat settings/ns-temptargets.json | jq -c -C '.[0] | { target: .targetBottom, duration: .duration, start: .created_at }'
+    # delete any local-temptarget files last modified more than 24h ago
+    find settings/local-temptarget* -mmin +1440 -exec rm {} \;
+    echo -n "Merging local temptargets: "
+    cat settings/local-temptargets.json | jq -c -C '.[0] | { target: .targetBottom, duration: .duration, start: .created_at }'
+    jq -s '.[0] + .[1]|unique|sort_by(.created_at)|reverse' settings/ns-temptargets.json settings/local-temptargets.json > settings/temptargets.json
+    echo -n "Temptargets merged: "
+    cat settings/temptargets.json | jq -c -C '.[0] | { target: .targetBottom, duration: .duration, start: .created_at }'
+    oref0-get-profile settings/settings.json settings/bg_targets.json settings/insulin_sensitivities.json settings/basal_profile.json preferences.json settings/carb_ratios.json settings/temptargets.json --model=settings/model.json --autotune settings/autotune.json | jq . > settings/profile.json.new || die "Couldn't refresh profile"
+    if cat settings/profile.json.new | jq . | grep -q basal; then
+        mv settings/profile.json.new settings/profile.json
+    else
+        die "Invalid profile.json.new after refresh"
+    fi
 }
 
 # openaps report invoke monitor/carbhistory.json; oref0-meal monitor/pumphistory-merged.json settings/profile.json monitor/clock-zoned.json monitor/glucose.json settings/basal_profile.json monitor/carbhistory.json > monitor/meal.json.new; grep -q COB monitor/meal.json.new && mv monitor/meal.json.new monitor/meal.json; exit 0
 function ns_meal_carbs {
     #openaps report invoke monitor/carbhistory.json >/dev/null
-    nightscout ns $NIGHTSCOUT_HOST $API_SECRET carb_history > monitor/carbhistory.json
+    nightscout ns $NIGHTSCOUT_HOST $API_SECRET carb_history > monitor/carbhistory.json.new
+    cat monitor/carbhistory.json.new | jq .[0].carbs | egrep -q [0-9] && mv monitor/carbhistory.json.new monitor/carbhistory.json
     oref0-meal monitor/pumphistory-merged.json settings/profile.json monitor/clock-zoned.json monitor/glucose.json settings/basal_profile.json monitor/carbhistory.json > monitor/meal.json.new
     grep -q COB monitor/meal.json.new && mv monitor/meal.json.new monitor/meal.json
     echo -n "Refreshed carbhistory; COB: "

bin/oref0-online.sh

@@ -1,78 +1,251 @@
 #!/bin/bash
-echo; echo Starting oref0-online.
-# if we are connected to wifi but don't have an IP, try to get one
-if iwgetid -r wlan0 | egrep -q "[A-Za-z0-9_]+"; then
-    if ! ip route | grep default | grep -q wlan0; then
-        echo Attempting to renew wlan0 IP
-        sudo dhclient wlan0
+
+main() {
+    MACs=$@
+    HostAPDIP='10.29.29.1'
+    echo; echo Starting oref0-online at $(date).
+    # if we are connected to wifi but don't have an IP, try to get one
+    if iwgetid -r wlan0 | egrep -q "[A-Za-z0-9_]+"; then
+        if ! ip route | grep default | grep -q wlan0; then
+            if grep -q $(iwgetid -r wlan0) /tmp/bad_wifi; then
+                echo Not renewing wlan0 IP due to previous connectivity failure:
+                ls -la /tmp/bad_wifi
+            else
+                echo Attempting to renew wlan0 IP
+                sudo dhclient wlan0
+            fi
+        fi
+    fi
+	if ifconfig | egrep -q "wlan0" >/dev/null; then
+	#if [[ $(ip -4 -o addr show dev wlan0 | awk '{split($4,a,"/");print a[1]}') = $(print_local_ip wlan0) ]]; then
+		print_wifi_name
+        echo -n "At $(date) my local wifi IP is: "
+        print_local_ip wlan0
+	fi
+	if ifconfig | egrep -q "bnep0" >/dev/null; then
+		#if [[ $(ip -4 -o addr show dev bnep0 | awk '{split($4,a,"/");print a[1]}') = $(print_local_ip bnep0) ]]; then
+			print_bluetooth_name
+		#fi
+        echo -n "At $(date) my local Bluetooth IP is: "
+        print_local_ip bnep0
+	else
+		echo "At $(date) my Bluetooth PAN is not connected"
+	fi
+	echo -n "At $(date) my public IP is: "
+    if check_ip; then
+        stop_hotspot
+        if has_ip wlan0 && has_ip bnep0; then
+            # if online via BT w/o a DHCP IP, cycle wifi
+            if print_local_ip wlan0 | grep $HostAPDIP || ! has_ip wlan0; then
+                ifdown wlan0; ifup wlan0
+            fi
+        fi
+        # if online via wifi, disconnect BT
+        if has_ip wlan0 && ifconfig | egrep -q "bnep0" >/dev/null; then
+            bt_disconnect $MACs
+            #wifi_dhcp_renew
+        fi
+    else
+        echo
+        print_wifi_name
+        if ! has_ip wlan0; then
+            wifi_dhcp_renew
+        fi
+        if ! check_ip >/dev/null; then
+            bt_connect $MACs
+        fi
+        #print_wifi_name
+        if check_ip >/dev/null; then
+            # if we're online after activating bluetooth, shut down any local-access hotspot we're running
+            stop_hotspot
+			if ! print_local_ip wlan0 | egrep -q "[A-Za-z0-9_]+" >/dev/null; then
+				wifi_dhcp_renew
+			fi
+        else
+            # if we can't connect via BT, might as well try previously bad wifi networks again
+            rm /tmp/bad_wifi
+            # if we can't get online via wifi or bluetooth, start our own local-access hotspot
+            start_hotspot $@
+            # don't disconnect bluetooth when starting local-only hotspot
+        fi
+    fi
+    echo Finished oref0-online at $(date).
+}
+
+function print_bluetooth_name {
+    echo -n "At $(date) my Bluetooth is connected to "
+    grep Name /var/lib/bluetooth/*/*/info | awk -F = '{print $2}'
+    #echo ${MACs}
+}
+
+function print_wifi_name {
+    SSID=$(iwgetid -r wlan0 | tr -d '\n')
+    if [[ ! -z $SSID ]]; then
+        echo "At $(date) my wifi network name is $SSID"
+    else
+        echo "At $(date) my wifi is not connected"
     fi
-fi
-echo -n "At $(date) my local IP is: "
-ip -4 -o addr show dev wlan0 | awk '{split($4,a,"/");print a[1]}'
-ip -4 -o addr show dev bnep0 | awk '{split($4,a,"/");print a[1]}'
-echo
-echo -n "At $(date), my wifi network name is "
-iwgetid -r wlan0 | tr -d '\n'
-echo -n ", and my public IP is: "
-if curl --compressed -4 -s -m 15 checkip.amazonaws.com | awk -F , '{print $NF}' | egrep "^[12]*[0-9]*[0-9]\.[12]*[0-9]*[0-9]\.[12]*[0-9]*[0-9]\.[12]*[0-9]*[0-9]$"; then
-    # if we are back on wifi (and have connectivity to checkip.amazonaws.com), shut down bluetooth
-    if ( ifconfig | grep -A1 wlan0 | grep -q "inet addr" ) && ( ifconfig | grep -A1 bnep0 | grep -q "inet addr" ); then
-        echo "Back online via wifi; disconnecting BT $MAC"
-        ifdown bnep0
-        # loop over as many MACs as are provided as arguments
-        for MAC; do
+}
+
+function print_local_ip {
+    LOCAL_IP=$(ip -4 -o addr show dev $1 | awk '{split($4,a,"/");print a[1]}')
+    if [[ -z $LOCAL_IP ]]; then
+        echo unassigned
+    else
+        echo $LOCAL_IP
+    fi
+}
+
+function check_ip {
+    PUBLIC_IP=$(curl --compressed -4 -s -m 15 checkip.amazonaws.com | awk -F , '{print $NF}' | egrep "^[12]*[0-9]*[0-9]\.[12]*[0-9]*[0-9]\.[12]*[0-9]*[0-9]\.[12]*[0-9]*[0-9]$")
+    if [[ -z $PUBLIC_IP ]]; then
+        echo not found
+        return 1
+    else
+        echo $PUBLIC_IP
+    fi
+}
+
+function has_ip {
+    ifconfig | grep -A1 $1 | grep -q "inet addr"
+}
+
+function bt_connect {
+    # loop over as many MACs as are provided as arguments
+    for MAC; do
+        #echo -n "At $(date) my public IP is: "
+        if ! check_ip >/dev/null; then
+            echo; echo "No Internet access detected, attempting to connect BT to $MAC"
+            oref0-bluetoothup
             sudo bt-pan client $MAC -d
-        done
-        echo "and getting new wlan0 IP"
+            sudo bt-pan client $MAC
+            #echo  "Attempt to get bnep0 IP :"
+            sudo dhclient bnep0
+            if ifconfig | egrep -q "bnep0" >/dev/null; then
+                echo -n "Connected to Bluetooth with IP: "
+                print_local_ip bnep0
+            fi
+            # if we couldn't reach the Internet over wifi, but (now) have a bnep0 IP, release the wifi IP/route
+            if has_ip wlan0 && has_ip bnep0 && ! grep -q $HostAPDIP /etc/network/interfaces; then
+                # release the wifi IP/route but *don't* renew it, in case it's not working
+                sudo dhclient wlan0 -r
+                iwgetid -r wlan0 >> /tmp/bad_wifi
+            fi
+            #echo
+        fi
+    done
+}
+
+function bt_disconnect {
+    echo "Disconnecting BT $MAC"
+    ifdown bnep0
+    # loop over as many MACs as are provided as arguments
+    for MAC; do
+        sudo bt-pan client $MAC -d
+    done
+}
+
+function wifi_dhcp_renew {
+    if grep -q $(iwgetid -r wlan0) /tmp/bad_wifi; then
+        echo Not renewing wlan0 IP due to previous connectivity failure:
+        ls -la /tmp/bad_wifi
+    else
+        echo; echo "Getting new wlan0 IP"
         ps aux | grep -v grep | grep -q "dhclient wlan0" && sudo killall dhclient
         sudo dhclient wlan0 -r
         sudo dhclient wlan0
     fi
-else
-    echo
-    echo -n "At $(date), my wifi network name is "
-    iwgetid -r wlan0 | tr -d '\n'
-    echo -n ", and my public IP is: "
-    # loop over as many MACs as are provided as arguments
-    if ! curl --compressed -4 -s -m 15 checkip.amazonaws.com | awk -F , '{print $NF}' | egrep "^[12]*[0-9]*[0-9]\.[12]*[0-9]*[0-9]\.[12]*[0-9]*[0-9]\.[12]*[0-9]*[0-9]$"; then
-        echo
-        for MAC; do
-            echo -n "At $(date) my public IP is: "
-            if ! curl --compressed -4 -s -m 15 checkip.amazonaws.com | awk -F , '{print $NF}' | egrep "^[12]*[0-9]*[0-9]\.[12]*[0-9]*[0-9]\.[12]*[0-9]*[0-9]\.[12]*[0-9]*[0-9]$"; then
-                echo; echo -n "Error, connecting BT to $MAC"
-                oref0-bluetoothup
-                sudo bt-pan client $MAC -d
-                sudo bt-pan client $MAC
-                echo -n ", getting bnep0 IP"
-                sudo dhclient bnep0
-                # if we couldn't reach the Internet over wifi, but we have a bnep0 IP, release the wifi IP/route
-                if ( ifconfig | grep -A1 wlan0 | grep -q "inet addr" ) && ( ifconfig | grep -A1 bnep0 | grep -q "inet addr" ); then
-                    echo -n " and releasing wifi IP"
-                    sudo dhclient wlan0 -r
-                    echo
-                    echo Sleeping for 2 minutes before trying wifi again
-                    sleep 120
-                fi
-                echo
-            fi
-        done
-        echo
+}
+
+function stop_hotspot {
+    if grep -q $HostAPDIP /etc/network/interfaces || iwconfig wlan0 | grep Mode:Master; then
+        echo "Shutting down local-only hotspot"
+        echo "Attempting to stop hostapd"
+        /etc/init.d/hostapd stop
+        echo "Attempting to stop dnsmasq"
+        /etc/init.d/dnsmasq stop
+        echo "Activating client config"
+        ifdown wlan0
+        cp /etc/network/interfaces.client /etc/network/interfaces
+        ifup wlan0
+        echo "Renewing IP Address for wlan0"
+        dhclient_restart
+    else
+        echo -n "At $(date) my local hotspot is not running"
+        if ! cat preferences.json | jq -e .offline_hotspot >/dev/null; then
+            echo " (and not enabled in preferences.json)"
+        else
+            echo
+        fi
     fi
-    echo -n "At $(date), my wifi network name is "
-    iwgetid -r wlan0 | tr -d '\n'
-    echo -n ", and my public IP is: "
-    # if we still can't get online, try cycling networking as a last resort
-    if ! curl --compressed -4 -s -m 15 checkip.amazonaws.com | awk -F , '{print $NF}' | egrep "^[12]*[0-9]*[0-9]\.[12]*[0-9]*[0-9]\.[12]*[0-9]*[0-9]\.[12]*[0-9]*[0-9]$"; then
-        echo; echo "Error, cycling networking "
-        sudo /etc/init.d/networking stop
+}
+
+function stop_cycle {
+    stop_hotspot
+    echo "Cycling wlan0"
+    ifdown wlan0; ifup wlan0
+}
+
+
+function start_hotspot {
+    echo
+    if ls /tmp/disable_hotspot; then
+        stop_cycle
+    elif ! ls preferences.json 2>/dev/null >/dev/null \
+        || ! cat preferences.json | jq -e .offline_hotspot >/dev/null; then
+        echo "Offline hotspot not enabled in preferences.json"
+        stop_cycle
+    elif [[ -z $1 ]]; then
+        echo "No BT MAC provided: not activating local-only hotspot"
+        echo "Cycling wlan0"
+        ifdown wlan0; ifup wlan0
+    elif grep -q $HostAPDIP /etc/network/interfaces \
+        && ifconfig wlan0 | grep -q $HostAPDIP; then
+        echo "Local hotspot is running."
+        service hostapd status > /dev/null || service hostapd restart
+        service dnsmasq status > /dev/null || service dnsmasq restart
+    elif ! ls /etc/network/interfaces.ap 2>/dev/null >/dev/null; then
+        echo "Local-only hotspot not configured"
+        stop_cycle
+    else
+        echo "Unable to connect via wifi or Bluetooth; activating local-only hotspot"
+        echo "Killing wpa_supplicant"
+        #killall wpa_supplicant
+        wpa_cli terminate
+        echo "Shutting down wlan0"
+        ifdown wlan0
+        echo "Activating AP config"
+        cp /etc/network/interfaces.ap /etc/network/interfaces
+        ifup wlan0
+        echo "Attempting to start hostapd"
+        /etc/init.d/hostapd start
+        echo "Attempting to start dnsmasq"
+        service udhcpd stop
+        /etc/init.d/dnsmasq start
+        systemctl daemon-reload
+        #echo "Stopping networking"
+        #/etc/init.d/networking stop
+        #echo "Starting networking"
+        #/etc/init.d/networking start
         sleep 5
-        sudo /etc/init.d/networking start
-        echo "and getting new wlan0 IP"
-        ps aux | grep -v grep | grep -q "dhclient wlan0" && sudo killall dhclient
-        sudo dhclient wlan0 -r
-        sudo dhclient wlan0
+        echo "Setting IP Address for wlan0"
+        /sbin/ifconfig wlan0 $HostAPDIP netmask 255.255.255.0 up
     fi
-fi
-# restart avahi every minute to keep mDNS working properly
-#/etc/init.d/avahi-daemon restart
-echo Finished oref0-online.
+}
+
+function dhclient_restart {
+    ps aux | grep -v grep | grep -q "dhclient wlan0" && sudo killall dhclient
+    sudo dhclient wlan0 -r
+    sudo dhclient wlan0
+}
+
+function restart_networking {
+    echo; echo "Error, cycling networking "
+    sudo /etc/init.d/networking stop
+    sleep 5
+    sudo /etc/init.d/networking start
+    echo "and getting new wlan0 IP"
+    dhclient_restart
+}
+
+main "$@"