Merge pull request #501 from dOpensource/v0.72

V0.72

CHANGELOG.md

@@ -5,6 +5,11 @@
 
 [//]: # (END_SECTION HEADER)
 [//]: # (START_SECTION COMMITS
+0ad6e3d9fad4a59b025b47c8fec57523e8fa0a25
+28309c1a2d37d8ec3b3e3bfdbce1afda1c33ca28
+e0bbbfa860e57f9549736149142c9df8be72433b
+a659fedb44ef2a844282d3de0d5503c567837948
+baa6415f1d630b695cd28de742ed609eb65f1399
 bc560f0fd419cbe271e22c0801e64e92a9469397
 0209c81fa9302517ac674725cefc1f4799d70d28
 c850b78988eed29591fcacbc01d4c26f2f49c22a
@@ -2172,6 +2177,115 @@ a72121b9551921aa3dced32d943c6034ba318f82
 ce6c5aac0db5476dc496c34388e4f9ce2c4b86e5
 b46b1e64f06f448bde78b98e3ae8228ce5f96067
 END_SECTION COMMITS)
+[//]: # (START_SECTION 0ad6e3d9fad4a59b025b47c8fec57523e8fa0a25)
+### Update Documentation for v0.72
+
+> Commit: [0ad6e3d9fad4a59b025b47c8fec57523e8fa0a25](https://github.com/dOpensource/dsiprouter/commit/0ad6e3d9fad4a59b025b47c8fec57523e8fa0a25)  
+> Date: Mon, 3 Apr 2023 15:52:09 -0400  
+> Author: Tyler Moore (tmoore@goflyball.com)  
+> Committer: Tyler Moore (tmoore@goflyball.com)  
+> Signed: Tyler Moore (devopsec) <tmoore@goflyball.com>  
+
+
+- update boostrap command
+- update changelog
+
+
+---
+
+[//]: # (END_SECTION 0ad6e3d9fad4a59b025b47c8fec57523e8fa0a25)
+[//]: # (START_SECTION 28309c1a2d37d8ec3b3e3bfdbce1afda1c33ca28)
+### v0.72 Release Bug Fix
+
+> Commit: [28309c1a2d37d8ec3b3e3bfdbce1afda1c33ca28](https://github.com/dOpensource/dsiprouter/commit/28309c1a2d37d8ec3b3e3bfdbce1afda1c33ca28)  
+> Date: Mon, 3 Apr 2023 14:09:23 -0400  
+> Author: Tyler Moore (tmoore@goflyball.com)  
+> Committer: Tyler Moore (tmoore@goflyball.com)  
+> Signed: Tyler Moore (devopsec) <tmoore@goflyball.com>  
+
+
+- fix bad hash algorithm implementation in the CLI functions
+- fix schema migration issues on `dsip_settings` table
+- fix edge cases where bootstrapping failed
+- # Please enter the commit message for your changes. Lines starting
+- # with '#' will be ignored, and an empty message aborts the commit.
+- #
+- # On branch v0.72
+- # Your branch is up to date with 'origin/v0.72'.
+- #
+- # Changes to be committed:
+- #	modified:   dsiprouter/dsip_lib.sh
+- #	modified:   resources/upgrade/v0.72/scripts/bootstrap.sh
+- #	modified:   resources/upgrade/v0.72/scripts/migrate.sh
+- #
+
+
+---
+
+[//]: # (END_SECTION 28309c1a2d37d8ec3b3e3bfdbce1afda1c33ca28)
+[//]: # (START_SECTION e0bbbfa860e57f9549736149142c9df8be72433b)
+### Bootstrap Fixes
+
+> Commit: [e0bbbfa860e57f9549736149142c9df8be72433b](https://github.com/dOpensource/dsiprouter/commit/e0bbbfa860e57f9549736149142c9df8be72433b)  
+> Date: Mon, 3 Apr 2023 10:32:08 -0400  
+> Author: Tyler Moore (tmoore@goflyball.com)  
+> Committer: Tyler Moore (tmoore@goflyball.com)  
+> Signed: Tyler Moore (devopsec) <tmoore@goflyball.com>  
+
+
+- change bootstrap to use tag instead of branch
+- fix python versioning
+- fix boostrap variable scoping
+- # Please enter the commit message for your changes. Lines starting
+- # with '#' will be ignored, and an empty message aborts the commit.
+- #
+- # Date:      Mon Apr 3 10:32:08 2023 -0400
+- #
+- # On branch v0.72
+- # Your branch is up to date with 'origin/v0.72'.
+- #
+- # Changes to be committed:
+- #	modified:   dsiprouter.sh
+- #	modified:   resources/upgrade/v0.72/scripts/bootstrap.sh
+- #	modified:   resources/upgrade/v0.72/scripts/migrate.sh
+- #
+
+
+---
+
+[//]: # (END_SECTION e0bbbfa860e57f9549736149142c9df8be72433b)
+[//]: # (START_SECTION a659fedb44ef2a844282d3de0d5503c567837948)
+### Merge Changes From Master (#500)
+
+> Commit: [a659fedb44ef2a844282d3de0d5503c567837948](https://github.com/dOpensource/dsiprouter/commit/a659fedb44ef2a844282d3de0d5503c567837948)  
+> Date: Mon, 3 Apr 2023 10:10:11 -0400  
+> Author: Tyler Moore (tmoore@goflyball.com)  
+> Committer: GitHub (noreply@github.com)  
+> Signed:   
+
+
+- Co-authored-by: Mack Hendricks <mack@dopensource.com>
+
+
+---
+
+[//]: # (END_SECTION a659fedb44ef2a844282d3de0d5503c567837948)
+[//]: # (START_SECTION baa6415f1d630b695cd28de742ed609eb65f1399)
+### Update The Changelog
+
+> Commit: [baa6415f1d630b695cd28de742ed609eb65f1399](https://github.com/dOpensource/dsiprouter/commit/baa6415f1d630b695cd28de742ed609eb65f1399)  
+> Date: Mon, 3 Apr 2023 09:12:46 -0400  
+> Author: Tyler Moore (tmoore@goflyball.com)  
+> Committer: Tyler Moore (tmoore@goflyball.com)  
+> Signed: Tyler Moore (devopsec) <tmoore@goflyball.com>  
+
+
+- update `CHANGELOG.md`
+
+
+---
+
+[//]: # (END_SECTION baa6415f1d630b695cd28de742ed609eb65f1399)
 [//]: # (START_SECTION bc560f0fd419cbe271e22c0801e64e92a9469397)
 ### Fixed permissions again
 
@@ -3364,11 +3478,11 @@ END_SECTION COMMITS)
 > Signed:   
 
 
-- * Update cdrs.sql
+- * Update cdrs.sql
 
-- Adjusted field types to address bug [#464](https://github.com/dOpensource/dsiprouter/issues/464)
+- Adjusted field types to address bug [#464](https://github.com/dOpensource/dsiprouter/issues/464)
 
-- ---------
+- ---------
 
 - Co-authored-by: Tyler Moore <tmoore@goflyball.com>
 
@@ -4888,11 +5002,11 @@ END_SECTION COMMITS)
 > Signed:   
 
 
-- RTPEngine fails to build on Debian 10. 
-- Added cmake dependency and libavcodec-extra. 
-- libavcodec-extra - is recommended by RTPEngine github on debian installs to support wider codecs
+- RTPEngine fails to build on Debian 10. 
+- Added cmake dependency and libavcodec-extra. 
+- libavcodec-extra - is recommended by RTPEngine github on debian installs to support wider codecs
 
-- You can not build using 
+- You can not build using 
 - `dsiprouter.sh install -rtp` or `dsiprouter.sh install -all -servernat` without getting the RTPEngine failed to install error.
 
 
@@ -8382,7 +8496,7 @@ END_SECTION COMMITS)
 > Signed:   
 
 
-- Disable removing "#" comments tags and only remove ";" comment tags instead, otherwise, installing RTPENGINE under CentOS breaks rsyslog.service
+- Disable removing "#" comments tags and only remove ";" comment tags instead, otherwise, installing RTPENGINE under CentOS breaks rsyslog.service
 
 - By the way, dsiprouter logging is still broken even after this fix, I suspect there is another issue, possible with sysloginit.py  but have not been able to figure out yet.
 
@@ -10467,7 +10581,7 @@ END_SECTION COMMITS)
 > Signed:   
 
 
-- Typo Corrections!
+- Typo Corrections!
 - Feel free to just fix this yourself instead of accepting this PR, it might honestly be faster :)
 
 

docs/source/user/upgrade.rst

@@ -17,7 +17,7 @@ You can upgrade from 0.70 by doing the following
 
 .. code-block:: bash
 
-   curl -s https://raw.githubusercontent.com/dOpensource/dsiprouter/v0.72/resources/upgrade/v0.72/scripts/bootstrap.sh | bash -s upgrade -rel v0.72
+   curl -s https://raw.githubusercontent.com/dOpensource/dsiprouter/v0.72/resources/upgrade/v0.72/scripts/bootstrap.sh | bash
 
 3. Login to the dSIPRouter UI to validate that the upgrade was successful.  
 

dsiprouter.sh

@@ -2837,11 +2837,11 @@ function removeInitService() {
 
 function upgrade() {
     local BS_SCRIPT_URL="https://github.com/dOpensource/dsiprouter/tree/${UPGRADE_RELEASE}/resources/upgrade/${UPGRADE_RELEASE}/scripts/bootstrap.sh"
-    local BOOTSTRAPPING_UPGRADE=${BOOTSTRAPPING_UPGRADE:-0}
+    export BOOTSTRAPPING_UPGRADE=${BOOTSTRAPPING_UPGRADE:-0}
 
     # check if the new function definitions need bootstrapped prior to upgrade
     if (( $BOOTSTRAPPING_UPGRADE == 0 )) && curl -sf -I "$BS_SCRIPT_URL" -o /dev/null; then
-        curl -s "$BS_SCRIPT_URL" | bash -s upgrade -rel ${UPGRADE_RELEASE}
+        curl -s "$BS_SCRIPT_URL" | bash
         return $?
     fi
 

dsiprouter/dsip_lib.sh

@@ -28,10 +28,10 @@ DSIP_PROJECT_DIR=${DSIP_PROJECT_DIR:-$(git rev-parse --show-toplevel 2>/dev/null
 export DSIP_PROJECT_DIR=${DSIP_PROJECT_DIR:-$(dirname $(dirname $(readlink -f "$BASH_SOURCE")))}
 
 # reuse credential settings from python files (exported for later usage)
-SALT_LEN=${SALT_LEN:-$(grep -m 1 -oP 'SALT_LEN[ \t]+=[ \t]+\K[0-9]+' ${DSIP_PROJECT_DIR}/gui/util/security.py)}
-DK_LEN_DEFAULT=${DK_LEN_DEFAULT:-$(grep -m 1 -oP 'DK_LEN_DEFAULT[ \t]+=[ \t]+\K[0-9]+' ${DSIP_PROJECT_DIR}/gui/util/security.py)}
-CREDS_MAX_LEN=${CREDS_MAX_LEN:-$(grep -m 1 -oP 'CREDS_MAX_LEN[ \t]+=[ \t]+\K[0-9]+' ${DSIP_PROJECT_DIR}/gui/util/security.py)}
-HASH_ITERATIONS=${HASH_ITERATIONS:-$(grep -m 1 -oP 'HASH_ITERATIONS[ \t]+=[ \t]+\K[0-9]+' ${DSIP_PROJECT_DIR}/gui/util/security.py)}
+export SALT_LEN=${SALT_LEN:-$(grep -m 1 -oP 'SALT_LEN[ \t]+=[ \t]+\K[0-9]+' ${DSIP_PROJECT_DIR}/gui/util/security.py)}
+export DK_LEN_DEFAULT=${DK_LEN_DEFAULT:-$(grep -m 1 -oP 'DK_LEN_DEFAULT[ \t]+=[ \t]+\K[0-9]+' ${DSIP_PROJECT_DIR}/gui/util/security.py)}
+export CREDS_MAX_LEN=${CREDS_MAX_LEN:-$(grep -m 1 -oP 'CREDS_MAX_LEN[ \t]+=[ \t]+\K[0-9]+' ${DSIP_PROJECT_DIR}/gui/util/security.py)}
+export HASH_ITERATIONS=${HASH_ITERATIONS:-$(grep -m 1 -oP 'HASH_ITERATIONS[ \t]+=[ \t]+\K[0-9]+' ${DSIP_PROJECT_DIR}/gui/util/security.py)}
 export HASHED_CREDS_ENCODED_MAX_LEN=${HASHED_CREDS_ENCODED_MAX_LEN:-$(grep -m 1 -oP 'HASHED_CREDS_ENCODED_MAX_LEN[ \t]+=[ \t]+\K[0-9]+' ${DSIP_PROJECT_DIR}/gui/util/security.py)}
 export AESCTR_CREDS_ENCODED_MAX_LEN=${AESCTR_CREDS_ENCODED_MAX_LEN:-$(grep -m 1 -oP 'AESCTR_CREDS_ENCODED_MAX_LEN[ \t]+=[ \t]+\K[0-9]+' ${DSIP_PROJECT_DIR}/gui/util/security.py)}
 
@@ -1116,16 +1116,18 @@ function parseDBConnURI() {
 }
 export -f parseDBConnURI
 
-# $1 == number of characters to get
-# output: string of random printable characters
+# usage:    urandomChars [options] [args]
+# options:  -f <filter> == characters to allow
+# args:     $1 == number of characters to get
+# output:   string of random printable characters
 function urandomChars() {
 	local LEN=32 FILTER="a-zA-Z0-9"
 
     while (( $# > 0 )); do
     	# last arg is length
-        if (( $# == 1 )) && [[ -z "$CREDS" ]]; then
-			LEN="$1"
-			shift
+        if (( $# == 1 )); then
+            LEN="$1"
+            shift
             break
         fi
 
@@ -1230,7 +1232,13 @@ function hashCreds() {
 
 	# python native version
 	# no external dependencies other than vanilla python3
-	${PYTHON} -c "import hashlib,binascii; print(binascii.hexlify(hashlib.pbkdf2_hmac('sha512', '$CREDS'.encode('utf-8'), '$SALT'.encode('utf-8'), iterations=$HASH_ITERATIONS, dklen=$DK_LEN)).decode('utf-8'));"
+	${PYTHON} <<EOPYTHON
+import hashlib,binascii
+creds='$CREDS'.encode('utf-8')
+salt='$SALT'.encode('utf-8')
+hash=hashlib.pbkdf2_hmac('sha512', creds, salt, iterations=$HASH_ITERATIONS, dklen=$DK_LEN) + salt
+print(binascii.hexlify(hash).decode('utf-8'))
+EOPYTHON
 	# bash native version
 	# currently too slow for production usage
 	#${DSIP_PROJECT_DIR}/dsiprouter/pbkdf2.sh 'sha512' "$CREDS" "$SALT" "$HASH_ITERATIONS" 4

resources/git/hooks/post-commit

@@ -153,7 +153,7 @@ createChangelog() {
     # otherwise we create new changelog from scratch
     NEW_COMMIT_HASHES=$(git --no-pager log --no-merges --format='%H')
     if [[ -e "$CHANGELOG_FILE" ]]; then
-        OLD_COMMIT_HASHES=$(getMDCommentSection 'COMMITS')
+        OLD_COMMIT_HASHES=$(cat "$CHANGELOG_FILE" | getMDCommentSection 'COMMITS')
         if [[ -n "$OLD_COMMIT_HASHES" ]]; then
             NEW_CHANGELOG=0
 

resources/upgrade/v0.72/scripts/bootstrap.sh

@@ -1,15 +1,10 @@
 #!/usr/bin/env bash
 
 export BOOTSTRAPPING_UPGRADE=1
-export SALT_LEN='16'
-export DK_LEN_DEFAULT='48'
-export CREDS_MAX_LEN='64'
-export HASH_ITERATIONS='10000'
-export HASHED_CREDS_ENCODED_MAX_LEN='128'
-export AESCTR_CREDS_ENCODED_MAX_LEN='160'
+export DSIP_PROJECT_DIR='/tmp/dsiprouter'
+TAG_NAME='v0.72-rel'
+REPO_URL='https://github.com/dOpensource/dsiprouter.git'
 rm -f /etc/dsiprouter/.requirementsinstalled
-rm -rf /tmp/dsiprouter 2>/dev/null
-git clone --depth 1 -b v0.72 https://github.com/dOpensource/dsiprouter.git /tmp/dsiprouter
-ln -sf /tmp/dsiprouter/resources/upgrade /opt/dsiprouter/resources/upgrade
-. /tmp/dsiprouter/dsiprouter/dsip_lib.sh
-. /tmp/dsiprouter/dsiprouter.sh upgrade -rel v0.72
+rm -rf "$DSIP_PROJECT_DIR" 2>/dev/null
+git clone --depth 1 -b "$TAG_NAME" "$REPO_URL" "$DSIP_PROJECT_DIR"
+${DSIP_PROJECT_DIR}/dsiprouter.sh upgrade -rel v0.72