Feature Request: List all known devices with name, mac, ip, landevice-ID and list available filter profiles for use with SETPROFILE

[PureIncompetence]

First of all thank you very much for maintaining this very helpful piece of software. I'm using it regularly.

I was really exited that the latest version allows to move devices to certain internet profiles. When I tried to use it I found its documentation rather lacking. With dev devname profile as parameters I wasnt sure what values exactly to use. Perhaps a hint that landevice01235-type dev IDs and filtprof01235-type profile IDs must be specified would be helpful for users not as throughoutly informed.

After finding out what I was looking for I struggled to find the needed IDs until I remembered about the web browser developer console. Using that it was possible to extract the needed IDs one by one but found that way rather tedious. Hence I kept looking for the specific URLs to the pages containing the relevant information and automated the extraction of said information:

#!/bin/bash
# the login procedure is adapted from https://github.com/Tscherno/Fritzbox.sh

FritzBoxURL="http://fritz.box"
Username=""
Passwd=""
FRITZLOGIN="/login_sid.lua"

login(){
	htmlLoginPage=$(curl -s "$FritzBoxURL$FRITZLOGIN")
	SessionInfoChallenge=$(sed -e 's#<Challenge>#§#' -e 's#</Challenge>#§#' <<< "$htmlLoginPage" | awk -F'§' '{print $2}')
	SessionInfoSID=$(sed -e 's#<SID>#§#' -e 's#</SID>#§#' <<< "$htmlLoginPage" | awk -F'§' '{print $2}')
	if [ "$SessionInfoSID" = "0000000000000000" ]; then
		CPSTR="$SessionInfoChallenge-$Passwd"  # Combine Challenge and Passwd
		MD5=$(echo -n "$CPSTR" | sed -e 's,.,&\n,g' | tr '\n' '\0' | md5sum | grep -o "[0-9a-z]\{32\}")  # here the MD5 checksum is calculated
		RESPONSE="$SessionInfoChallenge-$MD5"
		GETDATA="?username=$Username&response=$RESPONSE"
		SID=$(curl -s "$FritzBoxURL$FRITZLOGIN$GETDATA" | sed -e 's#<SID>#§#' -e 's#</SID>#§#' | awk -F'§' '{print $2}')
	else
		SID=$SessionInfoSID
	fi
}

# gets a json object containing an array of all known (offline and online) devices
curl -s "$FritzBoxURL/data.lua" --data-raw "xhr=1&sid=${SID}&page=netDev&xhrId=cleanup&useajax=1&no_sidrenew="
# formats said json object to a list with IPv4, mac, landevice-ID and DeviceName on one line per device sorted by IPv4
curl -s "$FritzBoxURL/data.lua" --data-raw "xhr=1&sid=${SID}&page=netDev&xhrId=cleanup&useajax=1&no_sidrenew=" | jq -r '.data.active[],.data.passive[] | "\(.ipv4.ip) \(.mac) \(.UID) \(.name)"' | sort -h

# gets the html from the filters-page of the Fritzbox-WebUI and extracts the filters with filtprof-ID and name one filter per Line
curl -s "$FritzBoxURL/data.lua" --data-raw "xhr=1^&sid=${SID}^&page=kidPro" | grep -Eo 'value="(filtprof[0-9]+)" class="[^"]+"[ a-z]*data-name="([0-9a-zA-Z_-]+)"' | sed -Ee 's/value="([^"]+)".*data-name="([^"]+)"/\1 \2/g'

Example output of the devices list:

192.168.178.2 00:1A:2B:3C:4D:5E landevice2185 Device1
192.168.178.3 00:1A:2B:3C:4D:5F landevice32561 Device2
192.168.178.5 00:1A:2B:3C:4D:3E landevice2728 device-15

Example output of the filters list:

filtprof2 Profil1
filtprof15 Profil2

Maybe this would be helpful as a basis to implement such a feature.
(Another but probably much more difficult to implement idea would be to parse these lists and allow the user to refer to device and filter by name only, in case there are only unique names. Also the device could be referred to by its mac address as that should be unique, too.)

[jhubig]

Could you please check the latest version? Sufficient for you?

[PureIncompetence]

Thank you for implementing my suggestion.
What I've read in your commits looks very promising. Sadly it does not entirely work with my Fritzbox. It does not detect my Profiles correctly, to be precise.
Running fritzBoxShell.sh LISTPROFILES returns the following, even though I do have multiple custom profiles defined:

Retrieving available filter profiles from Fritz!Box...

Available Filter Profiles:
==========================
No custom profiles found. Default profiles available:

ID    Profile Name              Description                                       
--------------------------------------------------------------------------------
0/1   Standard                  Unrestricted internet access                      
2     Blocked                   No internet access                                

Configure custom profiles in Fritz!Box web interface under:
Internet > Filters > Parental Controls

Usage with SETPROFILE:
  ./fritzBoxShell.sh SETPROFILE <device_name> <device_id> <profile_id>
  Example: ./fritzBoxShell.sh SETPROFILE "Johns Phone" "12345" "2"

To get device IDs, use: ./fritzBoxShell.sh LISTDEVICES

./fritzBoxShell.sh LISTDEVICES does output N/A for every one of my devices in the columns LAN-Dev-ID and Profile. Everything else is listed correctly. (It would be a nice2have-feature to automatically set the Device Name column's width to the width of the longest device name.)

fritzBoxShell.sh DEVICEPROFILES lists only a very small subset of my known devices which does however include devices configured with each one of my profiles.

My profiles are configured through and visible at (in the Fritzbox-webui) Internet -> Filter -> Access Profiles, in case that matters.
One of my profile names contains a german umlaut. May that be the cause of this behaviour?

Another Idea for a feature I came up with (which I should probably describe in a seperate feature request, though), would be the option to delete dhcp reservations through FritzBoxShell.sh, and possibly also to rename devices and reassign ip addresses. Since the Fritzbox-webui is getting slower/buggier by the minute (at least in my experience), it would be a great relieve being able to avoid it for such routine tasks. Would that be feasible?

[jhubig]

Hello,

could you please try with the latest commit (https://github.com/jhubig/FritzBoxShell/archive/refs/heads/master.zip). Seems to work but we need some further testing. Unfortunately it is also quite slow to display all the devices. Depending on how many you actually have.

Try first the fritzBoxShell.sh LISTPROFILES.

Then check fritzBoxShell.sh DEVICEPROFILES. This should give you all active devices with their profiles. Please check and let me know. Thanks.

For the other topic, please raise another issue to trace it. Thanks

[PureIncompetence]

Hi,
sorry for the late reply.

I tested the latest master as of today.

./fritzBoxShell.sh LISTPROFILES now works as expected in so far as it correctly lists the configured profiles. What I wondered is where you get the descriptions (of the default profiles) from. Did you write them? I'm asking because in my specific case they arent 100% accurate, as I denied internet access to all devices in my default profile (1) so only devices I manually move to unrestricted (3) are allowed to access the internet. I assume the descriptions to be accurate for the default settings, though.

./fritzBoxShell.sh DEVICEPROFILES now lists the devices with their profiles, though it is awfully slow in doing so. I do have a lot of devices (>100), but it takes like 10-30s per device to show up. It also seems to kind of time out(?) [Edit: No, it doesn't time out. The 38 shown devices are the ones which were online then.] after listing roughly 35-40 currently online devices because after listing them it closes its output with the following 2 lines:

No active devices found with profile information.
This might indicate that parental controls are not configured.

Edit: I measured the time needed for it to run once: 5m 59s to list 38 devices with their profiles -> about 10s per device.

./fritzBoxShell.sh LISTDEVICES (also being very slow despite using parallel processing) now shows Lan-Dev-IDs and Profiles for for some (but not for all) devices. Interestingly some (6 out of >100) show a LAN-Dev-ID, but no Profile.
Edit: Just for reference, I also measured the execution time for this command: 4m 42s for listing all devices.

I also tried to move a device based on the output of ./fritzBoxShell.sh LISTPROFILES and ./fritzBoxShell.sh DEVICEPROFILES and it didn't succeed. The script returned Gerät mydevice (39) in Profil 1 verschoben (device mydevice (39) moved to profile 1), but a subsequent check showed that it remained in its old profile.