Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ensure that CrowCam works on DSM 7.x #77

Open
tfabris opened this issue Mar 17, 2024 · 18 comments
Open

Ensure that CrowCam works on DSM 7.x #77

tfabris opened this issue Mar 17, 2024 · 18 comments

Comments

@tfabris
Copy link
Owner

tfabris commented Mar 17, 2024 •
edited
Loading

Currently the CrowCam code is tested and working on Synology DSM version 6.2.

There is a DSM 7.x version available, and 6.2 will reach end-of-life soon. I received an email that said support for DSM 6.2 will end on October 1st, 2024. I should make sure to upgrade my Synology to version 7 and ensure that CrowCam works on it.

I had an email conversation with Alex Santantonio from https://oldtownhome.com - He's using the CrowCam code to run his OspreyCam on YouTube at https://www.youtube.com/@OldTownHomeBlog/streams - He says that he needed to make some changes to the script to make it compatible with DSM 7.x. I'm trying to get a hold of him again to get some tips from him.

My guess is that his code and my code have diverged enough at this point that a merge would be difficult, but I'd certainly like to see Alex's changes that he made to make it DSM 7.x compatible.
@quog
Copy link

quog commented Jul 15, 2024

One issue:
IsServiceUp in CrowCamHelperFunctions.sh uses synoservicectl which doesn't exist anymore in DSM 7.2
There is
sudo synosystemctl get-active-status pkgctl-SurveillanceStation.service
but it needs sudo which is an issue.
I commented out the service check on line 918 of CrowCam.sh

@quog
Copy link

quog commented Jul 15, 2024

Second issue (I'm debugging as I go) - I had to replace
authWebCall="auth.cgi?api=SYNO.API.Auth&version=1&method=login&account=$username&passwd=$password&session=SurveillanceStation&format=cookie"
with
authWebCall="entry.cgi?api=SYNO.API.Auth&version=3&method=login&account=$username&passwd=$password&session=SurveillanceStation&format=cookie"
in CrowCamHelperFunctions.sh line 1100

@quog
Copy link

quog commented Jul 15, 2024

One other issue, ping is no longer able to run unless you sudo - you get
ping: socket: Operation not permitted
Two solutions:
chmod +s /bin/ping
which I don't like as it will potentially be lost with a DSM upgrade
So I changed line 650 of CrowCam.sh to be
if nslookup $ThisTestSite 8.8.8.8 >/dev/null
as nslookup still works for a user, 8.8.8.8 resolves as dns.google, and nslookup returns appropriately for the if statement. And by having the server parameter, it means internal DNS caching is bypassed.

@quog
Copy link

quog commented Jul 15, 2024 •
edited
Loading

FYI - the stream is at https://www.youtube.com/channel/UCIo6tmAPHtFduP3of7quqOw/live

@tfabris
Copy link
Owner Author

tfabris commented Jul 15, 2024

@quog - Thank you so very much!

Regarding "IsServiceUp" and "synoservicectl" - I looks like this was originally done as bulletproofing, just to make sure that someone who was running this script truly met the requirements. By which I mean, being on a Synology with Surveillance Station installed and running. I don't think that level of bulletproofing is truly necessary, and since it doesn't work any more, it makes sense to skip it. So indeed I will entirely remove the code which calls synoservicectl.

Regarding the "auth.cgi" versus "entry.cgi" change, and the move from v1 api to v3 api: I'll have to research the difference between those CGIs for the API endpoint. If you have more information about what the difference is between "auth" and "entry", let me know, otherwise I'll have a scrape around the docs about it. Additionally, I'm going to need to figure out why I needed to back down to V1 of the API. I made a note to myself about having had troubles with the V3 API, and I'm going to need to do some experimenting to find out exactly what those troubles were. I should have left myself better reminders, so that I know what to look for.

Regarding the change of "ping" to "nslookup" - This seems like a logical change on the face of it, but I need to understand it in more detail. In particular, I need to be absolutely certain that I'm not getting a cached version of the DNS from my local router when I use nslookup in favor of ping. You say that it should work, and I believe you, but sometimes I've seen other network commands lie to me before and feed me cached data (even when their docs say they shouldn't), and I have to fall back to using actual ping to get real time information. I also don't understand why sudo is going to be required to run ping if the script is running under root. When I set these things up on my synology in my Task Scheduler, the script is already running under root and sudo shouldn't be needed. Are you sure that sudo is really required to run ping, you're not just debugging the script under some lower account? Have a look at https://github.com/tfabris/CrowCam/blob/master/README.md#usage-and-troubleshooting and see if the notes there about running it with sudo and running under root in the task manager are applying to your situation. I don't have V7 installed yet, so I don't know if the task manager works any differently, so you might have more information on this than me.

Thanks again! Let me know if you find anything else.

@quog
Copy link

quog commented Jul 15, 2024

Hi Tony,

Those were the only changes needed to make it work on my client's NAS running DSM 7.2.1. Note: I have the scheduled tasks running as an admin user, but not as root - and that seems to be working (at least so far.) It's only been running for a couple of hours so I will see what happens tomorrow.

And yes, ping requires root or sudo now. The change to using nslookup was primarily because my client often had internet issues, and I didn't want to remove that check altogether. By giving two parameters to nslookup, you are telling it to talk to the DNS server in the second parameter which in my example is outside their network. So unless something like a firewall is proxying DNS, you are pretty much guaranteed to have a network connection if that command works.

I found the new Synology APIs here: https://global.download.synology.com/download/Document/Software/DeveloperGuide/Os/DSM/All/enu/DSM_Login_Web_API_Guide_enu.pdf

Thanks for the great code.

Regards
Damian

tfabris added a commit that referenced this issue Jul 16, 2024
@tfabris
Remove synoservicectl checks
66a314c 
GitHub Issue #77 - According to contributor @quog, the command "synoservicectl" has been removed from Synology DSM version 7.x.

I only used synoservicectl to check the running status of the Surveillance Station service, and I only checked it for making sure that the user met the requirements of running Surveillance Station. So it is safe to completely remove the code which references synoservicectl. Now the code will not check to see if the user is running Surveillance Station.

This change removes all code that uses the following commands and variables:

serviceName
synoservicectl
IsServiceUp
currentServiceState
@tfabris
Copy link
Owner Author

tfabris commented Jul 16, 2024

@quog - Understood, thanks so much! Your comments are very helpful. Since my NAS is still running DSM 6.x, I don't have a way to test on DSM 7.x at the moment. My intention is to upgrade my NAS to a newer model which is already running DSM 7.x, and then migrate the data and scripts over to it, but I haven't done that yet.

I've updated the code to remove synoservicectl already. Regarding the other two changes...

Clarifying questions:

  • If the scripts had been configured to run from the Synology Task Scheduler as "root" with the "bash" command line (as described in the configuration instructions here), then would the existing "ping" command have worked without modification?

  • After you modified the the "WebApiAuth" function to use "entry.cgi" and "version=3" to log in, did the other API commands still work without modification? Examples:

    • CrowCamHelperFunctions.sh
      • "IsStreamRunning"
      • "StartStream"
      • "StopStream"
      • "CreateNewStream"
    • CrowCam.sh
      • Three instances of calls to "entry.cgi?api=SYNO.SurveillanceStation.YoutubeLive"

    Did those API calls continue to work, unmodified, with the "version=1" parameter still in them?

Thanks again!

@tfabris
Copy link
Owner Author

tfabris commented Jul 16, 2024

Also, @quog - attached to this message is a test script containing a prototype of the "fixed" version of the WebApiAuth function. Would you mind helping me out, and running this test file from the console and sending me its output? It should run from a Linux or Mac shell prompt, and it should be run from a computer on the same network as the NAS. You have to edit three variables at the top of the file to point to your server running DSM 7.x, and its username, rename the file from .txt to .sh, and chmod it to be executable. The output should look similar this, except yours should try to authenticate to version 7 instead of version 6:

DSM Authentication Test - Querying http://192.168.0.88:5000/webapi to see if it is DSM 6.x.
DSM Authentication Test - Version Response: {"data":{"SYNO.API.Auth":{"maxVersion":6,"minVersion":1,"path":"auth.cgi"}},"success":true}.
DSM Authentication Test - DSM version appears to be 6 - Result: {"data":{"SYNO.API.Auth":{"maxVersion":6,"minVersion":1,"path":"auth.cgi"}},"success":true}.
DSM Authentication Test - Querying http://192.168.0.88:5000/webapi to see if it is DSM 7.x.
DSM Authentication Test - Query did not succeed - Result: {"error":{"code":102},"success":false}.
DSM Authentication Test - Attempting to authenticate to DSM version 6.
DSM Authentication Test - Auth response: {"success":true}.

Script should be attached below. Thanks!
DsmAuthTest.txt

@tfabris
Copy link
Owner Author

tfabris commented Jul 17, 2024

Also, @quog - Can you tell if logging to the Synology log (the function "LogMessage" in CrowCamHelperFunctions.sh) is working on DSM 7.x?

tfabris added a commit that referenced this issue Jul 17, 2024
@tfabris
Authenticate with DSM API differently
6c8c9fc 
GitHub issue #77: Fix the DSM API authentication login so that it works on both DSM6 and DSM7. Detect if the NAS is running DSM v6/earlier or v7/later by trying to do the api queries indicated by their authentication docs. Doesn't quite work the way they said it would work because they moved the commands between different CGI files from DSM6 to DSM7. Use this to our advantage to detect which side of that line we're on, and change the authentication API string depending on what we found out.
@tfabris
Copy link
Owner Author

tfabris commented Jul 17, 2024

OK, I believe I've made the necessary changes to the master branch code to make it work under DSM 7.x, as long as it's run as "root".

I was able to do some limited testing on DSM 7.2 by logging into their online "demo" of DSM 7.2 from the Synology main web site. I see that the logging function still works, my tests to detect DSM6 or DSM7 are working, etc. I wasn't able to actually test whether the authentication API command works, though, because when you log into their online demo, you don't know the password for the user "demouser" and it terminates your session if you try to add new users or change the demouser password.

I haven't yet updated the "ping" command, because I have strong feelings about it needing to truly be a "ping" and not some other command. The trick of using nslookup instead of ping is a good trick, but it's not exactly the same thing. In particular, I'd like to have the option to change the configuration so that it pings some other server other than a DNS server to perform the network test. There could theoretically be situations where you would want that. Leaving it as a true ping gives it that flexibility.

@quog - If you are able, please try the new master branch version of the CrowCam code on your 7.2 server and tell me if it's working OK for you. The only important changed files are "CrowCam.sh" and "CrowCamHelperFunctions.sh", and you can leave your configuration files as they are. But you will need to reconfigure the tasks in Task Scheduler so that they run under the user "root" in order for the ping command to work.

@quog
Copy link

quog commented Jul 18, 2024 via email

@quog
Copy link

quog commented Jul 18, 2024 via email

@quog
Copy link

quog commented Jul 18, 2024 via email

@quog
Copy link

quog commented Jul 18, 2024 via email

@tfabris
Copy link
Owner Author

tfabris commented Jul 18, 2024 •
edited
Loading

@quog - Thanks so much for testing those things! I'm also stumped by why the authentication didn't work. As far as I can tell, the auth string in my code ends up being the same as the auth string in your (working) code and should therefore work.

If you run the new master branch version of the CrowCam code (keep your working code safe somewhere), if the auth string still fails, please contact me by email (tfabris at gmail) and we can work out what's wrong without spamming this GitHub thread.

@tfabris
Copy link
Owner Author

tfabris commented Jul 18, 2024 •
edited
Loading

@quog - If the test code failed but the new Master branch code works, then one possibility could be that the password is obtained differently in my test code than in the main code. In the test code it's just a variable assignment at the top of the file:

password="myPassWord"

In the main code it reads the value from a different file on the disk:

read username password < "api-creds"

One possibility is that you simply mistyped the username and/or password in the test code, whereas it's correct in the "api-creds" file. A more likely possibility is that password contains some kind of special character that Bash interprets in a special way when performing a variable assignment, but the "read" command doesn't care about. Obviously don't tell me your password, but does the password contain something like a percent sign, a dollar sign, square or curly brackets, forward or back slashes, single or double quotes, or backticks? For example, the code password="pas$word" would definitely not work as intended. If that's the case, maybe it will work fine in the final code, and only fail in the test code.

@quog
Copy link

quog commented Jul 18, 2024

Hi Tony

The issue was special characters in the password. I changed the DsmAuthTest.sh to use read and it works.

bash DsmAuthTest.sh
DSM Authentication Test - Querying http://127.0.0.1:5000/webapi to see if it is DSM 6.x.
DSM Authentication Test - Version Response: {"data":{"SYNO.API.Auth":{"maxVersion":7,"minVersion":1,"path":"entry.cgi"}},"success":true}.
DSM Authentication Test - DSM version appears to be 6 - Result: {"data":{"SYNO.API.Auth":{"maxVersion":7,"minVersion":1,"path":"entry.cgi"}},"success":true}.
DSM Authentication Test - Querying http://127.0.0.1:5000/webapi to see if it is DSM 7.x.
DSM Authentication Test - DSM version appears to be 7 - Result: {"data":{"SYNO.API.Auth":{"maxVersion":7,"minVersion":1,"path":"entry.cgi"}},"success":true}.
DSM Authentication Test - Attempting to authenticate to DSM version 7.
DSM Authentication Test - Auth response: {"data":{"did":"REDACTED","sid":"REDACTED"},"success":true}.

@tfabris
Copy link
Owner Author

tfabris commented Jul 19, 2024

@quog - That is fantastic, thank you so much. You’ve been a great help. Let me know if you encounter any other problems.

I’m going to leave this bug open until I can run DSM 7.x myself and double check a few things. Other than that, I think this bug should be fixed. (Crossing my fingers.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@quog @tfabris