Scanner Plugin for FM-DX-Webserver
This plugin provides scanning functions for the FM-DX web server.
- Important security update (Thanks to PE5PVB!)
- Download the last repository as a zip
- Unpack all files from the plugins folder to ..fm-dx-webserver-main\plugins\
- Stop or close the fm-dx-webserver
- Start/Restart the fm-dx-webserver with "npm run webserver" on node.js console, check the console informations
- Activate the scanner plugin in the settings
- Stop or close the fm-dx-webserver
- Start/Restart the fm-dx-webserver with "npm run webserver" on node.js console, check the console informations
- Configure your personal settings in the automatically created scanner.json (in the folder: ../fm-dx-webserver-main/plugins_configs)
- Stop or close the fm-dx-webserver
- Start/Restart the fm-dx-webserver with "npm run webserver" on node.js console, check the console informations
You can find detailed documentation here: https://highpoint.fmdx.org/manuals/Scanner-Documentation.html
To create or modify the scanner configuration file (scanner.json) please use the Scanner Wizard. All options are stored and described there.
- Auto Scan Mode with the options is only usable with ADMIN- oder TUNE-Athentification !!!
- By briefly pressing the Auto Scan button you start/stop the automatic scanning process. Pressing the button longer opens or closes the scanner's setting options
- The automatic antenna switching only works if more than 1 antenna is configured in the web server !!!
- For ESP32 receivers (e.g. TEF6686) the plugin can uses the newly integrated firmware scan and search function. However, the prerequisite is the installation of the latest PE5PVB firmware version.
- Auto scanner and logging preferences can be configured in the scanner.json under /plugin_configs
- In the HTML file you can klick on the header fields to sort the column or you enter a term in the search field
- In order to use the live stream link in the log file, you must register at fmscan.org. When you open a link for the first time, you have to authenticate yourself with it
- Since all FMLIST log entries are created automatically, with FMLIST Autolog mode "on" the manual log button is hidden on the web interface, in the Autolog mode “auto”, the manual log button is only hidden on the web interface during the autoscan process
- If there are several web servers, it makes sense to use a central server to register the logs that have already been sent. The CanLogServer can provide this functionality. When the server is used, the log interval set in the scanner.json is inactive because the log interval set for the server takes precedence!
- The computer's standard sound output is used for acoustic signaling during the scanning process
- For FMDX scanning operation, we recommend reducing the defaultScanHoldTime to 2-3 seconds and setting Autoscan_PE5PVB_Mode: false
- To use the fast spectrum scan, the spectrum graph plugin must be installed. The SpectrumLimiterValue variable can be used to set an upper limit for the filter of strong transmitters. Transmitters that exceed this value are filtered out. Use the variable Spectrum PlusMinus Value to set a signal strength of a strong/local station independently of the Spectrum Limiter Value, where the neighboring channels (+/- 0.1 MHz) should be filtered out. To automatically recreate the spectrum after each frequency scan, the rescanDelay variable in the SpectrumGraph.json must be set to 0 !!!
- Difference Scan (extension of spectrum scan): The SpectrumChangeValue (dBf/dBµV) indicates in which +/- range the signal must differ from the previous scan for the frequency to be used. Only frequencies with this change are then scanned
- When GPS data is received, the location is updated dynamically (GPS receiver and GPS plugin required!)
- To use the fm-dx-monitor set the OnlyScanHoldTime 'on' and choose a minimum defaultScanHoldTime of 5 seconds
- Whitelist entries are processed with 0.01 MHZ increments
- URDS CSV Log protocol and Map Viewer Button can be activated with CSVcreate option in the configuration settings
- for manual or automatic upload the URDS CSV log protocol, please install the URDS Upload Plugin
- Attention! To use dynamic sensitivity calibration, the reference frequency entered must be within the web server's approved frequency range. For example, 87.1 is within the range 87.0 - 108.0 MHz. The range to be scanned must be defined using the variables tuningLowerLimit and tuningUpperLimit in the scanner.json file and must also be within the web server's approved frequency range.
- If the value is changed via the user interface, the updated value is used for all subsequent scans
- Multiple configuration files can now be created and selected via a drop-down menu in the web server settings (admin login required!). All settings are loaded on the fly, including subsequent changes to the configuration files. Additional configuration files must be named as follows: scanner_file1.json, scanner_file2.json, etc. They will then appear in the dropdown menu as file1, file2. The default file refers to the scanner.json file. The dropdown menu in the webserver settings will only be displayed if there are multiple scanner configuration files in the plugin_configs folder!
- The scanner can also be controlled directly via the keyboard: Press U or u for "search up" / Press D or d for "search down" / Press S or s to start/stop automatic scanning
The scanner plugin contains various blacklist options and a whitelist function. The necessary files with sample data are included in the plugin package. The whitelist.txt file can be used to store frequencies (e.g., 89.800, 89.400, 100.80, separated by spaces) that are scanned exclusively in the "Whitelist" scan mode. Likewise, the blacklist.txt file can be used to define frequencies that are not scanned (i.e., skipped and therefore not logged) in the "Blacklist," "SpectrumBL," and "DifferenceBL" scan modes. The blacklist_log.txt file is used to explicitly exclude frequencies and/or PI codes from being entered into the log files. This also applies to logs with the manual search (<< >>). Using blacklist_fmlist.txt, frequencies and/or PI codes can be additionally excluded from FMLIST publication. The use of the blacklist or whitelist must be configured in the scanner.json configuration file.
- When you start the auto scanner in PE5PVB mode, the frequency freezes for a few seconds
- With newer node.js libraries there may be problems installing the speaker module. If acoustic signaling is desired, the node.js version must be downgraded!
If you have any questions, would like to report problems, or have suggestions for improvement, please feel free to contact me! You can reach me by email at highpoint2000@googlemail.com. I look forward to hearing from you!
History
- Addition of two further scan modes: "spectrumWL" and "differenceWL". These hybrid modes combine the speed of spectrum-diagram-based filtering with the exclusivity of the "whitelist.txt" file
- The standard whitelist scan mode has been reverted to its state prior to version 4.6c
- Sensitivity fix: A user interface bug that caused a (cumulative) increase in the sensitivity value when switching modes has been resolved by ensuring that only explicitly updated values are transmitted from the client to the server
- The cooldown for the Spectrum Graph Socket has been reduced to 3000 ms to improve the responsiveness of the scanning process
- Whitelist mode has been completely revamped and now utilizes the Spectrum plugin to pre-determine signal strength
- Autoscan Concurrency: Prevented scan activation or resumption during active user sessions via real-time connection checks and immediate timer cancellation.
- System Stability & UI: Resolved spectrum-empty search errors and optimized button layouts for narrow displays.
Many thanks to AmateurAudioDude for this code improvements!
- Fixed error in the identification of Polish radio stations
- Added StartScanByMUF (true/false) and StartScanByMUF_Region ('EU', 'NA', 'AU') variables to control the new automation. The scanner can now automatically start scanning when Sporadic-E MUF is detected, and safely revert to its default state (off or auto) when the MUF drops.
- Fixed an issue where interacting with the scanner's dropdown menus prevented the main web server's dropdown menus from opening
- This version now additionally saves the current status of the RDS AI Decoder (1 = on / 0 = off), as well as the PI and PS data identified by the decoder to the CSV and HTML logs. This allows for subsequent validation and analysis of the quality of the AI logs. Therefore, I recommend that all RDS AI Decoder users install both this update and the decoder update!
- The PS Log is reset with every frequency change
- Spaces in the PS are replaced by underscores
- The Search buttons << >> now support the Blacklist and Whitelist options for Scan Mode selection
- Map Viewer URL linked to new domain
- Fix potential scanner crash on profile switch (Thanks to AmateurAudioDude for PR)
- Hardware keyboard shortcuts added: You can now control the scanner directly via the keyboard: Press U or u for Search Up / Press D or d for Search Down / Press S or s to Start/Stop Autoscan
- Frequency Grid Snapping: The software search logic now automatically rounds the frequency to the nearest 100 kHz (or 10 kHz for the OIRT band) before calculating the next step. This prevents the scanner from maintaining a "crooked" offset (e.g., jumping from 96.23 to 96.33 instead of 96.30).
- PE5PVB Hardware Search Sync: When using the receiver's internal hardware search mode (Search_PE5PVB_Mode), the server now forces the tuner to snap to the rounded 100 kHz grid before sending the hardware search command (C1/C2). This guarantees the hardware search always starts from a clean frequency.
- Log Sanitization: All incoming frequencies from the tuner are now instantly rounded at the WebSocket handler level. This guarantees that no crooked 10 kHz frequencies (like 96.23) will ever be written to the CSV, HTML, FMLIST, or Logbook records.
- Providing DX event data to the plugin API (PR from AmateurAudioDude)
- Extra button has been added for directly validating the current URDS CSV file via the URDS Validator
- All the code for using the screenshot function through the DX-Alert plugin has been removed
Important note: Due to the diverse uses of the web server, especially the scanner, this function cannot currently be reliably implemented!
Fixed an issue with frequency restoration:
- The correct frequency is now restored when connecting to the web server and when stopping the scanner
- If a default frequency is configured on the web server, it is always restored
- If no default frequency is configured, the last used frequency is restored
Fixed an issue with the default sensitivity value (defaultSensitivityValue):
- When automatic calibration is active (SensitivityCalibrationFrequency is set), changes to the value in the JSON configuration are now taken into account when calculating the sensitivity calibration value
- When automatic calibration is disabled: The default sensitivity value applies only at scanner startup
If the value is changed via the user interface, the updated value is used for all subsequent scans. Multiple configuration files can now be created and selected via a drop-down menu in the web server settings (admin login required!). All settings are loaded on the fly, including subsequent changes to the configuration files. Additional configuration files must be named as follows: scanner_file1.json, scanner_file2.json, etc. They will then appear in the dropdown menu as file1, file2. The default file refers to the scanner.json file.
Note: The dropdown menu in the webserver settings will only be displayed if there are multiple scanner configuration files in the plugin_configs folder!
Multiple configuration files can now be created and selected via a drop-down menu in the web server settings (admin login required!). All settings are loaded on the fly, including subsequent changes to the configuration files. Additional configuration files must be named as follows: scanner_file1.json, scanner_file2.json, etc. They will then appear in the dropdown menu as file1, file2. The default file refers to the scanner.json file.
Note: The dropdown menu in the webserver settings will only be displayed if there are multiple scanner configuration files in the plugin_configs folder!
- Unit changes on the web server are now directly reflected in the scanner
- Changes to the scanner mode in the JSON configuration are now displayed in the web interface
- Problem with broadcast messages containing empty strings has been resolved
- Settings in scanner.json are now updated on the fly when changes are made
- Various code improvements and bug fixes
- Problems with specifying incorrect scan limits corrected
- Using the new plugin API to communicate with the web server and spectrum plugin -> Scanner can now also be used in Admin Lock mode (Web server and Spectrum plugin update required!)
- Automatic calibration adjusted
- Control of the locked web server via FMDX Connector built in
Thanks to AmateurAudioDude for the Pull Request!
- Using the new plugin API to communicate with the web server and spectrum plugin -> Scanner can now also be used in Admin Lock mode (Web server and Spectrum plugin update required!)
- Automatic calibration adjusted
Thanks to AmateurAudioDude for the Pull Request!
- Necessary adjustments for using the screenshot function of the DX-Alert plugin
- Control options for the TEF Logger App (v3.4) and FMDX Connector (v1.0) have been implemented
- Corrected client status and the starting and stopping of autoscan mode
- Fixed a bug in the unit of measurement switching
- Control options for the TEF Logger App (v3.4) and FMDX Connector (v1.0) have been implemented
- Extended URDS CSV protocol implemented (storage of station data for faster evaluation e.g. in the map viewer)
For URDS uploads, the uploader version from 1.0g (Version with dBµV Flag) upwards must be used!
- Scanner interface also works with strict tracking protection at Edge
For URDS uploads, the uploader version from 1.0g (Version with dBµV Flag) upwards must be used!
- Fixed a bug that caused the scan to stop at the upper band limit when the SensitivityCalibrationFrequency was not used
- Fixed incorrect display of signal strength unit in dropdown menu and console
- dBµV is now fixed Unit for CSV File
For URDS uploads, the uploader version from 1.0g (Version with dBµV Flag) upwards must be used!
- Added UUID check - Token required for publishing logs to FMLIST
- Default FMLIST_LogInterval set to 60 minutes
- Fixed error in log interval conversion
- Fixed FMLIST publishing without PI & PS
- PI codes with 0000 are now logged
- Column sorting in HTML template corrected
For URDS uploads, the uploader version from 1.0g (Version witht dBµV Flag) upwards must be used!
- Fixed problem with stopping at the upper band limit
- Code optimizations for dynamically noise level scan (please refer to the comments under important notes for setup!)
- New variable to define a reference frequency to dynamically scan the current noise level (please refer to the comments under important notes for setup!)
- Information about antenna switching hidden in the console
- Display of the sensitivity value and spectrum limiter value in the spectrum updated
- Skipping frequencies with signal levels below the signal value stored in the spectrum analysis
- Updating the signal level in the spectrum analysis with conductance values
- Update display is now in the web server settings under Plugins (Thanks to AmateurAudioDude for the Code)
- "Map all" link in HTML log brought forward
- Additional column “Mode” added in the HTML protocol (M – manual / A – autoscan)
- Design adjustments for the new mobile layout
- Button design created for SDR receiver
For URDS uploads, the uploader version from 1.0g (Version witht dBµV Flag) upwards must be used!
- Minor design work
- Blacklist function tidied up Minor design work (see Blacklist & Whitelist Options)
- Used antenna added to the HTML Log
- Error message hidden
- Additional values for sensitivity and scan hold time added
- Renamed variables for HTML logging
- Added a new variable for HTML logging of unidentified transmitters
- The signal strength units are now correctly logged, different from dBf
- Add a special blacklist function for the Logfile entries (see instructions!)
- Future spamming protection for the scanner plugin implemented (Thanks to AmateurAudioDude!)
- Bandwidth information changed to Hz
- Add a blacklist function for the FMLIST log entries (see instructions!)
- Fixed incorrect display of server name in FMLIST log
- Bug when disabling CSV log creation fixed
- Protocols with a distance >700km are now automatically marked as Es (Sporadic E) on FMLIST
- The signal strength indication in the FMLIST Log has been changed to dBµV
- The minimum distance for an automatic log entry is now 200 km and the repetition interval has been increased to 6 hours
- Added option to create the URSD CSV log file, this also activates a map viewer button to open and evaluate the log file
- Fixed an issue when stopping the scan at the upper band limit
- Removed duplicate message from FMLIST log
- Rollback Code optimization from Version 3.3b
- Fixed an issue when stopping the scan at the upper band limit
- Code optimization for PS detection performed (thanks to AmateurAudioDude)
- improved CSV logging mode for PS information
- Fixed an error when publishing a version 3.2c FMLIST entry
- Whitelist entries are now also processed with 0.01 MHZ increments
- Adding a new variable CSVcompletePS for logging with and without full PS information
- minor code corrections
- fixed problem with doubled autologged entry
- small design adjustment
For URDS uploads, the uploader version from 1.0g (Version witht dBµV Flag) upwards must be used!
- Fixed the problem with the background and the display of the “autoscan active” screen
- Corrected display of Auto Log button on smaller display
- Fixed temporary display of small scanner buttons during reload
- Start time for automatic scan increased from 5 seconds to 15 seconds
- Small code corrections regarding writing the log file
For URDS uploads, the uploader version from 1.0g (Version witht dBµV Flag) upwards must be used!
- URDS RDS data added (e.g RT, Station ID, ECC)
- Signal unit for URDS Log changed to dBµV
- Added PS filter for error-free URDS logs
- Added display of the current antenna and server short name for FMLIST logbook entries
- Bug with the red flashing chat symbol fixed
For URDS uploads, the uploader version from 1.0g (Version witht dBµV Flag) upwards must be used!
- Bug fixes
- duplicate messages removed
For URDS uploads, the uploader version from 1.0d upwards must be used!
- Bug fixes and code optimizations
- Added antenna switching for whitelist mode
For URDS uploads, the uploader version from 1.0d upwards must be used!
- The search bug has been fixed
- Search stop function when pressed repeatedly
- Switch for mandatory use of ScanHoldTime implemented (useful for fm-dx-monitor!)
- Column 30 in the urds log file removed (now supplemented with URDS Uploader from version 1.0d!)
- Logs with ? in the PS information are now logged
- various code and scanner optimizations
- XML protocol converted to URDS format (To use the protocol, please install the URDS Upload Plugin)
- Processing of GPS data (GPS Receiver & GPS plugin required!)
- Added acoustic signaling during scanning operation
- Daily update check for admin
- Scan algorithm revised
- Added fast spectrum scan with limiter and filter for strong station (spectrum graph plugin must be installed!)
- Added ultrafast difference scan option with limiter and filter for strong station (spectrum graph plugin must be installed!)
- Added tuningLowerLimit and tuningUpperLimit
- Starting frequencies above 74 MHz are rounded to 100 kHz during the autoscan
- bugfixing
- Adjustments for CanLogServer (Version 2.0)
- Automatic jump back of the background scanner to the initial frequency if no frequency is configured when loading the web server
- Option to use the CanLogServer (see important notes!)
- FMLIST integration for automatic logging (For details see configuration options and important notes!)
- Bugfixing
- Bugfixing
- Added options to set scanIntervalTime and scanBandwith in the configuration file
- Delay serialport connection loss check on startup (thanks to AmateurRadioDude)
- Added signal strength to logfile
- Blacklist and Whitelist can switch off in config file
- Fixed problem creating plugins_configs path
- bugfixing
- configPlugin.json is moved to ../fm-dx-webserver-main/plugins_configs/scanner.json
- Switches for blacklist and whitelist built into the config file
- HTML Language Tag set to English
- Adaptation of the web socket /extra to /data_plugins, index.js update is no longer needed from now on!
- New notification design (Toast Notification)
v2.5 (only works from web server version 1.2.6 - older versions must take the plugin version 1.3c oder 1.3d !!!)
- FIRST LOG MODE now displayed in the log file
v2.4 (only works from web server version 1.2.6 - older versions must take the plugin version 1.3c oder 1.3d !!!)
- Fixed configuration is now stored in configPlugin.json
v2.3 (only works from web server version 1.2.6 - older versions must take the plugin version 1.3c oder 1.3d !!!)
- 0.01 MHz Fixed error when exceeding OIRT upper limit
- Fixed bug when crossing the date line at UTC
- The station IDs of Polish radio stations are now identified via a separate database file
- Renamed maps.fmdx.pl to maps.fmdx.org and FMDX links to MAP links
- The MAP ALL link is now created dynamically and adapts to the log filter, and there are now distance restrictions in the log file
- The refresh interval of the log file has been increased to 10 seconds
- Fixed logging and autostart for PE5PVB mode
v2.2 (only works from web server version 1.2.6 - older versions must take the plugin version 1.3c oder 1.3d !!!)
- New layout for HTML logfile with search/sort Options, Toggle Button for auto refresh and dark mode
- Time display corrected for local time
- Scan step size implemented for OIRT band
- Default selection 87.5-108 MHz if no limits are set (caused problems in previous versions!)
v2.1 (only works from web server version 1.2.6 - older versions must take the plugin version 1.3c oder 1.3d !!!)
- Mobile control for autoscan
- CSV log files (RAW + filtered) and HTML log files (RAW + filtered) can be saved automatically in /web/logs
- Time in Logfiles can be set to UTC
- HTML-Logfiles has 5 seconds autorefresh inside
- Logfiles can be download with RDS-Logger CSV- & HTML Buttons (You need RDS-Logger Plugin from v1.5) or via DX-Alert email (You need DX-Alert Plugin v. 2.0a)
v2.0 (only works from web server version 1.2.6 - older versions must take the plugin version 1.3c oder 1.3d !!!)
- Automatic background scan when no user is connected or automatic start when the web server is started
- Automatic antenna switching on upper band limit
- Hide the control buttons when autoscan mode is active (blinking information!)
- Activating the scanner control function is done by holding down the auto scan buttons
- blacklist.txt and whitlist.txt are now loaded from the plugin path
- Server plugins are activated/deactivated via the web server GUI
- Proxy Server ready (Tnx to zer0_gravity)
v1.3e (only works from web server version 1.2.6 - older versions must take the plugin version 1.3c oder 1.3d !!!)
- compatible with changed websocket data in version 1.2.6
- Increase scan and search speed
v1.3d (only works from web server version 1.2.3 - older versions must take the plugin version 1.3c !!!)
- Problem with multiple connections (user online) fixed
- Fixed a bug due to missing band limits
- Autoscan mode button is visible again without login
- optimze scan performance
- longer RDS detection if PI-code detected
- optimze the blacklist processing
- Add a white list function (file /web/scanner/whitelist.txt must be created with the frequencies: 89.800 89.400 100.80 ... They can be written next to or below each other with spaces)
- separate software switches for auto scan and search mode << >> (Mixed Mode with PE5PVB Firmware)
- Added Auto Scan Mode for ALL Devices including settings (blacklist on/off, sensivity, scan hold time). To do this, set the query off PE5PVB Firmware in the header of the script to 'false'.
- Default values for mode, sensitivity and scan hold time can be set in the script
- blacklist on/off and sensivity have affect the manual search mode << >>
- To use the blacklist option, a file /web/scanner/blacklist.txt must be created (For example, the frequencies that should not be logged must be: 89.800 89.400 100.80 ... They can be written next to or below each other with spaces)
- Added Scan Sensitivity and Scanhold Time settings
- Design issues fixed
- Add a Auto Scan Mode for ESP32 receiver (Newewst PE5PVB ESP32 firmware (RC-Version) required!)
- Merging the functionalities of v1.0 and v1.0a (Switching in JS-Code)
- Direct use of the integrated scan function of the ESP32 receiver (PE5PVB ESP32 firmware required!)
- Fixed issue with incorrect number of users
- Plugin scan function