JS Debugger does not attach

[jokrug]

I made small changes to the meshagent and recompiled it. From time to time, the agent hangs and CPU load goes up to almost 100%. I wanted to use the JS debugger to track that down.
What I did:

• Copy duktape-debugger.js to the mesh directory on the target machine.
• Add "jsDebugPort=9091" and "showModuleNames=1" to meshagent.msh.
• Start the meshagent
  When I start Firefox on my pc and enter "http://myTargetIp:9091" I the "Built-in JS Debugger" opens.
  When I click "Attach" it sais "Attaching debugger..." and stucks in that state.
  When I click on "Pause" the meshagent prints "false, false, null" on the terminal.

Did I miss something? Do I need all sources of the meshagent on the target in order to debug it?

[krayon007]

I'll take a look at it. Normally you don't need to click attach.

but on a side note... If you are seeing CPU spikes, one thing you can do, is from the console tab, type the command fdsnapshot
That should show a list of all the descriptors and their state, as well as any timers that are set. Normally when there is a CPU spike, it's becuase there's a descriptor in there that is signaled but not getting serviced. This may help to see which descriptor is causing problems, to be able give a hint in what part of the agent is messing up.

[jokrug]

Thank you for the hint and yes, you are right. I do not have to click at "Attach". After a few seconds the "Attaching debugger" message appears without any action.
Please note, that we use the meshagent on a stripped down x86 linux with X, but not all programs and libraries are available or at the usual places. If you are not able to reproduce the problem, it might have to do with our special linux.

[aktxyz]

wow ... I was wondering how to debug the meshcore.js from the server ! I can't wait to try this.

The meshcore.js also has require statements like

• require('user-sessions')
• require('sysinfo')

are those actually resolving to js files somehow embedded in the agent exe?
from ... https://github.com/Ylianst/MeshAgent/modules

[aktxyz]

Hmm ... I tried this by copying the duktape-debugger.js (from the MeshAgent/modules folder in github)
to the same directory on my windows machine as the MeshAgent.exe (maybe that's not right?)

then updated the MeshAgent.msh to look like this ...

...
MeshServer=local
ignoreProxyFile=1
jsDebugPort=9091   << added this 
...

and tried connecting to http://localhost:9091 on after restarting the MeshAgent service ... but could not connect

soooo must be doing something wrong here?

[krayon007]

wow ... I was wondering how to debug the meshcore.js from the server ! I can't wait to try this.

The meshcore.js also has require statements like

• require('user-sessions')
• require('sysinfo')

are those actually resolving to js files somehow embedded in the agent exe? from ... https://github.com/Ylianst/MeshAgent/modules

Yes. If you look in ILibDuktape_Polyfills.c, you will find where some of the JS files are embedded in to agent. The module loader will first look at embedded JS files, then in the file system if it can't find it.

[krayon007]

Hmm ... I tried this by copying the duktape-debugger.js (from the MeshAgent/modules folder in github) to the same directory on my windows machine as the MeshAgent.exe (maybe that's not right?)

then updated the MeshAgent.msh to look like this ...

...
MeshServer=local
ignoreProxyFile=1
jsDebugPort=9091   << added this 
...

and tried connecting to http://localhost:9091 on after restarting the MeshAgent service ... but could not connect

soooo must be doing something wrong here?

You are correct in copying that JS file... That MSH option was really meant for running the agent in command line mode... To use the debugger with the background service, it's easer to do it this way:

1. copy the file as you did
2. from the console tab of the agent, enter this command, substituting the port number you want to use instead of 9999
   eval "attachDebugger({ webport: 9999 })"

Then direct your browser to http://localhost:9999 or whatever port you used...
The debugger is rather rudimentary, but it gets the job done, lol.

[krayon007]

By the way, if you pause the debugger, and happen to forget about it, the agent will automatically kill itself and restart, becuase it will think that a thread is stuck. I think the default timeout is 10 minutes, so if you leave the debugger paused for 10 minutes, you may find a log entry saved to disk saying "Microstack Thread STUCK", or something similar.

[aktxyz]

Well, I got the debugger working (Built-in JS Debugger html ui) ... but must be dense as I cannot find where to set an actual breakpoint?

[aktxyz]

Well, I got the debugger working (Built-in JS Debugger html ui) ... but must be dense as I cannot find where to set an actual breakpoint?

never mind !!! click on the line number, could have sworn I tried that

[aktxyz]

any trick to getting breakpoints to actually break? I am setting a breakpoint inside the 2 case 'cpuinfo' case blocks, then issuing a cpuinfo from the console in the meshcentral web ui ... and it does not break in the duktape js debugger

[krayon007]

One thing you can try is to put the breakpoint a line or two before you want it. I noticed sometimes the JS runtime's reported program counter is off a little compared to the source.

[jokrug]

• The module loader will first look at embedded JS files, then in the file system if it can't find it.

Is it realy in this order? Or does it first look in the file system?
If it would first load the embedded JS files, I would have to delete the embedded JS and recompile the agent, before I could test my own version of the files...

[krayon007]

There's no need to delete the embedded JS. The way it's encoded, it includes the timestamp of when it was encoded. So if you just put the modified JS file in the same folder, it will pick it up.

[aktxyz]

Is there anyway to turn on some extra agent side logging?

oh, and anyway from the meshcore.js to log to the agent console window ... like where this is logged ...

AutoRetry Connect in 2124 milliseconds
AutoRetry Connect in 3117 milliseconds
Connecting to: wss://192.168.86.22:443/agent.ashx
Connected.
Server verified meshcore... meshcore already running...
Server verified meshcore... meshcore already running...
Server verified meshcore... meshcore already running...
Server verified meshcore... meshcore already running...
Server verified meshcore... meshcore already running...

[krayon007]

Is there anyway to turn on some extra agent side logging?

oh, and anyway from the meshcore.js to log to the agent console window ... like where this is logged ...

AutoRetry Connect in 2124 milliseconds
AutoRetry Connect in 3117 milliseconds
Connecting to: wss://192.168.86.22:443/agent.ashx
Connected.
Server verified meshcore... meshcore already running...
Server verified meshcore... meshcore already running...
Server verified meshcore... meshcore already running...
Server verified meshcore... meshcore already running...
Server verified meshcore... meshcore already running...

Are you talking about the console tab in the browser? If so, then you can use sendConsoleText() inside meshcore.js. It takes a string, and will send it to the console tab of all browsers that are logged in with access to that device.

[krayon007]

Another way to redirect things, is to use the setdebug command in the console tab. That command will redirect where console.log() gets printed. You can have it go to stdout, which is the default, or you can redirect it to the console tab window..

As far as the extra logging, what type of logging are you referring to? There's quite a bit of logging available, thats hidden behind various flags...

[aktxyz]

I was seeing an exception, which ended up being an unhandled exception in my meshcore.js .... but in general any kind of logging that could help understand program flow ... it can be very helpful to grep your way thru an ocean of logging and refer back to the source to piece together flow ... I will look thru the repo for how to turn on the flag ... thanks !