-
Notifications
You must be signed in to change notification settings - Fork 666
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Debugger Transport APIs and Zephyr port #2247
Conversation
84bb335
to
8d6e461
Compare
@jimmy-huang some builds are failing (click "Details" below to learn more) |
* The default transport layer that uses TCP socket | ||
* other transports can be added like usb, bluetooth | ||
*/ | ||
jerry_debugger_transport_t *jerry_debugger_init_socket_transport(uint16_t tcp_port); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this be in jerryscript-port.h
?
It's specific to the socket transport implementation, no?
[EDIT] Ah, I guess you use the same function signature in the zephyr implementation as well.
Thinking about it again, it makes sense.
jerry-port/default/default-socket.c
Outdated
*/ | ||
jerry_debugger_transport_t* | ||
jerry_debugger_init_socket_transport(uint16_t tcp_port) /**< server port number */ | ||
{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps add an assert to make sure it's only called once (since the socket_transport
state is global/singleton)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this only intended to be called once? if you call debugger_cleanup() and then restart the the cycle with debugger_init(), and you would expect to init the transport() again? or at least okay to call it as long as it's a singleton.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm I see, ok, I guess it's ok.
There's nothing precluding one from creating a dynamically created transport (and having a transport-specific cleanup function to free/destroy the transport), so I'm happy.
8bb9dab
to
24785b4
Compare
targets/zephyr/src/jerry-port.c
Outdated
|
||
#include "jerryscript-port.h" | ||
|
||
#include "jrt.h" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is used from this header?
It's a private/internal header.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@martijnthe i was trying to fix the travis build erros, because i have to call JERRY_UNUSED on the unused port parameter when debugger is ifdef'ed out, but JERRY_UNUSED is not defined, but i think i don't need to call it in the zephyr port, only in the default-socket.c?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@martijnthe btw, I am still fixing some of these errors, i got around a bunch yesterday, but it turned out pulling in unistd.h for ssize_t is problematic on other platforms, so I think i will change it to use size_t only on the bytes send/received values, i mean it was initially copied from the socket calls, i think making it just unsigned is okay since we return the error values anyway, so no need to check -1.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds reasonable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
JERRY_UNUSED
Hmm I guess it's not part of the "API"
Maybe just use (void) variable_that_is_unused
?
@zherczeg could you take a look at the design? |
24ac32a
to
2609b6a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The general direction is ok, but I feel the transport and transport below websocket is somehow mixed. General functions should be moved to jerry_debugger.c from jerry_debugger_ws.c.
@@ -93,13 +93,18 @@ jerry_debugger_stop_at_breakpoint (bool enable_stop_at_breakpoint) /**< enable/d | |||
* Debugger server initialization. Must be called after jerry_init. | |||
*/ | |||
void | |||
jerry_debugger_init (uint16_t port) /**< server port number */ | |||
jerry_debugger_init (jerry_debugger_transport_t *transport_p) /**< transport */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this port specific definition part of the public API?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you mean the jerry_debugger_transport_t type? Yes, this is intended and part of the jerry_port_ api where each platform have to provide their own definition, hence why it was forward declared, so it can build.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately "it can build" is not a strong enough reason for something to be part of the public API.
While I am ok with having a structure for communication, the public API part needs to be independent of everything: no jerry port dependency, no compiler define dependency, etc.
I think the first question is: do we want multiple transport layer support, or the recv/send/close could be provided by jerry port.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zherczeg When I originally started the the port, it was a simpler patch to just port the socket APIs, but I think @martijnthe really like to support multiple transports, so we kinds modified it to take a struct of handlers. I mean we can submit a simpler patch to ports only socket calls and then a separate patch later to rework the APIs to support multiple transports, I think it's up to you and the JerryScript maintainers to make this kind of decision. Honestly, my goal is to at least get the websockets working on Zephyr as part of this effort. If we move the the struct definitions and APIs to from jerryscript-port to jerryscript-debugger.h, would that work for you?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this port specific definition part of the public API?
The definition should NOT be part of the public API, only the forward declaration should be. That way there is no dependency. (Until one actually want to use the debugger feature, in which case one has to pick an implementation which would contain the definition of the struct as well.)
I think the first question is: do we want multiple transport layer support
Yes, at least in our project we want to mix and match between 3 transports at run-time.
IMHO, a stronger reason for wanting a more dynamic transport interface as opposed to extending the "port surface" with static functions for the transport, is that static functions would make combining, composing and re-using transport implementations harder.
For example in our case, we want USB, BLE and a "pseudo transport" that just gathers the debug info (i.e. for printing human consumable exception traces). Imagine that someone already wrote each of these transports separately. Each transport would define the same set of functions/symbols. If I want to create a new transport that combines/chains/... them, I'd have to rename each function to avoid conflicts. Even if a simple "switch" between 2 transports would involve renaming each function.
@@ -17,6 +17,7 @@ | |||
#define JERRYSCRIPT_DEBUGGER_H | |||
|
|||
#include <stdbool.h> | |||
#include "jerryscript-port.h" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is definitely not good here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can change the init() to take a void*, thus avoid pulling in jerryscript-port.h
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or repeat the forward declaration?
Or move the forward declaration into an API header?
@zherczeg What's the best do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or repeat the forward declaration?
Or move the forward declaration into an API header and include that?
@zolWhat's the best do you think?
jerry-core/debugger/debugger-ws.c
Outdated
@@ -65,24 +60,20 @@ typedef struct | |||
* Close the socket connection to the client. | |||
*/ | |||
static void | |||
jerry_debugger_close_connection_tcp (bool log_error) /**< log error */ | |||
jerry_debugger_close_connection_transport (void) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a plan to log connection error in the future?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah, we can just change to all the handlers Apis to take the same log_error parameter
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like only this function has such an option.
jerry-core/debugger/debugger-ws.c
Outdated
if (!jerry_debugger_send_tcp ((const uint8_t *) text_p, strlen (text_p)) | ||
|| !jerry_debugger_send_tcp (request_buffer_p + sha1_length + 1, 27)) | ||
if (!jerry_debugger_send_transport ((const uint8_t *) text_p, strlen (text_p)) | ||
|| !jerry_debugger_send_transport (request_buffer_p + sha1_length + 1, 27)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is quite websocket specific. As far as I know websockets is available only through normal sockets.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i could leave it with just _tcp if it's ok, @martijnthe ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As far as I know websockets is available only through normal sockets.
At this point I'm regarding the websocket framing and handshaking to be part of the JrS debugger protocol as you yourself suggested a year ago: #1557 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It was a long time ago when I worked on this. But it seems a reasonable idea. Probably the handshake could be an optional feature.
*/ | ||
typedef struct jerry_debugger_transport_t jerry_debugger_transport_t; | ||
typedef struct jerry_debugger_config_t jerry_debugger_config_t; | ||
typedef struct jerry_debugger_conn_t jerry_debugger_conn_t; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need to forward declare these functions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
these needs to be forward declared because it's up to the jerry_port layers to provide the definitions for these, it will run into compile errors if debugger is not enabled.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The struct definitions itself is up to the implementation of the transport. Initially, @jimmy-huang had used void *
, but I suggested to use forward declarations. That way the compiler will still help with ensuring the correct types are used.
jerry-core/jcontext/jcontext.h
Outdated
uint8_t debugger_message_delay; /**< call receive message when reaches zero */ | ||
int debugger_connection; /**< holds the file descriptor of the socket communication */ | ||
jerry_debugger_transport_t *debugger_transport_p; /**< holds the pointer to the debugger transport */ | ||
jerry_debugger_conn_t *debugger_connection_p; /**< holds the pointer to the debugger connection */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to separate these two?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we can probably keep just the transport obj, and have the connection be part of the transport, @martijnthe ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I think so.
I don't see a use case for being able to distinguish the transport and the connection.
I guess in a multi-Jerry-context one could have multiple connections, sharing the same transport implementation. But one could work around that by just creating separate jerry_debugger_transport_t
s instead I guess.
It seems these types/functions are used by the debugger. If we want to stick to the websocket package format, perhaps the first step would be to move certain of these types into the main debugger, reducing the content of |
By the way the send / receive already working in different ways, since the send aware of the low-level websocket header format while it is stripped from the receive part. Perhaps the |
Re. keeping websocket packet framing: I'm OK with either, keeping it or moving it out. I do think that it's a tad bit wasteful / annoying if the transport itself also (inherently) takes care of packet framing (I'm thinking about BLE for example where every byte saved in the air is important). If we decide to move it out, then I agree, there's quite a bit of rework needed to separate the WS related stuff out and into the transport side of the fence. If we decide to keep WS and regard it as part of the JrS debug protocol, I'm not sure why it's needed to refactor the @jimmy-huang thoughts? @zherczeg So it sounds like a decision needs to be made about "websocket framing is part of JrS debug protocol": yes or no. Any other people that need to chime in to help make that decision? |
@martijnthe Yeah, personally, i rather keep the WS at least for now, because initially I think most of the platforms will just re-use the same WS protocol, and like you said, it's a lot of work for each port to add these WS. I think the WS and both the TCP socket can be consider as a transport, so for now if we just move the TCP logic into a tcp transport, then later we can then move the WS into a separate transport as well. But I am okay with either as well. |
I would first remove as much macros / types from debugger-ws.h as possible. Probably in separate PR-s first. JERRY_DEBUGGER_INIT_SEND_MESSAGE could go away easily (send should write this byte). The JERRY_DEBUGGER_SET_SEND_MESSAGE_SIZE_FROM_TYPE and |
@zherczeg where does jerry_debugger_send_header_t go? Should it be moved to debugger.h? or it's a internal thing that's in debugger-ws.c? I am guessing the former? |
@zherczeg I submitted another patch #2262 to remove some of these macros. Is the goal here to make the debugger-ws.c as the default transport and remove all the macros and structurs in debuggr-ws.h. What about these structs? jerry_debugger_send_header_t |
@jimmy-huang @zherczeg Alternative idea: instead of letting the transport specify the max header size, how about if the debugger instead would ask the transport for a pointer to a send buffer? The transport itself would take care of reserving space for a header. In case sending happens async, it could also save a copy to another buffer + lower the peak memory usage a little (perhaps not relevant though). Downside is that getting a buffer would become a function call instead of just getting a pointer + offset from the context, which in the simplest of projects would be at a fixed memory location. OTOH, there are only ~10 or so places, so in absolute .text size footprint it may be acceptable. I'm not feeling strongly about either, but wanted to share the idea. |
2609b6a
to
624541c
Compare
42f9a36
to
094dd10
Compare
@jimmy-huang I would not use .something fields. Simply create some functions such as |
I'm not sure if I understand. This PR removes the jcontext.h dependency from debugger-ws.c already? Would you like to see this PR split in 2 separate ones? If so, what should be left out? Please clarify. Thank you. |
d3ed618
to
6d3658c
Compare
@zherczeg @martijnthe Updated with jerry_debugger_set_transmit_sizes function |
6d3658c
to
d5a5985
Compare
@zherczeg Can take take a another look at the patch? |
2aa9821
to
bf2c3da
Compare
@zherczeg can you take a look at this patch to see if there's something else I should fix? |
The problem is not with the patch. The problem lies in nicely restructuring the debugger interface. I spent some hours to decouple the ws part from jcontext, but I was not satisfied with the results. It worked, but the code was ugly. Unfortunately this must be the first step. So this is the challenge here. If you have time, you can try it. |
JerryScript-DCO-1.0-Signed-off-by: Jimmy Huang jimmy.huang@intel.com
This patch adds the transport interfaces for the debugger and allows it to be ported to different platforms. JerryScript-DCO-1.0-Signed-off-by: Jimmy Huang jimmy.huang@intel.com
Move all generic debugger logic into debugger.c and remove as many calls to internal APIs as possible. JerryScript-DCO-1.0-Signed-off-by: Jimmy Huang jimmy.huang@intel.com
JerryScript-DCO-1.0-Signed-off-by: Jimmy Huang jimmy.huang@intel.com
JerryScript-DCO-1.0-Signed-off-by: Jimmy Huang jimmy.huang@intel.com
@zherczeg Ok, that's essentially what I am doing with one of the commits, so I separate it out as a separate patch, maybe this is what you are looking? |
Not exactly. I would like to be able to chain this at the top of other interfaces, e.g. tcp. Maybe move it to some jerry extension or port. I am not sure. |
Sorry, I don't quite understand, maybe if you can show me some of the work you are doing? I guess maybe it makes more sense for you to actually do the refactoring, since you guys are more familiar with the APIs? I'd be happy to help but our resource is kind limited so I am not able to put too much time into this, but I would at least want to get to a state where at least someone can easily port this interface to Zephyr or other platforms. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for chiming in late to this conversation but something doesn't feel right about the forward-declared types in the debugger API and/or the port API. I may not see the whole picture but I don't really grasp how a forward declared-only struct type would work with the ports. If the ports don't see the whole type, why do they need it at all? If the type is internal to the debugger implementation, why shall it be published? Any clarification is appreciated.
@@ -38,18 +38,12 @@ typedef struct | |||
/** | |||
* Byte data for evaluating expressions and receiving client source. | |||
*/ | |||
typedef struct | |||
typedef struct jerry_debugger_uint8_data_t |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change seems to be unjustified. Current code base adds names to those structs only that are recursive. Here, it's not the case.
*/ | ||
|
||
/* forward declaration */ | ||
struct jerry_debugger_uint8_data_t; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This forward declaration also seems to be unnecessary, especially in this public header. The type is only used in sources internal to jerry-core. If I see it right.
@zherczeg @akosthekiss Thanks for the feedback, been out for a while and just got back from vacation. We actually have a work-around building debugger on top of Zephyr's native posix network stack, since they are almost identical. So in our project, we have this custom header that get injected when debugger is turned on. I can probably submit a patch to the zephyr target as well, is this something you guys want? Since at this point, we don't need to support other transports besides tcp. This will not have any changes on the API too. https://github.com/intel/zephyr.js/blob/master/src/jerry-port/arpa/inet.h |
Unfortunately I have no time to work on this at the moment. The whole debugger interface needs a major redesign to satisfy all requirements. I would recommend to use your workaround until we can do this. |
https://github.com/zherczeg/jerryscript/tree/debugger_transport Something to start with. It is far from compiling though. This will take a long time to finish. |
Please check #2421 |
Hi @jimmy-huang, after #2426 lands you can freely add your low-level transmission protocols to JerryScript. Please check the patch if you have time. |
@zherczeg Yeah, thanks for the heads up, I am actually ramping up on another project atm, but I'll try to look at this later when I have time, if you think i should close this PR first, I am okay with it and I'll re-submit a pr later. |
closing |
This is patch to tackle issue #2229, basically we want the ability to have debugger support different platforms and different transports.
The current debugger implementation only supports BSD sockets, and it's not portable to different OSes that have different platform APIs. Besides, other platforms may want to use another transport layer like USB or Bluetooth instead of TCP sockets. The goal is to create a generic transport interface that different platforms can be used, and provide the actual implementation for running on their devices. This patch provides a struct that defines the function signature that each transport can be used to register their own implementation of the socket APIs using the jerry_port_xxx interface and a Zephyr port is provided as a sample.
This is a WIP patch and a reference implementation, but I would like use this as a starting point for discussion on how to improve the APIs from here, would like to have feedbacks from @zherczeg and @martijnthe and JerryScript team after reviewing them.