Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement flash.net.Socket (google code Issue 10) #49

Closed
zwetan opened this issue Dec 31, 2015 · 10 comments

Comments

Projects
None yet
2 participants
@zwetan
Copy link
Member

commented Dec 31, 2015

https://code.google.com/p/redtamarin/issues/detail?id=10


mimic the APi of flash.net.Socket
without the security limitation

eg> being able to connect anywhere, any ports, etc.


@ajwfrost

This comment has been minimized.

Copy link

commented Mar 17, 2016

Hi

This is what I've been working towards, and I wanted to quickly discuss the approach here. I've got some code that I've been executing in a Worker which will allow me to create and use a Socket object pretty much the same way that a normal (Flash/AIR) client would. There are a few pain points though:

  1. The Worker code is using a Timer (as is the Socket object) so that they can talk to each other. It works but it's not massively efficient. Communication is via the shared properties between the background worker and the primordial one.

  2. Currently the .abc for the Worker is embedded in my SWF (within the Socket class) using the Embed tag. I'm not entirely sure how I could make this work if the Socket class was then actually part of redtamarin itself (currently my application includes the Socket class directly so it's my SWF that ends up with the worker code embedded in it, which works fine). Potentially I could take the .abc file and turn it into a static bytearray definition but this is pretty laborious!)

  3. There are some properties which are meant to be read-only according to the Flash API but when using just ActionScript, it's impossible to implement these without having a writeable property on the class as well.

  4. It's a pain having to write all of the class definitions again! Minor point but copying stuff from the API documentation is fairly slow...

Anyway: I've got something working, using just AS3 code as you'd suggested rather than any native/C++ changes. The one hack is that I needed the Windows socket to be non-blocking.

However: my preference at this point would actually be to re-implement support for Sockets (and SocketServer) using a lot of native (i.e. C++) code and pthreads, probably drawing upon a lot of your extended VMPI functions but also with some OS-specific code where necessary (non-blocking-ness!). Then it can be implemented along the same lines as Adobe has it (and I'm thinking of a tool that will generate all the AS3 wrapping code automatically using airglobal.swc...)

This would likely then need us to have the event processing loop also in C++ so that the communications between threads and the event handling/socket data dispatching can be handled neatly. This is quite a lot of departure from your concepts of keeping everything in ActionScript and just having a low-level C-style API available from ActionScript itself, so I wanted your thoughts on all this before doing anything further!

Thanks

@zwetan

This comment has been minimized.

Copy link
Member Author

commented Mar 18, 2016

oK few things that I don't follow

The Worker code is using a Timer (as is the Socket object) so that they can talk to each other

not sure why you're doing it this way, even if the Socket class depends on EventDispatcher you could first implement it using call back and ignore the events, and later take care of the events by redirecting the callbacks to an event manager of some sort.

Currently the .abc for the Worker is embedded in my SWF (within the Socket class) using the Embed tag.

Are you testing this in the Redtamarin runtime ?
"as is" the way you explain it there is no way this could work in Redtamarin
maybe gives more details about the setup?

There are some properties which are meant to be read-only according to the Flash API

That's a small implementation details, don't focus on that.
Usually what I do is use the AVM2 namespace for those "read-only" properties
eg.

public function get hello():String
{

}

AVM2 function set Hello( value:String ):void
{

}

I've got something working, using just AS3 code as you'd suggested rather than any native/C++ changes.

so you used the C API ?

The one hack is that I needed the Windows socket to be non-blocking.

Why ? it does not make any sense to have only the Windows socket non-blocking and keep the Linux / Mac OS X blocking.
But again without actually being able to review the implementation there is not much more I can say here.

my preference at this point would actually be to re-implement support for Sockets (and SocketServer) using a lot of native (i.e. C++) code and pthreads

but you already have the C API, why do you need to rewrite that in C++ ?

drawing upon a lot of your extended VMPI functions but also with some OS-specific code where necessary (non-blocking-ness!).

First, the VMPI implementation already take care of the OS specific
look at WIN32_SocketStart, any call to C API socket functions use WIN32_SocketStart to be sure Winsock is initialised, something you don't have to do with Linux / Mac OS X.

Second, I'm not sure why you're insisting so much on the non-blocking but you don't need non-blocking sockets to make it work under Windows.

And non-blocking vs blocking sockets is just a configuration, it's not accessible in the C API but not that hard to add.

This would likely then need us to have the event processing loop also in C++ so that the communications between threads and the event handling/socket data dispatching can be handled neatly.

nope

Your problem is you try to tackle something too big and 2 different problem in a context which I think you don't fully understand.

The "event loop" problem and "implementing Socket like Flash" problem are not connected, pick one of the two problems but don't work on both.

With the current Redtamarin 0.4.1 you can perfectly implement flash.net.Socket without the need to add any native C++ code.

see this gist for example

look into httplib / HttpConnection
for sending / receiving data

all done in AS3 using the Redtamarin C API, no need to use C++
not need to deal with events, you can use function callbacks
things like the timeout on connection can be added later
etc.

@zwetan

This comment has been minimized.

Copy link
Member Author

commented Mar 18, 2016

oh and provided you installed the redtamarin-sdk and as3shebang the gist above will work under Windows / Mac OS X / Linux

that's why I would advise to reuse the C API
it's low-level and extremely powerful "as if" you had written the code in C

@ajwfrost

This comment has been minimized.

Copy link

commented Mar 18, 2016

Hi

To clarify: I'm using non-blocking sockets on both Windows and Linux. It's just that on Linux, it's possible to use fcntl() and the O_NONBLOCK definition in order to make it non-blocking. This isn't implemented/available on Windows, so a higher level API is needed. Currently though I've just hacked the windows implementation of VMPI_fcntl to make it work.

I am testing in the redtamarin runtime, but one that's been updated to allow for SWF files that contain embedded binary data (still haven't got github quite set up enough yet to allow me to upload those changes..) - but other than that, yes it's all using the C API.

I'm looking at your Socket/HttpConnection examples and from what I can see, these are going to be using blocking socket calls for connect() and receive() - so basically if you want to receive some data, you need to call a receive() function. I'm not 100% sure whether the receive() function would return immediately if there was no data waiting to be picked up from the socket: if it didn't then it wouldn't be much use in terms of how an event-driven application would be put together. If it does, then in order to provide events, you'd need a timer going round and polling to see if there was data that could be sent out to your callback (or event listener).

So the aim here (for me) is to reproduce the flash.net.Socket implementation, in terms of how this is used by enterprise applications. Using a worker and polling in this way is possible but not very elegant and may start to cause a problem if we've got 100k connections that are sending/receiving data, so I wanted to avoid this sort of polling and pseudo-event based system.

If I were to implement the Socket with some native code (it will be nominally C++ rather than C but just because it's a ScriptObject-derived class...) then the implementation would be more elegant and wouldn't need the Worker. Looking ahead to the ServerSocket implementation, the same sorts of arguments apply (and I definitely don't want to have a Worker per socket!). So once this has gone down into the C/C++ layer, it would then need to have some mechanism that triggers the events from this layer which implies an event loop that either is written in the C++ code, or that calls down to C++ in order to dispatch any waiting events.

Basically a bit more like how Adobe did it :-)

I am quite happy if you can explain though, how would you use just the C API in order to create a Socket object which is going to sit there passively listening for data and will then call your callback when data has arrived? In a scaleable and platform-independent way? As mentioned, I can do this (using non-blocking sockets which needs a tweak for this to work on Windows) but I'm not entirely sure how to embed my Worker code into redtamarin itself, or how this might work when I have thousands of sockets that potentially all need to be polled.

BTW just re-reading your comments, the first one was why I don't use callbacks rather than Timers to communicate between the primordial worker and the background one? I didn't think it was possible to use callbacks (or event handlers) between different workers at the moment, so they are each communicating via shared properties and using Timers to poll for changes in those properties. Again this is just to try to make sure we remain asynchronous.

I'll uploal my AS code here in case that helps:

  1. program.as is the background worker which communicates with the C API to do all the socket stuff
  2. Socket.as which has enough implementation to connect and then to listen for data, if it receives it though then it's just tracing this out (line 208) rather than triggering an event or allowing access to the data via the normal API. Still work in progress..

thanks

Andrew

@ajwfrost

This comment has been minimized.

Copy link

commented Mar 18, 2016

program.txt
Socket.txt
Uploading as .txt files, just need renaming to .as

@zwetan

This comment has been minimized.

Copy link
Member Author

commented Mar 18, 2016

Look, I already posted numerous answers but you seem to not listen and do only whatever you want and completely ignoring my gentle advices

I personally have no problem with that but here what gonna happen if you keep doing that

  • I just gonna ignore your comments
  • and you will have to fork your own version of redtamarin
  • there you will be able to do whatever you want
    but you will be on your own

Without being rude you are lacking a couple of things

  • you do not seem to understand how contribution to an open source project work
    Redtamarin is setup in a very specific way that you could call a "vision" or direction
    and I'm basically the benevolent dictator enforcing that vision or direction
    if what you propose goes against that clear direction it goes against the interest of the project
    and so it will be ignored
  • you post numerous problems into one big pile of text and that generate way too complicated issues
    you gonna have to discipline yourself to post 1 problem per issue
    I already mentioned that to you numerous time, you ignore the advise
    so right now I see your issues as a waste of time
  • when someone take the time to address all your issues, you are ignoring the answers
    that is simply not acceptable
  • this project rely on source code
    you don't post source code so it's pretty hard to follow your purpose here

again, the following

[Embed(source = "../../SocketWorker/src/program.abc", mimeType = "application/octet-stream")]
private static var ByteClass : Class;

and

var ba : ByteArray = new ByteClass;
_worker = WorkerDomain.current.createWorker(ba);

will not compile using ASC, and even if you compile it with MXMLC the generated SWF will not run inside the Redtamarin runtime

it's simple, the Redtamarin know only to read doABC2 SWF tags and will not read a Binary SWF tag

the Worker and WorkerDomain class does not have the same API as in Flash/AIR

there is no way to call something like WorkerDomain.current.createWorker() from Redtamarin unless you change the runtime source code

so how are you compiling this ? how are you testing this with Redtamarin?
have you installed the redtamarin-sdk ?
are you compiling your code with redbean?

You gonna have to stop to put everything and anything into a single issue

more in details

I am testing in the redtamarin runtime, but one that's been updated to allow for SWF files that contain embedded binary data

so you are not testing with the redtamarin runtime
I have no access to your modifications, so I can not help you

again, updating the runtime to read something else than doABC2 SWF tags is not wanted

so basically if you want to receive some data, you need to call a receive() function. I'm not 100% sure whether the receive() function would return immediately if there was no data waiting to be picked up from the socket: if it didn't then it wouldn't be much use in terms of how an event-driven application would be put together.

you seem to not fully understand how socket programming work
please read something like Beej's Guide to Network Programming

the way socket are working is

  • you send data
  • you flush
  • then you read data

you don't need the socket to be non-blocking for this to work

so in a flash.net.Socket implementation you simply do it from the flush() method
as soon as it is called you simply wait on the blocking receive(), sure you could go non-blocking and loop on receive(), but that does not change a lot

what you're mentioning show a lack of understanding about sockets

see beej's guide about recv()

recv() returns the number of bytes actually read into the buffer, or -1 on error (with errno set, accordingly.)

Wait! recv() can return 0. This can mean only one thing: the remote side has closed the connection on you! A return value of 0 is recv()'s way of letting you know this has occurred.

with the code I provided earlier in the gist it's pretty simple
use 2 buffers

private var _outputBuffer:ByteArray
private var _inputBuffer:ByteArray

every call to "write" functions add data to the output buffer
and when flush() is called you simply use the send() native function
to send the data

after flush() is called read the data from recv() and add it to the input buffer
on each recv() call you can dispatch a progress event for example
all the bytearray "read" functions would read from that input buffer

so this part

If it does, then in order to provide events, you'd need a timer going round and polling to see if there was data that could be sent out to your callback (or event listener).

no you don't need a timer

So the aim here (for me) is to reproduce the flash.net.Socket implementation

read and understand the gist I posted earlier, the implementation can be done in about 30mn to 1h

If I were to implement the Socket with some native code (it will be nominally C++ rather than C but just because it's a ScriptObject-derived class...) then the implementation would be more elegant and wouldn't need the Worker.

at this point I don't care

please do a fork, name the project whatever you want and do whatever you want in C++
wether it is considered elegant or not I could not care less

You are not listening to what I'm saying, I'm done spending time writing those long comments

I am quite happy if you can explain though, how would you use just the C API in order to create a Socket object

read the freaking gist I have sent earlier, all is there

I'll uploal my AS code here in case that helps:

you have apparently modified the redtamarin runtime
without the runtime changes how am I supposed to run those AS sources?
I can't

this specifically will not work

var ba : ByteArray = new ByteClass;
_worker = WorkerDomain.current.createWorker(ba);

you are trying to change way too many things in a lot of different places, all at the same time, without listening, I can not spend more time dealign with this, I don't know, go see on other open source projects how it goes but so far you are more distraction than help.

@ajwfrost

This comment has been minimized.

Copy link

commented Mar 18, 2016

Hi

Fair enough: yes I have been aware that the direction that I'd been looking at for this isn't quite the approach that you have for redtamarin, hence posting comments/queries rather than the code changes that I'd done (plus the fact I'm still struggling with github to get the changes off my computer and back up to the repo fork).

In terms of what I've done, yes I've taken the source code from redtamarin and then added stuff so that those examples compile, it can cope with SWF files built with mxmlc that contain embedded data, and I tweaked the AS3 API for Worker so it was matching the Flash one.

For what I need sockets for, it's not a simple case of "send a request then wait for a response" which is all I think I could get working with your socket.as sample, and what you seem to be suggesting above. I need to have the socket able to receive data from time to time from a remote location without me having to send anything further, which is where the idea of polling comes from. But not ideal.

At this point I will agree with you - apologies for the distractions! but I wanted to see whether the sorts of changes I was looking to make would be of benefit to redtamarin: it's fair enough if this isn't where you want to take it so I'll follow your advice and do stuff separately.

Thanks for the responses anyway, and for all the effort you're putting in to the AS3 community at the moment!

cheers

Andrew

@zwetan

This comment has been minimized.

Copy link
Member Author

commented Mar 18, 2016

How To Contribute

respect the goal and direction of the project
don't try to convince us to go into another direction
it will simply not work and we will show you the "fork" button.

@zwetan

This comment has been minimized.

Copy link
Member Author

commented Mar 18, 2016

and just one word about

For what I need sockets for, it's not a simple case of "send a request then wait for a response" which is all I think I could get working with your socket.as sample, and what you seem to be suggesting above. I need to have the socket able to receive data from time to time from a remote location without me having to send anything further, which is where the idea of polling comes from. But not ideal.

again learn about socket programming
you don't need the flash.net.Socket to do that
you don't need non-blocking sockets
you don't need workers
you don't need events either

you just need to understand how sockets work
because I can guarantee you it is a simple case

@zwetan zwetan referenced this issue Mar 23, 2016

Open

flash.net package #78

6 of 20 tasks complete
@zwetan

This comment has been minimized.

Copy link
Member Author

commented Mar 23, 2016

closing this issue as it became out of focus

@zwetan zwetan closed this Mar 23, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.