New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http server and http client in the standard library #910

Open
andrewrk opened this Issue Apr 11, 2018 · 14 comments

Comments

Projects
None yet
10 participants
@andrewrk
Member

andrewrk commented Apr 11, 2018

justification for putting them in the standard library: we want to use them for the package manager.

@ansarizafar

This comment has been minimized.

ansarizafar commented Apr 23, 2018

Check this
Lwan is an opensource high-performance & scalable web server https://lwan.ws/

@bheads

This comment has been minimized.

bheads commented May 7, 2018

I can understand putting in an http client for the package manager but an http server? Building a secure http server is a big project. And if its in the std lib people will use it and it will have to be supported. But an officially support http server reference project used by zig to host zig would be awesome, but I would avoid including it in the std lib.

I would also avoid including C projects directly in the std lib. One mistake in D was including Curl in the std lib, now they have to deal with Curl versioning and distributing curl libs with the application. Would be better having an official package for these kitchen sink items, then when we want to end support they can be forked off into the community (or left abandoned in git) without it affecting the std lib.

@costincaraivan

This comment has been minimized.

costincaraivan commented Jun 7, 2018

Agreed. Include a HTTP client and then everything else should come from community packages. If the community converges on a set of very popular, high quality libraries, then you pull those in and even then, for a language such as Zig you'd probably want them just "outside" of the stdlib, in set of libraries maintained by the language developers yet separate (a sort of recommended and supported libraries, but not necessarily with the same commitment for backwards compatibility as the stdlib, which generally is super strict).

The main reason is that you want to avoid the Python or Java situation where you have a ton of libraries which are either bad or completely obsolete but can't be removed.

@andrewrk

This comment has been minimized.

Member

andrewrk commented Jun 7, 2018

Right now the standard library exists for 2 reasons:

  • To test out the language and make sure language decisions are good
  • To eventually become the standard library we will ship with zig 1.0.0

So an HTTP server / client would certainly help for the first case. We can evaluate whether it belongs in std when we do the stdlib audit before 1.0.0.

@costincaraivan

This comment has been minimized.

costincaraivan commented Jun 7, 2018

Ah, ok. But don't forget that as more people start to use libraries, the backlash caused by removing something, even before 1.0, increases. I'd at least mark things clearly as "experimental" or such to clarify a bit, to minimize outcry.

@renatoathaydes

This comment has been minimized.

renatoathaydes commented Jun 7, 2018

I would suggest adding to the standard library a low-level http library based on RFC-7230, which defined simply the HTTP message format, basically, not semantics (e.g. meaning of HTTP verbs, headers etc. which is defined in other RFCs). This is simple and can be the foundation of higher level libraries, including http clients and servers, while still being useful on its own to accomplish the goals mentioned by @andrewrk.

I wrote a low-level HTTP library in Java (which I am thinking of translating to zig for the async support and speed) and it's not a lot of work. Higher level APIs to work with HTTP are hard to get right and most people have different preferences on what they should look like (which probably explains most languages having several different options - which is probably a good thing), hence those are better left to third-party libraries, IMHO.

@ziglang ziglang deleted a comment from cup Jun 7, 2018

@binary132

This comment has been minimized.

binary132 commented Jun 18, 2018

HTTP/2?


Separately,

This may be an unpopular position, but I have noticed that Go suffers from very weak a) websocket, b) terminal, and c) GUI packages. I realize there is a vast amount of work represented in that short sentence, but there was also an enormous amount of community interest in it, and a likewise enormous amount of community fragmentation around different half-baked solutions to these problems.

Rust has a similar fragmented set of community solutions. Everyone wants to write a parser, everyone wants to do their own async i/o, etc.

Perhaps the way to do this is to support a canonical set of packages, sort of a second-tier set of official repos that are still outside of the stdlib, but that will always be the go-to for Zig for that problem. Thinking out loud here.

@Wulfklaue

This comment has been minimized.

Wulfklaue commented Jul 7, 2018

If it helps - here are some languages that have HTTP either in core or standard:

https://crystal-lang.org/

Has default HTTP server build in.

https://dlang.org/

Relies on external Vibe.D for HTTP support.

Rust has a similar fragmented set of community solutions. Everyone wants to write a parser, everyone wants to do their own async i/o, etc.

Its a issue that all Languages share. Any language that has a package manager tends to fragment the community because everybody wants "their" code to be popular or they disagree with the authors of other packages or ...

Look at Go ... Sometimes 20 packages that do the same thing, just in different ways, different supports ( not counting the clones with minimal changes ). Npm ... same.

Languages need to have a tighter controle over there eco system. No Raft Protocol package? Somebody can write one, that follows the language library standard. That becomes a official standard library package.

Somebody want to improve it? Commit to the official version.
Its not maintained anymore... Allow for a new version "2.0" that is backwards compatible.

Keep tight control over the resources. What makes PHP so easy? Not just the language but also the fact that all official extensions are fast, secure and well documented. Its in general the stuff outside the standard library that has issues over time. Hell, you can use 10 year old library code and it will mostly still work.

Unlike languages that rely a lot on external 3th party plugins, where your old code need to use unmaintained code or it does not work or ... and then requires rewrites or whatever issues.

@ansarizafar

This comment has been minimized.

ansarizafar commented Aug 8, 2018

Is there any progress on this issue?

@andrewrk

This comment has been minimized.

Member

andrewrk commented Aug 8, 2018

Yep, lots of progress:

  • cancel semantics for coroutines are now sound and thread-safe
  • the stage2 compiler is now a successful proof-of-concept of zig's async/await syntax integrating perfectly into linux's epoll, macos's kqueue, and windows's IOCP.
  • we now have these building blocks:
    • std.event.Channel
    • std.event.Lock
    • std.event.RwLock
    • std.event.Future
    • std.event.Group
  • in implementing the file system watcher, I discovered a much better API for a tcp server, using a Channel for incoming sockets rather than an async fn handler. This solves the 1 catch @panic("TODO") that I had in the status quo tcp server implementation.
  • I'm working on #1294 which is a nearly complete PR for async/await file system API
  • @kristate is working on #1331 which implements some async/await networking API
@ansarizafar

This comment has been minimized.

ansarizafar commented Aug 8, 2018

Thanks for the update

@kristate

This comment has been minimized.

Contributor

kristate commented Aug 8, 2018

Once #1271 lands, first it will be great to get some HTTP action. We could even be faster to market than the rust guys with coroutine async.

@u007

This comment has been minimized.

u007 commented Sep 20, 2018

is context in place yet? i think context is important to allow cancellation or io interruption during call to make http cancellable. go context is a good example. i think it should be part of the std http package as well

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment