New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request: state of web development #105

Open
vindarel opened this Issue Jul 28, 2017 · 28 comments

Comments

Projects
None yet
6 participants
@vindarel
Contributor

vindarel commented Jul 28, 2017

For this I'd personally use https://stackoverflow.com/a/42838145/1506338 (and I'm very eager in writing it).

There would be examples, what's lacking, how to quickstart everything, references to libraries found in the Awesome list.

@PuercoPop

This comment has been minimized.

PuercoPop commented Jul 29, 2017

That SO post have several omissions. For example just form generators there is:

Mariano also has clavier for validation. The ORMs list fails to mention CL-SQL. Fukamachi has written [mito] recently

I also don't understand why one would mention wu-wei but not hunchentoot.

Maybe it would be a good idea to tackle this by sections first?

@vindarel

This comment has been minimized.

Contributor

vindarel commented Jul 29, 2017

Thanks for the links ! They were hard to find. Just found also 1forms

Django-like web (html) forms for Common Lisp, It can be used with various Common Lisp web servers and web frameworks such as caveman2. 1forms uses CLOS and is extensible by design.

but not touched for 2 years.

Sure this post has omissions but

  • it is not meant to be exhaustive, this task is left to the Awesome CL list and the State of CL ecosystem 2015 (both links given in the intro)
  • in my biased opinion (I'm the author there) it's the only place I know one can have an overview. Now we could do this here.

(it does mention Hunchentoot but puts emphasize on Clack)

Maybe it would be a good idea to tackle this by sections first?

Ok. And shall we make up a list/our TOC first ?

@can3p

This comment has been minimized.

Contributor

can3p commented Jul 29, 2017

I think Hunchentoot is in much better place in terms of documentation. If clack is put in the cookbook it needs a really extensive tutorial there

@PuercoPop

This comment has been minimized.

PuercoPop commented Jul 29, 2017

The subsections/divisions could be

  • Frameworks and Web Application Servers (Including websockets)
  • Serialization/data storage (SQL and NoSQL?)
  • Validation and form generation
  • Template libraries
  • Misc (Job queues, oauth, jwt, etc)

Do y'all think its ok? Have I missing an area?

@vindarel

This comment has been minimized.

Contributor

vindarel commented Jul 30, 2017

+1 for the division. You didn't note javascript integration, but ok to do this last. Also we can suppose we'll have a real databases page, so we would speak here only about integrating a DB with a web framework.

Those are some points I'd like addressed (what I've been researching on this topic and what blocked me while trying Caveman and Lucerne).

First of all, what application shall we write as an example throughout the page ? (as in the web scraping page)

If we present the application servers, what level of documentation do
we put ? I'd say just enough to get things done, and redirect to the project's documentation.

  • Web Frameworks

    • application models, database tables, web forms generation ("do we
      have to write 3 times our models ?") (this question the twitter clone does not answer) Looks like Caveman has models with the Datafly lib
    • how to access request parameters ? (ok)
    • and url parameters ? (not obvious nor in
      Caveman (very unclear) nor in Lucern, had to ask and have rely on Clack). (edit) clear in Snooze.
    • session & cookies support
    • templates engines
      • all the different kinds…
      • spinneret VS cl-who,…
  • Data storage (according we have a DB page), and here say how frameworks
    integrate the DBs. We would test everything, show bits of code and
    warn about limitations.

    • migrations: does Caveman have DB migrations ? The Crane ORM
      advertises to have, but they don't work with SQLite (issue), and they have other issues.
    • SQL, NoSQL… and bknr.datastore ? I'd like to try that :D

Important stuff:

  • deployment

    • creating an executable (an image) for a web app (doable but with a
      trick, see SO)
    • regular technics, examples (launching a swank server at the app startup ?)
    • Ceramic for a portable multiplatform app (Electron)
    • Caveman advertises hot reloading (with a Perl module). How well
      does it work ? Examples. Built in Hunchentoot ?
  • testing

    • how is it ? Lucerne advertises clack-errors (edit: snooze can either print stacktraces in the browser, let the debugger handle them or display a customed 404 page).
    • How can we test code with a database, does the framework create a test database ? (as in django)

Then, again later:

  • JavaScript integration
    • basics of Parenscript
    • interactive with trident-mode (Skewer) ?
    • ajax POCs (even if not production ready)
    • Panic (React) POC ?

shall we only point to the interesting projects for this JS part, with no code sample ? (like the Awesome CL list) Because all of this is unstable and changes frequently.

At the same time:

  • see also…
    • we would advertise real world examples: Quickdocs, Turtl, Potato,… not really the
      goal of a cookbook, but not either the place of the Awesome
      CL. Also, no need to be exhaustive.
@vindarel

This comment has been minimized.

Contributor

vindarel commented Aug 24, 2017

@Shinmera

This comment has been minimized.

Shinmera commented Aug 24, 2017

I'd appreciate it if you could also investigate, or in the very least mention, Radiance.

@pnathan

This comment has been minimized.

Member

pnathan commented Aug 28, 2017

The Lisp web framework world has forked and reworked itself a lot in the past half decade. I'm not wholly clear on the reasons, myself. I think we should exclude any web framework that isn't relatively stable at a 1.0+ level, for the reasons of sanity... if someone wants to do the indepth review/analysis of different web frameworks, they won't go to a Cookbook.

@Shinmera

This comment has been minimized.

Shinmera commented Aug 28, 2017

I'd also make sure that the framework has a couple of example applications already out there and has a sizeable amount of documentation to offer as well. I've seen all too many people frustrated in #lisp due to the lack of proper documentation or reference material.

@vindarel

This comment has been minimized.

Contributor

vindarel commented Nov 8, 2017

Greate example with ningle, closure-template, hot deploy with connection to a swank server, supervisor, jquery and pjax, watchdog autoreload: https://github.com/tarballs-are-good/quickutil/blob/master/quickutil-server/

@azzamsa

This comment has been minimized.

azzamsa commented Nov 14, 2017

I hope there will someone write tutorial about web development in CL,

I felt what Shinmera said

"I've seen all too many people frustrated in #lisp due to the lack of proper documentation or reference material."

As a beginner, I spent days and hours just to figure it out simple things, compared to other languages that even beginner can have so many tutorial and screencast.

It is too hard just to learn basic web development because the lack or resources. the things I see when I stuck

I hope there will be nice tutorial for web development in CL. as I seen many question asking about tutorials.

many people say learning emacs is difficult, but I think it's harder to learn web development with CL, for non-web development there are a lot of books about CL (let over lambda, on lisp, PCL, Successful lisp, Gentle Introduction to Symbolic Computation). But we do not have any completed books of web development in CL.

I hope there will be the book as soon as possible.

I hope I can write tutorials about web dev in CL when I finished learning it.

Thank you so much for everyone contributing to this Cookbook, thank you.

@azzamsa azzamsa referenced this issue Nov 14, 2017

Open

Tutorial #21

@pnathan

This comment has been minimized.

Member

pnathan commented Nov 14, 2017

I have asked the maintainer of Quickutils if he minds us using the system. Although it is MIT license or so, I'd prefer to be polite (and he's a nice guy, I've met him).

@pnathan

This comment has been minimized.

Member

pnathan commented Nov 14, 2017

@azzamsa http://www.adamtornhill.com/articles/lispweb.htm is a book about Lisp for the web. I don't know how current it is though.

@Shinmera - how stable IS the Lisp web world these days? have we all come to a stable web world or are we still in the forking/rewriting stage? when I've farted with Lisp web stuff in the last 2 years, I've just used raw hunchentoot because, well, it wasn't fancy enough to matter.

@pnathan

This comment has been minimized.

Member

pnathan commented Nov 14, 2017

( @vindarel I think this is the area the Cookbook should focus on over the next two months or so. I have vacations coming up, so I should have time to help out here, but I don't have deep knowledge of web frameworks and what to expect - I mostly don't use them in general)

@azzamsa

This comment has been minimized.

azzamsa commented Nov 14, 2017

@pnathan Thank you so much for the response, I've known you long before I learned lisp, I visited your blog several times. I know you from your answers on Stackoverflow. Thank you very much.

Yes, lispweb by Adam is the first book I know of Lisp web Development, but I think it does not suit me because I take us immediately to CLOS.

I also have contacted the author of Fullstacklisp and he said that he will continue working on it soon

If I finished learning web dev in CL, I will write my story about it, challenging and giving a little pain :)

@pnathan

This comment has been minimized.

Member

pnathan commented Nov 14, 2017

The author of Quickutils says:

Great idea! We are happy about that.

So let's see what we can do to port it into a Cookbook tutorial

@Shinmera

This comment has been minimized.

Shinmera commented Nov 14, 2017

@pnathan Radiance has been at 1.0 for a while now. It comes with full documentation, a bunch of pre-written applications to look at, and an in-depth introductory tutorial. It is pretty stable and I haven't done anything to it aside from bugfixes since release, nor do I intend to do anything that would cause compatibility breakage. I don't know about the other frameworks available, as I don't use them.

@azzamsa

This comment has been minimized.

azzamsa commented Nov 14, 2017

@Shinmera nice to meet you :)

pre-written applications: https://github.com/orgs/Shirakumo/teams/radiance/repositories give me 404

@Shinmera

This comment has been minimized.

Shinmera commented Nov 14, 2017

Bah, I guess that page is private to team members. This should work instead: https://github.com/Shirakumo?utf8=✓&q=radiance&type=&language=

@azzamsa

This comment has been minimized.

azzamsa commented Nov 14, 2017

why I just found this, Web search engine does not tell me about it . I need 2 days to find out my error using external css file in restas, and it solved yesterday when I use caveman.

Radiance has complete tutorials.

@vindarel

This comment has been minimized.

Contributor

vindarel commented Nov 14, 2017

@vindarel I think this is the area the Cookbook should focus on over the next two months or so.

Ok ! I'm still taking notes on the matter (here, just a TOC and deployment examples). I began a todobackend in Ningle (here, very basic) but it's a difficulty after another (changing headers… now I can't even run a lisp on my VPS. SBCL complains about "ensure_space" and I can't use another lisp… (edit: fixed).

Given the two pre-requisites from above, that I agree with, it didn't seem doable just yet:

I think we should exclude any web framework that isn't relatively stable at a 1.0+ level [pnathan]

that leaves… Caveman, Radiance, Ningle ? But Clack's doc is lacking.

I'd also make sure that the framework has a couple of example applications already out there and has a sizeable amount of documentation to offer as well.

that leaves… nothing, except Radiance ? I know 1 example for Caveman or Ningle. Would appreciate interesting examples (more complete than a hello world).

I think Caveman's README is not bad. We could do something similar for the Cookbook but there is no much added value. I'd like to add information: testing, deployment, DB migrations (if possible),… or maybe just to show a complete simple project is already helpful.

My pb with Radiance is that I don't fully understand where it lies, if it is more a publication framework (CMS) or a full-featured web framework (Flask/Django-like). Does it help in defining DB models, in writing forms, in writing migrations, in giving a testing environment, etc ? It also announces limitations:

Data types are also restricted to integers, floats, and strings. Despite these constraints, the database interface is sufficiently useful for most applications.

and the terminology is different, so it's a more difficult than others to comprehend.
And again, do we have to stick with cl-who ? Can we easily work with SQLite or Postgres ? etc

So let's see what we can do to port it into a Cookbook tutorial [about Quickutil]

Did you think about re-doing Quickutil in the Cookbook ? Not fully satisfactory to me because it doesn't use an ORM but raw SQL. It also uses closure-templates, that is maybe good but unusual for templates, I didn't encounter them before.
If it is about to base our example on Ningle: why not. But its doc doesn't introduce templates and DBs, so we'll have to find our way.
Shall we do a todobackend implementation ? (a todo-app)

Lastly, @pnathan, if you prefer to focus on other areas when you have time, that would still be great (error handling for example :D edit: PR of mine). I know you'll still help here in defining goals and proofreading :)


I didn't link to a new Weblocks branch (too alpha) but FYI its new Quickstart is pretty great: http://40ants.com/weblocks/quickstart.html simple and does backend and ajax-based clientside without JS.

@azzamsa

This comment has been minimized.

azzamsa commented Nov 14, 2017

maybe this help even little bit, https://www.slideshare.net/fukamachi/about-clack

@Shinmera

This comment has been minimized.

Shinmera commented Nov 15, 2017

if it is more a publication framework (CMS) or a full-featured web framework (Flask/Django-like).

It is both and neither at the same time, depending on how you use it.

Does it help in defining DB models

I don't know what you mean by this, but I assume the answer is no.

in writing forms

No, but iirc there are libraries for this.

in writing migrations

No.

in giving a testing environment

Again, not sure what you mean by this, but the answer is probably no.

It also announces limitations:

Radiance doesn't force you to do anything. You can use postmodern or whatever if the interfaces it provides are too restrictive.

And again, do we have to stick with cl-who

No, and I don't understand where you could have even got the idea from.

Can we easily work with SQLite or Postgres

Yes.

Radiance is different from most frameworks out there because it tries to do different things. I suggest you read some more about it.

@azzamsa

This comment has been minimized.

azzamsa commented Dec 28, 2017

@vindarel Sorry for asking, but what is the status for this (tutorial for web development).

I think we have to choose one of the web framework out there, then we make the tutorial or complete the documentation, for now we really need a good documentation for web app with CL.

After finishing with one framework, we can add other framework too.

@azzamsa

This comment has been minimized.

azzamsa commented Dec 28, 2017

This syllabus can give us some information, what we need to include in tutorial.

@Shinmera

This comment has been minimized.

Shinmera commented Dec 28, 2017

Radiance already has a lengthy tutorial out there if that's all you want. It isn't an introduction to web development, though, and thus doesn't cover JS, CSS, and HTML.

@azzamsa

This comment has been minimized.

azzamsa commented Dec 28, 2017

@Shinmera wow, that's cool, I think it's better if you put it in ReadtheDocs

@vindarel

This comment has been minimized.

Contributor

vindarel commented Dec 28, 2017

@azzamsa Given our prerequisites (stable version, real world examples), I'm not 100% keen on writing a tutorial here right now because there is no one method, I want to explore more possibilities (Radiance, Weblocks/Reblocks) and I don't want to write about Caveman or Ningle that I found awkward . So, as for web frameworks, we might need to rely on their own documentation. Thus kuddos to Radiance's documentation effort.

I carried on my efforts on the cookbook though, with two PRs, on condition handling and building executables, also for web apps, so it fills a gap for that tutorial. We also need a DB section (I tried Mito successfully and found Crane unfinished and buggy, and I see Radiance includes its own ubiquitous-based bindings).

Lately, as I wrote you I pretty liked Snooze (by Sly and Yasnippet author). Specifying and accessing arguments is straightforward, it deals nicely with errors (interactive debugger or print the stacktrace in the browser or print a custom 404), and its author responded. I take notes on lisp-journey/web-dev.

Now you have several options and you'll have to make a choice…

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment