Form data goes to `request.params`, `request.formData` is empty and crashing on iteration

[GoldsteinE]

I suppose that when data sent to server using POST, it shoud go to formData, not to params. Also crashing when trying to iterate is weird.

I use #head nim and jester 0.4.1

Code:

import jester, asyncdispatch

routes:
  post "/":
    echo request.params
    echo request.formData.len
    for name, value in request.formData.pairs:
      echo name
      echo value

Request:

$ curl -v -XPOST -d'test=boom' http://127.0.0.1:5000/
Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 5000 (#0)
> POST / HTTP/1.1
> Host: 127.0.0.1:5000
> User-Agent: curl/7.47.0
> Accept: */*
> Content-Length: 9
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 9 out of 9 bytes
< HTTP/1.1 502 Bad Gateway
< Content-Length: 5649
< Server: HttpBeast
< Date: Sat, 08 Jun 2019 12:22:23 GMT
< Content-Type: text/html;charset=utf-8
<
(traceback goes here)

Compilation/run log:

Hint: used config file '/home/agent/.choosenim/toolchains/nim-#head/config/nim.cfg' [Conf]
Hint: used config file '/home/agent/.choosenim/toolchains/nim-#head/config/config.nims' [Conf]
Hint: system [Processing]
Hint: widestrs [Processing]
Hint: io [Processing]
Hint: test [Processing]
Hint: jester [Processing]
Hint: net [Processing]
Hint: nativesockets [Processing]
Hint: os [Processing]
Hint: strutils [Processing]
Hint: parseutils [Processing]
Hint: math [Processing]
Hint: bitops [Processing]
Hint: macros [Processing]
Hint: algorithm [Processing]
Hint: unicode [Processing]
Hint: pathnorm [Processing]
Hint: osseps [Processing]
Hint: posix [Processing]
Hint: times [Processing]
Hint: options [Processing]
Hint: typetraits [Processing]
Hint: sets [Processing]
Hint: hashes [Processing]
Hint: strtabs [Processing]
Hint: re [Processing]
Hint: pcre [Processing]
Hint: rtarrays [Processing]
Hint: tables [Processing]
Hint: uri [Processing]
Hint: mimetypes [Processing]
Hint: asyncnet [Processing]
Hint: asyncdispatch [Processing]
Hint: heapqueue [Processing]
Hint: lists [Processing]
Hint: asyncstreams [Processing]
Hint: asyncfutures [Processing]
Hint: deques [Processing]
Hint: cstrutils [Processing]
Hint: selectors [Processing]
Hint: epoll [Processing]
Hint: md5 [Processing]
Hint: logging [Processing]
Hint: httpcore [Processing]
Hint: asyncfile [Processing]
Hint: macrocache [Processing]
Hint: json [Processing]
Hint: lexbase [Processing]
Hint: streams [Processing]
Hint: parsejson [Processing]
Hint: strformat [Processing]
Hint: errorpages [Processing]
Hint: htmlgen [Processing]
Hint: utils [Processing]
Hint: cgi [Processing]
Hint: cookies [Processing]
Hint: request [Processing]
Hint: httpbeast [Processing]
Hint: future [Processing]
Hint: osproc [Processing]
Hint: cpuinfo [Processing]
Hint: linux [Processing]
Hint: parser [Processing]
Hint: patterns [Processing]
/home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(1273, 9) Hint: Asynchronous route: match. [User]
CC: stdlib_assertions.nim
CC: stdlib_dollars.nim
CC: stdlib_io.nim
CC: stdlib_system.nim
CC: stdlib_parseutils.nim
CC: stdlib_math.nim
CC: stdlib_strutils.nim
CC: stdlib_pathnorm.nim
CC: stdlib_options.nim
CC: stdlib_times.nim
CC: stdlib_os.nim
CC: stdlib_nativesockets.nim
CC: stdlib_hashes.nim
CC: stdlib_net.nim
CC: stdlib_strtabs.nim
CC: stdlib_tables.nim
CC: stdlib_uri.nim
CC: stdlib_mimetypes.nim
CC: stdlib_heapqueue.nim
CC: stdlib_deques.nim
CC: stdlib_cstrutils.nim
CC: stdlib_asyncfutures.nim
CC: stdlib_asyncstreams.nim
CC: stdlib_selectors.nim
CC: stdlib_asyncdispatch.nim
CC: stdlib_md5.nim
CC: stdlib_logging.nim
CC: stdlib_httpcore.nim
CC: stdlib_asyncfile.nim
CC: ../../../../../../home/agent/.nimble/pkgs/jester-0.4.1/jester/private/errorpages.nim
CC: stdlib_cgi.nim
CC: ../../../../../../home/agent/.nimble/pkgs/jester-0.4.1/jester/private/utils.nim
CC: stdlib_future.nim
CC: ../../../../../../home/agent/.nimble/pkgs/httpbeast-0.2.2/httpbeast/parser.nim
CC: ../../../../../../home/agent/.nimble/pkgs/httpbeast-0.2.2/httpbeast.nim
CC: ../../../../../../home/agent/.nimble/pkgs/jester-0.4.1/jester/request.nim
CC: ../../../../../../home/agent/.nimble/pkgs/jester-0.4.1/jester.nim
CC: test.nim
Hint:  [Link]
Hint: operation successful (80370 lines compiled; 5.875 sec total; 112.074MiB peakmem; Debug Build) [SuccessX]
Hint: /mnt/c/Users/golds/Desktop/nbin/test  [Exec]
INFO Jester is making jokes at http://0.0.0.0:5000
Starting 1 threads
^CSIGINT: Interrupted by Ctrl-C.
Traceback (most recent call last)
/mnt/c/Users/golds/Desktop/nbin/test.nim(3) test
/home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(482) serve
/home/agent/.nimble/pkgs/httpbeast-0.2.2/httpbeast.nim(423) run
/home/agent/.nimble/pkgs/httpbeast-0.2.2/httpbeast.nim(286) eventLoop
/home/agent/.choosenim/toolchains/nim-#head/lib/pure/ioselects/ioselectors_epoll.nim(383) selectInto
SIGINT: Interrupted by Ctrl-C.
Error: execution of an external program failed: '/mnt/c/Users/golds/Desktop/nbin/test '
[1] agent@DESKTOP-RGR7528 /mnt/c/Users/golds/Desktop/nbin [15:18] $ nim c -r test.nim
Hint: used config file '/home/agent/.choosenim/toolchains/nim-#head/config/nim.cfg' [Conf]
Hint: used config file '/home/agent/.choosenim/toolchains/nim-#head/config/config.nims' [Conf]
Hint: system [Processing]
Hint: widestrs [Processing]
Hint: io [Processing]
Hint: test [Processing]
Hint: jester [Processing]
Hint: net [Processing]
Hint: nativesockets [Processing]
Hint: os [Processing]
Hint: strutils [Processing]
Hint: parseutils [Processing]
Hint: math [Processing]
Hint: bitops [Processing]
Hint: macros [Processing]
Hint: algorithm [Processing]
Hint: unicode [Processing]
Hint: pathnorm [Processing]
Hint: osseps [Processing]
Hint: posix [Processing]
Hint: times [Processing]
Hint: options [Processing]
Hint: typetraits [Processing]
Hint: sets [Processing]
Hint: hashes [Processing]
Hint: strtabs [Processing]
Hint: re [Processing]
Hint: pcre [Processing]
Hint: rtarrays [Processing]
Hint: tables [Processing]
Hint: uri [Processing]
Hint: mimetypes [Processing]
Hint: asyncnet [Processing]
Hint: asyncdispatch [Processing]
Hint: heapqueue [Processing]
Hint: lists [Processing]
Hint: asyncstreams [Processing]
Hint: asyncfutures [Processing]
Hint: deques [Processing]
Hint: cstrutils [Processing]
Hint: selectors [Processing]
Hint: epoll [Processing]
Hint: md5 [Processing]
Hint: logging [Processing]
Hint: httpcore [Processing]
Hint: asyncfile [Processing]
Hint: macrocache [Processing]
Hint: json [Processing]
Hint: lexbase [Processing]
Hint: streams [Processing]
Hint: parsejson [Processing]
Hint: strformat [Processing]
Hint: errorpages [Processing]
Hint: htmlgen [Processing]
Hint: utils [Processing]
Hint: cgi [Processing]
Hint: cookies [Processing]
Hint: request [Processing]
Hint: httpbeast [Processing]
Hint: future [Processing]
Hint: osproc [Processing]
Hint: cpuinfo [Processing]
Hint: linux [Processing]
Hint: parser [Processing]
Hint: patterns [Processing]
/home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(1273, 9) Hint: Asynchronous route: match. [User]
Hint:  [Link]
Hint: operation successful (80370 lines compiled; 3.096 sec total; 112.078MiB peakmem; Debug Build) [SuccessX]
Hint: /mnt/c/Users/golds/Desktop/nbin/test  [Exec]
INFO Jester is making jokes at http://0.0.0.0:5000
Starting 1 threads
DEBUG POST /
{"test": "boom"}
0
ERROR /mnt/c/Users/golds/Desktop/nbin/test.nim(3) test
/home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(482) serve
/home/agent/.nimble/pkgs/httpbeast-0.2.2/httpbeast.nim(423) run
/home/agent/.nimble/pkgs/httpbeast-0.2.2/httpbeast.nim(287) eventLoop
/home/agent/.nimble/pkgs/httpbeast-0.2.2/httpbeast.nim(219) processEvents
/home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(485) :anonymous
/home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(386) handleRequest
/home/agent/.choosenim/toolchains/nim-#head/lib/pure/asyncmacro.nim(311) dispatch
/home/agent/.choosenim/toolchains/nim-#head/lib/pure/asyncmacro.nim(34) dispatch_continue
/home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(285) dispatchIter
/home/agent/.choosenim/toolchains/nim-#head/lib/pure/asyncmacro.nim(311) match
/home/agent/.choosenim/toolchains/nim-#head/lib/pure/asyncmacro.nim(34) match_continue
/home/agent/.choosenim/toolchains/nim-#head/lib/pure/collections/tables.nim(1258) matchIter
/home/agent/.choosenim/toolchains/nim-#head/lib/system/fatal.nim(39) sysFatal
[[reraised from:
/mnt/c/Users/golds/Desktop/nbin/test.nim(3) test
/home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(482) serve
/home/agent/.nimble/pkgs/httpbeast-0.2.2/httpbeast.nim(423) run
/home/agent/.nimble/pkgs/httpbeast-0.2.2/httpbeast.nim(287) eventLoop
/home/agent/.nimble/pkgs/httpbeast-0.2.2/httpbeast.nim(219) processEvents
/home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(485) :anonymous
/home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(386) handleRequest
/home/agent/.choosenim/toolchains/nim-#head/lib/pure/asyncmacro.nim(311) dispatch
/home/agent/.choosenim/toolchains/nim-#head/lib/pure/asyncmacro.nim(37) dispatch_continue
/home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(285) dispatchIter
/home/agent/.choosenim/toolchains/nim-#head/lib/pure/asyncfutures.nim(353) read
]]
[[reraised from:
/mnt/c/Users/golds/Desktop/nbin/test.nim(3) test
/home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(482) serve
/home/agent/.nimble/pkgs/httpbeast-0.2.2/httpbeast.nim(423) run
/home/agent/.nimble/pkgs/httpbeast-0.2.2/httpbeast.nim(287) eventLoop
/home/agent/.nimble/pkgs/httpbeast-0.2.2/httpbeast.nim(219) processEvents
/home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(485) :anonymous
/home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(386) handleRequest
/home/agent/.choosenim/toolchains/nim-#head/lib/pure/asyncmacro.nim(311) handleRequestSlow
/home/agent/.choosenim/toolchains/nim-#head/lib/pure/asyncmacro.nim(37) handleRequestSlow_continue
/home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(336) handleRequestSlowIter
/home/agent/.choosenim/toolchains/nim-#head/lib/pure/asyncfutures.nim(353) read
]]
index 0 not in 0 .. -1
Async traceback:
  /mnt/c/Users/golds/Desktop/nbin/test.nim(3)                                       test
  /home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(482)                             serve
  /home/agent/.nimble/pkgs/httpbeast-0.2.2/httpbeast.nim(423)                       run
  /home/agent/.nimble/pkgs/httpbeast-0.2.2/httpbeast.nim(287)                       eventLoop
  /home/agent/.nimble/pkgs/httpbeast-0.2.2/httpbeast.nim(219)                       processEvents
  /home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(485)                             :anonymous
  /home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(386)                             handleRequest
  /home/agent/.choosenim/toolchains/nim-#head/lib/pure/asyncmacro.nim(311)          dispatch
  /home/agent/.choosenim/toolchains/nim-#head/lib/pure/asyncmacro.nim(34)           dispatch_continue
  /home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(285)                             dispatchIter
  /home/agent/.choosenim/toolchains/nim-#head/lib/pure/asyncmacro.nim(311)          match
  /home/agent/.choosenim/toolchains/nim-#head/lib/pure/asyncmacro.nim(34)           match_continue
  /home/agent/.choosenim/toolchains/nim-#head/lib/pure/collections/tables.nim(1258) matchIter
  /home/agent/.choosenim/toolchains/nim-#head/lib/system/fatal.nim(39)              sysFatal
  #[
    /mnt/c/Users/golds/Desktop/nbin/test.nim(3)                                       test
    /home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(482)                             serve
    /home/agent/.nimble/pkgs/httpbeast-0.2.2/httpbeast.nim(423)                       run
    /home/agent/.nimble/pkgs/httpbeast-0.2.2/httpbeast.nim(287)                       eventLoop
    /home/agent/.nimble/pkgs/httpbeast-0.2.2/httpbeast.nim(219)                       processEvents
    /home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(485)                             :anonymous
    /home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(386)                             handleRequest
    /home/agent/.choosenim/toolchains/nim-#head/lib/pure/asyncmacro.nim(311)          dispatch
    /home/agent/.choosenim/toolchains/nim-#head/lib/pure/asyncmacro.nim(37)           dispatch_continue
    /home/agent/.nimble/pkgs/jester-0.4.1/jester.nim(285)                             dispatchIter
    /home/agent/.choosenim/toolchains/nim-#head/lib/pure/asyncfutures.nim(353)        read
  ]#
Exception message: index 0 not in 0 .. -1
Exception type:

[GoldsteinE]

I noticed that x-www-form-urlencoded goes to params. But how do I extract uploaded file from request? And how to distinguish between form data and query params?

[dom96]

Hm, I'm on a slightly older version of Nim but I'm getting Http400 out of jester with your example. Are you sure this happens because of the iteration? Does it also happen when you just reply with "hello world" for example?

[GoldsteinE]

This code works fine:

import jester, asyncdispatch

routes:
  post "/":
    echo request.params
    echo request.formData.len
    resp "Boom!"
    #[
    for name, value in request.formData.pairs:
      echo name
      echo value
    ]#

[dom96]

That crash must have been due to some issue in Nim or Jester's old code. I tried it with Jester HEAD and Nim 1.0 and I'm getting this error: "Missing route action, did you forget to use resp in your route?" because this route doesn't return anything.

The reason x-www-form-urlencoded goes to params is because it has a clear "key" and "value" and these are parsed and then put into the params dictionary.

The formData is for a specific content type: https://github.com/dom96/jester/blob/master/jester/request.nim#L107