Python completion: HTTPError: 413 Client Error: Request Entity Too Large for url: <url>

[puremourning]

Problem

Completions don't work for python on large (ish) files since JediHTTP was merged

Detail

The new JediHTTP completer is throwing an error when getting completions in a moderately large module:

Here's the error from stderr:

2016-01-11 14:03:10,946 - ERROR - Exception from semantic completer (using general): Traceback (most recent call last):
  File "/mma/users/benj/.vim/bundle/YouCompleteMe/third_party/ycmd/ycmd/../ycmd/handlers.py", line 104, in GetCompletions
    .ComputeCandidates( request_data ) )
  File "/mma/users/benj/.vim/bundle/YouCompleteMe/third_party/ycmd/ycmd/../ycmd/completers/completer.py", line 168, in ComputeCandidates
    candidates = self._GetCandidatesFromSubclass( request_data )
  File "/mma/users/benj/.vim/bundle/YouCompleteMe/third_party/ycmd/ycmd/../ycmd/completers/completer.py", line 184, in _GetCandidatesFromSubclass
    raw_completions = self.ComputeCandidatesInner( request_data )
  File "/mma/users/benj/.vim/bundle/YouCompleteMe/third_party/ycmd/ycmd/../ycmd/completers/python/jedi_completer.py", line 217, in ComputeCandidatesInner
    for completion in self._JediCompletions( request_data ) ]
  File "/mma/users/benj/.vim/bundle/YouCompleteMe/third_party/ycmd/ycmd/../ycmd/completers/python/jedi_completer.py", line 221, in _JediCompletions
    return self._GetResponse( '/completions', request_data )[ 'completions' ]
  File "/mma/users/benj/.vim/bundle/YouCompleteMe/third_party/ycmd/ycmd/../ycmd/completers/python/jedi_completer.py", line 168, in _GetResponse
    response.raise_for_status()
  File "/mma/users/benj/.vim/bundle/YouCompleteMe/third_party/ycmd/third_party/requests/requests/models.py", line 837, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
HTTPError: 413 Client Error: Request Entity Too Large for url: http://127.0.0.1:58005/completions

An example file I get this in is git-p4 from the git distribution. On my system that's in /opt/rh/devtoolset-2/root/usr/libexec/git-core/git-p4 though it's unlikely to be there on most systems. It's 3388 lines long and 124K.

It works fine if I revert to prior to JediHTTP landing.

@vheon

Steps to reproduce (at least on my system):

• Open git-p4 from git's libexec/git-core directory
• add import pprint to the list of imports
• in p4_build_cmd add something like pprint. and observe the error in Vim and the log.

:version

:version
VIM - Vi IMproved 7.4 (2013 Aug 10, compiled Apr 15 2015 17:44:49)
Included patches: 1-701
Compiled by benj@ukfd-ltp2l
Huge version with X11-Motif GUI.  Features included (+) or not (-):
+acl             +farsi           +mouse_netterm   +syntax
+arabic          +file_in_path    +mouse_sgr       +tag_binary
+autocmd         +find_in_path    -mouse_sysmouse  +tag_old_static
+balloon_eval    +float           +mouse_urxvt     -tag_any_white
+browse          +folding         +mouse_xterm     -tcl
++builtin_terms  -footer          +multi_byte      +terminfo
+byte_offset     +fork()          +multi_lang      +termresponse
+cindent         +gettext         -mzscheme        +textobjects
+clientserver    -hangul_input    +netbeans_intg   +title
+clipboard       +iconv           +path_extra      +toolbar
+cmdline_compl   +insert_expand   -perl            +user_commands
+cmdline_hist    +jumplist        +persistent_undo +vertsplit
+cmdline_info    +keymap          +postscript      +virtualedit
+comments        +langmap         +printer         +visual
+conceal         +libcall         +profile         +visualextra
+cryptv          +linebreak       +python          +viminfo
+cscope          +lispindent      -python3         +vreplace
+cursorbind      +listcmds        +quickfix        +wildignore
+cursorshape     +localmap        +reltime         +wildmenu
+dialog_con_gui  -lua             +rightleft       +windows
+diff            +menu            -ruby            +writebackup
+digraphs        +mksession       +scrollbind      +X11
-dnd             +modify_fname    +signs           +xfontset
-ebcdic          +mouse           +smartindent     +xim
+emacs_tags      +mouseshape      -sniff           +xsmp_interact
+eval            +mouse_dec       +startuptime     +xterm_clipboard
+ex_extra        -mouse_gpm       +statusline      -xterm_save
+extra_search    -mouse_jsbterm   -sun_workshop    +xpm
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
  system gvimrc file: "$VIM/gvimrc"
    user gvimrc file: "$HOME/.gvimrc"
2nd user gvimrc file: "~/.vim/gvimrc"
    system menu file: "$VIMRUNTIME/menu.vim"
  fall-back for $VIM: "/mma/users/benj/proj/vim/root-6.5/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_MOTIF     -g -O2 -U_FORT
IFY_SOURCE -D_FORTIFY_SOURCE=1
Linking: gcc   -L/usr/local/lib -Wl,--as-needed -o vim  -lXp -lXmu -lXext -lXm -lXt
 -lSM -lICE -lXt -lX11 -lSM -lICE -ldl  -lm -ltinfo -lelf -lnsl  -lselinux  -ldl
 -L/usr/lib/python2.6/config -lpython2.6

Both the jediHTTP stdout and stderr log files are empty.

-- JediHTTP running at 127.0.0.1:52546
--   stdout log: /tmp/ycm_temp/jedihttp_52546_stdout.log
--   stderr log: /tmp/ycm_temp/jedihttp_52546_stderr.log

$>cat /tmp/ycm_temp/jedihttp_52546_stdout.log
$>cat  /tmp/ycm_temp/jedihttp_52546_stderr.log
$>

Explanation

Bottle has a max-size for request buffers. It is 100K by default. ycmd increases this to 1MB

ycmd has this code:

# num bytes for the request body buffer; request.json only works if the request
# size is less than this
bottle.Request.MEMFILE_MAX = 1000 * 1024

bottle has this code:

    MEMFILE_MAX = 102400

    def _get_body_string(self):
        ''' read body until content-length or MEMFILE_MAX into a string. Raise
            HTTPError(413) on requests that are to large. '''
        clen = self.content_length
        if clen > self.MEMFILE_MAX:
            raise HTTPError(413, 'Request to large')
        if clen < 0: clen = self.MEMFILE_MAX + 1
        data = self.body.read(clen)
        if len(data) > self.MEMFILE_MAX: # Fail fast
            raise HTTPError(413, 'Request to large')
        return data

I think we need something in JediHTTP for this? Agree?

[puremourning]

Can confirm adding the above to jedihttp/handlers.py fixes it.