Complete failure of CodeIntel for any Python3 file that includes both an F-String and an iterable unpack.

[will-ca]

Short Summary

I have a moderately large Python3 codebase in which Code Intelligence, Autocompletion, and Calltips have never worked, even for built-ins.

So far, I've managed to track the problematic code down to two expressions that were originally parts of instance methods in a single file.

Steps to Reproduce

1. Write, open, or paste any Python3 code that includes both an iterable unpacking and an F-String, such as the following:

"""
Short reproducer for Python3 CodeIntel failure in Komodo Edit.
(Komodo Edit, version 11.1.1, build 18206, platform linux-x86_64)

Having an F-String followed by an iterable unpacking in the same
file causes Code Intelligence, Call Tips, Autocompletion, etc. to
fail in the rest of the file and in all projects that import the
the file. This affects not just names defined in the file but also
built-ins and names in other modules, and it still occurs after
resetting the CodeIntel database, with the code in a package, with
the code copied and pasted, and in Safe Mode.

In the original code, the F-String and the iterable unpacking
were the return values of instance methods, and used different
arguments and types. The issue persisted after moving them into
module-level functions and changing the types of the iterables,
and then after turning them both into effect-less literal, module-
level expressions. Removing or commenting out either the F-String
or the iterable unpacking causes Code Intelligence to work again.
"""

class CallTipTest:
	testname = 5

f'{5}'

[*()]

2. Try to trigger an autocompletion or a calltip for a name that was not successfully loaded into the CodeIntel database prior to opening the code containing the F-String and the iterable unpacking. (I.E. Any name at all, including built-ins, if the code was all loaded at once, or any name that was defined after writing the F-String and the iterable unpacking if the F-String and the iterable unpacking were typed into an already-open file.)

Expected results

Code intelligence works. Both built-ins and defined names are available as autocompletion suggestions, both in the file itself and in any file that imports it.

For the reproducing code above, typing CallTip and CallTipTest. creates dropdowns that suggest CallTipTest and CallTipTest.testname respectively, and pressing CTRL+J does not create any error message in the notification panel.

Actual results

Code intelligence doesn't work. No new suggestions or dropdowns listing suggestions are created, whether defined names or built-ins, and whether in the file or in any file that imports it.

Pressing CTRL+J causes cryptic error messages such as following to be pushed to the notifications panel:

evaluating 'CallTipTest' at Text-1.txt#25: could not resolve first part of 'CallTipTest' (error determining completions

Platform Information

• Komodo Edit
• version 11.1.1, build 18206, platform linux-x86_64 (AUR)
• Manjaro (Up to date as of February 2020)

Additional Information

100% Pre-loading code intelligence database
                                         Thu Feb 06 2020 22:39:21 GMT+0000 (GMT)
--------------------------------------------------------------------------------
Pre-loading code intelligence database
                                         Thu Feb 06 2020 22:39:13 GMT+0000 (GMT)
  Pre-loading code intelligence database. This process will improve the speed of first time autocomplete and calltips. It typically takes less than a minute.
  Pre-loading standard library data...
  JavaScript standard library...
  Determining necessary updates...
  Adding javascript stdlib
  Node.js standard library...
  Determining necessary updates...
  Adding node.js stdlib
  PHP standard library...
  Determining necessary updates...
  Adding php-4.4 stdlib
  Determining necessary updates...
  Adding php-5.2 stdlib
  Determining necessary updates...
  Adding php-5.3 stdlib
  Determining necessary updates...
  Adding php-5.4 stdlib
  Determining necessary updates...
  Adding php-5.5 stdlib
  Determining necessary updates...
  Adding php-5.6 stdlib
  Determining necessary updates...
  Adding php-7.0 stdlib
  Perl 5.30 standard library...
  Determining necessary updates...
  Adding perl-5.22 stdlib
  Python 2.7 standard library...
  Determining necessary updates...
  Adding python-2.7 stdlib
  Python3 3.8 standard library...
  Determining necessary updates...
  Adding python3-3.5 stdlib
  Ruby 2.7 standard library...
  Determining necessary updates...
  Adding ruby-2.3 stdlib
  Pre-loading catalogs...
  Determining necessary catalog updates...
  Adding 'jquery.cix' API catalog
  Adding 'mochikit.cix' API catalog
  Adding 'dojo.cix' API catalog
  Adding 'xpcom.cix' API catalog
  Adding 'xbl.cix' API catalog
  Adding 'python3-pywin32.cix' API catalog
  Adding 'rails.cix' API catalog
  Adding 'yui.cix' API catalog
  Adding 'angular.cix' API catalog
  Adding 'ember.cix' API catalog
  Adding 'phpunit-5.cix' API catalog
  Adding 'pywin32.cix' API catalog
  Adding 'prototype.cix' API catalog
  Saving catalog indices...
  Code intelligence database pre-loaded.
--------------------------------------------------------------------------------
Pre-loading code intelligence database
                                         Thu Feb 06 2020 22:39:13 GMT+0000 (GMT)
--------------------------------------------------------------------------------
No completions found
                                         Thu Feb 06 2020 22:39:39 GMT+0000 (GMT)
--------------------------------------------------------------------------------
Scanning Python3 files in '/usr/lib/python3.8/site-packages'
                                         Thu Feb 06 2020 22:39:42 GMT+0000 (GMT)
--------------------------------------------------------------------------------
Scanning one directory
                                         Thu Feb 06 2020 22:39:41 GMT+0000 (GMT)
--------------------------------------------------------------------------------
Scanning one directory
                                         Thu Feb 06 2020 22:39:42 GMT+0000 (GMT)
--------------------------------------------------------------------------------
ParseError: Expected end, got '(.)'
                                         Thu Feb 06 2020 22:39:44 GMT+0000 (GMT)
--------------------------------------------------------------------------------
ParseError: Expected end, got '(name arguments)'
                                         Thu Feb 06 2020 22:39:43 GMT+0000 (GMT)
--------------------------------------------------------------------------------
Repeat notifications truncated, please check your Notifications panel
                                         Thu Feb 06 2020 22:39:44 GMT+0000 (GMT)
--------------------------------------------------------------------------------
ParseError: Expected end, got '(.)'
                                         Thu Feb 06 2020 22:39:44 GMT+0000 (GMT)
--------------------------------------------------------------------------------
aborted
                                         Thu Feb 06 2020 22:39:46 GMT+0000 (GMT)
--------------------------------------------------------------------------------
evaluating 'CallTipTest' at Text-1.txt#25: could not resolve first part of 'CallTipTest' (error determining completions)
                                         Thu Feb 06 2020 22:39:46 GMT+0000 (GMT)
--------------------------------------------------------------------------------
evaluating 'CallTipTest' at Text-1.txt#25: could not resolve first part of 'CallTipTest' (error determining completions)
                                         Thu Feb 06 2020 22:39:46 GMT+0000 (GMT)

CILog.txt

[will-ca]

I just tested and found this still happens on Komodo Edit, version 12.0.0, build 18431, platform linux-x86_64, Built on Tue Jan 14 18:20:37 2020.

[th3coop]

@will-ca , thorough report, thank you for the digging.

[th3coop]

IDE has a different implementation of code intelligence if you want to try that.