Implement literal dictionaries and lists.

[stuartarchibald]

As title.
Permits compilation of things like:

@njit
def foo():
  d = {'a': 1,
       'b': 2j,
       'c': 'STRING',
       'd': np.zeros(5),
       'e': ['list', 1, 'of', 2, 'mixed', 3, 'types']}

WIP, opened for CI.

[sklam]

From out-of-band discussion:

• This feature is pushing the limit of type inference
• We speculate that could be problem with over-specialization due to the capturing of initial-value as part of the type.
• Planned actions:
  • benchmark for problems after merging
  • fix the overspecialization in a new PR

[sklam]

I've done a first pass review. There are a few more details I want to look at.

[stuartarchibald]

479b425 is failing with:

FAIL: test_dict_unify (numba.tests.test_dictobject.TestLiteralStrKeyDict)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/runner/work/1/s/numba/tests/test_dictobject.py", line 1983, in test_dict_unify
    self.assertEqual(foo(100), 'd')
AssertionError: 'CAT' != 'd'
- CAT
+ d

think literal dictionary unification is probably wrong

[stuartarchibald]

#5946 (comment) is fixed by b17f841 on wards.

[stuartarchibald]

@sklam this still has over-specialisation issues but is probably in a position where it's close enough to working to be reviewable! Thanks.

[sklam]

The new changes are looking good. Feature-wise, it will address the use-cases for https://gist.github.com/sklam/4f08128860485485c7b3dbd2c9149e07.

I did notice that it is difficult to search for the added feature in the current doc. We should probably address this separately.

I will want to have another look at the specific details in the new overload_method.

[sklam]

untested

[sklam]

how do we need this?

[stuartarchibald]

think it's left over from trying to make literal dictionary types gracefully degrade, removed in 120ffce

[stuartarchibald]

Turns out it is needed: https://dev.azure.com/numba/numba/_build/results?buildId=6009&view=logs&j=12ecc1d6-2964-5850-b419-89f950957853&t=abce8f77-76e0-5291-fb7e-5ca9f390cd04&l=9914 IIRC typed.Dict uses the list dtype and not its initial value, but the initial value is part of the type, so casts such as the one failing won't work without it.

[stuartarchibald]

Reverted 120ffce in ad7343f

[sklam]

Cannot cast DictType[int32,int32]<iv={4: 2}> to DictType[int32,int32]<iv={1: 3}>:

why is that allowed? should it be unified to one without initial_value?

[stuartarchibald]

Think this stems from two problems:

1. No attempt at unification in the cast impl

   numba/numba/typed/typedobjectutils.py

   Lines 23 to 35 in e043ee7

   	@intrinsic
   	def _cast(typingctx, val, typ):
   	"""Cast *val* to *typ*
   	"""
   	def codegen(context, builder, signature, args):
   	[val, typ] = args
   	context.nrt.incref(builder, signature.return_type, val)
   	return val
   	# Using implicit casting in argument types
   	casted = typ.instance_type
   	_sentry_safe_cast(val, casted)
   	sig = casted(casted, typ)
   	return sig, codegen

2. DictType will only unify with another DictType if the other is "imprecise"

   numba/numba/core/types/containers.py

   Lines 636 to 643 in e043ee7

   	def unify(self, typingctx, other):
   	"""
   	Unify this with the *other* dictionary.
   	"""
   	# If other is dict
   	if isinstance(other, DictType):
   	if not other.is_precise():
   	return self

[sklam]

smoking at numba_smoketest_cpu_49

[sklam]

numba_smoketest_cpu_49 is failing but due to entry-point test (probably a llvmlite version pinning problem again).
I don't see other failures from my random sample.

Update: confirmed that all failed tests are entry-point tests due to llvmlite version pinning; thus this PR has essentially passed our smoketest.