ParseError: Duplicate top-level identifier

[nirradi]

upgraded from 2021.11.2 to pytype-2021.11.18,
getting this difficult to understand error:

File "common.py", line 3, in <module>: Couldn't import pyi for 'interfaces' [pyi-error]
    File: "interfaces.pyi", line None
  ParseError: Duplicate top-level identifier(s): datetime

[martindemello]

is the pyi file somewhere we can take a look at it?

the error message is saying that there are two top-level symbols in the module that clash with each other, e.g.

f = ... # type: int
def f(x): ...

we would welcome any suggestions to make it clearer.

[nirradi]

its a private repo, here's the top of it with some redaction

# (generated with --quick)

import abc
import datetime
import enum
from typing import Any, Callable, Dict, List, Mapping, Optional, Set, Tuple, Type, TypeVar, Union

XXXXXX = Dict[str, Any]
XXXXXX = Callable[[Dict[str, Any], XXXXXX], XXXXXX]

ABC: Type[abc.ABC]
DataFrame: Any
Flag: Type[enum.Flag]
date: Type[datetime.date]
datetime: Type[datetime.datetime]
time: Type[datetime.time]

here are the original imports in that file

from abc import ABC, abstractmethod
from enum import Flag
from datetime import datetime, time, date
from typing import List, Optional, Union, Callable, Dict
from pandas.core.frame import DataFrame
from dateparser import parse

[csudcy]

Yeah, I'm seeing the same issue on 2021.11.18.
Workaround: Downgrade to 2021.11.12 (we don't see the issue in this version anyway).

Definitely the from datetime import datetime that's tripping it up - if I change the import to e.g. from datetime import datetime as datetime2, the error is no longer thrown & the pyi file looks like this:

import datetime
...
datetime2: Type[datetime.datetime]
...

Also, import datetime.datetime (and using datetime.datetime in code) works (though I'm hoping neither of these is the actual solution 😅 ).

[martindemello]

confirmed, thanks for the report! this is caused by the new --gen-stub-imports feature, which puts imports in the module's aliases list, so we now have an alias datetime pointing to the imported module and a constant datetime pointing to datetime.datetime. we probably shouldn't generate an alias for the import since it's a pure from-import.

[rchen152]

I've committed a fix for this, but I had to put it behind a feature flag because it was causing some weird internal test failures. I also just discovered that the feature flag isn't working properly because it isn't getting passed to the type inference step =/ The second issue should be pretty easy to fix, so starting with next week's release, you should be able to pass --fix-module-collisions to pytype to fix this bug. Once I sort out the internal tests, I'll flip the flag to True by default.

[csudcy]

Awesome, thanks for looking at it @rchen152 👍

Would it be worth re-opening this issue until the tests are fixed (or at least until the --fix-module-collisions flag is good)?

[rchen152]

Ah, yeah, I should have reopened the issue when I realized the flag wasn't working - sorry about that! But the flag issue should be fixed with today's release (2021.11.29), so you can now pass --fix-module-collisions to pytype to fix this bug.