More crashes in class scoped imports

[JelleZijlstra]

In Python 3 (but not Python 2), it is illegal to do import * in class body, but mypy allows it.

It also crashes if you do class X: from os import *:

% python -m mypy -c 'class Capybara: from os import *' --show-traceback
<string>:1: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.940+dev.1de5e55fff22b023d54aa700d630befd162db20e
Traceback (most recent call last):
  File "/Users/jelle/.pyenv/versions/3.9.4/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/jelle/.pyenv/versions/3.9.4/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/jelle/py/mypy/mypy/__main__.py", line 34, in <module>
    console_entry()
  File "/Users/jelle/py/mypy/mypy/__main__.py", line 12, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "/Users/jelle/py/mypy/mypy/main.py", line 96, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/Users/jelle/py/mypy/mypy/main.py", line 173, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/Users/jelle/py/mypy/mypy/build.py", line 180, in build
    result = _build(
  File "/Users/jelle/py/mypy/mypy/build.py", line 256, in _build
    graph = dispatch(sources, manager, stdout)
  File "/Users/jelle/py/mypy/mypy/build.py", line 2722, in dispatch
    process_graph(graph, manager)
  File "/Users/jelle/py/mypy/mypy/build.py", line 3066, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/Users/jelle/py/mypy/mypy/build.py", line 3158, in process_stale_scc
    mypy.semanal_main.semantic_analysis_for_scc(graph, scc, manager.errors)
  File "/Users/jelle/py/mypy/mypy/semanal_main.py", line 79, in semantic_analysis_for_scc
    process_functions(graph, scc, patches)
  File "/Users/jelle/py/mypy/mypy/semanal_main.py", line 233, in process_functions
    process_top_level_function(analyzer,
  File "/Users/jelle/py/mypy/mypy/semanal_main.py", line 274, in process_top_level_function
    deferred, incomplete, progress = semantic_analyze_target(target, state, node, active_type,
  File "/Users/jelle/py/mypy/mypy/semanal_main.py", line 326, in semantic_analyze_target
    analyzer.refresh_partial(refresh_node,
  File "/Users/jelle/py/mypy/mypy/semanal.py", line 416, in refresh_partial
    self.accept(node)
  File "/Users/jelle/py/mypy/mypy/semanal.py", line 5199, in accept
    node.accept(self)
  File "/Users/jelle/py/mypy/mypy/nodes.py", line 730, in accept
    return visitor.visit_func_def(self)
  File "/Users/jelle/py/mypy/mypy/semanal.py", line 607, in visit_func_def
    for arg in defn.arguments:
AttributeError: arguments
<string>:1: : note: use --pdb to drop into pdb

[hauntsaninja]

Okay, so while #12023 fixed a bunch of crashes, it also resulted in some new crashes. It also looks like there's some kind of caching bug at play, so make sure to use --no-incremental if you're seeing weirdness when debugging.

λ mypy -c 'class Foo: from os import *' --show-traceback --no-incremental
<string>:1: error: Method must have at least one argument
<string>:1: error: An overloaded function outside a stub file must have an implementation
<string>:517: error: Name "overload" is not defined
<string>:528: error: Name "overload" is not defined
<string>:539: error: Name "overload" is not defined
<string>:550: error: Name "overload" is not defined
<string>:561: error: Name "overload" is not defined
<string>:572: error: Name "overload" is not defined
<string>:583: error: Name "overload" is not defined
<string>:1: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.940+dev.1de5e55fff22b023d54aa700d630befd162db20e
Traceback (most recent call last):
  File "/Users/shantanu/.virtualenvs/mypy-fusj/bin/mypy", line 33, in <module>
    sys.exit(load_entry_point('mypy', 'console_scripts', 'mypy')())
  File "/Users/shantanu/dev/mypy/mypy/__main__.py", line 12, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "/Users/shantanu/dev/mypy/mypy/main.py", line 96, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/Users/shantanu/dev/mypy/mypy/main.py", line 173, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/Users/shantanu/dev/mypy/mypy/build.py", line 180, in build
    result = _build(
  File "/Users/shantanu/dev/mypy/mypy/build.py", line 256, in _build
    graph = dispatch(sources, manager, stdout)
  File "/Users/shantanu/dev/mypy/mypy/build.py", line 2722, in dispatch
    process_graph(graph, manager)
  File "/Users/shantanu/dev/mypy/mypy/build.py", line 3066, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/Users/shantanu/dev/mypy/mypy/build.py", line 3158, in process_stale_scc
    mypy.semanal_main.semantic_analysis_for_scc(graph, scc, manager.errors)
  File "/Users/shantanu/dev/mypy/mypy/semanal_main.py", line 79, in semantic_analysis_for_scc
    process_functions(graph, scc, patches)
  File "/Users/shantanu/dev/mypy/mypy/semanal_main.py", line 233, in process_functions
    process_top_level_function(analyzer,
  File "/Users/shantanu/dev/mypy/mypy/semanal_main.py", line 274, in process_top_level_function
    deferred, incomplete, progress = semantic_analyze_target(target, state, node, active_type,
  File "/Users/shantanu/dev/mypy/mypy/semanal_main.py", line 326, in semantic_analyze_target
    analyzer.refresh_partial(refresh_node,
  File "/Users/shantanu/dev/mypy/mypy/semanal.py", line 416, in refresh_partial
    self.accept(node)
  File "/Users/shantanu/dev/mypy/mypy/semanal.py", line 5199, in accept
    node.accept(self)
  File "/Users/shantanu/dev/mypy/mypy/nodes.py", line 564, in accept
    return visitor.visit_overloaded_func_def(self)
  File "/Users/shantanu/dev/mypy/mypy/semanal.py", line 751, in visit_overloaded_func_def
    self.analyze_overloaded_func_def(defn)
  File "/Users/shantanu/dev/mypy/mypy/semanal.py", line 778, in analyze_overloaded_func_def
    types, impl, non_overload_indexes = self.analyze_overload_sigs_and_impl(defn)
  File "/Users/shantanu/dev/mypy/mypy/semanal.py", line 825, in analyze_overload_sigs_and_impl
    item.accept(self)
  File "/Users/shantanu/dev/mypy/mypy/nodes.py", line 822, in accept
    return visitor.visit_decorator(self)
  File "/Users/shantanu/dev/mypy/mypy/semanal.py", line 1095, in visit_decorator
    dec.func.accept(self)
  File "/Users/shantanu/dev/mypy/mypy/nodes.py", line 730, in accept
    return visitor.visit_func_def(self)
  File "/Users/shantanu/dev/mypy/mypy/semanal.py", line 629, in visit_func_def
    self.analyze_func_def(defn)
  File "/Users/shantanu/dev/mypy/mypy/semanal.py", line 636, in analyze_func_def
    self.update_function_type_variables(defn.type, defn)
  File "/Users/shantanu/dev/mypy/mypy/semanal.py", line 737, in update_function_type_variables
    fun_type.variables = a.bind_function_type_variables(fun_type, defn)
  File "/Users/shantanu/dev/mypy/mypy/typeanal.py", line 979, in bind_function_type_variables
    assert var_node, "Binding for function type variable not found within function"
AssertionError: Binding for function type variable not found within function
<string>:1: : note: use --pdb to drop into pdb

It looks like the "copy" is a little strong, since mypy now looks for typevars (and presumably the overload symbol and other decorators) in the file in which things were imported. But we do need at least enough of a copy of the symbol that we can associate set the info attribute on it. If we want to continue barrelling down this path, we need to stub out at least the body of the constructed node.

Jukka had suggested making this a blocking error. I'd prefer not to do that, because a blocking error is only slightly better than a crash and this would prevent me from using mypy on a codebase at work. Maybe we just issue a non-blocking error and create an Any node.

[hauntsaninja]

I realise I sort of hijacked this issue. Opened #12198 for the low-pri issue.

[JelleZijlstra]

Do you know what the exact conditions are under which it now crashes? Is it importing a function from a stub?

[hauntsaninja]

Multiple things are bad:

[case testClassScopeImportWithError]
class Foo:
    from mod import meth1
    from mod import meth2

[file mod.pyi]
from typing import Any, TypeVar, overload

@overload
def meth1(self: Any, y: int) -> int: ...
@overload
def meth1(self: Any, y: str) -> str: ...

T = TypeVar("T")
def meth2(self: Any, y: T) -> T: ...

It's basically copy-pasting the definition of the function into the class, so then mypy can't look up the typevar and doesn't know what overload is.