Merge remote-tracking branch 'upstream/master' into typevar2

Author: hauntsaninja

docs/source/command_line.rst

@@ -58,12 +58,20 @@ for full details, see :ref:`running-mypy`.
     For instance, to avoid discovering any files named `setup.py` you could
     pass ``--exclude '/setup\.py$'``. Similarly, you can ignore discovering
     directories with a given name by e.g. ``--exclude /build/`` or
-    those matching a subpath with ``--exclude /project/vendor/``.
-
-    Note that this flag only affects recursive discovery, that is, when mypy is
-    discovering files within a directory tree or submodules of a package to
-    check. If you pass a file or module explicitly it will still be checked. For
-    instance, ``mypy --exclude '/setup.py$' but_still_check/setup.py``.
+    those matching a subpath with ``--exclude /project/vendor/``. To ignore
+    multiple files / directories / paths, you can combine expressions with
+    ``|``, e.g ``--exclude '/setup\.py$|/build/'``.
+
+    Note that this flag only affects recursive directory tree discovery, that
+    is, when mypy is discovering files within a directory tree or submodules of
+    a package to check. If you pass a file or module explicitly it will still be
+    checked. For instance, ``mypy --exclude '/setup.py$'
+    but_still_check/setup.py``.
+
+    In particular, ``--exclude`` does not affect mypy's :ref:`import following
+    <follow-imports>`. You can use a per-module :confval:`follow_imports` config
+    option to additionally avoid mypy from following imports and checking code
+    you do not wish to be checked.
 
     Note that mypy will never recursively discover files and directories named
     "site-packages", "node_modules" or "__pycache__", or those whose name starts

docs/source/config_file.rst

@@ -254,6 +254,10 @@ section of the command line docs.
     ``error``.  For explanations see the discussion for the
     :option:`--follow-imports <mypy --follow-imports>` command line flag.
 
+    Using this option in a per-module section (potentially with a wildcard,
+    as described at the top of this page) is a good way to prevent mypy from
+    checking portions of your code.
+
     If this option is used in a per-module section, the module name should
     match the name of the *imported* module, not the module containing the
     import statement.
@@ -924,10 +928,10 @@ Instead of using a ``mypy.ini`` file, a ``pyproject.toml`` file (as specified by
 
   * Boolean values should be all lower case
 
-Please see the `TOML Documentation`_ for more details and information on 
+Please see the `TOML Documentation`_ for more details and information on
 what is allowed in a ``toml`` file. See `PEP 518`_ for more information on the layout
 and structure of the ``pyproject.toml`` file.
-  
+
 Example ``pyproject.toml``
 **************************
 

docs/source/more_types.rst

@@ -1151,3 +1151,148 @@ section of the docs has a full description with an example, but in short, you wi
 need to give each TypedDict the same key where each value has a unique
 unique :ref:`Literal type <literal_types>`. Then, check that key to distinguish
 between your TypedDicts.
+
+
+User-Defined Type Guards
+************************
+
+Mypy supports User-Defined Type Guards
+(:pep:`647`).
+
+A type guard is a way for programs to influence conditional
+type narrowing employed by a type checker based on runtime checks.
+
+Basically, a ``TypeGuard`` is a "smart" alias for a ``bool`` type.
+Let's have a look at the regular ``bool`` example:
+
+.. code-block:: python
+
+  from typing import List
+
+  def is_str_list(val: List[object]) -> bool:
+    """Determines whether all objects in the list are strings"""
+    return all(isinstance(x, str) for x in val)
+
+  def func1(val: List[object]) -> None:
+      if is_str_list(val):
+          reveal_type(val)  # Reveals List[object]
+          print(" ".join(val)) # Error: incompatible type
+
+The same example with ``TypeGuard``:
+
+.. code-block:: python
+
+  from typing import List
+  from typing import TypeGuard  # use `typing_extensions` for Python 3.9 and below
+
+  def is_str_list(val: List[object]) -> TypeGuard[List[str]]:
+      """Determines whether all objects in the list are strings"""
+      return all(isinstance(x, str) for x in val)
+
+  def func1(val: List[object]) -> None:
+      if is_str_list(val):
+          reveal_type(val)  # List[str]
+          print(" ".join(val)) # ok
+
+How does it work? ``TypeGuard`` narrows the first function argument (``val``)
+to the type specified as the first type parameter (``List[str]``).
+
+.. note::
+
+  Narrowing is
+  `not strict <https://www.python.org/dev/peps/pep-0647/#enforcing-strict-narrowing>`_.
+  For example, you can narrow ``str`` to ``int``:
+
+  .. code-block:: python
+
+    def f(value: str) -> TypeGuard[int]:
+        return True
+
+  Note: since strict narrowing is not enforced, it's easy
+  to break type safety.
+
+  However, there are many ways a determined or uninformed developer can
+  subvert type safety -- most commonly by using cast or Any.
+  If a Python developer takes the time to learn about and implement
+  user-defined type guards within their code,
+  it is safe to assume that they are interested in type safety
+  and will not write their type guard functions in a way
+  that will undermine type safety or produce nonsensical results.
+
+Generic TypeGuards
+------------------
+
+``TypeGuard`` can also work with generic types:
+
+.. code-block:: python
+
+  from typing import Tuple, TypeVar
+  from typing import TypeGuard  # use `typing_extensions` for `python<3.10`
+
+  _T = TypeVar("_T")
+
+  def is_two_element_tuple(val: Tuple[_T, ...]) -> TypeGuard[Tuple[_T, _T]]:
+      return len(val) == 2
+
+  def func(names: Tuple[str, ...]):
+      if is_two_element_tuple(names):
+          reveal_type(names)  # Tuple[str, str]
+      else:
+          reveal_type(names)  # Tuple[str, ...]
+
+Typeguards with parameters
+--------------------------
+
+Type guard functions can accept extra arguments:
+
+.. code-block:: python
+
+  from typing import Type, Set, TypeVar
+  from typing import TypeGuard  # use `typing_extensions` for `python<3.10`
+
+  _T = TypeVar("_T")
+
+  def is_set_of(val: Set[Any], type: Type[_T]) -> TypeGuard[Set[_T]]:
+      return all(isinstance(x, type) for x in val)
+
+  items: Set[Any]
+  if is_set_of(items, str):
+      reveal_type(items)  # Set[str]
+
+TypeGuards as methods
+---------------------
+
+ A method can also serve as the ``TypeGuard``:
+
+.. code-block:: python
+
+  class StrValidator:
+      def is_valid(self, instance: object) -> TypeGuard[str]:
+          return isinstance(instance, str)
+
+  def func(to_validate: object) -> None:
+      if StrValidator().is_valid(to_validate):
+          reveal_type(to_validate)  # Revealed type is "builtins.str"
+
+.. note::
+
+  Note, that ``TypeGuard``
+  `does not narrow <https://www.python.org/dev/peps/pep-0647/#narrowing-of-implicit-self-and-cls-parameters>`_
+  types of ``self`` or ``cls`` implicit arguments.
+
+  If narrowing of ``self`` or ``cls`` is required,
+  the value can be passed as an explicit argument to a type guard function:
+
+  .. code-block:: python
+
+    class Parent:
+        def method(self) -> None:
+            reveal_type(self)  # Revealed type is "Parent"
+            if is_child(self):
+                reveal_type(self)  # Revealed type is "Child"
+
+    class Child(Parent):
+        ...
+
+    def is_child(instance: Parent) -> TypeGuard[Child]:
+        return isinstance(instance, Child)

docs/source/running_mypy.rst

@@ -384,6 +384,10 @@ We do not recommend using ``skip`` unless you know what you are doing:
 while this option can be quite powerful, it can also cause many
 hard-to-debug errors.
 
+Adjusting import following behaviour is often most useful when restricted to
+specific modules. This can be accomplished by setting a per-module
+:confval:`follow_imports` config option.
+
 
 .. _mapping-paths-to-modules:
 

mypy/applytype.py

@@ -5,7 +5,7 @@
 from mypy.expandtype import expand_type
 from mypy.types import (
     Type, TypeVarId, TypeVarType, CallableType, AnyType, PartialType, get_proper_types,
-    TypeVarLikeType, ProperType
+    TypeVarLikeType, ProperType, ParamSpecType
 )
 from mypy.nodes import Context
 
@@ -19,6 +19,8 @@ def get_target_type(
     skip_unsatisfied: bool
 ) -> Optional[Type]:
     # TODO(shantanu): fix for ParamSpecType
+    if isinstance(tvar, ParamSpecType):
+        return None
     assert isinstance(tvar, TypeVarType)
     values = get_proper_types(tvar.values)
     if values:

mypy/checker.py

@@ -466,6 +466,13 @@ def check_overlapping_overloads(self, defn: OverloadedFuncDef) -> None:
         # At this point we should have set the impl already, and all remaining
         # items are decorators
 
+        if self.msg.errors.file in self.msg.errors.ignored_files:
+            # This is a little hacky, however, the quadratic check here is really expensive, this
+            # method has no side effects, so we should skip it if we aren't going to report
+            # anything. In some other places we swallow errors in stubs, but this error is very
+            # useful for stubs!
+            return
+
         # Compute some info about the implementation (if it exists) for use below
         impl_type: Optional[CallableType] = None
         if defn.impl:

mypy/checkexpr.py

@@ -18,7 +18,7 @@
     Type, AnyType, CallableType, Overloaded, NoneType, TypeVarType, TypeGuardType,
     TupleType, TypedDictType, Instance, ErasedType, UnionType,
     PartialType, DeletedType, UninhabitedType, TypeType, TypeOfAny, LiteralType, LiteralValue,
-    is_named_instance, FunctionLike,
+    is_named_instance, FunctionLike, ParamSpecType,
     StarType, is_optional, remove_optional, is_generic_instance, get_proper_type, ProperType,
     get_proper_types, flatten_nested_unions
 )
@@ -4471,14 +4471,16 @@ def merge_typevars_in_callables_by_name(
             target = freshen_function_type_vars(target)
 
             rename = {}  # Dict[TypeVarId, TypeVar]
-            for tvdef in target.variables:
-                name = tvdef.fullname
+            for tv in target.variables:
+                name = tv.fullname
                 if name not in unique_typevars:
                     # TODO(shantanu): fix for ParamSpecType
-                    assert isinstance(tvdef, TypeVarType)
-                    unique_typevars[name] = tvdef
-                    variables.append(tvdef)
-                rename[tvdef.id] = unique_typevars[name]
+                    if isinstance(tv, ParamSpecType):
+                        continue
+                    assert isinstance(tv, TypeVarType)
+                    unique_typevars[name] = tv
+                    variables.append(tv)
+                rename[tv.id] = unique_typevars[name]
 
             target = cast(CallableType, expand_type(target, rename))
         output.append(target)