TypeVars cannot refer to type variables

[danr]

We define a type variable A:

A = TypeVar('A')

The following gives the error Invalid type A:

IterA = TypeVar('IterA', Iterator[A], Iterable[A])

[gvanrossum]

Confimed. But I'm not sure whether this is a bug or an understandable restriction on the type system.

How were you intending to use IterA? Note that the form TypeVar(name, t1, t2, ...) is intended to define a type variable that gets substituted with exactly t1, t2, etc., but here, because your restrictions are themselves generic, there's not really a single type for each of t1 etc.

Perhaps you're better off using a type alias? Those can be generic, e.g.

IterA = Union[Iterator[A], Iterable[A]]

(Note that when using a generic type alias, you should append a type parameter, e.g. IterA[T], IterA[int] or IterA[Any].)

[drakedevel]

I just ran into this today -- here's a stripped-down example of my use-case for this:

from typing import Any, Generic, Sequence, TypeVar

T = TypeVar('T')
ST = TypeVar('ST', bound=Sequence[Any])
# want: ST = TypeVar('ST', bound=Sequence[T])


class Abstract(Generic[T, ST]):
    def _helper(self) -> ST:
        ...

    def method(self) -> ST:
        return self._helper()


class Impl(Abstract[int, bytes]):
    def _helper(self) -> bytes:
        return b''


def dummy(x: Abstract[T, ST]) -> None:
    reveal_type(x.method().__len__)
    reveal_type(x.method()[0])


reveal_type(Impl().method())

The output of running mypy on this is:

example.py:22: error: Revealed type is 'def () -> builtins.int'
example.py:23: error: Revealed type is 'Any'
example.py:26: error: Revealed type is 'builtins.bytes*'

Ideally, mypy would be able to infer that the sequence is of T, rather than Any. If I try the commented-out line for the definition of ST, I get the following:

example.py:5: error: Invalid type "example4.T"
example.py:22: error: Revealed type is 'def () -> builtins.int'
example.py:23: error: Revealed type is 'T?'
example.py:26: error: Revealed type is 'builtins.bytes*'

In Java, for example, I can create this relationship by writing the template parameters as <T, IT extends Iterable<T>> but as far as I know no equivalent syntax exists in mypy.

[pkch]

This is my use case for this issue (and also for #3148):

# type of values provided by user
UserValue = TypeVar('UserValue')

# abstract class; defines methods sufficient for building this data structure
class MutableStructure(Generic[UserValue]): ...

# a particular implementation, generic in user value type
class MyStructure(MutableStructure[UserValue]): ...

# type of data structure being created
S = TypeVar('S', bound=MutableStructure[UserValue])  # fail (issue #2756)

# should work for any MutableStructure subclasses
def from_string(cls: Type[S], s: str) -> S:  # fail (issue #3148)
    return cls()

[pkch]

It might be worth explicitly stating in PEP 484 that type variables are not permitted inside TypeVar arguments. I think it's a reasonable restriction, at least for now.

[gvanrossum]

Sure, send a PR to the peps repo.

[JukkaL]

This doesn't seem useful enough to be worth the extra complexity, so I'm closing the issue.

[gjulianm]

Hello, I would like to add to the discussion here. It definitely seems to be a complex issue but I think the use case is not that uncommon. In my case I am having issues with generic containers in a graph class, something like this simplified example:

T = TypeVar('T')

class Node(Generic[T]):
    @property 
    def key(self) -> T: 
        ...

# What I can do:

class Graph(Generic[T]):
    def __init__(self):
    	self._nodes = []  # type: List[Node[T]]
        
    def build_edge(self, src: T, dst: T):
    	self._nodes.append(Node(src))
    	self._nodes.append(Node(dst))

# What I would like to do:

TNode = TypeVar('TNode', bound=Node[T])

class Graph(Generic[TNode]):
    def __init__(self):
    	self._nodes = []  # type: List[TNode]
        
    def build_edge(self, src: T, dst: T):
        # Default implementation, can be overridden 
        # by child classes
    	self._nodes.append(Node(src))
    	self._nodes.append(Node(dst))

The second approach would allow a subclass of Graph to use an specialization of the Node class without issues.

I think people will run into this issue whenever there are subclasses that utilize generics that can be specialized. Maybe it's not the most common use case, but it is a situation that works well with regular Python and where typing can help clarify a lot the code and the types that are being assumed in each function.

[ProfFan]

I encountered exactly the same problem as @gjulianm 's. This is a quite common pattern in anything related to implementing trees and graphs of any kind (mine is a binary search tree).

I don't know if there are any other ways to implement this, but please share if you know.

BTW, all my attempts end up as the same error message.
Googling is very hard, so I'll leave the error message below for everyone else Googling.

Argument of type "_K@_V" cannot be assigned to parameter "__object" of type "_K@FastTree" in function "append"
  Type "_K@_V" cannot be assigned to type "_K@FastTree"

Argument of type _K @ _V cannot be assigned to parameter __object of type _K @ FastTree in function append
  Type _K @ _V cannot be assigned to type _K @ FastTree