Document library modules

[JukkaL]

Currently there is no documentation for Python standard library modules, other than a list of supported modules in the wiki. Users will have to rely on Python documentation. This is generally fine, but sometimes it's not obvious how some things translate to mypy. Additionally some Python library features are not supported yet in the stubs.

We should generate automatically HTML documentation based on the library stubs. They don't have to duplicate the Python library reference (at least initially; this would be good to have later in the project). There should be at least the following information per module:

• all variables, functions, types and methods defined in the module, including their static types
• link to relevevant information in the Python library reference (version 3.2)
• additional notes about things that are different from Python or that might not be obvious to the reader (these notes could be stored in a separate file and merged automatically when building the HTML files)

Additionally there should be a contents page that lists all supported modules.

[gvanrossum]

Do we still need this now that mypy is no longer a separate language?

[JukkaL]

I think that having searchable, formatted, cross referenced and pretty online documentation generated from stubs would still be quite useful. Currently users can browse typeshed, but it's not optimal for various reasons:

• It's cumbersome to find the stub for a particular class such as dict (GitHub search produces a lot of noise).
• The files on GitHub have no syntax highlighting.
• The stub files don't have consistent formatting and include comments for internal use, which make them noisy and hard to read.
• There are no links from types to definitions. For example, I'd like to be able to click a type in an annotation to see how that type is defined.

I doubt that this should have high priority, but something like this could be pretty valuable. Some parts of the original issue description likely won't make sense any more, though. Also, perhaps this issue should be moved to typeshed.

[elazarg]

So this issue is the same as #517?

(The syntax highlighting part has been solved)

[JukkaL]

Yeah, closing #517. #517 has the idea of generating Sphinx documentation, but it's basically the same.

[theodoretliu]

Hey, I'd like to revive and close out this issue. Just so I understand, the desire here is to read the Python standard library typing stubs and then automatically generate mypy documentation for them?

[ilevkivskyi]

@theodoretliu TBH I am not sure what is the desired scope/goal here. I would rather propose you to work on couple issues related to subsystems you will need for the numeric types. For example #4975 and #4441 (probably in this order because the latter is more complex and bigger).

If you really want to work on this one, then @JukkaL is the best person to give more context here.

[theodoretliu]

@ilevkivskyi Okay, happy to start working on those two as well! I'll still offer to help on this issue in parallel since it seems important (and is the oldest issue)

[JukkaL]

I think that the original problem is still relevant, but there are many potential ways to approach this:

1. Searchable HTML generated from typeshed stubs (what was proposed originally). This has the drawback of not necessarily being discoverable or easily accessible, and requires maintenance.
2. Integrate type annotations into Python stdlib docs. This may be hard to pull off, and doesn't directly help with third-party packages. If we fork the stdlib docs, this will have the problems of discoverability and accessibility mentioned above (and maintenance).
3. Something integrated into reveal_type. For example, reveal_type(module) could perhaps print out all the types in a module. We could also use syntax highlighting and pretty multi-line formatting in the output of reveal_type.
4. Something integrated into the mypy command line. We could have something like mypy --describe re.sub that would print out the signature of a function (or all signatures within module/class). To make it easier to use, maybe only a short name such as sub would be required and mypy could somehow look for definitions in all modules visible to it (this is easy to do in the mypy daemon). As above, the output should use syntax highlighting and pretty formatting.
5. Command line integration that can be easily used from editor integrations (from PyCharm or VS Code, for example). It could take a source file path, line and column and produce information that the editor integration can use, such as HTML.

I think that 3, 4 and 5 are the most promising approaches. Hosted HTML documentation probably isn't worth it.

[emmatyping]

I think a potentially easy way to accomplish this would be to use something like https://github.com/agronholm/sphinx-autodoc-typehints but instead of pulling types from the stdlib sources, pull them from typeshed.

Though I also think being able to search a type from the mypy command line would be very handy, and not too difficult to accomplish (famous last words).

[hauntsaninja]

IMHO, the best experience for this kind of thing is having it in IDE. Pyright/Pylance in VSCode is currently working really well for me (get types on hover / as suggestions as you type + go to definition). Unless it's something you want to work on, I vote we just close this issue.

[emmatyping]

IMHO, the best experience for this kind of thing is having it in IDE.

There are plenty of people who don't use IDEs that provide type information. And even though I use such a program I often wish I could look at the type signature when looking at the Python docs.

That being said I think a web based solution is best, so I don't think we need to make this a mypy issue.