Skip to content

unittest: Improve self.assert(Not)AlmostEqual(s) #8066

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Jun 14, 2022
Merged

unittest: Improve self.assert(Not)AlmostEqual(s) #8066

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
1b7ac44
`unittest`: Improve `self.assertAlmostEqual`
AlexWaygood Jun 12, 2022
615e997
Fix pyright complaints
AlexWaygood Jun 12, 2022
704eda1
Avoid regression with `datetime`, add a test
AlexWaygood Jun 12, 2022
33c02d2
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 12, 2022
70c2420
Improve tests
AlexWaygood Jun 12, 2022
93df0e0
One more
AlexWaygood Jun 12, 2022
389f686
Take care of the deprecated aliases as well
AlexWaygood Jun 12, 2022
3a30d0b
Merge remote-tracking branch 'origin/master' into almostequal
AlexWaygood Jun 13, 2022
a227de6
Get all tests to pass correctly
AlexWaygood Jun 13, 2022
6ed6d96
Fix
AlexWaygood Jun 13, 2022
761a483
Unused import
AlexWaygood Jun 13, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions stdlib/_typeshed/__init__.pyi
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,9 @@ class SupportsAdd(Protocol[_T_contra, _T_co]):
class SupportsRAdd(Protocol[_T_contra, _T_co]):
def __radd__(self, __x: _T_contra) -> _T_co: ...

class SupportsSub(Protocol[_T_contra, _T_co]):
def __sub__(self, __x: _T_contra) -> _T_co: ...

class SupportsDivMod(Protocol[_T_contra, _T_co]):
def __divmod__(self, __other: _T_contra) -> _T_co: ...

Expand Down
57 changes: 35 additions & 22 deletions stdlib/unittest/case.pyi
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,19 +1,32 @@
import datetime
import logging
import sys
import unittest.result
from _typeshed import Self
from _typeshed import Self, SupportsDunderGE, SupportsSub
from collections.abc import Callable, Container, Iterable, Mapping, Sequence, Set as AbstractSet
from contextlib import AbstractContextManager
from types import TracebackType
from typing import Any, AnyStr, ClassVar, Generic, NamedTuple, NoReturn, Pattern, TypeVar, overload
from typing import (
Any,
AnyStr,
ClassVar,
Generic,
NamedTuple,
NoReturn,
Pattern,
Protocol,
SupportsAbs,
SupportsRound,
TypeVar,
overload,
)
from typing_extensions import ParamSpec
from warnings import WarningMessage

if sys.version_info >= (3, 9):
from types import GenericAlias

_T = TypeVar("_T")
_S = TypeVar("_S", bound=SupportsSub[Any, Any])
_E = TypeVar("_E", bound=BaseException)
_FT = TypeVar("_FT", bound=Callable[..., Any])
_P = ParamSpec("_P")
Expand Down Expand Up @@ -62,6 +75,8 @@ def skipUnless(condition: object, reason: str) -> Callable[[_FT], _FT]: ...
class SkipTest(Exception):
def __init__(self, reason: str) -> None: ...

class _SupportsAbsAndDunderGE(SupportsDunderGE, SupportsAbs[Any], Protocol): ...

class TestCase:
failureException: type[BaseException]
longMessage: bool
Expand Down Expand Up @@ -165,33 +180,35 @@ class TestCase:
self, logger: str | logging.Logger | None = ..., level: int | str | None = ...
) -> _AssertLogsContext[None]: ...

@overload
def assertAlmostEqual(self, first: _S, second: _S, places: None, msg: Any, delta: _SupportsAbsAndDunderGE) -> None: ...
@overload
def assertAlmostEqual(
self, first: float, second: float, places: int | None = ..., msg: Any = ..., delta: float | None = ...
self, first: _S, second: _S, places: None = ..., msg: Any = ..., *, delta: _SupportsAbsAndDunderGE
) -> None: ...
@overload
def assertAlmostEqual(
self,
first: datetime.datetime,
second: datetime.datetime,
first: SupportsSub[_T, SupportsAbs[SupportsRound[object]]],
second: _T,
places: int | None = ...,
msg: Any = ...,
delta: datetime.timedelta | None = ...,
delta: None = ...,
) -> None: ...
@overload
def assertNotAlmostEqual(self, first: float, second: float, *, msg: Any = ...) -> None: ...
def assertNotAlmostEqual(self, first: _S, second: _S, places: None, msg: Any, delta: _SupportsAbsAndDunderGE) -> None: ...
@overload
def assertNotAlmostEqual(self, first: float, second: float, places: int | None = ..., msg: Any = ...) -> None: ...
@overload
def assertNotAlmostEqual(self, first: float, second: float, *, msg: Any = ..., delta: float | None = ...) -> None: ...
def assertNotAlmostEqual(
self, first: _S, second: _S, places: None = ..., msg: Any = ..., *, delta: _SupportsAbsAndDunderGE
) -> None: ...
@overload
def assertNotAlmostEqual(
self,
first: datetime.datetime,
second: datetime.datetime,
first: SupportsSub[_T, SupportsAbs[SupportsRound[object]]],
second: _T,
places: int | None = ...,
msg: Any = ...,
delta: datetime.timedelta | None = ...,
delta: None = ...,
) -> None: ...
def assertRegex(self, text: AnyStr, expected_regex: AnyStr | Pattern[AnyStr], msg: Any = ...) -> None: ...
def assertNotRegex(self, text: AnyStr, unexpected_regex: AnyStr | Pattern[AnyStr], msg: Any = ...) -> None: ...
Expand Down Expand Up @@ -249,14 +266,10 @@ class TestCase:
) -> None: ...
@overload
def failUnlessRaises(self, exception: type[_E] | tuple[type[_E], ...], msg: Any = ...) -> _AssertRaisesContext[_E]: ...
def failUnlessAlmostEqual(self, first: float, second: float, places: int = ..., msg: Any = ...) -> None: ...
def assertAlmostEquals(
self, first: float, second: float, places: int = ..., msg: Any = ..., delta: float = ...
) -> None: ...
def failIfAlmostEqual(self, first: float, second: float, places: int = ..., msg: Any = ...) -> None: ...
def assertNotAlmostEquals(
self, first: float, second: float, places: int = ..., msg: Any = ..., delta: float = ...
) -> None: ...
failUnlessAlmostEqual = assertAlmostEqual
assertAlmostEquals = assertAlmostEqual
failIfAlmostEqual = assertNotAlmostEqual
assertNotAlmostEquals = assertNotAlmostEqual
def assertRegexpMatches(self, text: AnyStr, regex: AnyStr | Pattern[AnyStr], msg: Any = ...) -> None: ...
def assertNotRegexpMatches(self, text: AnyStr, regex: AnyStr | Pattern[AnyStr], msg: Any = ...) -> None: ...
@overload
Expand Down
29 changes: 29 additions & 0 deletions test_cases/stdlib/test_unittest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# pyright: reportUnnecessaryTypeIgnoreComment=true

import unittest
from datetime import datetime, timedelta
from decimal import Decimal
from fractions import Fraction

case = unittest.TestCase()

case.assertAlmostEqual(2.4, 2.41)
case.assertAlmostEqual(Fraction(49, 50), Fraction(48, 50))
case.assertAlmostEqual(datetime(1999, 1, 2), datetime(1999, 1, 2, microsecond=1), delta=timedelta(hours=1))
case.assertAlmostEqual(datetime(1999, 1, 2), datetime(1999, 1, 2, microsecond=1), None, "foo", timedelta(hours=1))
case.assertAlmostEqual(Decimal("1.1"), Decimal("1.11"))
case.assertAlmostEqual(2.4, 2.41, places=8)
case.assertAlmostEqual(2.4, 2.41, delta=0.02)
case.assertAlmostEqual(2.4, 2.41, None, "foo", 0.02)

case.assertAlmostEqual(2.4, 2.41, places=9, delta=0.02) # type: ignore[call-overload]
case.assertAlmostEqual("foo", "bar") # type: ignore[call-overload]
case.assertAlmostEqual(datetime(1999, 1, 2), datetime(1999, 1, 2, microsecond=1)) # type: ignore[arg-type]

case.assertNotAlmostEqual(Fraction(49, 50), Fraction(48, 50))
case.assertNotAlmostEqual(datetime(1999, 1, 2), datetime(1999, 1, 2, microsecond=1), delta=timedelta(hours=1))
case.assertNotAlmostEqual(datetime(1999, 1, 2), datetime(1999, 1, 2, microsecond=1), None, "foo", timedelta(hours=1))

case.assertNotAlmostEqual(2.4, 2.41, places=9, delta=0.02) # type: ignore[call-overload]
case.assertNotAlmostEqual("foo", "bar") # type: ignore[call-overload]
case.assertNotAlmostEqual(datetime(1999, 1, 2), datetime(1999, 1, 2, microsecond=1)) # type: ignore[arg-type]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple thoughts about our tests in general:

  • The tests work very well for pull requests like this one :) I was going to write my own tests as part of reviewing this, but it was nice that they already existed.
  • I ran this test file with Python, making sure to get a runtime error whenever there's a type ignore comment. Ideally we would automate that somehow, but it doesn't seem to be worth the effort because we don't have that many tests.
  • The error codes in tests are pretty much noise to me. I ignore them. I don't really care whether "wrong argument types" is a call-overload or arg-type error.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran this test file with Python, making sure to get a runtime error whenever there's a type ignore comment. Ideally we would automate that somehow, but it doesn't seem to be worth the effort because we don't have that many tests.

Oh, interesting idea. Tests like this file especially are basically very doctest-esque, just without the >>> prompt.

The error codes in tests are pretty much noise to me. I ignore them. I don't really care whether "wrong argument types" is a call-overload or arg-type error.

Yes, and compounding this problem, there's the fact that we generally only include tests for the trickiest functions to write stubs for. These are usually the ones where mypy ends up emitting really weird error codes. E.g. loads of the ones in test_sum.py are [list-item], and that's really meaningless; we shouldn't be asserting that mypy should emit such a meaningless error code (and there's no real reason why we should be giving mypy special treatment in these tests either). It seemed like a good idea at the time, but it's probably time to call it a day on the error-codes-in-tests.