BUG: Hash and compare tuple subclasses as builtin tuples #59286
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
matiaslindgren
changed the title
matiaslindgren
changed the title
matiaslindgren
changed the title
mroeschke
reviewed
Jul 22, 2024
pandas/tests/libs/test_hashtable.py
Outdated
| assert table.get_item(nan2) == 42 | ||
| with pytest.raises(KeyError, match=None) as error: | ||
| table.get_item(other) | ||
| assert str(error.value) == str(other) |
There was a problem hiding this comment.
Instead could you use the match argument instead of setting it to None?
There was a problem hiding this comment.
This line should be removed now that you added the match argument
There was a problem hiding this comment.
Yeah, that makes more sense. I'll fix the other tests too while I'm at it.
mroeschke
reviewed
Jul 22, 2024
mroeschke
added
Indexing
mroeschke
reviewed
Jul 22, 2024
Co-authored-by: Matthew Roeschke <[email protected]>
mroeschke
approved these changes
Jul 22, 2024
|
Thanks @matiaslindgren |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.Relaxing the
PyTuple_CheckExactin khash_python.h toPyTuple_Checkwould allow the usage of types created withcollections.namedtupleas index keys everywheretupleis expected.It is worth noting this would change the behaviour of all client code expecting custom
__eq__and__hash__functions intuplesubclasses to be respected.This also removes the non-deterministic behaviour in cases where
collections.namedtupletypes andtuples are used interchangeably.I traced the intermittent
KeyErrors to this part inPyObjectHashTable.I did not verify this, but I suspect the custom tuple hasher leads to a scenario where builtin
tuples are hashed using the custom hasher, while alltuplesubclasses are hashed with their own__hash__, resulting in different hash values.For small hash tables, the resulting index might point to the correct bucket just by coincidence.
A similar issue, but for different types, was reported in #39585.