Return google symptoms to using current pandas version, with mitigations #1497
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dshemetov
reviewed
Sep 30, 2022
| @@ -37,7 +37,7 @@ def generate_transition_matrix(geo_res): | |||
| if geo_res == "hrr": | |||
| map_df["population"] = map_df["population"] * map_df["weight"] | |||
|
|
|||
| aggregated_pop = map_df.groupby(geo_res).sum().reset_index() | |||
| aggregated_pop = map_df.groupby(geo_res).sum(numeric_only=True).reset_index() | |||
There was a problem hiding this comment.
Wondering about the numeric_only=True. In the function arg type hint, it says there's no default, but in the docs below that it says it defaults to bool=True. Guessing that it actually tries to use NANs by default?
There was a problem hiding this comment.
I changed this because I'm getting a warning about to-be-deprecated behavior. Thought I'd just throw it in here.
FutureWarning: The default value of numeric_only in DataFrameGroupBy.sum is deprecated.
In a future version, numeric_only will default to False. Either specify numeric_only or select
only columns which should be valid for the function.
There was a problem hiding this comment.
The current default does actually seem to be numeric_only=True
There was a problem hiding this comment.
Ah thanks for that! Curious why they're moving away from numeric_only=True.
krivard
approved these changes
Oct 3, 2022
google_symptoms/tests/test_data/state_data_20220916-20220924.csv
Outdated
Show resolved
Hide resolved
|
@krivard This is ready to merge. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Followup to #1496.
Adapt pipeline to work correctly with new version of pandas. Add test to check for nulled-out rows. Fails when using pandas 1.4.0 and passes when using 1.3.5, which matches expected behavior.
Turn on
numeric_only=Trueto suppress future deprecation warning.Changelog
setup.pypull.pygeo.pyFixes
DataFrame.append().dbdatetodatetime64to prevent data from being nulled out.Bug probably related to the datetime-like changes in 1.4.0 and an issue they fixed, pandas-dev/pandas#42921