Skip to content

CI run coverage on multiple builds #40394

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 36 commits into from
Mar 17, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
ec0479b
upload coverage
MarcoGorelli Mar 12, 2021
339ffe4
add pytest-cov
MarcoGorelli Mar 12, 2021
e1e973e
debug coverage
MarcoGorelli Mar 12, 2021
3669e85
cobertura
MarcoGorelli Mar 12, 2021
630722b
indent
MarcoGorelli Mar 12, 2021
8df6f80
wip
MarcoGorelli Mar 12, 2021
95623e4
use correct path in print_skipped.py
MarcoGorelli Mar 12, 2021
7ea05a0
full run
MarcoGorelli Mar 12, 2021
cc1f37f
try moving to ghactions
MarcoGorelli Mar 12, 2021
753c3ac
wip
MarcoGorelli Mar 12, 2021
25629d6
stage posix file
MarcoGorelli Mar 12, 2021
21b0682
coverage: true
MarcoGorelli Mar 12, 2021
36f96e9
fixup yml error
MarcoGorelli Mar 12, 2021
213a06f
PYTEST_WORKERS
MarcoGorelli Mar 12, 2021
d0f4569
remove pytest-azurepipelines
MarcoGorelli Mar 12, 2021
fff0217
valid yaml
MarcoGorelli Mar 12, 2021
d6eefae
pin pytest-cov
MarcoGorelli Mar 12, 2021
e9ba4be
install extra
MarcoGorelli Mar 12, 2021
e5e0087
install extra
MarcoGorelli Mar 12, 2021
231def7
wip
MarcoGorelli Mar 13, 2021
7c5d5d4
wip
MarcoGorelli Mar 13, 2021
318e206
flexi
MarcoGorelli Mar 13, 2021
ff9a9b7
wip
MarcoGorelli Mar 13, 2021
c938acf
wip
MarcoGorelli Mar 13, 2021
0ba4e76
cov in numpydev
MarcoGorelli Mar 13, 2021
ff4a09f
cov in numpydev
MarcoGorelli Mar 13, 2021
9338780
cov in numpydev
MarcoGorelli Mar 13, 2021
3e0321b
cov file
MarcoGorelli Mar 13, 2021
815fd04
put the rest
MarcoGorelli Mar 13, 2021
97d0dc0
remove linux job from azure-pipelines.yml
MarcoGorelli Mar 13, 2021
ba9f12d
remove linux job from azure-pipelines.yml
MarcoGorelli Mar 13, 2021
7f5052e
remove linux job from azure-pipelines.yml
MarcoGorelli Mar 13, 2021
d804bcd
dont fail fast
MarcoGorelli Mar 13, 2021
dbfb5b5
dont fail fast
MarcoGorelli Mar 13, 2021
66f46df
resize differently
MarcoGorelli Mar 13, 2021
a56b0a8
share ci/deps/
MarcoGorelli Mar 15, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .github/workflows/database.yml
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,13 @@ jobs:
- name: Print skipped tests
run: python ci/print_skipped.py

- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
with:
flags: unittests
name: codecov-pandas
fail_ci_if_error: false

Linux_py37_cov:
runs-on: ubuntu-latest
defaults:
Expand Down Expand Up @@ -174,7 +181,6 @@ jobs:
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
with:
files: /tmp/test_coverage.xml
flags: unittests
name: codecov-pandas
fail_ci_if_error: true
93 changes: 93 additions & 0 deletions .github/workflows/posix.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
name: Posix

on:
push:
branches: [master]
pull_request:
branches:
- master
- 1.2.x

env:
PYTEST_WORKERS: "auto"
PANDAS_CI: 1

jobs:
pytest:
runs-on: ubuntu-latest
defaults:
run:
shell: bash -l {0}
strategy:
matrix:
settings: [
[actions-37-minimum_versions.yaml, "not slow and not network and not clipboard", "", "", "", "", ""],
[actions-37.yaml, "not slow and not network and not clipboard", "", "", "", "", ""],
[actions-37-locale_slow.yaml, "slow", "language-pack-it xsel", "it_IT.utf8", "it_IT.utf8", "", ""],
[actions-37-slow.yaml, "slow", "", "", "", "", ""],
[actions-38.yaml, "not slow and not network and not clipboard", "", "", "", "", ""],
[actions-38-slow.yaml, "slow", "", "", "", "", ""],
[actions-38-locale.yaml, "not slow and not network", "language-pack-zh-hans xsel", "zh_CN.utf8", "zh_CN.utf8", "", ""],
[actions-38-numpydev.yaml, "not slow and not network", "xsel", "", "", "deprecate", "-W error"],
[actions-39.yaml, "not slow and not network and not clipboard", "", "", "", "", ""]
]
fail-fast: false
env:
COVERAGE: true
ENV_FILE: ci/deps/${{ matrix.settings[0] }}
PATTERN: ${{ matrix.settings[1] }}
EXTRA_APT: ${{ matrix.settings[2] }}
LANG: ${{ matrix.settings[3] }}
LC_ALL: ${{ matrix.settings[4] }}
PANDAS_TESTING_MODE: ${{ matrix.settings[5] }}
TEST_ARGS: ${{ matrix.settings[6] }}

steps:
- name: Checkout
uses: actions/checkout@v1

- name: Cache conda
uses: actions/cache@v1
env:
CACHE_NUMBER: 0
with:
path: ~/conda_pkgs_dir
key: ${{ runner.os }}-conda-${{ env.CACHE_NUMBER }}-${{
hashFiles('${{ env.ENV_FILE }}') }}

- name: Extra installs
run: sudo apt-get update && sudo apt-get install -y libc6-dev-i386 ${{ env.EXTRA_APT }}

- uses: conda-incubator/setup-miniconda@v2
with:
activate-environment: pandas-dev
channel-priority: flexible
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't get the environment to resolve with strict

environment-file: ${{ env.ENV_FILE }}
use-only-tar-bz2: true

- name: Build Pandas
uses: ./.github/actions/build_pandas

- name: Test
run: ci/run_tests.sh
if: always()

- name: Build Version
run: pushd /tmp && python -c "import pandas; pandas.show_versions();" && popd

- name: Publish test results
uses: actions/upload-artifact@master
with:
name: Test results
path: test-data.xml
if: failure()

- name: Print skipped tests
run: python ci/print_skipped.py

- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
with:
flags: unittests
name: codecov-pandas
fail_ci_if_error: false
5 changes: 0 additions & 5 deletions azure-pipelines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,6 @@ jobs:
name: macOS
vmImage: macOS-10.14

- template: ci/azure/posix.yml
parameters:
name: Linux
vmImage: ubuntu-16.04
Comment on lines -20 to -23
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved to .github/workflows/posix.yml


- template: ci/azure/windows.yml
parameters:
name: Windows
Expand Down
64 changes: 0 additions & 64 deletions ci/azure/posix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,71 +14,7 @@ jobs:
CONDA_PY: "37"
PATTERN: "not slow and not network"

${{ if eq(parameters.name, 'Linux') }}:
py37_minimum_versions:
ENV_FILE: ci/deps/azure-37-minimum_versions.yaml
CONDA_PY: "37"
PATTERN: "not slow and not network and not clipboard"

py37:
ENV_FILE: ci/deps/azure-37.yaml
CONDA_PY: "37"
PATTERN: "not slow and not network and not clipboard"

py37_locale_slow:
ENV_FILE: ci/deps/azure-37-locale_slow.yaml
CONDA_PY: "37"
PATTERN: "slow"
LANG: "it_IT.utf8"
LC_ALL: "it_IT.utf8"
EXTRA_APT: "language-pack-it xsel"

py37_slow:
ENV_FILE: ci/deps/azure-37-slow.yaml
CONDA_PY: "37"
PATTERN: "slow"

py38:
ENV_FILE: ci/deps/azure-38.yaml
CONDA_PY: "38"
PATTERN: "not slow and not network and not clipboard"

py38_slow:
ENV_FILE: ci/deps/azure-38-slow.yaml
CONDA_PY: "38"
PATTERN: "slow"

py38_locale:
ENV_FILE: ci/deps/azure-38-locale.yaml
CONDA_PY: "38"
PATTERN: "not slow and not network"
# pandas does not use the language (zh_CN), but should support different encodings (utf8)
# we should test with encodings different than utf8, but doesn't seem like Ubuntu supports any
LANG: "zh_CN.utf8"
LC_ALL: "zh_CN.utf8"
EXTRA_APT: "language-pack-zh-hans xsel"

py38_np_dev:
ENV_FILE: ci/deps/azure-38-numpydev.yaml
CONDA_PY: "38"
PATTERN: "not slow and not network"
TEST_ARGS: "-W error"
PANDAS_TESTING_MODE: "deprecate"
EXTRA_APT: "xsel"

py39:
ENV_FILE: ci/deps/azure-39.yaml
CONDA_PY: "39"
PATTERN: "not slow and not network and not clipboard"

steps:
- script: |
if [ "$(uname)" == "Linux" ]; then
sudo apt-get update
sudo apt-get install -y libc6-dev-i386 $EXTRA_APT
fi
displayName: 'Install extra packages'

Comment on lines -75 to -81
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these linux ones are all moved to .github/workflows/posix.yml

- script: echo '##vso[task.prependpath]$(HOME)/miniconda3/bin'
displayName: 'Set conda path'

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ dependencies:
# tools
- cython>=0.29.21
- pytest>=5.0.1
- pytest-cov
- pytest-xdist>=1.21
- hypothesis>=3.58.0
- pytest-azurepipelines

# pandas dependencies
- beautifulsoup4=4.6.0
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ dependencies:
# tools
- cython=0.29.21
- pytest=5.0.1
- pytest-cov
- pytest-xdist>=1.21
- hypothesis>=3.58.0
- pytest-azurepipelines
- psutil

# pandas dependencies
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ dependencies:
# tools
- cython>=0.29.21
- pytest>=5.0.1
- pytest-cov
- pytest-xdist>=1.21
- hypothesis>=3.58.0
- pytest-azurepipelines

# pandas dependencies
- beautifulsoup4
Expand Down
2 changes: 1 addition & 1 deletion ci/deps/azure-37.yaml → ci/deps/actions-37.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ dependencies:
# tools
- cython>=0.29.21
- pytest>=5.0.1
- pytest-cov
- pytest-xdist>=1.21
- hypothesis>=3.58.0
- pytest-azurepipelines

# pandas dependencies
- botocore>=1.11
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ dependencies:
# tools
- cython>=0.29.21
- pytest>=5.0.1
- pytest-cov
- pytest-xdist>=1.21
- pytest-asyncio>=0.12.0
- hypothesis>=3.58.0
- pytest-azurepipelines

# pandas dependencies
- beautifulsoup4
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ dependencies:

# tools
- pytest>=5.0.1
- pytest-cov
- pytest-xdist>=1.21
- hypothesis>=3.58.0
- pytest-azurepipelines

# pandas dependencies
- pytz
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ dependencies:
# tools
- cython>=0.29.21
- pytest>=5.0.1
- pytest-cov
- pytest-xdist>=1.21
- hypothesis>=3.58.0

Expand Down
2 changes: 1 addition & 1 deletion ci/deps/azure-38.yaml → ci/deps/actions-38.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ dependencies:
# tools
- cython>=0.29.21
- pytest>=5.0.1
- pytest-cov
- pytest-xdist>=1.21
- hypothesis>=3.58.0
- pytest-azurepipelines

# pandas dependencies
- numpy
Expand Down
2 changes: 1 addition & 1 deletion ci/deps/azure-39.yaml → ci/deps/actions-39.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ dependencies:
# tools
- cython>=0.29.21
- pytest>=5.0.1
- pytest-cov
- pytest-xdist>=1.21
- hypothesis>=3.58.0
- pytest-azurepipelines

# pandas dependencies
- numpy
Expand Down
9 changes: 1 addition & 8 deletions ci/run_tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,7 @@ if [[ "not network" == *"$PATTERN"* ]]; then
fi

if [ "$COVERAGE" ]; then
COVERAGE_FNAME="/tmp/test_coverage.xml"
COVERAGE="-s --cov=pandas --cov-report=xml:$COVERAGE_FNAME"
COVERAGE="-s --cov=pandas --cov-report=xml"
fi

# If no X server is found, we use xvfb to emulate it
Expand All @@ -30,9 +29,3 @@ fi

echo $PYTEST_CMD
sh -c "$PYTEST_CMD"

if [[ "$COVERAGE" && $? == 0 && "$TRAVIS_BRANCH" == "master" ]]; then
echo "uploading coverage"
echo "bash <(curl -s https://codecov.io/bash) -Z -c -f $COVERAGE_FNAME"
bash <(curl -s https://codecov.io/bash) -Z -c -f $COVERAGE_FNAME
fi
3 changes: 2 additions & 1 deletion codecov.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
codecov:
branch: master

notify:
after_n_builds: 10
Comment on lines +3 to +4
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is so that codecov doesn't report on insufficient coverage prematurely

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this related to why codecov often gives an Access Denied?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without this, then Codecov will report coverage as soon as one job finishes. But if that job isn't comprehensive, then it'll show e.g. "coverage 30% less than target of 80%, failed". Like this, it'll wait for multiple jobs to be done and will merge their reports

comment: false

coverage:
Expand Down
2 changes: 1 addition & 1 deletion pandas/plotting/_matplotlib/misc.py
Original file line number Diff line number Diff line change
Expand Up @@ -246,7 +246,7 @@ def f(t):
# appropriately. Take a copy of amplitudes as otherwise numpy
# deletes the element from amplitudes itself.
coeffs = np.delete(np.copy(amplitudes), 0)
coeffs.resize(int((coeffs.size + 1) / 2), 2)
coeffs = np.resize(coeffs, (int((coeffs.size + 1) / 2), 2))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was throwing an error during CI:

____________________ TestDataFramePlots.test_andrews_curves ____________________
[gw0] linux -- Python 3.8.8 /usr/share/miniconda/envs/pandas-dev/bin/python

self = <pandas.tests.plotting.test_misc.TestDataFramePlots object at 0x7f3ae76442b0>
iris =      SepalLength  SepalWidth  PetalLength  PetalWidth            Name
0            5.1         3.5          1.4       ...      2.3  Iris-virginica
149          5.9         3.0          5.1         1.8  Iris-virginica

[150 rows x 5 columns]

    def test_andrews_curves(self, iris):
        from matplotlib import cm
    
        from pandas.plotting import andrews_curves
    
        df = iris
        # Ensure no UserWarning when making plot
        with tm.assert_produces_warning(None):
>           _check_plot_works(andrews_curves, frame=df, class_column="Name")

pandas/tests/plotting/test_misc.py:158: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pandas/tests/plotting/common.py:625: in _check_plot_works
    raise err
pandas/tests/plotting/common.py:618: in _check_plot_works
    for ret in gen_plots(f, fig, **kwargs):
pandas/tests/plotting/common.py:645: in _gen_two_subplots
    yield f(**kwargs)
pandas/plotting/_misc.py:271: in andrews_curves
    return plot_backend.andrews_curves(
pandas/plotting/_matplotlib/misc.py:282: in andrews_curves
    y = f(t)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

t = array([-3.14159265, -3.11001886, -3.07844506, -3.04687127, -3.01529747,
       -2.98372368, -2.95214988, -2.92057608, ...900229,  2.92057608,  2.95214988,  2.98372368,
        3.01529747,  3.04687127,  3.07844506,  3.11001886,  3.14159265])

    def f(t):
        x1 = amplitudes[0]
        result = x1 / np.sqrt(2.0)
    
        # Take the rest of the coefficients and resize them
        # appropriately. Take a copy of amplitudes as otherwise numpy
        # deletes the element from amplitudes itself.
        coeffs = np.delete(np.copy(amplitudes), 0)
>       coeffs.resize(int((coeffs.size + 1) / 2), 2)
E       ValueError: cannot resize an array that references or is referenced
E       by another array in this way.
E       Use the np.resize function or refcheck=False

pandas/plotting/_matplotlib/misc.py:249: ValueError


# Generate the harmonics and arguments for the sin and cos
# functions.
Expand Down