Skip to content

Use circleci to build conda packages #1154

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 38 commits into from
Sep 4, 2018
Merged

Use circleci to build conda packages #1154

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
fa6e350
Try adding a conda_build circleci job
jonmmease Sep 3, 2018
afdf1fc
Fix yaml indentation, enable 3.7 core
jonmmease Sep 3, 2018
f758339
Use separate conda installation for build
jonmmease Sep 3, 2018
a48b237
Add cacheing, remove noarch
jonmmease Sep 3, 2018
0f40e6b
Comment out conda test section for now
jonmmease Sep 3, 2018
cc68745
Add conda-verify dependency
jonmmease Sep 3, 2018
2f3d6b3
Fix conda build path
jonmmease Sep 3, 2018
ebfe52e
Add python version to conda_packages directory
jonmmease Sep 3, 2018
df868bb
Refactor conda logic into new create_conda_packages.sh script
jonmmease Sep 3, 2018
49da39d
Make script executable and Python 2.7, 3.5, and 3.6
jonmmease Sep 3, 2018
8c6ea10
Conda build from source directory (not PyPI) and get version from set…
jonmmease Sep 3, 2018
680d659
Fix conda root for switch to miniconda docker image
jonmmease Sep 3, 2018
471f6b6
Add branch filters for conda packaging workflows.
jonmmease Sep 3, 2018
b1b4ae6
Fix yaml indentation
jonmmease Sep 3, 2018
79dca7a
Fix yaml indentation (try again)
jonmmease Sep 3, 2018
1c9308c
Remove filter from 3.6
jonmmease Sep 3, 2018
dcfa24b
`filers` is not a list entry
jonmmease Sep 3, 2018
4c3f998
Add nodejs as build dependency
jonmmease Sep 3, 2018
b649fce
Restore Python 3.6 branch filters
jonmmease Sep 3, 2018
d4db34e
Commit compiled JavaScript for widget. This was CI package build
jonmmease Sep 3, 2018
9c8918e
Fix conda_packages directory name and install zip utility
jonmmease Sep 3, 2018
279a17f
Remove sudo
jonmmease Sep 3, 2018
901466e
copy don't move to keep from making conda-bld directory inconsistent
jonmmease Sep 3, 2018
cdda87f
Install zip with conda instead of apt (apt was failing to find it)
jonmmease Sep 3, 2018
675a5af
Re-enable all tests and add `requires` dependencies for conda builds
jonmmease Sep 4, 2018
7a2723c
Update contributing instructions with new anaconda channel upload
jonmmease Sep 4, 2018
e15486e
Remove old meta.yaml
jonmmease Sep 4, 2018
0e2ec99
Restore import tests in meta.yaml
jonmmease Sep 4, 2018
0538bce
Circle tests didn't run. Tre removing comment and reverting regular e…
jonmmease Sep 4, 2018
d2f5b37
Remove surrounding quotes for job names
jonmmease Sep 4, 2018
d91cb3c
That did it: Now re-enable all jobs and dependencies
jonmmease Sep 4, 2018
53358e0
Remove comment?
jonmmease Sep 4, 2018
1fabe0f
Comment non 2.7 jobs
jonmmease Sep 4, 2018
d91e314
That worked: uncomment 3.5
jonmmease Sep 4, 2018
92d910b
Fix Python version for conda 3.6
jonmmease Sep 4, 2018
6dd2ce0
Oh! We don't run plot_ly tests on python 3.5
jonmmease Sep 4, 2018
6e46222
... or Python 3.6
jonmmease Sep 4, 2018
becd440
... or orca job on Python 3.6
jonmmease Sep 4, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
179 changes: 147 additions & 32 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ version: 2

jobs:
# Core
"python-2.7-core":
python-2.7-core:
docker:
- image: circleci/python:2.7-stretch-node-browsers
environment:
Expand All @@ -18,7 +18,7 @@ jobs:
command: 'tox -e py27-core'
no_output_timeout: 20m

"python-3.4-core":
python-3.4-core:
docker:
- image: circleci/python:3.4-stretch-node-browsers
environment:
Expand All @@ -34,7 +34,7 @@ jobs:
command: 'tox -e py34-core'
no_output_timeout: 20m

"python-3.5-core":
python-3.5-core:
docker:
- image: circleci/python:3.5-stretch-node-browsers
environment:
Expand All @@ -50,7 +50,7 @@ jobs:
command: 'tox -e py35-core'
no_output_timeout: 20m

"python-3.6-core":
python-3.6-core:
docker:
- image: circleci/python:3.6-stretch-node-browsers
environment:
Expand All @@ -66,7 +66,7 @@ jobs:
command: 'tox -e py36-core'
no_output_timeout: 20m

"python-3.7-core":
python-3.7-core:
docker:
- image: circleci/python:3.7-stretch-node-browsers
environment:
Expand All @@ -83,7 +83,7 @@ jobs:
no_output_timeout: 20m

# Optional
"python-2.7-optional":
python-2.7-optional:
docker:
- image: circleci/python:2.7-stretch-node-browsers
environment:
Expand All @@ -103,7 +103,7 @@ jobs:
command: 'tox -e py27-optional'
no_output_timeout: 20m

"python-3.4-optional":
python-3.4-optional:
docker:
- image: circleci/python:3.4-stretch-node-browsers
environment:
Expand All @@ -123,7 +123,7 @@ jobs:
command: 'tox -e py34-optional'
no_output_timeout: 20m

"python-3.5-optional":
python-3.5-optional:
docker:
- image: circleci/python:3.5-stretch-node-browsers
environment:
Expand All @@ -143,7 +143,7 @@ jobs:
command: 'tox -e py35-optional'
no_output_timeout: 20m

"python-3.6-optional":
python-3.6-optional:
docker:
- image: circleci/python:3.6-stretch-node-browsers
environment:
Expand All @@ -163,7 +163,7 @@ jobs:
command: 'tox -e py36-optional'
no_output_timeout: 20m

"python-3.7-optional":
python-3.7-optional:
docker:
- image: circleci/python:3.7-stretch-node-browsers
environment:
Expand All @@ -184,7 +184,7 @@ jobs:
no_output_timeout: 20m

# Plot.ly
"python-2.7-plot_ly":
python-2.7-plot_ly:
docker:
- image: circleci/python:2.7-stretch-node-browsers
environment:
Expand All @@ -200,7 +200,7 @@ jobs:
command: 'tox -e py27-plot_ly'
no_output_timeout: 20m

"python-3.4-plot_ly":
python-3.4-plot_ly:
docker:
- image: circleci/python:3.4-stretch-node-browsers
environment:
Expand All @@ -216,7 +216,7 @@ jobs:
command: 'tox -e py34-plot_ly'
no_output_timeout: 20m

"python-3.7-plot_ly":
python-3.7-plot_ly:
docker:
- image: circleci/python:3.7-stretch-node-browsers
environment:
Expand All @@ -232,7 +232,7 @@ jobs:
command: 'tox -e py37-plot_ly'
no_output_timeout: 20m

"python-2-7-orca":
python-2-7-orca:
docker:
- image: circleci/node:10.9-stretch-browsers
environment:
Expand Down Expand Up @@ -262,7 +262,7 @@ jobs:
- store_artifacts:
path: plotly/tests/test_orca/images/linux/failed

"python-3-5-orca":
python-3-5-orca:
docker:
- image: circleci/node:10.9-stretch-browsers
environment:
Expand Down Expand Up @@ -292,7 +292,7 @@ jobs:
- store_artifacts:
path: plotly/tests/test_orca/images/linux/failed

"python-3-7-orca":
python-3-7-orca:
docker:
- image: circleci/node:10.9-stretch-browsers
environment:
Expand Down Expand Up @@ -322,24 +322,139 @@ jobs:
- store_artifacts:
path: plotly/tests/test_orca/images/linux/failed


python-2-7-conda:
docker:
- image: continuumio/miniconda3:4.5.4
environment:
PYTHON_VERSION: 2.7

steps:
- checkout
- run:
name: Create packages
command: .circleci/create_conda_packages.sh
- store_artifacts:
path: artifacts

python-3-5-conda:
docker:
- image: continuumio/miniconda3:4.5.4
environment:
PYTHON_VERSION: 3.5

steps:
- checkout
- run:
name: Create packages
command: .circleci/create_conda_packages.sh
- store_artifacts:
path: artifacts

python-3-6-conda:
docker:
- image: continuumio/miniconda3:4.5.4
environment:
PYTHON_VERSION: 3.6

steps:
- checkout
- run:
name: Create packages
command: .circleci/create_conda_packages.sh
- store_artifacts:
path: artifacts

python-3-7-conda:
docker:
- image: continuumio/miniconda3:4.5.4
environment:
PYTHON_VERSION: 3.7

steps:
- checkout
- run:
name: Create packages
command: .circleci/create_conda_packages.sh
- store_artifacts:
path: artifacts


workflows:
version: 2
build:
jobs:
- "python-2.7-core"
- "python-3.4-core"
- "python-3.5-core"
- "python-3.6-core"
- "python-3.7-core"
- "python-2.7-optional"
- "python-3.4-optional"
- "python-3.5-optional"
- "python-3.6-optional"

- python-2.7-core
- python-3.4-core
- python-3.5-core
- python-3.6-core
- python-3.7-core
- python-2.7-optional
- python-3.4-optional
- python-3.5-optional
- python-3.6-optional
# 3.7 optional disabled due to current shapely incompatibility
# - "python-3.7-optional"
- "python-2.7-plot_ly"
- "python-3.4-plot_ly"
- "python-3.7-plot_ly"
- "python-2-7-orca"
- "python-3-5-orca"
- "python-3-7-orca"
# - python-3.7-optional
- python-2.7-plot_ly
- python-3.4-plot_ly
- python-3.7-plot_ly
- python-2-7-orca
- python-3-5-orca
- python-3-7-orca

- python-2-7-conda:
requires:
- python-2.7-core
- python-2.7-optional
- python-2.7-plot_ly
- python-2-7-orca

filters:
branches:
only:
- master
- /release_.*/
- /conda_build_.*/

- python-3-5-conda:
requires:
- python-3.5-core
- python-3.5-optional
# - python-3.5-plot_ly
- python-3-5-orca

filters:
branches:
only:
- master
- /release_.*/
- /conda_build_.*/

- python-3-6-conda:
requires:
- python-3.6-core
- python-3.6-optional
# - python-3.6-plot_ly
# - python-3-6-orca

filters:
branches:
only:
- master
- /release_.*/
- /conda_build_.*/

- python-3-7-conda:
requires:
- python-3.7-core
# - python-3.7-optional
- python-3.7-plot_ly
- python-3-7-orca

filters:
branches:
only:
- master
- /release_.*/
- /conda_build_.*/
17 changes: 17 additions & 0 deletions .circleci/create_conda_packages.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Install conda dependencies
conda install -y conda-build conda-verify zip

# Perform build
conda build --python $PYTHON_VERSION recipe/

# Convert to other architectures
mkdir -p ./conda_packages_${PYTHON_VERSION}/linux-64/
cp /opt/conda/conda-bld/linux-64/plotly-*.tar.bz2 ./conda_packages_${PYTHON_VERSION}/linux-64/
conda convert -p linux-32 ./conda_packages_${PYTHON_VERSION}/linux-64/plotly-*.tar.bz2 -o ./conda_packages_${PYTHON_VERSION}/
conda convert -p osx-64 ./conda_packages_${PYTHON_VERSION}/linux-64/plotly-*.tar.bz2 -o ./conda_packages_${PYTHON_VERSION}/
conda convert -p win-64 ./conda_packages_${PYTHON_VERSION}/linux-64/plotly-*.tar.bz2 -o ./conda_packages_${PYTHON_VERSION}/
conda convert -p win-32 ./conda_packages_${PYTHON_VERSION}/linux-64/plotly-*.tar.bz2 -o ./conda_packages_${PYTHON_VERSION}/

# zip up packages into artifacts directory
mkdir artifacts
zip -r artifacts/conda_packages_${PYTHON_VERSION}.zip ./conda_packages_${PYTHON_VERSION}/
15 changes: 15 additions & 0 deletions .circleci/install_miniconda_and_build_.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/usr/bin/env bash

if [ ! -d $HOME/miniconda/ ]; then
# Download miniconda
wget http://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh

chmod +x miniconda.sh

# Install miniconda
./miniconda.sh -b -p $HOME/miniconda

# Create environment
# PYTHON_VERSION=3.6
$HOME/miniconda/bin/conda install conda-build conda-verify
fi
3 changes: 0 additions & 3 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,6 @@ plotly/presentation_objs/

js/node_modules/

# Compiled javascript
plotlywidget/static/

.pytest_cache

# virtual envs
Expand Down
Loading