Skip to content

Refactor validators #5173

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 82 commits into
base: release-6.1.0rc0
Choose a base branch
from

Conversation

bmaranville
Copy link
Contributor

Code PR

  • I have read through the contributing notes and understand the structure of the package. In particular, if my PR modifies code of plotly.graph_objects, my modifications concern the codegen files and not generated files.
  • I have added tests (if submitting a new feature or correcting a bug) or
    modified existing tests.
  • For a new feature, I have added documentation examples in an existing or
    new tutorial notebook (please see the doc checklist as well).
  • I have added a CHANGELOG entry if fixing/changing/adding anything substantial.
  • For a new feature or a change in behaviour, I have updated the relevant docstrings in the code to describe the feature or behaviour (please see the doc checklist as well).

Refactor of validators

This refactor is based on release-6.0.1rc0

Helps address #1208

Motivation

The large number of files in the plotly/validators folder slows down package installation, esp. on systems where files are inspected for viruses during install (e.g. Windows)

The majority of the code is duplicated across these files as well, as there are e.g. >650 instances of nearly-identical classes inheriting _plotly_utils.basevalidators.BooleanValidator

Mitigation

Things to note, in the current validator arrangement:

  1. Nearly all the validator classes in plotly.validators inherit directly from classes in _plotly_utils.basevalidators
  2. None of them override any methods at all, they just provide custom kwargs to __init__
  3. Almost all access to the validators in the codebase is through the plotly.validator_cache.ValidatorCache.get_validator(parent_name, prop_name)
  4. All these validator python source modules are generated by the plotly build system, by running python commands.py codegen

In this PR, the following changes are made:

  1. During codegen, instead of generating and writing thousands of python files to plotly/validators/..., we create a single JSON file store at plotly/validators/_validators.json and for each (parent_name, prop_name) combination we add an entry to the store with
    • "superclass": the name of the validator class to instantiate
    • "params": the custom kwargs to use when instantiating the validator class
  2. The ValidatorCache.get_validator method is modified so that rather than calculating an import path and then importing a validator class from the file tree, it looks up the custom kwargs and validator class name from the JSON store, then creates (and caches) a new instance of the validator class from
    basevalidators, with the custom init kwargs, finally returning the instance
  3. Usages of direct import of validator classes from plotly.validators in the codebase are refactored to use the ValidatorCache instead (there were only a handful of these cases)

The net result is that the number of files to be installed is reduced from 14981 to 1652.

LiamConnors and others added 30 commits March 25, 2025 10:21
@gvwilson
Copy link
Contributor

this is looking good - thank you very much @bmaranville. We won't try to get it into the 6.1 release, but it will definitely be part of 6.2.

@gvwilson gvwilson self-assigned this May 12, 2025
@gvwilson gvwilson added feature something new P1 needed for current cycle community community contribution labels May 12, 2025
@gvwilson
Copy link
Contributor

Note: plotly.py 6.1.0rc0 is 7549710 bytes as a .tar.gz; these changes get that down to 6601687 bytes, which is a 12.5% reduction on top of the reductions in #4978. cc @alexcjohnson

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community community contribution feature something new P1 needed for current cycle
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants