Speed up client initialization with lazy loading

[jhamon]

Problem

In the upcoming release we want to take a dependency on the Assistants plugin. But we don't want this dependency to come at a cost of degraded performance for users who are not using the features in the plugin.

Solution

To accomplish these goals I wanted to do some significant refactoring without breaking any existing functionality.

• Existing integration tests need to pass without major modification

Initialization performance

To assess the load time for the pinecone package, I used a built-in package called importtime.

poetry run python3 -X importtime -c "from pinecone import Pinecone; pc = Pinecone(api_key='foo')" 2> main.log

Then I visualized the results using a new dev-dependency called tuna

poetry run tuna main.log

These steps can be used to show that before any refactoring, the initialization time was more than 300ms(!)

After refactoring to make PluginAware lazy, and also restructure code related to operations on indexes, collections, inference, etc to take advantage of lazy loading we can improve the client initialization time very significantly. This is a big improvement because it means users will no longer need to wait to load a bunch of code for features they are not using.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update
• Infrastructure change (CI configs, etc)
• Non-code change (docs, etc)
• None of the above: (explain here)

Test Plan

Describe specific steps for validating this change.