⚠ This page is served via a proxy. Original site: https://github.com
This service does not collect credentials or authentication data.
Skip to content

Conversation

@eh-steve
Copy link

@eh-steve eh-steve commented Jan 27, 2026

Implement lazy loading to speed up import times for packages that use this library

Closes #544

Title

Improve import time by lazy-loading cloud provider modules

Description

Problem

Importing cloudpathlib currently takes ~435ms because it eagerly loads all cloud provider SDKs (google-cloud-storage, boto3, azure-storage-blob) at import time, even if the user only needs one provider.

This is problematic for:

  • CLI tools where startup latency matters
  • Serverless functions where cold start time is critical
  • Applications that only use one cloud provider but pay the import cost for all three

Benchmarks

Measured with Python 3.12 (5 runs, mean ± std):

Scenario Current With Lazy Loading Improvement
import cloudpathlib 435ms ± 24ms 22ms ± 1ms 20x faster
from cloudpathlib import S3Path 410ms ± 3ms 23ms ± 0ms 18x faster
from cloudpathlib import GSPath 410ms ± 3ms 24ms ± 1ms 17x faster
from cloudpathlib import AzureBlobPath 413ms ± 7ms 41ms ± 1ms 10x faster

Additionally, the current implementation loads all three SDKs even when importing just one path class. With lazy loading, only the SDK for the provider being used gets loaded.

Proposed Solution

Implement lazy loading via __getattr__ in the __init__.py files:

  • cloudpathlib/__init__.py
  • cloudpathlib/s3/__init__.py
  • cloudpathlib/gs/__init__.py
  • cloudpathlib/azure/__init__.py

This defers loading cloud SDKs until they are actually accessed, while preserving static type hints via TYPE_CHECKING blocks.

Example

Before:
import cloudpathlib # Takes ~435ms, loads all SDKsAfter:
import cloudpathlib # Takes ~22ms, no SDKs loaded
from cloudpathlib import S3Path # Only loads boto3 when S3Path is accessed

Implementation Notes

  • Use __getattr__ with TYPE_CHECKING blocks for static type hints (IDE support preserved)
  • Fix absolute import in cloudpath.py (from cloudpathlib.enumsfrom .enums)
  • Move anypath import to function-local in cloudpath.py to avoid circular imports
  • Add tests to verify lazy loading behavior and prevent regressions

Contributor checklist:

  • I have read and understood CONTRIBUTING.md
  • Confirmed an issue exists for the PR, and the text Closes #issue appears in the PR summary (e.g., Closes #123).
  • Confirmed PR is rebased onto the latest base
  • Confirmed failure before change and success after change
  • Any generic new functionality is replicated across cloud providers if necessary
  • Tested manually against live server backend for at least one provider
  • Added tests for any new functionality
  • Linting passes locally
  • Tests pass locally
  • Updated HISTORY.md with the issue that is addressed and the PR you are submitting. If the top section is not `## UNRELEASED``, then you need to add a new section to the top of the document for your change.

Implements lazy loading via __getattr__ for all cloud provider classes.
This defers loading heavy cloud SDKs (google-cloud-storage, boto3,
azure-storage-blob) until they are actually accessed, significantly
reducing import time for applications that only use a subset of providers.

Key changes:
- cloudpathlib/__init__.py: Use __getattr__ to lazy-load all path/client classes
- cloudpathlib/s3/__init__.py: Use __getattr__ to lazy-load S3Client, S3Path
- cloudpathlib/gs/__init__.py: Use __getattr__ to lazy-load GSClient, GSPath
- cloudpathlib/azure/__init__.py: Use __getattr__ to lazy-load AzureBlobClient, AzureBlobPath
- cloudpathlib/cloudpath.py: Change absolute import to relative; move anypath import to function-local
- tests/test_import_time.py: Add tests verifying lazy loading behavior

TYPE_CHECKING blocks preserve static type hints for IDE support.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve import time by lazy-loading cloud provider modules

1 participant