[GOOD FIRST ISSUE] Refactor: Dynamically Register Adapter Configurations

[raphael-intugle]

---

name: Good First Issue
about: A beginner-friendly task perfect for first-time contributors
title: '[GOOD FIRST ISSUE] Refactor: Dynamically Register Adapter Configurations'
labels: 'good first issue'
assignees: ''

Welcome! 👋

This is a beginner-friendly issue perfect for first-time contributors to the Intugle project. We've designed this task to help you get familiar with our codebase while making a meaningful contribution.

Task Description

The DataSetData type in src/intugle/adapters/models.py is currently a hardcoded Union of all supported data configurations (e.g., pd.DataFrame | DuckdbConfig | SnowflakeConfig | DatabricksConfig). This makes the system less extensible, as new adapter configurations need to be manually added to this Union.

This task involves refactoring the AdapterFactory to dynamically register adapter configuration types. Instead of a hardcoded Union, the DataSetData type should be constructed dynamically based on the configurations registered by each adapter. This will make the system more pluggable and easier to extend with new data sources.

Why This Matters

Dynamically registering adapter configurations improves the modularity and extensibility of the Intugle library. It reduces the need for manual updates when new adapters are added and promotes a more robust, plug-and-play architecture. This is crucial for maintaining a scalable and adaptable codebase.

What You'll Learn

• Understanding Python's typing module and Union types.
• Working with Intugle's AdapterFactory and adapter registration mechanism.
• Implementing dynamic type construction.
• Improving code extensibility and maintainability.

Step-by-Step Guide

Prerequisites

• Python 3.10+ installed
• Git basics (clone, commit, push, pull request)
• Read our CONTRIBUTING.md guide

Setup Instructions

1. Fork and clone the repository

   git clone https://github.com/YOUR_USERNAME/data-tools.git
   cd data-tools

2. Create a virtual environment

   python -m venv .venv
   source .venv/bin/activate  # On Windows: .venv\Scripts\activate

3. Install dependencies

   pip install -e ".[dev]"

4. Create a new branch

   git checkout -b refactor/dynamic-adapter-configs

Implementation Steps

1. Modify AdapterFactory to store config types:

   • In src/intugle/adapters/factory.py, modify the AdapterFactory.register method to also accept and store the DatabricksConfig, DuckdbConfig, SnowflakeConfig (and pd.DataFrame for PandasAdapter) classes themselves, not just the checker and creator functions. You might need a new dictionary to store these config types.

2. Dynamically build DataSetData:

   • In src/intugle/adapters/models.py, remove the hardcoded DataSetData Union.
   • Instead, create a function or a mechanism within AdapterFactory (or a new utility module) that can dynamically construct this Union type by gathering all registered config types.
   • This might involve using typing.Union and types.new_class or similar approaches to create the type dynamically at runtime.

3. Update AdapterFactory.create method:

   • Ensure the create method can still correctly identify the appropriate adapter based on the input data type, now that DataSetData is dynamic.

4. Adjust existing adapters:

   • Modify the register functions in src/intugle/adapters/types/databricks/databricks.py, src/intugle/adapters/types/duckdb/duckdb.py, src/intugle/adapters/types/pandas/pandas.py, and src/intugle/adapters/types/snowflake/snowflake.py to pass their respective configuration classes (e.g., DatabricksConfig, DuckdbConfig, SnowflakeConfig, pd.DataFrame) to the AdapterFactory.register method.

5. Ensure backward compatibility:

   • The goal is to make the system more dynamic without breaking existing functionality. All current tests should still pass.

Files to Modify

• File: src/intugle/adapters/factory.py
  • Change: Modify register method to store config types; add logic to dynamically build DataSetData.
• File: src/intugle/adapters/models.py
  • Change: Remove hardcoded DataSetData Union; potentially import a dynamic type builder.
• File: src/intugle/adapters/types/databricks/databricks.py
  • Change: Update register function to pass DatabricksConfig.
• File: src/intugle/adapters/types/duckdb/duckdb.py
  • Change: Update register function to pass DuckdbConfig.
• File: src/intugle/adapters/types/pandas/pandas.py
  • Change: Update register function to pass pd.DataFrame (or a wrapper if needed).
• File: src/intugle/adapters/types/snowflake/snowflake.py
  • Change: Update register function to pass SnowflakeConfig.

Testing Your Changes

1. Run existing unit tests:

   pytest tests/

   Ensure all tests, especially those related to DataSet initialization and adapter creation, still pass.

2. Add a new test case:

   • Create a dummy adapter (similar to the SQLite adapter issue) and verify that its configuration type is correctly registered and included in the dynamically built DataSetData.

Submitting Your Work

1. Commit your changes

   git add .
   git commit -m "refactor: Dynamically register adapter configurations"

2. Push to your fork

   git push origin refactor/dynamic-adapter-configs

3. Create a Pull Request

   • Go to the original repository
   • Click "Pull Requests" → "New Pull Request"
   • Select your branch
   • Fill out the PR template
   • Reference this issue with "Fixes #ISSUE_NUMBER"

Expected Outcome

• The DataSetData type is dynamically generated based on registered adapters.
• New adapters can be added without manually updating the DataSetData Union.
• All existing functionality and tests remain intact.

Definition of Done

• AdapterFactory modified to store and dynamically build DataSetData.
• DataSetData in src/intugle/adapters/models.py is no longer hardcoded.
• All existing adapter register functions updated.
• All existing tests pass.
• New test case added to verify dynamic registration.
• Code follows project style guidelines
• No new linter warnings
• Documentation updated (if needed)
• Pull request submitted

Resources

• Project Documentation
• CONTRIBUTING.md
• Python Style Guide (PEP 8)
• Python typing module documentation

Need Help?

Don't hesitate to ask questions! We're here to help you succeed.

• Comment below with your questions
• Join our Discord for real-time support
• Tag maintainers: @raphael-intugle

Skills You'll Use

• Python basics
• Git and GitHub
• Testing with pytest (optional)
• Advanced Python typing
• Object-Oriented Programming (OOP)

---

Thank you for contributing to Intugle!

Tips for Success:

• Take your time and read through everything carefully
• Don't be afraid to ask questions
• Test your changes before submitting
• Have fun! 🎉

[cnaples79]

I opened a PR for this.

[raphael-intugle]

Great job! Will test it out and get back to you soon