Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor path #3355

Merged
merged 17 commits into from Jul 12, 2023
Merged

Refactor path #3355

merged 17 commits into from Jul 12, 2023

Conversation

yinweisu
Copy link
Collaborator

@yinweisu yinweisu commented Jun 26, 2023

Issue #, if available:

Description of changes:

  • Refactor path logic in Tabular/TimeSeries, so that parent only will track relative paths to its children and passing correct path to its children when init/load
  • Trainer now stores model's relative path as a list, and dynamically construct them to a path.
  • Replaced usage os os.path.sep + str concatenation in path with os.path.join to avoid bugs
  • Cross OS check for tabular
  • Cross OS check for timeseries

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@yinweisu yinweisu added the model list checked You have updated the model list after modifying multimodal unit tests/docs label Jun 27, 2023
@github-actions
Copy link

github-actions bot commented Jul 6, 2023

Job PR-3355-705d09d is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-3355/705d09d/index.html

@Innixma Innixma added enhancement New feature or request priority: 0 Maximum priority labels Jul 7, 2023
@Innixma Innixma added this to the 0.9 Release milestone Jul 7, 2023
@github-actions
Copy link

github-actions bot commented Jul 7, 2023

Job PR-3355-18ef70f is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-3355/18ef70f/index.html

@yinweisu yinweisu marked this pull request as ready for review July 10, 2023 17:08
@yinweisu yinweisu requested review from Innixma and shchur July 10, 2023 17:08
Copy link
Contributor

@Innixma Innixma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, this is an excellent and much needed refactor!

Copy link
Collaborator

@shchur shchur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR, this looks great!

@yinweisu yinweisu merged commit 1cd5f98 into autogluon:master Jul 12, 2023
32 checks passed
@yinweisu yinweisu deleted the refactor_path branch July 12, 2023 17:30
ddelange added a commit to ddelange/autogluon that referenced this pull request Jul 26, 2023
* 'master' of https://github.com/awslabs/autogluon:
  Fix crash when `time_limit` is None and overwritten via `max_time_limit` (autogluon#3418)
  Upgrade torch 2.0 (autogluon#3404)
  Fix FastAI crash edge-case (autogluon#3416)
  Add log for fit time adjustments (autogluon#3408)
  [fix][eda] Anomaly Detection - disable bps_flag for SUOD (autogluon#3406)
  [timeseries] Remove all references to MXNet (autogluon#3396)
  Remove ray lightning (autogluon#3398)
  [AutoMM] Add unit tests for trimming sequence lengths (autogluon#3399)
  [FIX] fix detection tutorial index (autogluon#3397)
  Refactor path (autogluon#3355)
  [Doc] Add AutoMM FAQs (autogluon#3388)
  [AutoMM] Fix the input keys of categorical MLP (autogluon#3384)
  [AutoMM] Clean up text processor (autogluon#3383)
  [AutoMM] Support customizing use_fast for AutoTokenizer (autogluon#3379)
  vw version bump (autogluon#3373)
  0.8.2 post release (autogluon#3377)
@eromoe
Copy link

eromoe commented Aug 25, 2023

Similar error:
ValueError: path is on mount 'C:', start on mount 'E:'

I run code in E , and set path to C

TabularPredictor(label='y', eval_metric='precision', path=autgoluon_model_path)
ValueError                                Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_17032/2841266324.py in <module>
----> 1 m1 = train(X1_train, y1_train, X1_test, y1_test)

~\AppData\Local\Temp/ipykernel_17032/2534130983.py in train(X_train, y_train, X_test, y_test)
     42         m = model_cls(eval_metric=eval_metric, path=new_model_path)
     43
---> 44         m.fit(X_train, y_train)
     45
     46         if X_test.shape[0]:

e:\workspace\github_me\mlquant\mlquant\my_auto.py in fit(self, X, y, **kwargs)
     30
     31     def fit(self, X, y, **kwargs):
---> 32         return super().fit(X.assign(y=y), hyperparameters=self.hyperparameters, verbosity=0, **kwargs)
     33
     34     def predict_proba(self, data, model=None,):

~\anaconda3\lib\site-packages\autogluon\core\utils\decorators.py in _call(*args, **kwargs)
     29         def _call(*args, **kwargs):
     30             gargs, gkwargs = g(*other_args, *args, **kwargs)
---> 31             return f(*gargs, **gkwargs)
     32
     33         return _call

~\anaconda3\lib\site-packages\autogluon\tabular\predictor\predictor.py in fit(self, train_data, tuning_data, time_limit, presets, hyperparameters, feature_metadata, infer_limit, infer_limit_batch_size, fit_weighted_ensemble, calibrate_decision_threshold, num_cpus, num_gpus, **kwargs)
    984             aux_kwargs["fit_weighted_ensemble"] = False
    985         self.save(silent=True)  # Save predictor to disk to enable prediction and training after interrupt
--> 986         self._learner.fit(
    987             X=train_data,
    988             X_val=tuning_data,

~\anaconda3\lib\site-packages\autogluon\tabular\learner\abstract_learner.py in fit(self, X, X_val, **kwargs)
    157             raise AssertionError("Learner is already fit.")
    158         self._validate_fit_input(X=X, X_val=X_val, **kwargs)
--> 159         return self._fit(X=X, X_val=X_val, **kwargs)
    160
    161     def _fit(

~\anaconda3\lib\site-packages\autogluon\tabular\learner\default_learner.py in _fit(self, X, X_val, X_unlabeled, holdout_frac, num_bag_folds, num_bag_sets, time_limit, infer_limit, infer_limit_batch_size, verbosity, **trainer_fit_kwargs)
    133             time_limit_trainer = None
    134
--> 135         trainer = self.trainer_type(
    136             path=self.model_context,
    137             problem_type=self.label_cleaner.problem_type_transform,

~\anaconda3\lib\site-packages\autogluon\core\trainer\abstract_trainer.py in __init__(self, path, problem_type, eval_metric, num_classes, quantile_levels, low_memory, feature_metadata, k_fold, n_repeats, sample_weight, weight_evaluation, save_data, random_state, verbosity)
    102     ):
    103         self.path = path
--> 104         self.path = PathConverter.to_relative(self.path)
    105         self.problem_type = problem_type
    106         self.feature_metadata = feature_metadata

~\anaconda3\lib\site-packages\autogluon\common\utils\path_converter.py in to_relative(path)
     54             return path
     55         os_path_sep = PathConverter.os_path_sep()
---> 56         path_relative = os.path.relpath(path)
     57         if path.endswith(os_path_sep):
     58             if not path_relative.endswith(os_path_sep):

~\anaconda3\lib\ntpath.py in relpath(path, start)
    701         path_drive, path_rest = splitdrive(path_abs)
    702         if normcase(start_drive) != normcase(path_drive):
--> 703             raise ValueError("path is on mount %r, start on mount %r" % (
    704                 path_drive, start_drive))
    705

ValueError: path is on mount 'C:', start on mount 'E:'

@Innixma Innixma modified the milestones: 0.9 Release, 1.0 Release Oct 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request model list checked You have updated the model list after modifying multimodal unit tests/docs priority: 0 Maximum priority
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants