quick example possible?

[KonradHoeffner]

Even on an i9-12900k without a GPU the fastest example estimates 4 hours of duration. Is it possible to create an example that can be run faster or to modify one of the existing ones?

[Ayuei]

I think I can potentially package an index prebuilt for download. Also, which example are you referring to?

[KonradHoeffner]

I think it was both finetune_biencoder and hparam_tuning that estimated around 4 hours or slightly more of runtime.

I am not sure if integrated Intel GPUs are supported by https://www.intel.com/content/www/us/en/developer/tools/oneapi/optimization-for-pytorch.html but if yes that could be worth investigating as well.

[Ayuei]

I added a limit flag for the two examples you mentioned 08e8e3b

Run the examples again but with --limit 100 or any number less than the total number of samples.

As for the integrated intel CPUs/GPUs, I added a commit to allowing hooking into the fit/training loop. So you could do something like this:

    # Snippet from biencoder finetuning

   import intel_extension_for_pytorch as ipex
   
   def ipex_hook(self, **kwargs):
        param_optimizer = list(loss_model.named_parameters())

       no_decay = ['bias', 'LayerNorm.bias', 'LayerNorm.weight']
       optimizer_grouped_parameters = [
                {'params': [p for n, p in param_optimizer if not any(nd in n for nd in no_decay)], 'weight_decay': 0.01},
                {'params': [p for n, p in param_optimizer if any(nd in n for nd in no_decay)], 'weight_decay': 0.0}
        ]

        optimizer = optimizer_class()

        optimizer = kwargs['optimizer_class'](optimizer_grouped_parameters, lr = 2e-5) 
        model, optimizer = ipex.optimize(self.model, optimizer=optimizer)
         self.model = model

        # Wrapper needed as Sentence Transformer backend library will attempt to create it's own optimizer
        def optimizer_wrapper(*args, **kwargs):
              return optimizer

    trainer = SentenceTransformerTrainer(
        # Reuse the dataset with lazy static, so we don't have to do the preprocessing repeatedly
        dataset=load_dataset(limit),
        hparams_config=HparamConfig.from_json("hparam_cfg.json"),
        evaluator_fn=evaluation.BinaryClassificationEvaluator,
    )
   
    # Overload with our prehook
    trainer.prehook = ipex_hook

    trainer.fit()