Skip to content

[FSTORE-1160] Remove wait_for_job in all tutorials #232

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 21 commits into from
Feb 12, 2024
Merged

[FSTORE-1160] Remove wait_for_job in all tutorials #232

merged 21 commits into from
Feb 12, 2024

Conversation

@Maxxx-zh
Copy link
Contributor

@Maxxx-zh Maxxx-zh commented Feb 7, 2024

No description provided.

@Maxxx-zh Maxxx-zh requested a review from davitbzh February 7, 2024 17:12
@manu-sj manu-sj self-requested a review February 8, 2024 09:49
@manu-sj
Copy link
Collaborator

manu-sj commented Feb 8, 2024 •
edited
Loading

  1. File integrations/federated-offline-query/federated-offline-query.ipynb:263 has wait_for_job and I think this can also be removed.
  2. Files integrations/monitoring/feature_statistics_monitoring.ipynb and integrations/wandb/2_feature_view_creation.ipynb has wait_for_job in create_train_validation_test_split function and I think this can also be removed.
  3. For credit_scores tutorials wait_for_job is not removed for the fifth feature group insert is this intentional?

manu-sj
manu-sj approved these changes Feb 9, 2024
View reviewed changes
advanced_tutorials/recommender-system/1_feature_engineering.ipynb
Copy link
Collaborator

@manu-sj manu-sj Feb 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should leave the wait_for_job in this tutorial since ranking_df is calculated based on perviously inserted feature groups. If we don't wait and the user runs compute_ranking_dataset then the returned dataframe will be empty and insertion will fail.

advanced_tutorials/citibike/2_citibike_feature_pipeline.ipynb
Copy link
Collaborator

@manu-sj manu-sj Feb 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just something I noticed while testing is that this tutorial does not work with master but works in app.

@davitbzh davitbzh merged commit ed0c662 into logicalclocks:master Feb 12, 2024
manu-sj added a commit that referenced this pull request Sep 17, 2024
@manu-sj @Maxxx-zh
@steffengr @jimdowling @robzor92 @javierdlrm @davitbzh @kennethmhc @rvanbruggen @rcnnnghm @SirOibaf
Merge master branch into dev branch to sync them (#280)
51ac075 
* [FSTORE-1081] GitHub Actions Fix (#220)

Github actions

* Add notebook for federated offline query (#221)

Co-authored-by: Steffen Grohsschmiedt <steffen@logicalclocks.com>

* Update README.md

* [FSTORE-1055] Pandas 2.1.x compatibility (#225)

* Upload predictor script to Resources folder (#228)

* [FSTORE-1129] Tutorials Update (#224)

* Minor updates to all tutorials - variable naming, etc

* [FSTORE-1141] Mage-AI Tutorial (#226)

* Mage Tutorial for fraud use case

* [FSTORE-1160] Remove wait_for_job in all tutorials (#232)

Remove wait_for_job in all tutorials

* [FSTORE-1231] README for the MageAI tutorial (#233)

* Update README.md

* [FSTORE-1196] Neo4j tutorial (#234)

Neo4j tutorial

* [FSTORE-612] Update Feature Monitoring tutorial (#236)

* [FSTORE-1196][APPEND] Link from README to Neo4j tutorial (#235)

 Link from README to Neo4j tutorial

* [FSTORE-1239] Rewrite Recommendation System tutorial using new Embeddings API (#237)

* Recommender example re-written with the 3.7 API containing embeddings.

* [FSTORE-1239] [APPEND] - Embeddings API Explanation (#238)

Embeddings API Explanation

* [FSTORE-1259] Add new change notification for feature groups example notebook (#240)

* Add new change notification for feature groups example notebook

* [FSTORE-1196][APPEND] Neo4j tutorial fixes (#241)

* neo4j tutorial fixes

* [FSTORE-1262] Tutorials for similarity search (#242)

* Added tutorials for similarity search for feature groups and feature views

* neo4j tutorial working on serverless (#243)

neo4j tutorial working on serverless

* [FSTORE-1196] [APPEND]  Instructions for Transformer  (#244)

 add clarification on feature view and training data version for transformer

* [FSTORE-1239] [APPEND] - Remove deprecated tensorflow-addons library (#245)

* remove deprecated tensorflow-addons

Co-authored-by: davitbzh <davit.bzhalava@gmail..com>

* [FSTORE-1298] add requirements.txt to vector embeddings (#248)

* add requirements.txt to vector embeddings. refactor knn search to allow to be run for just search

---------

Co-authored-by: Jim Dowling <jim@hopsworks.ai>

* [FSTORE-1186] Adding tutorial for Polars (#246)

Polars Integration into Hopsworks

* [FSTORE-1207] AirQuality LLM project (#250)

* Function Calling & AirQuality FunctionCalling Chatbot

* [FSTORE-1280] Tutorial about Opensearch integration with LangChain (#251)

* add notebook

* rename folder

* add prerequisite step

* use get_feature_group

* AirQuality OpenAI API Support (#253)

* notebook for saving model (#247)

* [FSTORE-1107] Pyspark streaming Tutorial (#227)

* Draft pyspark streaming

* adding init.py for module

* adding code to download libaries in hopsworks jupyter

* adding topic for testing

* adding notebook for creating simulated data stream

* correcting imports

* working base pyspark streaming

* adding all pipeline pyspark streaming

* updated tutorial

* fixed pyspark streaming

* clearning all outputs from notebooks

* renaming filenames in tutorial

* adding online inference pipeline instead of batch inference

* pyspark streaming online inference with deployments

* moving polars and pyspark streaming tutorial to integrations folder

* Updating readme based on moved polars tutorials and adding Pyspark streaming into readme

* AML Tutorial (#255)

* use model dimension (#254)

* Add a snowflake example, with snowflake as a data source to FGs in Hopsworks (#257)

Co-authored-by: Jim Dowling <jim@hopsworks.ai>

* [FSTORE-1186][APPEND] Adding tutorial for Polar

* ChequeDetection (#260)

* Feature pipeline with Snowflake as a data source (#258)

* Add a snowflake example, with snowflake as a data source to FGs in Hopsworks and fix polars tutorial

---------

Co-authored-by: Jim Dowling <jim@hopsworks.ai>

* Cosmetic improvements and spelling errors (#256)

* Data (#261)

* [FSTORE-1385][APPEND] Data Loading Optimization (#262)

* Data Loading Optimization

* [FSTORE-1395] Update the Langchain usage in Fraud Cheque Detection (#263)

* Update the Langchain usage in Fraud Cheque Detection

* [FSTORE-1394] Update the Langchain usage in AirQuality (#264)

* Update the Langchain usage in AirQuality

* Update the date of the test questions

* StrOutputParser Usage

* [FSTORE-1396] Tutorials Update: Save/load XGBoost model as json file (#265)

* Replace pickling xgboost models with saving them as json objects for the model registry

* [FSTORE-1404] LLM PDF Tutorial (#266)

* LLM PDF Search Tutorial  using RAG and Fine-Tuning

* [FSTORE-1408] LLM PDFs README (#267)

* Readme for pdf LLMs

* Update README.md

Added reference to video, that you can use a private trainer model.

* Update README.md

Fix titles

* Update README.md

Nicer readme for llm-pdfs

* [FSTORE-1422] Fraud Cheque Detection README (#268)

* Create README.md

* Update README.md

* Typo

* VideoLink

* Update README.md

Minor grammatical changes.

---------

Co-authored-by: rcnnnghm <Raymond.Cunningham@gmail.com>

* Add AzureSQL example (#270)

* Move feature_monitoring tutorial to api_examples (#271)

* [FSTORE-1437] Hospital Wait Time Tutorial (#272)

* hospital waiting time forecasting tutorial

* Move similarity search examples (#275)

* TikTok Recommendation System (#277)

---------

Co-authored-by: Maksym Zhytnikov <63515947+Maxxx-zh@users.noreply.github.com>
Co-authored-by: Steffen Grohsschmiedt <gitbhub@steffeng.eu>
Co-authored-by: Steffen Grohsschmiedt <steffen@logicalclocks.com>
Co-authored-by: Jim Dowling <jim@logicalclocks.com>
Co-authored-by: Robin Andersson <robin@hopsworks.ai>
Co-authored-by: Javier de la Rúa Martínez <javier@logicalclocks.com>
Co-authored-by: Marco Pellegrino <marco@hopsworks.ai>
Co-authored-by: davitbzh <44586065+davitbzh@users.noreply.github.com>
Co-authored-by: kennethmhc <kennethmhc@users.noreply.github.com>
Co-authored-by: davitbzh <davit.bzhalava@gmail..com>
Co-authored-by: Jim Dowling <jim@hopsworks.ai>
Co-authored-by: Rik Van Bruggen <2995654+rvanbruggen@users.noreply.github.com>
Co-authored-by: rcnnnghm <Raymond.Cunningham@gmail.com>
Co-authored-by: Fabio Buso <buso.fabio@gmail.com>
manu-sj added a commit that referenced this pull request Sep 17, 2024
@manu-sj @Maxxx-zh
@steffengr @jimdowling @robzor92 @javierdlrm @davitbzh @kennethmhc @rvanbruggen @rcnnnghm @SirOibaf
[FSTORE-1411] On-Demand Transformations (#278)
7bc62cd 
* [FSTORE-1081] GitHub Actions Fix (#220)

Github actions

* Add notebook for federated offline query (#221)

Co-authored-by: Steffen Grohsschmiedt <steffen@logicalclocks.com>

* Update README.md

* [FSTORE-1055] Pandas 2.1.x compatibility (#225)

* Upload predictor script to Resources folder (#228)

* [FSTORE-1129] Tutorials Update (#224)

* Minor updates to all tutorials - variable naming, etc

* [FSTORE-1141] Mage-AI Tutorial (#226)

* Mage Tutorial for fraud use case

* [FSTORE-1160] Remove wait_for_job in all tutorials (#232)

Remove wait_for_job in all tutorials

* [FSTORE-1231] README for the MageAI tutorial (#233)

* Update README.md

* [FSTORE-1196] Neo4j tutorial (#234)

Neo4j tutorial

* [FSTORE-612] Update Feature Monitoring tutorial (#236)

* [FSTORE-1196][APPEND] Link from README to Neo4j tutorial (#235)

 Link from README to Neo4j tutorial

* [FSTORE-1239] Rewrite Recommendation System tutorial using new Embeddings API (#237)

* Recommender example re-written with the 3.7 API containing embeddings.

* [FSTORE-1239] [APPEND] - Embeddings API Explanation (#238)

Embeddings API Explanation

* [FSTORE-1259] Add new change notification for feature groups example notebook (#240)

* Add new change notification for feature groups example notebook

* [FSTORE-1196][APPEND] Neo4j tutorial fixes (#241)

* neo4j tutorial fixes

* [FSTORE-1262] Tutorials for similarity search (#242)

* Added tutorials for similarity search for feature groups and feature views

* neo4j tutorial working on serverless (#243)

neo4j tutorial working on serverless

* [FSTORE-1196] [APPEND]  Instructions for Transformer  (#244)

 add clarification on feature view and training data version for transformer

* [FSTORE-1239] [APPEND] - Remove deprecated tensorflow-addons library (#245)

* remove deprecated tensorflow-addons

Co-authored-by: davitbzh <davit.bzhalava@gmail..com>

* [FSTORE-1298] add requirements.txt to vector embeddings (#248)

* add requirements.txt to vector embeddings. refactor knn search to allow to be run for just search

---------

Co-authored-by: Jim Dowling <jim@hopsworks.ai>

* [FSTORE-1186] Adding tutorial for Polars (#246)

Polars Integration into Hopsworks

* [FSTORE-1207] AirQuality LLM project (#250)

* Function Calling & AirQuality FunctionCalling Chatbot

* [FSTORE-1280] Tutorial about Opensearch integration with LangChain (#251)

* add notebook

* rename folder

* add prerequisite step

* use get_feature_group

* AirQuality OpenAI API Support (#253)

* notebook for saving model (#247)

* [FSTORE-1107] Pyspark streaming Tutorial (#227)

* Draft pyspark streaming

* adding init.py for module

* adding code to download libaries in hopsworks jupyter

* adding topic for testing

* adding notebook for creating simulated data stream

* correcting imports

* working base pyspark streaming

* adding all pipeline pyspark streaming

* updated tutorial

* fixed pyspark streaming

* clearning all outputs from notebooks

* renaming filenames in tutorial

* adding online inference pipeline instead of batch inference

* pyspark streaming online inference with deployments

* moving polars and pyspark streaming tutorial to integrations folder

* Updating readme based on moved polars tutorials and adding Pyspark streaming into readme

* AML Tutorial (#255)

* use model dimension (#254)

* Add a snowflake example, with snowflake as a data source to FGs in Hopsworks (#257)

Co-authored-by: Jim Dowling <jim@hopsworks.ai>

* [FSTORE-1186][APPEND] Adding tutorial for Polar

* ChequeDetection (#260)

* Feature pipeline with Snowflake as a data source (#258)

* Add a snowflake example, with snowflake as a data source to FGs in Hopsworks and fix polars tutorial

---------

Co-authored-by: Jim Dowling <jim@hopsworks.ai>

* Cosmetic improvements and spelling errors (#256)

* Data (#261)

* [FSTORE-1385][APPEND] Data Loading Optimization (#262)

* Data Loading Optimization

* [FSTORE-1395] Update the Langchain usage in Fraud Cheque Detection (#263)

* Update the Langchain usage in Fraud Cheque Detection

* [FSTORE-1394] Update the Langchain usage in AirQuality (#264)

* Update the Langchain usage in AirQuality

* Update the date of the test questions

* StrOutputParser Usage

* [FSTORE-1396] Tutorials Update: Save/load XGBoost model as json file (#265)

* Replace pickling xgboost models with saving them as json objects for the model registry

* [FSTORE-1404] LLM PDF Tutorial (#266)

* LLM PDF Search Tutorial  using RAG and Fine-Tuning

* [FSTORE-1408] LLM PDFs README (#267)

* Readme for pdf LLMs

* Update README.md

Added reference to video, that you can use a private trainer model.

* Update README.md

Fix titles

* Update README.md

Nicer readme for llm-pdfs

* [FSTORE-1422] Fraud Cheque Detection README (#268)

* Create README.md

* Update README.md

* Typo

* VideoLink

* Update README.md

Minor grammatical changes.

---------

Co-authored-by: rcnnnghm <Raymond.Cunningham@gmail.com>

* Add AzureSQL example (#270)

* Move feature_monitoring tutorial to api_examples (#271)

* [FSTORE-1437] Hospital Wait Time Tutorial (#272)

* hospital waiting time forecasting tutorial

* inital changes

* fixing fraud online

* Move similarity search examples (#275)

* TikTok Recommendation System (#277)

* fixing electricity training pipelines

---------

Co-authored-by: Maksym Zhytnikov <63515947+Maxxx-zh@users.noreply.github.com>
Co-authored-by: Steffen Grohsschmiedt <gitbhub@steffeng.eu>
Co-authored-by: Steffen Grohsschmiedt <steffen@logicalclocks.com>
Co-authored-by: Jim Dowling <jim@logicalclocks.com>
Co-authored-by: Robin Andersson <robin@hopsworks.ai>
Co-authored-by: Javier de la Rúa Martínez <javier@logicalclocks.com>
Co-authored-by: Marco Pellegrino <marco@hopsworks.ai>
Co-authored-by: davitbzh <44586065+davitbzh@users.noreply.github.com>
Co-authored-by: kennethmhc <kennethmhc@users.noreply.github.com>
Co-authored-by: davitbzh <davit.bzhalava@gmail..com>
Co-authored-by: Jim Dowling <jim@hopsworks.ai>
Co-authored-by: Rik Van Bruggen <2995654+rvanbruggen@users.noreply.github.com>
Co-authored-by: rcnnnghm <Raymond.Cunningham@gmail.com>
Co-authored-by: Fabio Buso <buso.fabio@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@manu-sj manu-sj manu-sj approved these changes

@davitbzh davitbzh Awaiting requested review from davitbzh

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Maxxx-zh @manu-sj @davitbzh