AITools

AI Tools, Examples

This folder contains Jupyter notebook examples of AI tools, including LLMs, Transformers, vector databases. The notebooks are intended to be run using GPU resources.
To use GPU resources in SWAN, you need to

• Access SWAN from you browser: https://swan.cern.ch
• Select a software stack with GPU
  • To get the latest version of the tools used here select the 'bleeding edge' software stack

Contact: Luca.Canali@cern.ch

Transformers library for text, image, and speech

This is to illustrate the use of the Transformers library from Hugging Face for LLM, Natural Language Processing (NLP), image, and speech tasks.

• Transformers for text classification
• Transformers for image classifier
• Stable diffusion with transformers
• Transformers for speech recognition

Large Language Models

These notebooks provide examples of how to use LLMs in notebook environments for tests and prototyping

• Transformers Large Language Models
• LangChain LLMs

Semantic search with Vector Databases and LLM

Semantic search allows to query a set of documents. This example notebook shows how to create vector embeddings, store them in a vector database, and perform semantic queries enhanced with LLM.

• Semantic search with Vector Databases and LLM