Code notebook for python package learning.
emoji | connotation | tag |
---|---|---|
🌕 or ✅ | experienced | :full_moon: |
🌔 or ✅ | advanced | :moon: |
🌓 or ✅ | well-grounded | :first_quarter_moon: |
🌒 or ⬜ | understanding | :waxing_crescent_moon: |
🌑 or ⬜ | strange | :new_moon: |
integrated machine/deep/reinforcement learning toolkits.
- 🌓 pytorch : open source deep learning framework commonly used in academia.
- 🌓 tensorflow : open source deep learning framework commonly used in industry.
- 🌑 paddle : baidu open source deep learning platform derived from industry practices.
- 🌑 ncnn : tencent high performance neural network forward computing framework optimized for mobiles.
- 🌒 gym : a toolkit for developing and comparing reinforcement learning algorithms.
- 🌑 ACME : a research framework for reinforcement learning by DeepMind.
- 🌓 sklearn : machine learning library.
- 🌑 autokeras : automl system based on keras.
- 🌑 pytorch_geometric : geometric deep learning extension library for pytorch.
- 🌓 textbrewer: nlp knowledge distillation toolkit.
- 🌓 OpenPrompt : prompt tuning toolkit developed by Tsinghua.
- 🌑 OpenAttack : an open-source package for texual adversarial attack.
- 🌑 PyCID: a python library for causal influence diagrams.
NLP/CV/etc. packages.
- 🌔 torchvision : vision processing toolkit in pytorch.
- 🌓 cv2 : opencv in python.
- 🌓 pillow : python imaging library.
-
🌓 torchtext : natural language data processing for pytorch.
-
🌓 nltk : english natural language toolkit by UPenn.
for chinese, we should import Stanford NLP Chinese package.
-
🌓 ltp : chinese language technology platform by HIT.
-
🌓 gensim : topic modelling for humans (semantic vector).
-
🌓 glove : toy python implementation of glove.
-
🌓 hugging face : sota pre-trained language model toolkit.
mainly for pre-trained and transformer, etc.
-
🌓 fairseq : a seq2seq learning toolkit for pytorch by fair.
-
🌑 EDA : easy data augmentation techniques for text.
-
🌑 pattern : simpler to get started than nltk.
-
🌑 pyenchant : easy access to dictionaries
-
🌒 allennlp : natural language processing toolkit based on pytorch by Allen.
mainly for ELMo, KnowBERT, etc.
-
🌑 seqeval : evaluation for sequence labelling.
-
🌑 sentence transformers : compute dense vector representations for sentences, paragraphs, and images.
-
🌒 (corenlp, spacy, jieba) : chinese processing, language processing, etc.
we put gensim and glove together in this code notebook.
- 🌑 torchaudio : audio processing toolkit in pytorch.
data processing and computing.
- 🌓 pandas : high performance and user-friendly data structure and analysis tool.
- 🌔 matplotlib : comprehensive library for creating static, animated and interactive visualizations in python.
- 🌓 numpy : scientific computing.
- 🌓 scipy : open-source software for mathematics, science, and engineering.
programming aids.
- 🌒 tqdm : progress bar.
- 🌑 labml : training visualization on the mobile phone.
- 🌑 wandb: experiment results management.
- 🌑 TensorSensor:visualization of errors about tensor.
- 🌑 optuna: automatic hyperparameter searching.
- 🌑 alfred: visualization of annotations.
- 🌓 threading : multithreading.
- 🌓 multiprocessing : multiprocessing.
- 🌓 deepspeed : microsoft training acceleration toolkit.
- 🌑 bminf : a low-resource inference package for large-scale pretrained language models (PLMs).
We put threading and multiprocessing together in this code notebook.
- 🌑 glob : file path manager.
- 🌑 shutil : advanced file operation module.
- 🌑 re : regex.
- 🌑 elasticsearch : search and analysis engine.
- 🌓 scrapy : web crawler.
- 🌑 pyqt : GUI.
- 🌑 pyinstaller : application packaging.
- 🌒 os : access os-related functions.
- 🌑 sys : access interpreter-related variables and functions.
- .py file is not runnable : we view it as a codebook composed by numerous fragments, but not an executable file (there is no strong relation between every two fragments).