{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":387192484,"defaultBranch":"main","name":"Hetu","ownerLogin":"Hsword","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2021-07-18T14:14:59.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/6723331?v=4","public":true,"private":false,"isOrgOwned":false},"refInfo":{"name":"","listCacheKey":"v0:1702478511.0","currentOid":""},"activityList":{"items":[{"before":"41f7b275f57c355ce33ae972d360d8c2c2c306f4","after":"a1cb1bcaf236c9f1d452733b105997d50c16c9df","ref":"refs/heads/main","pushedAt":"2023-12-18T21:03:41.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"Hsword","name":"Hsword","path":"/Hsword","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6723331?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":null,"after":"41f7b275f57c355ce33ae972d360d8c2c2c306f4","ref":"refs/heads/diffusers","pushedAt":"2023-12-13T14:41:51.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"Hsword","name":"Hsword","path":"/Hsword","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6723331?s=80&v=4"},"commit":{"message":"Implement Galvatron-2 Lite. (#72)\n\n* Remove old version Galvatron.\r\n\r\n* Implement Galvatron-2 core.\r\n\r\n* Implement Galvatron-2 example models.\r\n\r\n* Revision on Galvatron-2.","shortMessageHtmlLink":"Implement Galvatron-2 Lite. (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2022425998\" data-permission-text=\"Title is private\" data-url=\"https://github.com/Hsword/Hetu/issues/72\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/Hsword/Hetu/pull/72/hovercard\" href=\"https://github.com/Hsword/Hetu/pull/72\">#72</a>)"}},{"before":"11300294f695b6c5ed6893676678ef7e97670b2c","after":"41f7b275f57c355ce33ae972d360d8c2c2c306f4","ref":"refs/heads/main","pushedAt":"2023-12-04T02:43:09.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Hsword","name":"Hsword","path":"/Hsword","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6723331?s=80&v=4"},"commit":{"message":"Implement Galvatron-2 Lite. (#72)\n\n* Remove old version Galvatron.\r\n\r\n* Implement Galvatron-2 core.\r\n\r\n* Implement Galvatron-2 example models.\r\n\r\n* Revision on Galvatron-2.","shortMessageHtmlLink":"Implement Galvatron-2 Lite. (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2022425998\" data-permission-text=\"Title is private\" data-url=\"https://github.com/Hsword/Hetu/issues/72\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/Hsword/Hetu/pull/72/hovercard\" href=\"https://github.com/Hsword/Hetu/pull/72\">#72</a>)"}},{"before":"6a501c9a57b036c19341122668e5678b55881173","after":"11300294f695b6c5ed6893676678ef7e97670b2c","ref":"refs/heads/main","pushedAt":"2023-12-03T08:22:59.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ccchengff","name":"Fangcheng Fu","path":"/ccchengff","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19160265?s=80&v=4"},"commit":{"message":"Merge embedmem branch (#71)\n\n* implement operators, strategies, and scripts for automatic parallelism\r\n\r\n* implement compressed embedding and debug\r\n\r\n* debug for integer values on hetu\r\n\r\n* implement MDEmbedding\r\n\r\n* implement embedding pruning in training\r\n\r\n* implement inference for pruning method\r\n\r\n* implement quantization embedding with bugs\r\n\r\n* debug quantization\r\n\r\n* implement forward part of ROBE\r\n\r\n* implement ROBE\r\n\r\n* add test_sgd_robe in test_optimizer.py\r\n\r\n* implement +1 / -1\r\n\r\n* implement ROBE-Z\r\n\r\n* update conda environment file\r\n\r\n* seperate features' embedding\r\n\r\n* implement wandb logger\r\n\r\n* improve scripts\r\n\r\n* move deduplicate into embedding gradient op\r\n\r\n* improve separated optimizer update ops; also separate state variables\r\n\r\n* simplify dataloader\r\n\r\n* improve dataloader to continue  training from part of an epoch\r\n\r\n* make beta1t and beta2t variable ops to save states\r\n\r\n* implement INT16 quantization for embedding\r\n\r\n* debug and improve other quantization functions\r\n\r\n* implement HetuRandomState and replace all previous random seed\r\n\r\n* implement scheduler for embedding layers; implement coo format for sparse\r\n\r\n* try re-write autodim (not stable, need further debug)\r\n\r\n* implement adam optimizer for embeddings (check quantize later)\r\n\r\n* dealing avazu codes\r\n\r\n* improve ctr datasets\r\n\r\n* debug optimize sparse update op in memory plan for MDEmbedding\r\n\r\n* implement simple tensor train embedding\r\n\r\n* improve compoemb to support different compression rate\r\n\r\n* debug autodim method\r\n\r\n* make tt-rec using multi embedding\r\n\r\n* reconstruct robe to enable adam etc.\r\n\r\n* implement use multi dhe; improve all solving using binary search\r\n\r\n* improve deeplight and generalize switch inference trainer\r\n\r\n* reconstruct trainer for embedding layers\r\n\r\n* improve dpq to support use_multi\r\n\r\n* implement mgqe\r\n\r\n* implement raw adaptive embedding\r\n\r\n* enable ops to support multiple data types if necessary for embedding lookup (only implement for gpu)\r\n\r\n* debug and implement adapt embedding\r\n\r\n* set use multi for mgqe, adapt\r\n\r\n* implement kernels and raw embedding layer for ALPT (need to implement trainer later)\r\n\r\n* decouple unique indices op from deduplicate op\r\n\r\n* implement alpt\r\n\r\n* improve save and load; make it more stable\r\n\r\n* implement pep the first training stage\r\n\r\n* improve alpt and seeds\r\n\r\n* implement pep (without considering compress rate) and improve existing methods\r\n\r\n* enable pep to reach compress rate\r\n\r\n* debug alpt\r\n\r\n* refine the retrain logic for autodim and pep: in retrain stage, re-init the parameters of deep model\r\n\r\n* implement autosrh method; debug deeplight, pep, robe, dhe, mgqe, adapt\r\n\r\n* implement optembed\r\n\r\n* md -- auto calculate alpha according to compress rate\r\n\r\n* improve md alpha calculation process\r\n\r\n* remove redundant flag in md\r\n\r\n* enable using validation or test data; debug adaptive embedding\r\n\r\n* debug multiple-stage methods\r\n\r\n* implement adagrad and amsgrad sparse update op\r\n\r\n* implement wdl, deepfm, dcn models\r\n\r\n* debug ctr models for avazu dataset\r\n\r\n* debug early stopping\r\n\r\n* debug early stop, embedding layer, binary cross op\r\n\r\n* debug bce loss and implement raw ncf\r\n\r\n* debug autosrh\r\n\r\n* minor improvement (no effects)\r\n\r\n* enable autodim to ignore second order, and reset without re-init\r\n\r\n* debug adapt\r\n\r\n* enable alpt to have different lr for scale\r\n\r\n* add cub and thrust into conda environment\r\n\r\n* debug broadcast op\r\n\r\n* debug load data\r\n\r\n* support criteo-tb (with bugs)\r\n\r\n* debug new dataloader\r\n\r\n* debug dataloader and enable drop last\r\n\r\n* enable shuffling in dataloader\r\n\r\n* debug and add some tests to modify skewness\r\n\r\n* debug ttrec: fixed ttcore initializer in embedding module\r\n\r\n* enable using criteo-tb dataset\r\n\r\n* align ncf precision\r\n\r\n* move to test directory\r\n\r\n* implement mf and ncf for ml20m, amazonbooks datasets\r\n\r\n* modify rating models to fit all compression methods\r\n\r\n* MAE implementation\r\n\r\n* deeplight mae loss\r\n\r\n* methods update\r\n\r\n* modified for ctr tasks\r\n\r\n* debug adaptembedding\r\n\r\n* format python documents\r\n\r\n* debug full embedding when dim=64 (int -> size_t)\r\n\r\n* debug and improve autosrh\r\n\r\n* remove embedding layer type in scripts\r\n\r\n* simplify autosrh\r\n\r\n* debug autosrh testing phase\r\n\r\n* enable load ckpt with corresponding metrics\r\n\r\n* improve deeplight trainer\r\n\r\n* improve autodim\r\n\r\n* refine trainers codes\r\n\r\n* debug and improve optembed\r\n\r\n* replace autodim alpha optimizer to sgd\r\n\r\n* debug autodim not reset retrain\r\n\r\n* improve deeplight with smaller buffer size\r\n\r\n* add start meta for deeplight saving\r\n\r\n* debug and improve dpq\r\n\r\n* add exact split flag into adapt\r\n\r\n* add threshold flag for inter-feature compression\r\n\r\n* complement threshold usage\r\n\r\n* add criteo 2-core dataset\r\n\r\n* add sparsified and densified criteo 2-core datasets\r\n\r\n* implement more skewed and less skewed criteo dataset\r\n\r\n* debug datasets; densification needs to hash the features with near frequency\r\n\r\n* debug initialization\r\n\r\n* debug criteo2core and implement avazu2core and variants\r\n\r\n* add figure script\r\n\r\n* improve README for Embedding Compression EA\r\n\r\n* add configurations of methods\r\n\r\n* rearrange the directories to decouple Embedding Memory Compression EA project\r\n\r\n* improve README of EA\r\n\r\n* implement deduplication method for compression\r\n\r\n* add embedding details to log name\r\n\r\n* add embedding details to ckpt_dir\r\n\r\n* fix a bug in dedup\r\n\r\n* implement all the compressor for retrieval-llm\r\n\r\n* add timer for compressor\r\n\r\n* debug for reconstruct_n\r\n\r\n* remove local path\r\n\r\n* debug deduplicate\r\n\r\n* debug for faiss-gpu\r\n\r\n* debug pq compression ratio\r\n\r\n* debug and improve memory usage of mgqe\r\n\r\n* improve readme and add explanation for ttrec\r\n\r\n* improve readme\r\n\r\n* accelerate dedup\r\n\r\n* improve timing decompress batch\r\n\r\n* debug compilation after merging\r\n\r\n* fix the deletion in merging\r\n\r\n* debug executor after merging\r\n\r\n* improve README for Embedding Compression\r\n\r\n* remove \"daim\" keywords; add Thrust into prerequisite\r\n\r\n---------\r\n\r\nCo-authored-by: Weiyu Chen <1830915329@qq.com>\r\nCo-authored-by: hitwh1*********pg <1248406609@qq.com>\r\nCo-authored-by: lhy101 <2000012918@stu.pku.edu.cn>","shortMessageHtmlLink":"Merge embedmem branch (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2012348195\" data-permission-text=\"Title is private\" data-url=\"https://github.com/Hsword/Hetu/issues/71\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/Hsword/Hetu/pull/71/hovercard\" href=\"https://github.com/Hsword/Hetu/pull/71\">#71</a>)"}},{"before":"a9dd1340efc3160bae4e5d746fe3d7780fe6b36a","after":"6a501c9a57b036c19341122668e5678b55881173","ref":"refs/heads/main","pushedAt":"2023-06-23T03:59:54.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"Hsword","name":"Hsword","path":"/Hsword","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6723331?s=80&v=4"},"commit":{"message":"Update COMMITTERS.md","shortMessageHtmlLink":"Update COMMITTERS.md"}}],"hasNextPage":false,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAADzUELXAA","startCursor":null,"endCursor":null}},"title":"Activity · Hsword/Hetu"}