{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":643793540,"defaultBranch":"main","name":"Awesome-Efficient-LLM","ownerLogin":"horseee","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2023-05-22T07:07:49.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/22924514?v=4","public":true,"private":false,"isOrgOwned":false},"refInfo":{"name":"","listCacheKey":"v0:1684945695.006468","currentOid":""},"activityList":{"items":[{"before":"df0b19aa3b228eea3ae6f0cd2ffc44aeb9a739a5","after":"ece891721c12f7e0b4fca9cf50c5728bc2d27be6","ref":"refs/heads/main","pushedAt":"2024-05-12T07:01:22.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"[ADD] Clover: Regressive Lightweight Speculative Decoding with Sequential Knowledge & Accelerating Speculative Decoding using Dynamic Speculation Length","shortMessageHtmlLink":"[ADD] Clover: Regressive Lightweight Speculative Decoding with Sequen…"}},{"before":"50b1b9d2a2e8f3754d5110c551f62e702d545a91","after":"df0b19aa3b228eea3ae6f0cd2ffc44aeb9a739a5","ref":"refs/heads/main","pushedAt":"2024-05-12T06:56:20.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"ADD Switchable Decision: Dynamic Neural Generation Networks & Kangaroo: Lossless Self-Speculative Decoding via Double Early Exiting & Mixture-of-Depths: Dynamically allocating compute in transformer-based language models","shortMessageHtmlLink":"ADD Switchable Decision: Dynamic Neural Generation Networks &amp; Kangaro…"}},{"before":"9f676a7aca85f327c8894a306e930698d4918f0d","after":"50b1b9d2a2e8f3754d5110c551f62e702d545a91","ref":"refs/heads/main","pushedAt":"2024-05-12T06:49:13.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"[ADD] KV-Runahead: Scalable Causal LLM Inference by Parallel Key-Value Cache Generation & You Only Cache Once: Decoder-Decoder Architectures for Language Models & QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving","shortMessageHtmlLink":"[ADD] KV-Runahead: Scalable Causal LLM Inference by Parallel Key-Valu…"}},{"before":"248785fffdd1127df56924c94aa6870457101b8a","after":"9f676a7aca85f327c8894a306e930698d4918f0d","ref":"refs/heads/main","pushedAt":"2024-05-07T18:37:28.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"[ADD] Dependency-Aware Semi-Structured Sparsity of GLU Variants in Large Language Models & Efficient and Economic Large Language Model Inference with Attention Offloading & Structural Pruning of Pre-trained Language Models via Neural Architecture Search","shortMessageHtmlLink":"[ADD] Dependency-Aware Semi-Structured Sparsity of GLU Variants in La…"}},{"before":"735555f2527c5961d9ed4948ba2b026a3a184d89","after":"248785fffdd1127df56924c94aa6870457101b8a","ref":"refs/heads/main","pushedAt":"2024-05-07T18:27:34.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"[ADD] Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment & Compressing Long Context for Enhancing RAG with AMR-based Concept Distillation & Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning","shortMessageHtmlLink":"[ADD] Enabling High-Sparsity Foundational Llama Models with Efficient…"}},{"before":"d11a4c8786c87765e2ddfc23b024ca8f7b23297a","after":"735555f2527c5961d9ed4948ba2b026a3a184d89","ref":"refs/heads/main","pushedAt":"2024-05-02T07:48:33.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"Merge branch 'main' of https://github.com/horseee/Awesome-Efficient-LLM","shortMessageHtmlLink":"Merge branch 'main' of <a href=\"https://github.com/horseee/Awesome-Efficient-LLM\">https://github.com/horseee/Awesome-Efficient-LLM</a>"}},{"before":"cd781d42c090523919a09422edd4db8bcc6b4833","after":"d11a4c8786c87765e2ddfc23b024ca8f7b23297a","ref":"refs/heads/main","pushedAt":"2024-05-02T07:38:17.000Z","pushType":"pr_merge","commitsCount":3,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"[pull request #17] Hierarchical Context Merging: Better Long Context Understanding for Pre-trained LLMs\n\nAdd HOMER: Hierarchical Context Merging: Better Long Context Understanding for Pre-trained LLMs","shortMessageHtmlLink":"[pull request <a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2268052925\" data-permission-text=\"Title is private\" data-url=\"https://github.com/horseee/Awesome-Efficient-LLM/issues/17\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/horseee/Awesome-Efficient-LLM/pull/17/hovercard\" href=\"https://github.com/horseee/Awesome-Efficient-LLM/pull/17\">#17</a>] Hierarchical Context Merging: Better Long Context …"}},{"before":"be5b9739d8daae165fd763f19e984f05339659fe","after":"cd781d42c090523919a09422edd4db8bcc6b4833","ref":"refs/heads/main","pushedAt":"2024-05-02T07:36:37.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"Merge branch 'main' of https://github.com/horseee/Awesome-Efficient-LLM","shortMessageHtmlLink":"Merge branch 'main' of <a href=\"https://github.com/horseee/Awesome-Efficient-LLM\">https://github.com/horseee/Awesome-Efficient-LLM</a>"}},{"before":"6aa0db70c95a0e164dd732f474288f69b77ffdb2","after":"be5b9739d8daae165fd763f19e984f05339659fe","ref":"refs/heads/main","pushedAt":"2024-04-28T05:40:32.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":"05ab4cfe01e196c02295d925c5fd7d79002b4b4f","after":"6aa0db70c95a0e164dd732f474288f69b77ffdb2","ref":"refs/heads/main","pushedAt":"2024-04-28T05:39:00.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"[ADD] Layer Skip: Enabling Early Exit Inference and Self-Speculative Decoding","shortMessageHtmlLink":"[ADD] Layer Skip: Enabling Early Exit Inference and Self-Speculative …"}},{"before":"5a7a524b76ad672e08eaf95292261be966811636","after":"05ab4cfe01e196c02295d925c5fd7d79002b4b4f","ref":"refs/heads/main","pushedAt":"2024-04-26T04:22:44.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"[ADD] A Survey on Efficient Inference for Large Language Models & Mélange: Cost Efficient Large Language Model Serving by Exploiting GPU Heterogeneity & Expert Router: Orchestrating Efficient Language Model Inference through Prompt Classification","shortMessageHtmlLink":"[ADD] A Survey on Efficient Inference for Large Language Models &amp; Mél…"}},{"before":"31d726ea63f01f06c1359fc24e25559a76abc184","after":"5a7a524b76ad672e08eaf95292261be966811636","ref":"refs/heads/main","pushedAt":"2024-04-26T04:19:10.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"[ADD] Rethinking LLM Memorization through the Lens of Adversarial Compression & Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing & How Good Are Low-bit Quantized LLaMA3 Models? An Empirical Study","shortMessageHtmlLink":"[ADD] Rethinking LLM Memorization through the Lens of Adversarial Com…"}},{"before":"dcc7deb4c9b6cf2bb6188173bd3474255ec7941c","after":"31d726ea63f01f06c1359fc24e25559a76abc184","ref":"refs/heads/main","pushedAt":"2024-04-26T04:08:05.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"[ADD] Lossless and Near-Lossless Compression for Foundation Models & Plug-and-Play: An Efficient Post-training Pruning Method for Large Language Models & XC-Cache: Cross-Attending to Cached Context for Efficient LLM Inference","shortMessageHtmlLink":"[ADD] Lossless and Near-Lossless Compression for Foundation Models &amp; …"}},{"before":"3c493d6a09cefbe1086bf254cffd2db4563ccf6a","after":"dcc7deb4c9b6cf2bb6188173bd3474255ec7941c","ref":"refs/heads/main","pushedAt":"2024-04-22T07:49:49.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"[ADD] Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length & decoupleQ: Towards 2-bit Post-Training Uniform Quantization via decoupling Parameters into Integer and Floating Points","shortMessageHtmlLink":"[ADD] Megalodon: Efficient LLM Pretraining and Inference with Unlimit…"}},{"before":"ab2a2eac858c8238730ba346c746afc42e372b9c","after":"3c493d6a09cefbe1086bf254cffd2db4563ccf6a","ref":"refs/heads/main","pushedAt":"2024-04-21T17:50:10.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"[ADD] Parallel Decoding via Hidden Transfer for Lossless Large Language Model Acceleration","shortMessageHtmlLink":"[ADD] Parallel Decoding via Hidden Transfer for Lossless Large Langua…"}},{"before":"e6108942433fcbd0997cb1881293448b4cad9670","after":"ab2a2eac858c8238730ba346c746afc42e372b9c","ref":"refs/heads/main","pushedAt":"2024-04-17T11:01:44.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"[ADD] Intuition-aware Mixture-of-Rank-1-Experts for Parameter Efficient Finetuning","shortMessageHtmlLink":"[ADD] Intuition-aware Mixture-of-Rank-1-Experts for Parameter Efficie…"}},{"before":"340e53d1a0b33ba8be0f10c1bab54fb346980a45","after":"e6108942433fcbd0997cb1881293448b4cad9670","ref":"refs/heads/main","pushedAt":"2024-04-17T10:56:31.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"[ADD] Post-Semantic-Thinking: A Robust Strategy to Distill Reasoning Capacity from Large Language Models & CATS: Contextually-Aware Thresholding for Sparsity in Large Language Models & LoongServe: Efficiently Serving Long-context Large Language Models with Elastic Sequence Parallelism","shortMessageHtmlLink":"[ADD] Post-Semantic-Thinking: A Robust Strategy to Distill Reasoning …"}},{"before":"a508614142018116aff4cd37022ba83c89b04ecc","after":"340e53d1a0b33ba8be0f10c1bab54fb346980a45","ref":"refs/heads/main","pushedAt":"2024-04-17T10:50:01.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"update layout","shortMessageHtmlLink":"update layout"}},{"before":"5c7a157958d10b739de6e2f452d70bf8cd4dda9e","after":"a508614142018116aff4cd37022ba83c89b04ecc","ref":"refs/heads/main","pushedAt":"2024-04-17T10:48:10.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"update layout","shortMessageHtmlLink":"update layout"}},{"before":"67b4f213154ab5ceb38ef854e3d06eddc420ac93","after":"5c7a157958d10b739de6e2f452d70bf8cd4dda9e","ref":"refs/heads/main","pushedAt":"2024-04-17T10:46:39.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"[ADD] Towards Fast Inference: Exploring and Improving Blockwise Parallel Drafts & Lossless Acceleration of Large Language Model via Adaptive N-gram Parallel Decoding & Self-Selected Attention Span for Accelerating Large Language Model Inference","shortMessageHtmlLink":"[ADD] Towards Fast Inference: Exploring and Improving Blockwise Paral…"}},{"before":"9c6b99798a619a032d8dd24cec25b70067be46eb","after":"67b4f213154ab5ceb38ef854e3d06eddc420ac93","ref":"refs/heads/main","pushedAt":"2024-04-17T10:39:58.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"[ADD] Quantization of Large Language Models with an Overdetermined Basis & LoRAP: Transformer Sub-Layers Deserve Differentiated Structured Compression for Large Language Models & Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models","shortMessageHtmlLink":"[ADD] Quantization of Large Language Models with an Overdetermined Ba…"}},{"before":"acb393b022028e5b20fa5fe2540dbe581124fc69","after":"9c6b99798a619a032d8dd24cec25b70067be46eb","ref":"refs/heads/main","pushedAt":"2024-04-14T14:14:00.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"[ADD] Accelerating Inference in Large Language Models with a Unified Layer Skipping Strategy","shortMessageHtmlLink":"[ADD] Accelerating Inference in Large Language Models with a Unified …"}},{"before":"a06141c0d43254e237694a4f5fcc53be6eee17df","after":"acb393b022028e5b20fa5fe2540dbe581124fc69","ref":"refs/heads/main","pushedAt":"2024-04-10T13:05:22.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"[ADD] Adapting LLMs for Efficient Context Processing through Soft Prompt Compression & Multilingual Brain Surgeon: Large Language Models Can be Compressed Leaving No Language Behind & Dense Training, Sparse Inference: Rethinking Training of Mixture-of-Experts Language Models & Increased LLM Vulnerabilities from Fine-tuning and Quantization","shortMessageHtmlLink":"[ADD] Adapting LLMs for Efficient Context Processing through Soft Pro…"}},{"before":"60f215f663a5ef089f351feb81cc401353ecf68d","after":"a06141c0d43254e237694a4f5fcc53be6eee17df","ref":"refs/heads/main","pushedAt":"2024-04-10T12:52:01.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"Merge branch 'main' of https://github.com/horseee/Awesome-Efficient-LLM","shortMessageHtmlLink":"Merge branch 'main' of <a href=\"https://github.com/horseee/Awesome-Efficient-LLM\">https://github.com/horseee/Awesome-Efficient-LLM</a>"}},{"before":"c2578359414bb057b7427cba5f26c48710a71fd3","after":"60f215f663a5ef089f351feb81cc401353ecf68d","ref":"refs/heads/main","pushedAt":"2024-04-08T05:19:12.000Z","pushType":"pr_merge","commitsCount":2,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"Merge pull request #15 from wutaiqiang/main\n\nadd two papers","shortMessageHtmlLink":"Merge pull request <a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2230266868\" data-permission-text=\"Title is private\" data-url=\"https://github.com/horseee/Awesome-Efficient-LLM/issues/15\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/horseee/Awesome-Efficient-LLM/pull/15/hovercard\" href=\"https://github.com/horseee/Awesome-Efficient-LLM/pull/15\">#15</a> from wutaiqiang/main"}},{"before":"8ed6277c5463fb96029ee9a103518d2bfa3c501e","after":"c2578359414bb057b7427cba5f26c48710a71fd3","ref":"refs/heads/main","pushedAt":"2024-04-07T09:02:39.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"Merge branch 'main' of https://github.com/horseee/Awesome-Efficient-LLM","shortMessageHtmlLink":"Merge branch 'main' of <a href=\"https://github.com/horseee/Awesome-Efficient-LLM\">https://github.com/horseee/Awesome-Efficient-LLM</a>"}},{"before":"728174a1e0657fd98276233d624384e01506b612","after":"8ed6277c5463fb96029ee9a103518d2bfa3c501e","ref":"refs/heads/main","pushedAt":"2024-04-04T05:32:19.000Z","pushType":"pr_merge","commitsCount":2,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"Merge pull request #14 from bokyeong1015/main\n\nAdd github for shortened-llama","shortMessageHtmlLink":"Merge pull request <a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2223219926\" data-permission-text=\"Title is private\" data-url=\"https://github.com/horseee/Awesome-Efficient-LLM/issues/14\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/horseee/Awesome-Efficient-LLM/pull/14/hovercard\" href=\"https://github.com/horseee/Awesome-Efficient-LLM/pull/14\">#14</a> from bokyeong1015/main"}},{"before":"033d6881ef01b7954bfad3fb19cb7473d203d265","after":"728174a1e0657fd98276233d624384e01506b612","ref":"refs/heads/main","pushedAt":"2024-04-03T16:24:58.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"update the layout","shortMessageHtmlLink":"update the layout"}},{"before":"2aec471cec32fbd1b6a342094c431def38299475","after":"033d6881ef01b7954bfad3fb19cb7473d203d265","ref":"refs/heads/main","pushedAt":"2024-04-03T16:22:47.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"add missing figures and update the layout","shortMessageHtmlLink":"add missing figures and update the layout"}},{"before":"a05993c4240863a115c617d3f644cb9e68ff27fd","after":"2aec471cec32fbd1b6a342094c431def38299475","ref":"refs/heads/main","pushedAt":"2024-04-03T16:18:18.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"horseee","name":"Horseee","path":"/horseee","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22924514?s=80&v=4"},"commit":{"message":"[ADD] Gecko: Versatile Text Embeddings Distilled from Large Language Models & DiJiang: Efficient Large Language Models through Compact Kernelization & Accurate Block Quantization in LLMs with Outliers","shortMessageHtmlLink":"[ADD] Gecko: Versatile Text Embeddings Distilled from Large Language …"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAER435JgA","startCursor":null,"endCursor":null}},"title":"Activity · horseee/Awesome-Efficient-LLM"}