Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?


Failed to load latest commit information.

Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors

Thomas Hartvigsen  Swami Sankaranarayanan  Hamid Palangi  Yoon Kim  Marzyeh Ghassemi

Official implementation of Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors (NeurIPS 2023).


Please feel free to email Tom or raise an issue with this repository and we'll get back to you as soon as possible.


  1. Create a virtual environment (we use conda)
    conda env create --name grace_env --file environment.yml
  2. Activate the virtual environment
    conda activate grace_env
  3. Install the repository
    pip install -e .


The QA experiments use data linked by the MEND repository. Per their instructions, you can download the data for NQ and zsRE from their Google Drive link and unzip each sub-directory into grace/data. SCOTUS and Hallucination data are handled through huggingface.

Running experiments

Experiments are run using Experiment settings and hyperparameters are chosen using hydra. While more examples are available in ./scripts/, three representative experiments can be run as follows:

Editing GPT2-XL on Hallucination with GRACE

python grace/ experiment=hallucination model=gpt2xl editor=grace

Editing BERT on SCOTUS with GRACE

python grace/ experiment=scotus model=bert editor=grace

Editing T5 on zsRE with GRACE

python grace/ experiment=qa model=t5small editor=grace

Repository Roadmap

  • ./scripts/ contains handy shell scripts for starting and running experiments in slurm.
  • ./notebooks/ contains a simple example of editing a model with GRACE.
  • ./ckpts/ will contain checkpoints of your edited models if you choose to checkpoint models.
  • ./data/ will contain downloaded datasets if you choose to cache data yourself instead of relying on HuggingFace.
  • ./grace/ contains the source code to GRACE


Please use the following to cite this work:

  title={Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors},
  author={Hartvigsen, Thomas and Sankaranarayanan, Swami and Palangi, Hamid and Kim, Yoon and Ghassemi, Marzyeh},
  booktitle={Advances in Neural Information Processing Systems},


[NeurIPS'23] Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors






No releases published


No packages published