From 32dec6becb228ebc5d5e3ec0428f4962f4ec08e3 Mon Sep 17 00:00:00 2001 From: Mark Saroufim Date: Wed, 11 Oct 2023 17:05:21 -0700 Subject: [PATCH 1/2] Update README.md --- examples/pt2/README.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/examples/pt2/README.md b/examples/pt2/README.md index dbffc749ec7..ffcd6a174db 100644 --- a/examples/pt2/README.md +++ b/examples/pt2/README.md @@ -46,6 +46,17 @@ opt_mod = torch.compile(mod) torchserve takes care of 4 and 5 for you while the remaining steps are your responsibility. You can do the exact same thing on the vast majority of TIMM or HuggingFace models. +### Note + +`torch.compile()` is a JIT compiler and JIT compilers generally have a startup cost. If that's an issue for you make sure to populate these two environment variables + +``` +import os + +os.environ["TORCHINDUCTOR_CACHE_DIR"] = "1" +os.environ["TORCHINDUCTOR_FX_GRAPH_CACHE"] = "/path/to/directory" # replace with your desired path +``` + ## torch.export.export Export your model from a training script, keep in mind that an exported model cannot have graph breaks. From 9ccc3aeb8078ace46252fbd352b8159d02bb6d93 Mon Sep 17 00:00:00 2001 From: Mark Saroufim Date: Wed, 11 Oct 2023 17:16:10 -0700 Subject: [PATCH 2/2] Update README.md --- examples/pt2/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/pt2/README.md b/examples/pt2/README.md index ffcd6a174db..0758b089af6 100644 --- a/examples/pt2/README.md +++ b/examples/pt2/README.md @@ -48,7 +48,7 @@ torchserve takes care of 4 and 5 for you while the remaining steps are your resp ### Note -`torch.compile()` is a JIT compiler and JIT compilers generally have a startup cost. If that's an issue for you make sure to populate these two environment variables +`torch.compile()` is a JIT compiler and JIT compilers generally have a startup cost. If that's an issue for you make sure to populate these two environment variables to improve your warm starts. ``` import os