Commits on Nov 2, 2023

1. Add meta functions for {}_embedding{}_codegen_forward_{}{}_cuda (pyto… …

   …rch#2094)
   
   Summary:
   
   For example, this hits split_embedding_codegen_lookup_rowwise_adagrad_function among others.
   
   There are a few other ways to write this and I'm open to other options.  This version seemed the shortest to write, at the cost of some code duplication.  The general strategy is I took embedding_forward_split_template.cu, copy pasted it into embedding_forward_split_meta_template.cpp, deleted all the CUDA specific stuff, and made it register a Meta implementation instead of a CUDA implementation.  Most parts of the original were excised except the kernel itself, where all of the error checking and output tensor calculation is kept.
   
   The biggest annoyance is the code duplication for the error checking, which are copy pasted with essentially no modifications from the original CUDA kernel.  I could potentially template the original logic so I can transclude it in two places, but that would also have made this diff harder to understand.  I'm willing to do this if someone tells me to.  Another approach would have been to implement the meta function entirely in Python (this is what I did in my original prototype), but to get it to handle all of the optimizer permutations seemed like a big pain.
   
   Reviewed By: zou3519
   
   Differential Revision: D50653995

   

   ezyang authored and facebook-github-bot committed Nov 2, 2023
   Configuration menu

   • View commit details
   • Copy full SHA for 903de3c
   • Browse repository at this point

   Copy the full SHA

   903de3c View commit details

   Browse the repository at this point in the history