Add support codegen2

[michaelfeil]

---

1. General Description

This PR adds support to convert models from Codegen-2 to GPT-J.
It does not modify the functionality of the existing converter. The PR seems quite small, but took me hours of debugging to figure out that the architecture of codegen2 is actually fully compatible for the large (7, 16B versions).
For the smaller models, a different permutation order is required, because the smaller models (1, 3_7B versions)were trained on another TPU setting.

2. Changes proposed in this PR:

1. CodeGen2 compatibility #202

Resolves: #202

3. How to evaluate:

1. Describe how to evaluate such that it may be reproduced by the reviewer (s).
   1.

2. Self assessment:

• Successfully build locally docker-compose build:
• Successfully tested the full solution locally
• Integrated this PR from here: Adding support for transformers - Salesforce/CodeGen architecture OpenNMT/CTranslate2#1230 and ran the conversion once. Don't have enough capacity to fully end-to-end test the whole solution for this framework.

[moyix]

Oh this is wonderful! I didn't realize CodeGen2 was still a standard GPT-J model! I'll try to test this out as soon as possible and get it merged :)

[michaelfeil]

Can it be merged? :)

[liasece]

Great job. Has anyone experienced the Codegen-2? Is it worth upgrading?

[pgarba]

Any update ? Would really like to try this out

[pai4451]

Also interested if anyone experienced the Codegen-2. Is it worth upgrading?
The CodeGen-2 model card also emphasizes its ability to do infilling. Is it possible to do it on the fauxpilot fastertransformer backend?
It seems that Salesforce just introduce another Codegen family - CodeGen2.5

[richjohnson-wwt]

Is it possible to test this PR by pulling the branch, following the build steps, and editing the setup.sh to add the codegen2.5 model as an option?

[michaelfeil]

CodeGen 2.5 is based on LLama architecture, no longer on CodeGen architecture.

[pai4451]

@michaelfeil Hi, I read the blog of CodeGen 2.5 and Salesforce indeed serve it and evaluate the latency on NVIDIA triton server. Do you know how to serve CodeGen 2.5 with triton? I feel like there’s other ways to make the CodeGen based model supported instead of converting to GPT-J.

[michaelfeil]

I would back for tutorials on how to run llama-2-7b on triton, and start from there.

[michaelfeil]

@Hoekz Should i close this one in favor of #230 ?

[fdegier]

Closed in favor of #230