Create a manifest file for LLM inference.

[navveenb]

Story

As a user I want to calculate the environmental impact of LLM inference.

Rationale

LMs have exploded in popularity in 2023 and it is therefore desirable to have them represented in the IF universe. This idea is to generate a manifest file that could be used to calculate the energy and carbon consumed during LLM inference

Scope

This is a manifest file. See here for examples and here for a manifest writing tutorial.

Implementation guidelines

• The manifest MUST be a single .yaml file
• The manifest file MUST have all the required fields described here
• The manifest MUST define the necessary inputs required for the calculation
• The manifest MUST include all necessary models in a pipeline
• The manifest MUST define the expected outputs and their units
• The manifest SHOULD include comments showing which models are necessary but currently unavailable
• The manifest SHOULD include some documentation in a separate README explaining the execution flow, any known problems or missing components.

Difficulty

• 4/5

Impact

• 4/5

Examples and resources

• IMPL writing guide