Large language models need a lot of hardware resources to be trained and also to run inferences. What are the hardware requirements for training GPT-J, OPT and Bloom and what are the requirements to run inference?
GPT-J and GPT-3 are two large language models, while OPT and Bloom are not specific models, so I am not sure which ones you are referring to. However, I can give you some general information on the hardware requirements for training and inference of large language models.

Training large language models requires a significant amount of computing resources, including CPU, RAM, storage, and specialized hardware such as GPUs or TPUs. GPT-3, for example, was trained on a supercomputer with 175 billion parameters, which required over 3,000 GPUs and consumed over 400,000 core hours of computation. GPT-J is a smaller model than GPT-3, with 6 billion parameters, but still requires a considerable amount of computational resources. The exact hardware requirements for training GPT-J are not known, as it is a community-built model that has been trained on various hardware setups.

Running inference, or using a trained language model to generate text or perform other tasks, also requires substantial computing resources, but generally not as much as training. The specific hardware requirements for inference depend on the size and complexity of the model, as well as the application and the desired response time. Inference can be performed on CPUs or specialized hardware such as GPUs or TPUs, with the latter typically providing better performance for large models. Many cloud providers offer pre-configured virtual machines or containers with hardware optimized for running inference on large language models.
Is there any language model that isn't so intensive? More specifically is there any language model you are aware of that a normal person could reasonably run on their own hardware, and when I say a normal person's hardware I mean a custom built computer that costs less than $5,000.