Add a rake task to generate an 'available model information' markdown guide. #71

keithrbennett · 2025-03-26T11:45:58Z

Addresses issue #80 .

This script illustrates the access of the various models, and can be used to produce a human-friendly tabular markdown display of the models, grouped by type. Here is the current output of the script:

Model List

This is a list of all the models available in the RubyLLM library, by type:

Chat Models

ID	Type	Name	Provider	Context	MaxTok	Family	In$/M	Out$/M
aqa	chat	Model that performs Attributed Question Answering.	gemini	7168	1024	aqa	0.00	0.00
babbage-002	chat	Babbage 002	openai	16385	16384	babbage	0.50	1.50
chat-bison-001	chat	PaLM 2 Chat (Legacy)	gemini	4096	1024	other	0.08	0.30
chatgpt-4o-latest	chat	ChatGPT-4o Latest	openai	128000	16384	gpt4o	0.50	1.50
claude-2.0	chat	Claude 2.0	anthropic	200000	4096	claude2	3.00	15.00
claude-2.1	chat	Claude 2.1	anthropic	200000	4096	claude2	3.00	15.00
claude-3-5-haiku-20241022	chat	Claude 3.5 Haiku	anthropic	200000	8192	claude35_haiku	0.80	4.00
claude-3-5-sonnet-20240620	chat	Claude 3.5 Sonnet (Old)	anthropic	200000	8192	claude35_sonnet	3.00	15.00
claude-3-5-sonnet-20241022	chat	Claude 3.5 Sonnet (New)	anthropic	200000	8192	claude35_sonnet	3.00	15.00
claude-3-7-sonnet-20250219	chat	Claude 3.7 Sonnet	anthropic	200000	8192	claude37_sonnet	3.00	15.00
claude-3-haiku-20240307	chat	Claude 3 Haiku	anthropic	200000	4096	claude3_haiku	0.25	1.25
claude-3-opus-20240229	chat	Claude 3 Opus	anthropic	200000	4096	claude3_opus	15.00	75.00
claude-3-sonnet-20240229	chat	Claude 3 Sonnet	anthropic	200000	4096	claude3_sonnet	3.00	15.00
davinci-002	chat	Davinci 002	openai	16385	16384	davinci	0.50	1.50
deepseek-chat	chat	DeepSeek V3	deepseek	64000	8192	chat	0.27	1.10
deepseek-reasoner	chat	DeepSeek R1	deepseek	64000	8192	reasoner	0.55	2.19
gemini-1.0-pro-vision-latest	chat	Gemini 1.0 Pro Vision	gemini	12288	4096	gemini10_pro	0.50	1.50
gemini-1.5-flash	chat	Gemini 1.5 Flash	gemini	1000000	8192	gemini15_flash	0.15	0.60
gemini-1.5-flash-001	chat	Gemini 1.5 Flash 001	gemini	1000000	8192	gemini15_flash	0.15	0.60
gemini-1.5-flash-001-tuning	chat	Gemini 1.5 Flash 001 Tuning	gemini	16384	8192	gemini15_flash	0.15	0.60
gemini-1.5-flash-002	chat	Gemini 1.5 Flash 002	gemini	1000000	8192	gemini15_flash	0.15	0.60
gemini-1.5-flash-8b	chat	Gemini 1.5 Flash-8B	gemini	1000000	8192	gemini15_flash_8b	0.08	0.30
gemini-1.5-flash-8b-001	chat	Gemini 1.5 Flash-8B 001	gemini	1000000	8192	gemini15_flash_8b	0.08	0.30
gemini-1.5-flash-8b-exp-0827	chat	Gemini 1.5 Flash 8B Experimental 0827	gemini	1000000	8192	gemini15_flash_8b	0.08	0.30
gemini-1.5-flash-8b-exp-0924	chat	Gemini 1.5 Flash 8B Experimental 0924	gemini	1000000	8192	gemini15_flash_8b	0.08	0.30
gemini-1.5-flash-8b-latest	chat	Gemini 1.5 Flash-8B Latest	gemini	1000000	8192	gemini15_flash_8b	0.08	0.30
gemini-1.5-flash-latest	chat	Gemini 1.5 Flash Latest	gemini	1000000	8192	gemini15_flash	0.15	0.60
gemini-1.5-pro	chat	Gemini 1.5 Pro	gemini	2000000	8192	gemini15_pro	2.50	10.00
gemini-1.5-pro-001	chat	Gemini 1.5 Pro 001	gemini	2000000	8192	gemini15_pro	2.50	10.00
gemini-1.5-pro-002	chat	Gemini 1.5 Pro 002	gemini	2000000	8192	gemini15_pro	2.50	10.00
gemini-1.5-pro-latest	chat	Gemini 1.5 Pro Latest	gemini	2000000	8192	gemini15_pro	2.50	10.00
gemini-2.0-flash	chat	Gemini 2.0 Flash	gemini	1048576	8192	gemini20_flash	0.10	0.40
gemini-2.0-flash-001	chat	Gemini 2.0 Flash 001	gemini	1048576	8192	gemini20_flash	0.10	0.40
gemini-2.0-flash-exp	chat	Gemini 2.0 Flash Experimental	gemini	1048576	8192	gemini20_flash	0.10	0.40
gemini-2.0-flash-exp-image-generation	chat	Gemini 2.0 Flash (Image Generation) Experimental	gemini	1048576	8192	gemini20_flash	0.10	0.40
gemini-2.0-flash-lite	chat	Gemini 2.0 Flash-Lite	gemini	1048576	8192	gemini20_flash_lite	0.08	0.30
gemini-2.0-flash-lite-001	chat	Gemini 2.0 Flash-Lite 001	gemini	1048576	8192	gemini20_flash_lite	0.08	0.30
gemini-2.0-flash-lite-preview	chat	Gemini 2.0 Flash-Lite Preview	gemini	1048576	8192	gemini20_flash_lite	0.08	0.30
gemini-2.0-flash-lite-preview-02-05	chat	Gemini 2.0 Flash-Lite Preview 02-05	gemini	1048576	8192	gemini20_flash_lite	0.08	0.30
gemini-2.0-flash-thinking-exp	chat	Gemini 2.0 Flash Thinking Experimental 01-21	gemini	1048576	65536	gemini20_flash	0.10	0.40
gemini-2.0-flash-thinking-exp-01-21	chat	Gemini 2.0 Flash Thinking Experimental 01-21	gemini	1048576	65536	gemini20_flash	0.10	0.40
gemini-2.0-flash-thinking-exp-1219	chat	Gemini 2.0 Flash Thinking Experimental	gemini	1048576	65536	gemini20_flash	0.10	0.40
gemini-2.0-pro-exp	chat	Gemini 2.0 Pro Experimental	gemini	2097152	8192	other	0.08	0.30
gemini-2.0-pro-exp-02-05	chat	Gemini 2.0 Pro Experimental 02-05	gemini	2097152	8192	other	0.08	0.30
gemini-exp-1206	chat	Gemini Experimental 1206	gemini	2097152	8192	other	0.08	0.30
gemini-pro-vision	chat	Gemini 1.0 Pro Vision	gemini	12288	4096	other	0.08	0.30
gemma-3-27b-it	chat	Gemma 3 27B	gemini	131072	8192	other	0.08	0.30
gpt-3.5-turbo	chat	GPT-3.5-Turbo	openai	16385	4096	gpt35	0.50	1.50
gpt-3.5-turbo-0125	chat	GPT-3.5-Turbo 0125	openai	4096	4096	gpt35	0.50	1.50
gpt-3.5-turbo-1106	chat	GPT-3.5-Turbo 1106	openai	4096	4096	gpt35	0.50	1.50
gpt-3.5-turbo-16k	chat	GPT-3.5-Turbo 16k	openai	16385	4096	gpt35	0.50	1.50
gpt-3.5-turbo-instruct	chat	GPT-3.5-Turbo Instruct	openai	4096	4096	gpt35_instruct	0.50	1.50
gpt-3.5-turbo-instruct-0914	chat	GPT-3.5-Turbo Instruct 0914	openai	4096	4096	gpt35_instruct	0.50	1.50
gpt-4	chat	GPT-4	openai	4096	4096	gpt4	0.50	1.50
gpt-4-0125-preview	chat	GPT-4-0125 Preview	openai	8192	8192	gpt4	0.50	1.50
gpt-4-0613	chat	GPT-4-0613	openai	8192	8192	gpt4	0.50	1.50
gpt-4-1106-preview	chat	GPT-4-1106 Preview	openai	4096	4096	gpt4	0.50	1.50
gpt-4-turbo	chat	GPT-4-Turbo	openai	128000	4096	gpt4_turbo	0.50	1.50
gpt-4-turbo-2024-04-09	chat	GPT-4-Turbo 20240409	openai	128000	4096	gpt4_turbo	0.50	1.50
gpt-4-turbo-preview	chat	GPT-4-Turbo Preview	openai	128000	4096	gpt4_turbo	0.50	1.50
gpt-4.5-preview	chat	GPT-4.5 Preview	openai	4096	4096	gpt4	0.50	1.50
gpt-4.5-preview-2025-02-27	chat	GPT-4.5 Preview 20250227	openai	4096	4096	gpt4	0.50	1.50
gpt-4o	chat	GPT-4o	openai	128000	16384	gpt4o	0.50	1.50
gpt-4o-2024-05-13	chat	GPT-4o 20240513	openai	128000	16384	gpt4o	0.50	1.50
gpt-4o-2024-08-06	chat	GPT-4o 20240806	openai	128000	16384	gpt4o	0.50	1.50
gpt-4o-2024-11-20	chat	GPT-4o 20241120	openai	128000	16384	gpt4o	0.50	1.50
gpt-4o-audio-preview	chat	GPT-4o-Audio Preview	openai	128000	16384	gpt4o_audio	0.50	1.50
gpt-4o-audio-preview-2024-10-01	chat	GPT-4o-Audio Preview 20241001	openai	128000	16384	gpt4o_audio	0.50	1.50
gpt-4o-audio-preview-2024-12-17	chat	GPT-4o-Audio Preview 20241217	openai	128000	16384	gpt4o_audio	0.50	1.50
gpt-4o-mini	chat	GPT-4o-Mini	openai	128000	16384	gpt4o_mini	0.50	1.50
gpt-4o-mini-2024-07-18	chat	GPT-4o-Mini 20240718	openai	128000	16384	gpt4o_mini	0.50	1.50
gpt-4o-mini-audio-preview	chat	GPT-4o-Mini Audio Preview	openai	128000	16384	gpt4o_mini_audio	0.50	1.50
gpt-4o-mini-audio-preview-2024-12-17	chat	GPT-4o-Mini Audio Preview 20241217	openai	128000	16384	gpt4o_mini_audio	0.50	1.50
gpt-4o-mini-realtime-preview	chat	GPT-4o-Mini Realtime Preview	openai	128000	16384	gpt4o_mini_realtime	0.50	1.50
gpt-4o-mini-realtime-preview-2024-12-17	chat	GPT-4o-Mini Realtime Preview 20241217	openai	128000	16384	gpt4o_mini_realtime	0.50	1.50
gpt-4o-mini-search-preview	chat	GPT-4o-Mini Search Preview	openai	128000	16384	gpt4o_mini	0.50	1.50
gpt-4o-mini-search-preview-2025-03-11	chat	GPT-4o-Mini Search Preview 20250311	openai	128000	16384	gpt4o_mini	0.50	1.50
gpt-4o-mini-transcribe	chat	GPT-4o-Mini Transcribe	openai	128000	16384	gpt4o_mini	0.50	1.50
gpt-4o-realtime-preview	chat	GPT-4o-Realtime Preview	openai	128000	16384	gpt4o_realtime	0.50	1.50
gpt-4o-realtime-preview-2024-10-01	chat	GPT-4o-Realtime Preview 20241001	openai	128000	16384	gpt4o_realtime	0.50	1.50
gpt-4o-realtime-preview-2024-12-17	chat	GPT-4o-Realtime Preview 20241217	openai	128000	16384	gpt4o_realtime	0.50	1.50
gpt-4o-search-preview	chat	GPT-4o Search Preview	openai	128000	16384	gpt4o	0.50	1.50
gpt-4o-search-preview-2025-03-11	chat	GPT-4o Search Preview 20250311	openai	128000	16384	gpt4o	0.50	1.50
gpt-4o-transcribe	chat	GPT-4o Transcribe	openai	128000	16384	gpt4o	0.50	1.50
learnlm-1.5-pro-experimental	chat	LearnLM 1.5 Pro Experimental	gemini	32767	8192	other	0.08	0.30
o1	chat	O1	openai	4096	4096	o1	0.50	1.50
o1-2024-12-17	chat	O1-20241217	openai	200000	100000	o1	0.50	1.50
o1-mini	chat	O1-Mini	openai	128000	4096	o1_mini	0.50	1.50
o1-mini-2024-09-12	chat	O1-Mini 20240912	openai	128000	65536	o1_mini	0.50	1.50
o1-preview	chat	O1-Preview	openai	4096	4096	o1	0.50	1.50
o1-preview-2024-09-12	chat	O1-Preview 20240912	openai	4096	4096	o1	0.50	1.50
o1-pro	chat	O1-Pro	openai	4096	4096	o1	0.50	1.50
o1-pro-2025-03-19	chat	O1-Pro 20250319	openai	4096	4096	o1	0.50	1.50
o3-mini	chat	O3-Mini	openai	200000	100000	o3_mini	0.50	1.50
o3-mini-2025-01-31	chat	O3-Mini 20250131	openai	200000	100000	o3_mini	0.50	1.50
text-bison-001	chat	PaLM 2 (Legacy)	gemini	8196	1024	other	0.08	0.30

Image Models

ID	Type	Name	Provider	Context	MaxTok	Family	In$/M	Out$/M
dall-e-2	image	DALL-E-2	openai	4096	4096	dalle2	0.50	1.50
dall-e-3	image	DALL-E-3	openai	4096	4096	dalle3	0.50	1.50
imagen-3.0-generate-002	image	Imagen 3.0 002 model	gemini	480	8192	other	0.08	0.30

Audio Models

ID	Type	Name	Provider	Context	MaxTok	Family	In$/M	Out$/M
gpt-4o-mini-tts	audio	GPT-4o-Mini Tts	openai	128000	16384	gpt4o_mini	0.50	1.50
tts-1	audio	TTS-1	openai	4096	4096	tts1	0.50	1.50
tts-1-1106	audio	TTS-1 1106	openai	4096	4096	tts1	0.50	1.50
tts-1-hd	audio	TTS-1 HD	openai	4096	4096	tts1_hd	0.50	1.50
tts-1-hd-1106	audio	TTS-1 HD 1106	openai	4096	4096	tts1_hd	0.50	1.50
whisper-1	audio	Whisper 1	openai	4096	4096	whisper1	0.50	1.50

Embedding Models

ID	Type	Name	Provider	Context	MaxTok	Family	In$/M	Out$/M
embedding-001	embedding	Embedding 001	gemini	2048	1	embedding1	0.00	0.00
embedding-gecko-001	embedding	Embedding Gecko	gemini	1024	1	other	0.00	0.00
gemini-embedding-exp	embedding	Gemini Embedding Experimental	gemini	8192	1	other	0.00	0.00
gemini-embedding-exp-03-07	embedding	Gemini Embedding Experimental 03-07	gemini	8192	1	other	0.00	0.00
text-embedding-004	embedding	Text Embedding 004	gemini	2048	1	embedding4	0.00	0.00
text-embedding-3-large	embedding	Text Embedding 3 Large	openai	4096	4096	embedding3_large	0.50	1.50
text-embedding-3-small	embedding	Text Embedding 3 Small	openai	4096	4096	embedding3_small	0.50	1.50
text-embedding-ada-002	embedding	Text Embedding Ada 002	openai	4096	4096	embedding2	0.50	1.50

Moderation Models

ID	Type	Name	Provider	Context	MaxTok	Family	In$/M	Out$/M
omni-moderation-2024-09-26	moderation	Omni-Moderation 20240926	openai	4096	4096	moderation	0.50	1.50
omni-moderation-latest	moderation	Omni-Moderation Latest	openai	4096	4096	moderation	0.50	1.50

crmne

Thank you for your PR!

I would add to my comment that the guide should have the same style as the rest of the guides, and it should provide a link to the issues and pull request and encourage people to change the respective capabilities.rb files and run rake models:update and submit the new models.json.

exe/model_list_as_markdown

Add "Models by Provider", before there was only "Models by Type".

Mention models.json timestamp. Change "Model Capabilities" section to "Additional Model Information" and make the information there more helpful.

keithrbennett · 2025-04-01T03:03:05Z

Rubocop issues resolved.

Sorry about that, I forgot to read CONTRIBUTING.md before, but I just did.

crmne

Oh I just realized: this should be run in a ci step in docs.yml!

keithrbennett · 2025-04-02T15:07:38Z

Ok, I've modified docs.yml to call rake models:docs.

codecov · 2025-04-02T18:47:00Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.48%. Comparing base (5340c93) to head (ae957dd).
Report is 3 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main      #71   +/-   ##
=======================================
  Coverage   92.48%   92.48%           
=======================================
  Files          72       72           
  Lines        2687     2687           
  Branches      382      382           
=======================================
  Hits         2485     2485           
  Misses        202      202

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

keithrbennett added 2 commits March 26, 2025 00:35

Add model_list_as_markdown example script.

6227608

Add model_list_as_markdown script to /exe.

d1fabcc

crmne requested changes Mar 26, 2025

View reviewed changes

exe/model_list_as_markdown Outdated Show resolved Hide resolved

keithrbennett added 3 commits March 27, 2025 23:13

Move exe/model_list_as_markdown to Rake task lib/tasks/models_docs.rake.

708e630

Add "Models by Provider", before there was only "Models by Type".

Add sections on Contributing and Model Capabilities.

3893f31

Improve Contributing and Available Models sections.

77d32d5

keithrbennett marked this pull request as draft March 28, 2025 05:28

keithrbennett added 3 commits March 29, 2025 14:56

Indicate that "registry" is models.json file.

692cf81

Mention models.json timestamp. Change "Model Capabilities" section to "Additional Model Information" and make the information there more helpful.

Add "Family" to list of unlisted model attributes.

2804080

Move "Additional Model Information" section to top of document.

ae93809

keithrbennett changed the title ~~Add an example script to output the available models and most of their info as markdown.~~ Add a rake task to generate available model information as markdown. Mar 29, 2025

keithrbennett changed the title ~~Add a rake task to generate available model information as markdown.~~ Add a rake task to generate an 'available model information' markdown guide. Mar 29, 2025

keithrbennett marked this pull request as ready for review March 30, 2025 03:29

Add model counts to sections.

4d94df3

keithrbennett mentioned this pull request Mar 31, 2025

Automate the generation of available model information for the guides. #80

Closed

crmne and others added 2 commits March 31, 2025 17:09

Merge branch 'main' into examples

b5d4866

Address Rubocop complaints.

8a28637

crmne approved these changes Apr 1, 2025

View reviewed changes

crmne requested changes Apr 1, 2025

View reviewed changes

keithrbennett added 4 commits April 2, 2025 22:29

Modify docs.yml to call rake models:docs.

24e9ffc

Improve names.

3e7d3ca

Add 'examples' branch for testing.

b8384c6

Remove 'examples' branch, previously added only for testing.

6dcc2bc

Merge branch 'main' into examples

ae957dd

crmne added documentation Improvements or additions to documentation enhancement New feature or request labels Apr 2, 2025

crmne merged commit 4cfb7f4 into crmne:main Apr 2, 2025
7 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Add a rake task to generate an 'available model information' markdown guide. #71

Add a rake task to generate an 'available model information' markdown guide. #71

Uh oh!

keithrbennett commented Mar 26, 2025 •

edited

Loading

Uh oh!

crmne left a comment

Uh oh!

Uh oh!

keithrbennett commented Apr 1, 2025

Uh oh!

crmne left a comment

Uh oh!

keithrbennett commented Apr 2, 2025

Uh oh!

codecov bot commented Apr 2, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Add a rake task to generate an 'available model information' markdown guide. #71

Add a rake task to generate an 'available model information' markdown guide. #71

Uh oh!

Conversation

keithrbennett commented Mar 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Model List

Chat Models

Image Models

Audio Models

Embedding Models

Moderation Models

Uh oh!

crmne left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

keithrbennett commented Apr 1, 2025

Uh oh!

crmne left a comment

Choose a reason for hiding this comment

Uh oh!

keithrbennett commented Apr 2, 2025

Uh oh!

codecov bot commented Apr 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

keithrbennett commented Mar 26, 2025 •

edited

Loading

codecov bot commented Apr 2, 2025 •

edited

Loading