Releases: rhishi99/hermes-model-auto-switch
Release list
v1.2.0 - The switcher actually switches
��#� �v�1�.�2�.�0� �-� �T�h�e� �s�w�i�t�c�h�e�r� �a�c�t�u�a�l�l�y� �s�w�i�t�c�h�e�s�
�
�
�
�L�i�v�e� �v�a�l�i�d�a�t�i�o�n� �o�f� �v�1�.�1�.�0� �f�o�u�n�d� �t�h�e� �e�n�g�i�n�e� �s�e�l�e�c�t�i�n�g� ���0� �m�o�d�e�l�s��� �a�g�a�i�n�s�t� �a� �h�e�a�l�t�h�y� �r�o�u�t�e�r�.�
�
�F�o�u�r� �s�e�p�a�r�a�t�e� �b�u�g�s� �s�t�a�c�k�e�d� �u�p� �t�o� �p�r�o�d�u�c�e� �t�h�a�t�;� �e�a�c�h� �i�s� �f�i�x�e�d� �b�e�l�o�w�.� �V�e�r�i�f�i�e�d� �e�n�d� �t�o� �e�n�d�:� �a�
�
�d�r�y�-�r�u�n� �n�o�w� �s�c�o�r�e�s� �3�/�3� �a�c�t�i�v�e� �m�o�d�e�l�s� �h�e�a�l�t�h�y� �a�n�d� �r�e�p�o�r�t�s� �t�h�e� �s�w�i�t�c�h� �i�t� �w�o�u�l�d� �m�a�k�e�.�
�
�
�
�#�#� �F�i�x�e�d�
�
�
�
�-� ���S�c�o�r�i�n�g� �r�e�a�d� �a�n� �e�n�d�p�o�i�n�t� �w�i�t�h� �n�o� �d�a�t�a� �i�n� �i�t�.��� ��f�e�t�c�h�_�f�c�m�_�m�o�d�e�l�s�(�)�� �c�a�l�l�e�d�
�
� � ��G�E�T� �/�v�1�/�m�o�d�e�l�s��,� �w�h�i�c�h� �r�e�t�u�r�n�s� �o�n�l�y� �t�h�e� �r�o�u�t�e�r�'�s� �o�w�n� �a�l�i�a�s�e�s� �(��f�c�m��,� ��f�c�m�:�f�a�s�t�-�c�o�d�i�n�g��)�
�
� � �w�i�t�h� �n�o� ��s�c�o�r�e��,� ��l�a�t�e�n�c�y�_�m�s�� �o�r� ��p�r�o�v�i�d�e�r�� �f�i�e�l�d�.� �S�o� ��f�c�m�_�s�c�o�r�e�� �w�a�s� �a�l�w�a�y�s� �0� �a�n�d�
�
� � ��p�r�o�v�i�d�e�r�� �a�l�w�a�y�s� �f�e�l�l� �b�a�c�k� �t�o� ��'�n�v�i�d�i�a�'�� �-�-� �t�h�e� �4�0�%� �F�C�M� �w�e�i�g�h�t� �a�n�d� �t�h�e� �p�r�o�v�i�d�e�r�-�d�i�v�e�r�s�i�t�y�
�
� � �b�o�n�u�s� �w�e�r�e� �d�e�a�d� �c�o�d�e�,� �a�n�d� �t�h�e� �t�w�o� �a�l�i�a�s�e�s� �w�e�r�e� �s�c�o�r�e�d� �a�s� �i�f� �t�h�e�y� �w�e�r�e� �r�e�a�l� �m�o�d�e�l�s�.� �I�t� �n�o�w�
�
� � �r�e�a�d�s� ��G�E�T� �/�s�e�t�s��,� �w�h�i�c�h� �c�a�r�r�i�e�s� �t�h�e� �a�c�t�i�v�e� �s�e�t�'�s� �r�e�a�l� �m�o�d�e�l� �i�d�s�,� �p�r�o�v�i�d�e�r� �a�n�d� �p�r�i�o�r�i�t�y�.�
�
� � �P�r�i�o�r�i�t�y� �b�e�c�o�m�e�s� �t�h�e� �F�C�M� �s�c�o�r�e� �(�p�r�i�o�r�i�t�y� �1� �=� �1�0�0�,� �-�1�5� �p�e�r� �s�t�e�p�)�,� �t�h�a�t� �o�r�d�e�r�i�n�g� �b�e�i�n�g� �t�h�e�
�
� � �o�n�l�y� �h�e�a�l�t�h� �s�i�g�n�a�l� �F�C�M� �e�x�p�o�s�e�s�.�
�
�
�
�-� ���T�h�e� �h�e�a�l�t�h� �c�h�e�c�k� �p�r�o�b�e�d� �m�o�d�e�l�s� �n�o�b�o�d�y� �r�o�u�t�e�s� �t�o�.��� ��_�D�E�F�A�U�L�T�_�M�O�D�E�L�S�_�T�O�_�M�O�N�I�T�O�R�� �w�a�s� �a�
�
� � �h�a�r�d�c�o�d�e�d� �l�i�s�t� �t�h�a�t� �h�a�d� �d�r�i�f�t�e�d� �a�w�a�y� �f�r�o�m� �t�h�e� �r�o�u�t�e�r�'�s� �a�c�t�u�a�l� �s�e�t� �-�-� �o�n�e� �m�o�d�e�l� �i�n� �c�o�m�m�o�n�
�
� � �o�u�t� �o�f� �f�i�v�e�.� �T�h�e� �s�w�i�t�c�h�e�r� �n�o�w� �p�a�s�s�e�s� �t�h�e� �F�C�M� �a�c�t�i�v�e� �s�e�t� �t�h�r�o�u�g�h� �t�h�e� �c�h�e�c�k�e�r�'�s� �e�x�i�s�t�i�n�g�
�
� � ��N�I�M�_�M�O�D�E�L�S�_�T�O�_�M�O�N�I�T�O�R�� �o�v�e�r�r�i�d�e�.� �A�n� �e�x�p�l�i�c�i�t� �e�n�v� �v�a�l�u�e� �s�t�i�l�l� �w�i�n�s�;� �t�h�e� �h�a�r�d�c�o�d�e�d� �l�i�s�t� �i�s�
�
� � �u�s�e�d� �o�n�l�y� �w�h�e�n� �t�h�e� �d�a�e�m�o�n� �i�s� �u�n�r�e�a�c�h�a�b�l�e�.�
�
�
�
�-� ���A�n�y� �m�o�d�e�l� �m�i�s�s�i�n�g� �f�r�o�m� �t�h�e� �c�u�r�a�t�e�d� �S�W�E� �t�a�b�l�e� �w�a�s� �s�i�l�e�n�t�l�y� �r�e�j�e�c�t�e�d�.��� ��g�e�t�_�s�w�e�_�s�c�o�r�e�(�)��
�
� � �r�e�t�u�r�n�s� �a� �c�o�n�s�e�r�v�a�t�i�v�e� ��5�0�.�0�� �f�o�r� �m�o�d�e�l�s� �i�t� �h�a�s� �n�e�v�e�r� �s�e�e�n�,� �i�n�d�i�s�t�i�n�g�u�i�s�h�a�b�l�e� �f�r�o�m� �a�
�
� � �g�e�n�u�i�n�e� �5�0�,� �a�n�d� �t�h�e� �h�a�r�d� �f�i�l�t�e�r� ��s�w�e�_�s�c�o�r�e� �>�=� �7�1�� �t�h�e�n� �d�r�o�p�p�e�d� �i�t�.� �E�v�e�r�y� �n�e�w�l�y� �r�e�l�e�a�s�e�d�
�
� � �m�o�d�e�l� �w�a�s� �t�h�e�r�e�f�o�r�e� �i�n�e�l�i�g�i�b�l�e� �b�y� �d�e�f�a�u�l�t�.� �A�d�d�e�d� ��h�a�s�_�s�w�e�_�s�c�o�r�e�(�)��;� �m�o�d�e�l�s� �w�i�t�h� �n�o� �s�c�o�r�e�
�
� � �a�r�e� �n�o�w� �s�c�o�r�e�d� �n�e�u�t�r�a�l�l�y� �a�t� �t�h�e� �t�h�r�e�s�h�o�l�d� �a�n�d� �j�u�d�g�e�d� �o�n� �h�e�a�l�t�h� �a�l�o�n�e�.� �A� �m�o�d�e�l� �w�i�t�h� �a�
�
� � ��r�e�a�l�� �s�u�b�-�t�h�r�e�s�h�o�l�d� �s�c�o�r�e� �i�s� �s�t�i�l�l� �f�i�l�t�e�r�e�d� �o�u�t�.�
�
�
�
�-� ���R�a�t�e� �l�i�m�i�t�i�n�g� �w�a�s� �r�e�c�o�r�d�e�d� �a�s� �m�o�d�e�l� �f�a�i�l�u�r�e�.��� �O�n�e� �H�T�T�P� �4�2�9� �a�m�o�n�g� �t�h�r�e�e� �p�r�o�b�e�s� �p�u�s�h�e�d�
�
� � ��f�a�i�l�u�r�e�_�r�a�t�e�� �t�o� �0�.�3�3�,� �p�a�s�t� ��M�A�X�_�F�A�I�L�U�R�E�_�R�A�T�E��,� �m�a�r�k�i�n�g� �a� �w�o�r�k�i�n�g� �m�o�d�e�l� �d�e�a�d� �-�-� �a�n�d� �o�n�
�
� � �f�r�e�e�-�t�i�e�r� �k�e�y�s� �t�h�e� �c�h�e�c�k�e�r� �t�r�i�p�s� �t�h�a�t� �l�i�m�i�t� �b�y� �i�t�s�e�l�f� �(�4� �c�o�n�c�u�r�r�e�n�t� �p�r�o�b�e�s� �x� �3� �r�e�q�u�e�s�t�s�)�.�
�
� � �4�2�9�s� �a�r�e� �n�o�w� �r�e�t�r�i�e�d� �w�i�t�h� �b�a�c�k�o�f�f� �a�n�d�,� �i�f� �s�t�i�l�l� �t�h�r�o�t�t�l�e�d�,� �e�x�c�l�u�d�e�d� �f�r�o�m� �t�h�e� �f�a�i�l�u�r�e� �r�a�t�e�
�
� � �e�n�t�i�r�e�l�y� �r�a�t�h�e�r� �t�h�a�n� �c�o�u�n�t�e�d� �a�g�a�i�n�s�t� �t�h�e� �m�o�d�e�l�.� �T�h�e� �l�e�a�r�n�i�n�g� �D�B� �r�e�c�o�r�d�s� �t�h�e� �c�h�e�c�k�e�r�'�s�
�
� � �r�a�t�e� �i�n�s�t�e�a�d� �o�f� �r�e�c�o�m�p�u�t�i�n�g� �i�t�s� �o�w�n�,� �s�o� �t�h�r�o�t�t�l�i�n�g� �n�o� �l�o�n�g�e�r� �p�o�i�s�o�n�s� �t�h�e� �h�i�s�t�o�r�y�.�
�
�
�
�#�#� �A�d�d�e�d�
�
�
�
�-� ���R�e�t�i�r�e�d�-�m�o�d�e�l� �d�e�t�e�c�t�i�o�n� �a�n�d� �a�u�t�o�-�r�e�p�a�i�r�.��� �N�V�I�D�I�A� �r�e�t�u�r�n�s� �H�T�T�P� �4�1�0� �w�i�t�h� �a�n� �E�O�L� �d�a�t�e� �f�o�r�
�
� � �m�o�d�e�l�s� �p�a�s�t� �e�n�d�-�o�f�-�l�i�f�e� �(�a�n�d� �4�0�4� �f�o�r� �m�o�d�e�l�s� �g�a�t�e�d� �o�f�f� �y�o�u�r� �a�c�c�o�u�n�t�)�;� �t�h�e�s�e� �w�e�r�e� �p�r�e�v�i�o�u�s�l�y�
�
� � �i�n�d�i�s�t�i�n�g�u�i�s�h�a�b�l�e� �f�r�o�m� �a� �t�r�a�n�s�i�e�n�t� �e�r�r�o�r�,� �s�o� �t�h�e� �r�o�u�t�e�r� �s�a�t� �p�o�i�n�t�i�n�g� �a�t� �m�o�d�e�l�s� �t�h�a�t� �n�o�
�
� � �l�o�n�g�e�r� �e�x�i�s�t�.� �P�r�o�b�e�s� �a�r�e� �n�o�w� �c�l�a�s�s�i�f�i�e�d� ��r�e�t�i�r�e�d�� �/� ��t�h�r�o�t�t�l�e�d�� �/� ��h�e�a�l�t�h�y�� �/� ��u�n�h�e�a�l�t�h�y��,�
�
� � �p�r�o�b�i�n�g� �s�t�o�p�s� �o�n� �t�h�e� �f�i�r�s�t� �4�1�0�,� �a�n�d� �r�e�t�i�r�e�d� �m�o�d�e�l�s� �a�r�e� �r�e�p�o�r�t�e�d� �i�n� ��r�e�t�i�r�e�d�_�m�o�d�e�l�s��.�
�
� � �W�h�e�n� �a�n�y� �a�r�e� �f�o�u�n�d�,� �t�h�e� �s�w�i�t�c�h�e�r� �r�u�n�s� ��f�r�e�e�-�c�o�d�i�n�g�-�m�o�d�e�l�s� �-�-�s�y�n�c�-�s�e�t� �<�a�c�t�i�v�e� �s�e�t�>�� �t�o�
�
� � �r�e�d�i�s�c�o�v�e�r� �a�n�d� �l�i�v�e�-�p�r�o�b�e� �r�e�p�l�a�c�e�m�e�n�t�s�,� �r�e�s�t�a�r�t�s� �t�h�e� �F�C�M� �d�a�e�m�o�n� �(��-�-�s�y�n�c�-�s�e�t�� �r�e�w�r�i�t�e�s� �t�h�e�
�
� � �c�o�n�f�i�g� �f�i�l�e�,� �b�u�t� �t�h�e� �d�a�e�m�o�n� �s�e�r�v�e�s� �a�n� �i�n�-�m�e�m�o�r�y� �c�o�p�y� �a�n�d� �h�a�s� �n�o� �r�e�l�o�a�d� �e�n�d�p�o�i�n�t�)�,� �a�n�d�
�
� � �r�e�-�p�r�o�b�e�s� �o�n�c�e�.� �D�i�s�a�b�l�e� �w�i�t�h� ��F�C�M�_�A�U�T�O�_�R�E�S�Y�N�C�=�f�a�l�s�e��;� ��-�-�d�r�y�-�r�u�n�� �n�e�v�e�r� �t�r�i�g�g�e�r�s� �i�t�.�
�
�
�
�-� ����t�e�s�t�s�/�t�e�s�t�_�s�c�o�r�i�n�g�.�p�y���� �� �e�l�i�g�i�b�i�l�i�t�y� �r�e�g�r�e�s�s�i�o�n�s�:� �h�e�a�l�t�h�y� �m�o�d�e�l�s� �w�i�t�h� �u�n�k�n�o�w�n� �S�W�E� �m�u�s�t�
�
� � �b�e� �s�e�l�e�c�t�a�b�l�e�,� �F�C�M� �p�r�i�o�r�i�t�y� �m�u�s�t� �d�r�i�v�e� �t�h�e� �s�c�o�r�e�,� �r�e�a�l� �s�u�b�-�t�h�r�e�s�h�o�l�d� �s�c�o�r�e�s� �m�u�s�t� �s�t�i�l�l� �b�e�
�
� � �f�i�l�t�e�r�e�d�,� �u�n�h�e�a�l�t�h�y� �m�o�d�e�l�s� �m�u�s�t� �n�e�v�e�r� �b�e� �e�l�i�g�i�b�l�e�.�
�
�
�
�-� ����t�e�s�t�s�/�t�e�s�t�_�h�e�a�l�t�h�_�c�l�a�s�s�i�f�y�.�p�y���� �� �p�r�o�b�e� �c�l�a�s�s�i�f�i�c�a�t�i�o�n�:� �a� �4�2�9� �m�u�s�t� �n�o�t� �c�o�n�d�e�m�n� �a� �w�o�r�k�i�n�g�
�
� � �m�o�d�e�l�,� �a�n� �a�l�l�-�t�h�r�o�t�t�l�e�d� �r�e�s�u�l�t� �m�u�s�t� �r�e�a�d� �a�s� �u�n�k�n�o�w�n� �r�a�t�h�e�r� �t�h�a�n� �u�n�h�e�a�l�t�h�y�,� �a� �4�1�0� �m�u�s�t� �m�a�r�k�
�
� � �t�h�e� �m�o�d�e�l� �r�e�t�i�r�e�d� �a�n�d� �s�t�o�p� �f�u�r�t�h�e�r� �p�r�o�b�e�s�,� �a�n�d� �r�e�a�l� �f�a�i�l�u�r�e�s� �m�u�s�t� �s�t�i�l�l� �m�a�r�k� �i�t� �u�n�h�e�a�l�t�h�y�.�
�
�
�
�-� �N�e�w� �e�n�v� �k�n�o�b�s�:� ��F�C�M�_�A�U�T�O�_�R�E�S�Y�N�C��,� ��F�C�M�_�A�C�T�I�V�E�_�S�E�T��,� ��F�C�M�_�C�L�I��.� �S�e�e� �S�E�T�U�P�.�m�d�.�
�
�
�
�#�#� �U�p�g�r�a�d�i�n�g�
�
�
�
�I�f� �y�o�u�r� �r�o�u�t�e�r� �s�e�t� �p�r�e�d�a�t�e�s� �t�h�i�s� �r�e�l�e�a�s�e� �i�t� �m�a�y� �b�e� �f�u�l�l� �o�f� �r�e�t�i�r�e�d� �m�o�d�e�l�s�.� �R�e�b�u�i�l�d� �i�t� �o�n�c�e�:�
�
�
�
����b�a�s�h� � �f�r�e�e�-�c�o�d�i�n�g�-�m�o�d�e�l�s� �-�-�s�y�n�c�-�s�e�t� �f�a�s�t�-�c�o�d�i�n�g� � �f�r�e�e�-�c�o�d�i�n�g�-�m�o�d�e�l�s� �-�-�d�a�e�m�o�n�-�s�t�o�p� �&�&� �f�r�e�e�-�c�o�d�i�n�g�-�m�o�d�e�l�s� �-�-�d�a�e�m�o�n�-�b�g� � ����
�
�
�
�T�h�e�n� �r�e�h�e�a�r�s�e�:� ��p�y�t�h�o�n� �s�c�r�i�p�t�s�/�h�y�b�r�i�d�_�a�u�t�o�_�s�w�i�t�c�h�.�p�y� �-�-�d�r�y�-�r�u�n�� �(�w�i�t�h�
�
��N�I�M�_�A�U�T�O�_�S�W�I�T�C�H�_�E�N�A�B�L�E�D�=�t�r�u�e��)�.� ��e�l�i�g�i�b�l�e�_�c�o�u�n�t�� �s�h�o�u�l�d� �b�e� �n�o�n�-�z�e�r�o�.�
�
�
�
�-�-�-�
�
�
�
�
v1.0.0 - Initial Release: Hermes Hybrid Auto-Switch
v1.0.0 - Initial Release: Hermes Hybrid Auto-Switch
Always have at least one working model with SWE > 71% in Hermes — combining NVIDIA NIM health monitoring, cross-provider FCM router daemon, and persistent learning database.
✨ Features
Core Engine
- Hybrid Auto-Switch (
scripts/hybrid_auto_switch.py): Combines NIM health checks, FCM router data, and learning DB for intelligent model selection - Learning Database (
scripts/model_learning_db.py): SQLite tracking SWE scores, success rates, latency, streaks, uptime, and selection outcomes - Atomic Config Updates (
scripts/nvidia_nim_config_update.py): Temp-file write → atomic replace → 10 backup retention with MoA-aware sync - NIM Health Checks (
scripts/nvidia_nim_health_check.py): Concurrent requests, configurable thresholds, ranked JSON output
Cross-Provider Failover (Layer 2)
- FCM Router Daemon (
free-coding-models): Runs onlocalhost:19280, handles per-request failover across NVIDIA NIM, Groq, Cerebras, GitHub Models, OpenRouter, Mistral, Cloudflare, SambaNova - Real-time health scores with circuit breakers
SWE Quality Guarantee
- Hard filter: Only models with SWE ≥ 71% are eligible
- Seeded database: 25+ models with known SWE-bench scores
- Measured learning: Tracks actual performance over time
MoA-Aware Configuration
Updates Hermes config.yaml atomically:
- Primary model (
model.default→fcm:fast-coding) - Fallback model
- MoA aggregator model
- MoA reference models
Cross-Platform Hooks
- Linux/macOS:
hooks/start_fcm_daemon.sh,hooks/stop_fcm_daemon.sh - Windows Batch:
hooks/start_fcm_daemon.bat,hooks/stop_fcm_daemon.bat - Windows PowerShell:
hooks/start_fcm_daemon.ps1,hooks/stop_fcm_daemon.ps1
Documentation
- Animated HTML architecture diagrams (
docs/index.html) with Tailwind CSS - Subagent delegation guide (
docs/subagent-delegation.md) - Complete setup guide (
SETUP.md)
Testing
- E2E pipeline test (
tests/test_e2e.py): Validates FCM daemon, NIM health, config update, learning DB - SWE gate unit test (
tests/test_swe_gate.py): Validates 71% threshold logic
CI/CD
- GitHub Actions (
.github/workflows/ci.yml): Lint (Ruff), type-check (MyPy), tests on Python 3.10/3.11/3.12 - Issue/PR templates for consistent contributions
- Contributing guide and Code of Conduct (Contributor Covenant v2.1)
🚀 Quick Start
# 1. Install FCM router
npm install -g free-coding-models
# 2. Install Python dependencies
pip install -e .[dev]
# 3. Configure
cp config/.env.example ~/.hermes/.env
# Edit ~/.hermes/.env with your NVIDIA_API_KEY
# 4. Start FCM daemon
./hooks/start_fcm_daemon.sh # Linux/macOS
# or hooks/start_fcm_daemon.ps1 # Windows
# 5. Run auto-switch (or schedule via cron)
python scripts/hybrid_auto_switch.py📦 Installation
pip install hermes-model-auto-switch
# Then use entry points:
hermes-auto-switch # Run hybrid auto-switch
hermes-nim-health # Run NIM health check
hermes-learning-db # Learning DB operations
hermes-config-update # Update Hermes config🏗️ Architecture
┌─────────────────────────────────────────────────────────────┐
│ LAYER 1: HERMES AGENT │
│ Reads config.yaml → uses fcm:fast-coding via localhost:19280│
└──────────────────────────┬──────────────────────────────────┘
▼
┌─────────────────────────────────────────────────────────────┐
│ LAYER 2: FCM ROUTER DAEMON (localhost:19280) │
│ Cross-Provider Failover: NVIDIA NIM → Groq → Cerebras... │
│ Per-request failover • Circuit breakers • Real-time scores │
└──────────────────────────┬──────────────────────────────────┘
▼
┌─────────────────────────────────────────────────────────────┐
│ LAYER 3: HYBRID AUTO-SWITCH CRON (Every 15 min) │
│ NIM Health + FCM Router + Learning DB → SWE > 71% filter │
│ Updates: primary, fallback, MoA aggregator, MoA refs │
└──────────────────────────┬──────────────────────────────────┘
▼
┌─────────────────────────────────────────────────────────────┐
│ LAYER 4: PERSISTENT STATE │
│ Learning DB (SQLite) + Hermes config.yaml (atomic + backup)│
└─────────────────────────────────────────────────────────────┘
📝 Changelog
- f0b78fc feat(core): hybrid auto-switch engine with learning DB and atomic config updates
- 6678736 feat(hooks): cross-platform FCM daemon lifecycle (sh/bat/ps1)
- e575939 test: e2e pipeline + SWE gate unit tests
- 2cee78f docs: architecture diagrams, setup guide, delegation docs
- chore: release prep - CI, contributing, packaging, pyproject.toml
🔗 Links
- Repository: https://github.com/rhishi99/hermes-model-auto-switch
- Documentation: See
docs/index.html(open in browser) - Issues: https://github.com/rhishi99/hermes-model-auto-switch/issues
Full Changelog: https://github.com/rhishi99/hermes-model-auto-switch/commits/v1.0.0