Repository navigation
v1.0.0 HVT Full Model FInally Released. #4
calisweetleaf
announced in
Announcements
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
The first production implementation of a trainable, rlhf capable, NGSST based model the HVT:v2 has been released. Training and RLHF pipeline are not fully prepared for public release. Due to internal development on the HVT:r-1 research variant, the training and dataset pipeline has been interrupted, so expect upcoming final decided, released production files for the hvt_v2. Where the first version was more of a demonstration, this version represents an actual display of potential. The r-1, research variant, of the harmonic vision transformer, will likely adopt a broader name due to increased capabalities. Internal testing showing serious potential, edge cases however must be smoothed out. This comes after the large disclosure of the recursive-categorical-framework, causing development monthly timeline of ngsst to be "off." This release includes hvt_v2.py, the training pipeline for quick validation and 2 dataset training, then a small dpo rl pipeline. This wont produce a SOTA model, it does however, along with the visualizations gathered, solidify the potential future of the Harmonic and Geometric State Space Transformer paradigm. Where rcf introduces an entire new ontology of frameworks, NGSST was set out for one goal, to match/go beyond average performance of a small GPT based MoE models. Training was done purely on CPU, Ryzen 5 2400g and 12gb ddr4 ram with integrated Vega 11 iGPU and took less than 1 hour to complete training. RL pipeline finished around 30 minutes.
Discussions are now opened. WIKI has been updated.
This discussion was created from the release v1.0.0 HVT Full Model FInally Released..
All reactions