Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix 'ValueError' in 'ct2-opennmt-py-converter' for Unsupported '--self_attn_type' #1647

Closed
wants to merge 1 commit into from

Conversation

chingizof
Copy link

Description:

This pull request resolves a ValueError when converting OpenNMT models to CTranslate2 using ct2-opennmt-py-converter. The issue was due to unsupported --self_attn_type scaled-dot-flash, with only scaled-dot being supported.

Solution:
Implemented a fix based on a suggestion from a discussion by vince62s on the OpenNMT forum, which successfully addresses the conversion issue.

Testing:
Verified the fix by converting models that previously triggered the error, ensuring the process now completes without issues.

Reference:

Solution inspired by a post on OpenNMT Forum.
This update should help users facing similar conversion problems.

@vince62s
Copy link
Member

Hi, my message on the forum was a hack not really a fix.
In fact a better fix is to replace (when applicable) "scaled-dot-flash" by "scaled-dot" because in onmt-py we added "scaled-dot-flash" to force decoding using flash attention but it has no real usage in ct2.

@vince62s
Copy link
Member

closing in favor of #1687

@vince62s vince62s closed this Apr 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants