Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to specify output language for CLI input #604

Closed
wants to merge 8 commits into from

Conversation

JFK
Copy link

@JFK JFK commented Jun 14, 2024

I am not sure if this is the best approach, but I wanted to propose it for consideration.

What this Pull Request (PR) does

Add option to specify output language for CLI input

  • Added --output-language (-L) argument to specify the output language for CLI input. Default is English.
  • Updated main function to include the output language instruction in the input text if specified.

This allows users to set the desired output language for responses directly from the command line.

Related issues

JFK#5

JFK and others added 8 commits June 14, 2024 13:58
…JSON output

- Updated print(transcript) to handle non-ASCII characters correctly.
- Modified print statements to use ensure_ascii=False in json.dumps for correct display of non-ASCII characters.
- Implemented exception handling to manage potential UnicodeEncodeErrors, ensuring fallback encoding to avoid garbled text.

This change enhances the readability of outputs containing non-ASCII characters and addresses potential encoding issues across different environments.
…scripts-to-prevent-garbled-output

Improved handling of non-ASCII characters in transcript printing and …
- Updated `README.md` to include instructions for using local openai-whisper.
- Added error handling to `ts.py` for cases where `import whisper` fails.
- Modified `ts.py` to support the use of the local openai-whisper model.
- Added new command-line arguments to `ts.py` for specifying language and using the local openai-whisper model.

To use the local openai-whisper, run:
pipx inject fabric openai-whisper

This update allows users to choose between the accuracy of the OpenAI API and the cost-effectiveness of local processing.
…-whisper-with-the-ts-command

Add support for local openai-whisper in ts command
- Added `--output-language` (`-L`) argument to specify the output language for CLI input. Default is English.
- Updated `main` function to include the output language instruction in the input text if specified.

This allows users to set the desired output language for responses directly from the command line.
…cify-additional-instructions-by-default-such-as-the-output-language

Add option to specify output language for CLI input
feat: Add `ft` command to fetch contents from URLs and local files

- Modified the `ft` command to accept both URLs and local file paths as input.
- Implemented file type detection to handle different file formats:
  - HTML from URLs
  - PDF files
  - DOCX files
  - PPTX files
  - XLSX files
- Used appropriate libraries to extract content from these formats and convert them into markdown.
- Provided an option to include links at the bottom of the markdown output when processing HTML content from URLs.

This enhancement increases the tool's flexibility and usability by enabling it to process and convert various content types into markdown format.
…tch-contents-from-urls-and-local-file

Refs: #7
@silverstreak
Copy link
Contributor

Would it not be possible to simply add a line in the system.md pattern file text indicating what the output language should be?

@JFK
Copy link
Author

JFK commented Jun 19, 2024

@silverstreak

Would it not be possible to simply add a line in the system.md pattern file text indicating what the output language should be?

I tried to add a line into system.md but it did not work...
Anyway I can understand English.

@silverstreak
Copy link
Contributor

I'll also try it at my end and let you know. But in general, if it works, then it should be fairly easy to provide an input parameter like --output-language and then programmatically append/prepend "The output should be in xxx language" to the system.md file text that is sent to the model.

@Gerkinfeltser
Copy link

... to the system.md file text that is sent to the model.

I've don't similarly with a context.md (& the -c flag) to good effect. I was getting some French responses prior but not when using my modified context file.

Side note: I saw someone mention adding functions to specify different context files. This seems like it would be super useful in the case of switching response languages per fabric command (among other things).

@silverstreak
Copy link
Contributor

silverstreak commented Jun 20, 2024

@Gerkinfeltser @JFK

I'll also try it at my end and let you know.

OK so here's the result of my investigation.

  1. --context option
    • The documentation on the --context option is quite sparse (almost non-existent)
    • Created context.md in ~/.config/fabric
    • Added this to it: # FINAL INSTRUCTION \n All the output should be in French.
    • Command used: pbpaste | fabric -c -p extract_wisdom
    • Outcome: nothing happens. The context file appears to be ignored
    • Note: maybe I'm using --context wrong
  2. Append a line to an existing pattern
    • Used extract_wisdom as an example
    • Appended the same line to the end of it: # FINAL INSTRUCTION \n All the output should be in French.
    • Outcome: output is in French

So I really think the most intuitive way to implement this feature (and actually, more generally, to add custom instructions to any pattern) is to do it this way: append whatever custom instructions you want to the system.md text that is sent to the model. Allow the user to input them via a file and using an option like --custom_instructions. My 2 cents.

@danielmiessler
Copy link
Owner

We're getting ready to migrate to Golang; please resubmit then if this hasn't been addressed. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants