Skip to content

Issues: NVIDIA/garak

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

322 Open 310 Closed
322 Open 310 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

garak scans a text embedding model question Further information is requested
#1117 opened Feb 27, 2025 by astefano
Empty Output Generating Hits bug Something isn't working
#1114 opened Feb 25, 2025 by JosephTLucas
Early Warning/Stopping on Suspected Bad Config
#1113 opened Feb 25, 2025 by JosephTLucas
tests: llava generators Interfaces with LLMs tests Testing-related
#1105 opened Feb 18, 2025 by leondz
Error running active probes bug Something isn't working
#1103 opened Feb 7, 2025 by milonbhattacharya
Probe: JailbreakV probes Content & activity of LLM probes
#1099 opened Feb 4, 2025 by erickgalinkin
@erickgalinkin
1
max_tokens missing in Single object
#1096 opened Feb 3, 2025 by dapascual
4
probe: ETA for VLMs new plugin Describes an entirely new probe, detector, generator or harness probes Content & activity of LLM probes
#1095 opened Feb 3, 2025 by leondz
FEAT: Set Global HF Model Directory for Detectors
#1094 opened Feb 2, 2025 by EricXQiu
detector: StringDetector matchtype fullmatch detectors work on code that inherits from or manages Detector new plugin Describes an entirely new probe, detector, generator or harness
#1084 opened Jan 17, 2025 by leondz
1
Add caching to GH actions environment for HF items housekeeping Repository and project maintenance
#1080 opened Jan 15, 2025 by leondz 25.02 Efficiency
Add dry run option that includes stats architecture Architectural upgrades generators Interfaces with LLMs
#1079 opened Jan 15, 2025 by leondz 25.02 Efficiency
Output estimated run cost at top of run architecture Architectural upgrades
#1078 opened Jan 15, 2025 by leondz 25.02 Efficiency
Order prompts within probes for max bag attack success rate architecture Architectural upgrades probes Content & activity of LLM probes
#1077 opened Jan 15, 2025 by leondz 25.02 Efficiency
Estimate # prompts, # input tokens per probe architecture Architectural upgrades
#1071 opened Jan 11, 2025 by leondz 25.02 Efficiency
Enable stopping when call (/token) budget hit architecture Architectural upgrades
#1070 opened Jan 11, 2025 by leondz 25.02 Efficiency
Automate updating the calibration process housekeeping Repository and project maintenance
#1069 opened Jan 10, 2025 by leondz 25.02 Efficiency
Harness that orders probes for expected attack success rate (ASR) according to bag architecture Architectural upgrades
#1068 opened Jan 10, 2025 by leondz 25.02 Efficiency
2
Add “real” multi-turn support to chat-enabled generators architecture Architectural upgrades generators Interfaces with LLMs
#1067 opened Jan 10, 2025 by leondz 25.02 Efficiency
probe: Adapt sandwich attack to auto-find effective languages architecture Architectural upgrades probes Content & activity of LLM probes
#1066 opened Jan 10, 2025 by leondz
2 tasks
25.02 Efficiency
Bayesian optimisation base class architecture Architectural upgrades probes Content & activity of LLM probes
#1065 opened Jan 10, 2025 by leondz 25.02 Efficiency
Pattern for probes that can react to generator/detector responses architecture Architectural upgrades probes Content & activity of LLM probes
#1064 opened Jan 10, 2025 by leondz 25.02 Efficiency
docs: model type and name mixed up documentation Improvements or additions to documentation
#1063 opened Jan 10, 2025 by dennisrall
1
support for python 3.13 architecture Architectural upgrades
#1062 opened Jan 8, 2025 by leondz
7 tasks
2
Include information about why detectors are skipped detectors work on code that inherits from or manages Detector quality-accuracy This affects result quality/reliability reporting Reporting, analysis, and other per-run result functions ux Interface & interaction improvements
#1061 opened Jan 7, 2025 by erickgalinkin 25.02 Efficiency
@erickgalinkin
ProTip! Exclude everything labeled bug with -label:bug.