GitHub Copilot crashes on the word "suicide"

[matthewrenze]

Select Topic Area

Bug

Body

Description:
When using GitHub Copilot in either PyCharm or VS Code, including the word "suicide" in your code causes GitHub Copilot to crash.

Steps to reproduce:

Just include the following code in your IDE

x = "suicide"

# Print x

Expected behavior:
Pressing [enter] after "# Print x" should trigger Copilot to complete the next line with "print(x)". However, the word suicide in the line above causes Copilot to stop working everywhere in the file.

Notes:
My best guess is that the additional safety guardrails used on Microsoft's Azure OpenAI service are blocking any calls or responses containing the word "suicide" for safety reasons.

I discovered this bug while performing a data analysis of the "Suicide Rates Overview 1985 to 2016" data set from Kaggle:
https://www.kaggle.com/datasets/russellyates88/suicide-rates-overview-1985-to-2016

[Akash1134]

Hi @matthewrenze ,

I have shared this with my team internally. However, we might not be able to provide much context as the topic is sensitive in nature. Thanks!

[matthewrenze]

@Akash1134 Thanks for the quick reply. I completely understand.

In the meantime, I just renamed the column to "risk", and everything works fine.

[github-actions[bot]]

🕒 Stale Discussion Alert 🕒

This Discussion has been labeled as stale by an automated system for having no activity in the last 60 days. Please consider one the following actions:

1️⃣ Close as Out of Date: If the topic is no longer relevant, close the Discussion as out of date at the bottom of the page.

2️⃣ Provide More Information: Share additional details or context — or let the community know if you've found a solution on your own.

3️⃣ Mark a Reply as Answer: If your question has been answered by a reply, mark the most helpful reply as the solution.

Note: This stale notification will only apply to Discussions with the Question label. To learn more, see our recent stalebot announcement.

Thank you for helping bring this Discussion to a resolution! 💬

[9021007]

I can confirm that this occurs. This is likely a side-effect of trying to ensure that Copilot doesn't suggest bad ideas, but a word blacklist is not a good way of doing this. Maybe pass the output through a binary classification program to detect if it's actually encouraging suicide, or if the program just happens to have the word suicide in it.