GitHub - eggsyntax/py-user-knowledge: Predicting demographics of users with GPT based on text they've written

eggsyntax / py-user-knowledge Public

Notifications You must be signed in to change notification settings
Fork 0
Star 2

Predicting demographics of users with GPT based on text they've written

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 22 Commits
.vscode		.vscode
results		results
.gitignore		.gitignore
README		README
calibration.py		calibration.py
conda_env.yml		conda_env.yml
conda_requirements.txt		conda_requirements.txt
loss.py		loss.py
okcupid.py		okcupid.py
openai_uk.py		openai_uk.py
persuade.py		persuade.py
prc.py		prc.py
py-user-knowledge.code-workspace		py-user-knowledge.code-workspace
test-demographics.py		test-demographics.py
uk01.py		uk01.py
utils.py		utils.py

Repository files navigation

Series of experiments testing how well LLMs (mainly GPT-3.5) can predict 
demographics from text (mainly OKCupid profiles).

To run from the command line:
- Make sure that OPENAI_API_KEY is defined in your environment
- Install packages (untested as yet, please let me know if you encounter difficulties): `conda install --file conda_requirements.txt`
- `python test-demographics.py`

NOTE: this is HORRIBLE CODE. This was my experiment with letting GPT-4 generate 
most of the individual functions, and then it's just patches on patches 
from there.

It suffers further from my initial naivete about typical ML conventions for 
eg data representation, so I'm munging data back and forth in a bunch of places.

Ideally I will rewrite it when I get time, but also when do I ever get time?

Caveat emptor.

Note that despite using temperature=0, the probability distribution predicted 
by GPT will vary somewhat between runs, so results will differ.