Skip to content

Commit

Permalink
Running tricky questions file
Browse files Browse the repository at this point in the history
  • Loading branch information
johnjosephhorton committed Jan 25, 2024
1 parent c00fd9a commit 037683d
Show file tree
Hide file tree
Showing 2 changed files with 19 additions and 0 deletions.
3 changes: 3 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,9 @@ integration-models:
integration-job-running:
pytest -v --log-cli-level=INFO integration/test_job_running.py

integration-tricky-questions:
pytest -v --log-cli-level=INFO integration/test_tricky_questions.py

#pytest --log-cli-level=INFO tests/test_JobRunning.p
lint: ## Run code linters (flake8, pylint, mypy).
mypy edsl
Expand Down
16 changes: 16 additions & 0 deletions integration/test_tricky_questions.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
from edsl.questions import QuestionFreeText
from edsl import Model


def test_failure():
# https://github.com/goemeritus/edsl/issues/78
m35 = Model("gpt-3.5-turbo")
m4 = Model("gpt-4-1106-preview")

q = QuestionFreeText(
question_name="personas",
question_text="Draft detailed narratives for 5 personas likely to provide diverse responses to a public opinion poll about AI.",
)

r_m35 = q.by(m35).run()
r_4 = q.by(m4).run()

0 comments on commit 037683d

Please sign in to comment.