Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

What does TEST_PAIRS_PATH refer to in the evaluate function in cli.py? #18

Closed
pk1130 opened this issue Jul 14, 2021 · 1 comment
Closed

What does TEST_PAIRS_PATH refer to in the evaluate function in cli.py? #18

pk1130 opened this issue Jul 14, 2021 · 1 comment

Comments

@pk1130
Copy link
Contributor

pk1130 commented Jul 14, 2021

Title.

Sidenote: First line of the evaluate function also has a console.log() that contains the data_path variable when it's not passed into the function. Since data_path is not being used in the function, that might have to be commented out or removed.

Thanks!
@pk1130 pk1130 changed the title What does TEST_PAIR_PATH refer to in the evaluate function in cli.py? What does TEST_PAIRS_PATH refer to in the evaluate function in cli.py? Jul 14, 2021
@pk1130 pk1130 mentioned this issue Jul 14, 2021
Closed
@pk1130
Copy link
Contributor Author

pk1130 commented Jul 16, 2021

Refers to the subject-item pairs that we want to test. In our case, subjects would be the models that have predicted on SQuAD and items would be the questions that they have predicted on. Resolved.

@pk1130 pk1130 closed this as completed Jul 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@pk1130