Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How much time will it take to run preprocess step? #21

Open
alan-ai-learner opened this issue Jul 27, 2021 · 2 comments
Open

How much time will it take to run preprocess step? #21

alan-ai-learner opened this issue Jul 27, 2021 · 2 comments

Comments

@alan-ai-learner
Copy link

No description provided.

@Impavidity
Copy link
Contributor

This is the output when I run python run.py preprocess experiments/spider-configs/gap-run.jsonnet.

As you can see, it takes 25mins to preprocess on the train set.
Screen Shot 2021-08-20 at 3 03 46 PM

Make sure that your Stanford NLP service is responsible.
If you did not see similar preprocessing time, one possible issue is your file system. If you store your databases on filesystems that will cache/commit every action you made (for recovery), the preprocessing will be very slow.

Let me know if you have other questions.

Peng

@alan-ai-learner
Copy link
Author

Okay.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants