Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parrallize the map/reduce chain #176

Closed
hwchase17 opened this issue Nov 23, 2022 · 4 comments
Closed

parrallize the map/reduce chain #176

hwchase17 opened this issue Nov 23, 2022 · 4 comments
Labels
03 enhancement Enhancement of existing functionality

Comments

@hwchase17
Copy link
Contributor

make the calls in the "map" part concurrently

@hwchase17 hwchase17 added the 03 enhancement Enhancement of existing functionality label Nov 23, 2022
@vshulman
Copy link

might be helpful to also make the intermediate outputs of the map step accessible to the caller

@devonbrackbill
Copy link

I understand the map/reduce chain to currently do 1 step of map and then "stuff" all of the results into 1 final query. I'm running into token limits on the final "stuff". Is there an obvious way to call map on the intermediate summaries until the input to the final "stuff" is below the token threshold?

@hwchase17
Copy link
Contributor Author

sorry @devonbrackbill i did not see this earlier! i think this should be addressed now - could you confirm?

@devonbrackbill
Copy link

Yes, this is addressed. Thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
03 enhancement Enhancement of existing functionality
Projects
None yet
Development

No branches or pull requests

3 participants