Add iterrows() and itertuples() DataFrame API, its usage is similar to pandas

[kxbin]

I'm sorry about this, because I want to squash multiple commits, and accidentally modified the branch name.
So that the PR was automatically closed unexpectedly.

For historical Conversation see PR #369

@sethmlarson Thanks for your comment
Based on these suggestions, I have completed the modification, And passed the lint and docs jobs.

Thanks to this change in #379, no need to convert _es_results_to_pandas into a generator now, because they have similar effects.
Now, the performance has been further improved and the logic is more concise.

Finally, the same 50,000 data sets, my test results are as follows:

ed.iterrows(),  It took a total of `19 seconds` after the iteration
ed.itertuples(), It took a total of `15 seconds` after the iteration
ed.to_pandas(), It took `15 seconds`

Closes #345

[elasticmachine]

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

[V1NAY8]

Thank You for all these changes.

• Can you once run pytest --doctest-modules eland/ tests/, doctests failed in my local
• Docstrings must follow https://numpydoc.readthedocs.io/en/latest/format.html

[sethmlarson]

Jenkins test this please

[kxbin]

Thank You for all these changes.

• Can you once run , doctests failed in my localpytest --doctest-modules eland/ tests/
• Docstrings must follow https://numpydoc.readthedocs.io/en/latest/format.html

Ok, wait a minute, let me try.

[kxbin]

I run pytest --doctest-modules eland/tests/ failed.

Raise a elasticsearch.exceptions.NotFoundError: no such index [flights]

Maybe it’s because I don’t have the ES_TEST_CLIENT environment

[V1NAY8]

If you are using linux and have docker installed
Do the following:

>>> sysctl -w vm.max_map_count=262144
>>> ELASTICSEARCH_VERSION=elasticsearch:7.x-SNAPSHOT .ci/run-elasticsearch.sh

leave the above terminal open

Open a different terminal and then

python -m tests.setup_tests

Then run tests

[sethmlarson]

I'd recommend only running the tests you need, some of the ML tests require a lot of resources.

[kxbin]

@V1NAY8 Thank you so much for the tutorial.

I successfully set up a test environment.
After further modification and testing, it can now pass the doctests.

[sethmlarson]

jenkins test this please

[kxbin]

Oops，It seems that the test command executed by Jenkins is not the same as the one I tested locally.

Jenkins executes this command:
python -m pytest --cov-report term-missing --cov=eland/ --doctest-modules --nbval eland/tests/

Let me modify it again, I probably know the problem.

[sethmlarson]

Awesome! Thanks for all the work you've put into this, I and others really excited to use this feature! 🤩

[V1NAY8]

LGTM :)

[sethmlarson]

jenkins test this please

[sethmlarson]

jenkins test this please

[sethmlarson]

jenkins test this please

[sethmlarson]

I made some modifications to your test suite to make it more concise and remove the no-ops. One more round of testing and this should be good to merge! ✨

[sethmlarson]

jenkins test this please

[sethmlarson]

Looks like CI is failing after removing the generator exhaustion inside of list(...). The itertuples doesn't look like a failure it's just a precision thing with floats. The iterrows failure looks stranger, will have to dive deeper. Going to check it out locally.

Btw you should do a git pull on your branch as I've made a handful of changes.

[sethmlarson]

jenkins test this please

[sethmlarson]

There was nothing wrong with the functionality, just the tests needed slightly different assert to make sure things worked. I think we're nearly there!

[sethmlarson]

Woo! 🎉 Thank you so much for this @kxbin! I'll mention your name next to this contribution in the next release changelog.

I hope you enjoyed contributing to Eland, if you've got time and are interested there are plenty of other issues to work on that I can assist with. You can also submit this contribution to the Elastic Contributor Program which recognizes community contributors and sends you prizes and swag too 🎁

[kxbin]

Woo! 🎉 Thank you so much for this @kxbin! I'll mention your name next to this contribution in the next release changelog.

I hope you enjoyed contributing to Eland, if you've got time and are interested there are plenty of other issues to work on that I can assist with. You can also submit this contribution to the Elastic Contributor Program which recognizes community contributors and sends you prizes and swag too 🎁

Thank you very much for your assistance.
I am very happy to be able to use these new features in eland in the future! 😋