-
Notifications
You must be signed in to change notification settings - Fork 3
Assignment 3 final
Contributing the different features to the Apache OpenSource project Airavata-MFT. We try to focus on the improvements with respect to adding new transport for the MFT and building different clients for the MFT. We will be the focusing on the test driven development where we want to improve the test coverage for the entire project. We are now concentrated on the different transport modules in the project.
Initial problem statement focused on building service mesh on top of Project 2 that we had built. We decided the change the approach during phase 1 as we reevaluated our course preference and we wanted to contribute to the apache project. Also, we initially thought that meaningful contribution to Airavata-MFT would be difficult given the time constraint.
We initiated the convesations on the developer list. In fact, team Pika Pika developed the new transport mechanism and we were following the conversation in the mailing list. We also what all that is required with that follow up. Below are the links to the conversations
New Transport Support: (FTP)
https://www.mail-archive.com/dev@airavata.apache.org/msg12645.html
New Client Support (Python)
https://www.mail-archive.com/dev@airavata.apache.org/msg12721.html
Improving the code coverage
https://www.mail-archive.com/dev@airavata.apache.org/msg12720.html
Intially we went through the architecture of the Apache Airavata MFT. We forked the repo from the Airavata-MFT and played around did some testing in the local. After resolving initial dependency issues we are able to successfully run the code in our local systems. As far as the investigation is concerned we saw the opportunity to develop the support for new transport. However, we could contibute to one transport although we wanted to contribute more improvements on transport. As far as developing additional client is concerned we were inspired by on of the other Airavata projects (Airavata Custos) where there are multiple language clients. We wanted to implement that in Airavata-MFT. As far as the code coverage is concerned, this was right in front of our noses. Except mft-core module there were no test cases for other modules.
Firstly we did a brainstrom of the MFT architecture. There were implementations of SCP, Local, S3 already. With a bit of code analysis we came to know what needs to be done. The code so modularized that understanding how to implement new transport was as clear as day.
contribution - https://github.com/apache/airavata-mft/commits?author=gkiran292
For client implementation we took inspiration from Apache Airavata Custos. However, implentation was a bit different as we wanted to generate the python code from stubs from the POM plugin. It is evident for java as it has maven support, integrating proto plugin for python plugin got us to do some serious digging. Although protobuf-maven-plugin mentions the way to integrate the plugin, it was not obvious. We had to build the plugin from the source to integrate that plugin in the project. Well, we can now integrate the other language plugins with protobuf-maven-plugin by building the proto plugin for thier respective languages from source.
contribution - https://github.com/apache/airavata-mft/pull/17/commits
For improving the code coverage we stuck to the basics where we tried with local and went ahead with the unit test cases as there had been none already
contribution - https://github.com/apache/airavata-mft/pull/18/commits
The MFT codebase is very well structured. Each and every module is so intuitive and well modularized. We especially appreciate how the stub information is passed on to different modules via the dependency. In fact in our previous projects we used to transport the proto files every time and it didn't occur to us to build it and transport it as library. This may be very small and subtle, but we are definitely taking forward this learning for the future.
Heads up! Apart from mailing discussion lot of things were suggested and exchanged in the git commit comments and conversations.
GOPI KIRAN https://www.mail-archive.com/dev@airavata.apache.org/msg12645.html
https://www.mail-archive.com/dev@airavata.apache.org/msg12696.html
https://www.mail-archive.com/dev@airavata.apache.org/msg12738.html
https://github.com/apache/airavata-mft/commits?author=gkiran292