Final work report of Google Summer of Code'21
17th May 2021, my proposal got selected for Google Summer of code'21 with Hydra Ecosystem, It was the most happiest moment in my life. My project was aimed to Build a Hydra-powered API for serving the loan portfolio managment with Open-Risk.
It was an amazing journey developing a project from scratch to mainting the repository, I have learned a lot of new thing, which are added in my developers bucket and will surely be heplful in the future. The idea behind the project is to build a prototype of "APIs for Fintech" with Hydra Ecosystem.
The Final completed project can be found here:
Project's Repo : https://github.com/HTTP-APIs/creditrisk-poc
Project is live @ http://34.145.188.116:8080/
We started with building a subset of POC with small Non-Performing Loan ontology. After the sucessfull completion of subset POC, we moved forward with the complete Non-Performing Loan ontlogy (All the classes and properties required for Loan portfolio data), where I needed to automate the creation of ApiDocumentation for which I have written two automation scripts:
- Vocab_generator: vocab_generator generates “NonPerformingLoan.jsonld” using rdflib and pyld libraries to parse & serialize owl ontology to JSON-LD with the @context.
- nplvocab_parser: nplvocab_parse parses all the classes & properties from“NonPerformingLoan.jsonld” and converts them to HydraClass & HydraClassProp.
With the help of there two scripts api_docwriter.py can generates the ApiDocumentation automatically according to Non-performing Loan ontology.
I have also developed Mock_portfolio_generator which is basically used as a mock_client to populate the database with the more realistic data, It automatically creates the object of the classes on the basis of ApiDocumentation.
Then I have Re-structured the complete repository according to the python standards and wrote tests using pytest for the API.
While working with the creditrisk-poc we encountered an issue that the class attributes (columns) in the database are of type VARCHAR by default for every property in hydrus. After a good discussion with the mentors it was decided that it's better to have attributes of a similar type as in the hydra property,
So my GSoC colleague implemented this feature in the hydrus and hydra-python-core, and then finally I was able to use it in creditrisk-poc successfully.
After the comlpletion of all these task I have tested the API on local system and got it reviewed with the mentors, now we were ready for the last stage i.e DEPLOYMENT 🚀
Here is the final Database schema with the foreign key relation.
With the help of my GSoC colleague we have dockerized the application and tested it, we have also changed the database from sqlite to postgres. Then I deployed the application Google Cloud Platform's compute engine and the API is live @ http://34.145.188.116:8080/ 🎉🎉
| PR Link | Description | Status |
|---|---|---|
| #1 | NPL Jsonld for POC | Merged ✅ |
| #3 | API Doc Created & hydrus setup. | Merged ✅ |
| #87 | HydraClass default endpoint set True | Merged ✅ |
| #89 | @context updated to W3C version | Merged ✅ |
| #2 | Create README.md | Merged ✅ |
| #7 | Repository re-structured. | Merged ✅ |
| #9 | Create integration testing code | Merged ✅ |
| #12 | Functionality tests added. | Merged ✅ |
| #13 | readme updated | Merged ✅ |
| #14 | Collateral class added | Merged ✅ |
| #8 | README.md updated & Functional doc added. | Merged ✅ |
| #17 | ApiDoc Automation script created & mock portfolio data generator refectored. | Merged ✅ |
| #19 | Owl -> Jsonld automation script created. | Merged ✅ |
| #21 | Datatypes for DB implemented | Merged ✅ |
| #22 | Readme & docs updated, test for iri collection added | Merged ✅ |
| #23 | Test doc added | Merged ✅ |
| #24 | Final ApiDoc created, nplo jsonld & mock_portfolio_generator refactored | Merged ✅ |
| #28 | hydrus version upadated | Merged ✅ |
| #32 | Final README Updated | Open |
| Issue link | Description |
|---|---|
| #4 | Create NonPerforming Jsonld vocabulary |
| #5 | Create API_Doc & Setup hydrus |
| #6 | [Discussion] Check datatype of properties in the database |
| #9 | Create integration testing code |
| #11 | Add collateral class |
| #74 | Change hydra core fetching in context. |
| #86 | HydraClass default endpoint should be True |
| #15 | Create Automated script for creating NPL Vocabulary |
| #16 | Refactor the mock_data_generator |
| #18 | Automate ontology to josnld creation |
| #20 | Update Readme & Add more docs |
| #33 | Fix the datatype of the properties in the NonPerformingLoan.jsonld |
It was an adventurous journey filled with lots of learning at each and every stage,I have learned a lot of things from structuring the repository to the deployment, It was mind-blowing experience working as student developer in Google Summer of Code, It was my dream which came true.
I am greatful to all my mentros for their continous guidance and clearing my each every doubt, I have gained alot of knowledge and experience, which reflects in my coding style, writing clean and understandable code.
Thankyou for sharing such an valuable experince and knowledge.
I will be an active member in the organization and will encourage the new comers to be the part of this growing organization.
I would also like to thanks my GSoC colleague Hasan Faraz Khan for being an amazing partner in this marvelous journey.
