Google Summer of Code @ Kornia

Welcome to the Google Summer of Code @ Kornia

Table of Contents

• What is Google Summer of Code 🌞 👩‍💻
• How to apply ? 🧑‍🎓
• Project Ideas List 💡 💡
  • Improve documentation and tutorials website
  • Camera calibration module
  • Better ecosystem with modern Local Features
  • SfM module
  • Probabilistic testing framework
• Timeline 📆
• How do I pass the GSoC evaluations ? 🚀

What is Google Summer of Code

more info: https://developers.google.com/open-source/gsoc/resources/downloads/GSoC2021Presentation.pdf

How to apply ?

1. Are you a student at a university in an eligible country ? [ READ ] 🧑‍🎓 🗺️
2. GO through the Project Ideas List list below 📄
3. Pre-Apply to this [ FORM ] AND join Kornia Slack [ JOIN ]
4. IF you are contacted by a mentor THEN write the project proposal ✍️
5. ELSE improve your skills and try next year 🔁
6. SUBMIT your project proposal through GSoC website !! (VERY IMPORTANT) ⚠️ ⚠️
7. The project admins will balance the applications ⚖️
8. IF you passed this process THEN Congratulations!! You're in !! 🎉 🎆
9. GO TO How do I pass the GSoC evaluations ? ➡️ 🚀

DISCLAIMERS:

• We won't consider any application from a student that hasn't been contacted by a mentor.
• Projects without a detailed schedule won't be considered. 📆
• The GSoC is a full-time internship; do not expect being contacted or if you are already working.
• A Project failure is not an option; we won't take that risk.
• The final application is on the GSoC site; otherwise your are out.
• Do not open useless pull requests to increase your git history; we know how to detect fake profiles.
• Pre-selected students might expect a screen interview.
• Google pays to the student; not kornia.org.
• If you are not notified by Google; you are not in.
• Read the GSoC student GUIDELINES & FAQ

Project Ideas List

1. IDEA: Improve documentation, tutorials website

   • Description: Review the documentation and the tutorials website. Re-organise existing tutorials, create new examples, improve testing on the tutorials, better link between functions and examples, benchmarks, etc.
   • Expected Outcomes:
     • A new reformatted website for tutorials and examples.
     • Proper benchmarkin suite.
   • Resources:
     • https://github.com/kornia/kornia-examples
     • https://github.com/kornia/tutorials
     • https://github.com/kornia/kornia-benchmark
   • Skills Required: PyTorch and basic knowledge about computer vision and Kornia
   • Possible Mentors: Edgar Riba, GridAI || Victor Vaquero, IVEX.ai.
   • Difficulty: Easy

2. IDEA: Camera calibration module

   • Description: Create an API with low level and possibly high level routines to perform camera calibration. Currently, OpenCV is one of the few libraries where this kind of algorithms can be found and easily used. This module can be oriented in two directions: intrinsic or extrinsic calibration. The goal is to have at least one of those algorithms implemented in a differentiable fashion, running in GPU and providing an easy way to develop new differentiable methods based on them.
   • Expected Outcomes:
     • Regarding the intrinsic calibration:
       • Merge in Kornia one differentiable method to estimate the intrinsic and undistortion parameters.
     • Regarding the extrinsic calibration:
       • Merge in Kornia one differentiable method to estimate the camera position wrt a given reference object in terms of rotation and translation.
   • Resources:
     • Camera resectioning in Wikipedia.
       • Intrinsics:
         • Opencv tutorial about camera calibration.
         • Zhang's camera calibration method is one of the most well known and accurate camera calibration methods.
       • Extrinsics:
         • OpenCV tutorials (one, two) about pose estimation.
         • PnP OpenCV function shows some different methods that could be implemented. The most basic method iteratively estimates the camera/object pose using Levenberg-Marquardt optimization.
   • Skills Required: Good geometry and linear algebra basis. Experience using PyTorch.
   • Possible Mentors: Luis Ferraz (PhD, Kognia Sports), Francesc Moreno (PhD, IRI).
   • Difficulty: Medium/Hard

3. IDEA: Better ecosystem with modern Local Features

   • Description: Add modern local features, e.g. R2D2, SuperPoint, DELF, D2Net, etc to the kornia.feature. Currently, there is no way practitioners could easily add learned local features to their code without cloning authors' implementation separately and spending time on code integration. Moreover, the official implementations of R2D2, SuperPoint have a restricted license, prohibiting its commercial usage. The student's goal is to adopt or re-implement the local feature and possibly re-train it from scratch following the description in the official paper.
   • Expected Outcomes:
     • new modern local features with pre-trained weights are integrated into kornia.feature
     • re-implementation is evaluated on standard benchmarks and has similar performance to the official implementation
   • Resources:
     • From handcrafted to deep local features: survey
     • SuperPoint paper,
     • SuperGlue,
     • R2D2
     • KeyNet,
     • KeyNet tensorflow implementation
     • D2Net pytorch implementation.
     • kornia.feature module documentation
   • Skills Required: Experience with training deep neural networks in pytorch. Access to GPU (can be done with a Google Colab).
   • Possible Mentors: Dmytro Mishkin, CTU in Prague
   • Difficulty: Medium/Hard.

4. IDEA: SfM module

   • Description: Implement differentiable structure-from-motion frontends (triangulation, camera resectioning) and backends (bundle adjustment, motion averaging).
   • Expected Outcomes:
     • An epipolar submodule implementing two-view epipolar geometry functionality.
     • A frontend submodule that scales existing feature matching routines to large image collections.
     • A backend submodule implementing routines for nonlinear least squares optimization (bundle adjustment).
   • Possible Mentors: Krishna Murthy, Mila || Edgar Riba, GridAI.
   • Difficulty: Hard.

5. IDEA: Probabilistic testing framework

   • Description: Write probabilistic testing for non-deterministic augmentations. Monte Carlo based testing of functions allows for testing.
   • Expected Outcomes:
     • Testable Reproducibility of random methods.
     • Testing Reproducibility across devices.
   • Possible Mentors: Anguelos, Jian.
   • Difficulty: Medium.

How do I pass the GSoC evaluations ?

The program duration is ~2months; we will be flexible but there are some major RULES.

‼️ To PASS the mid-term and final evaluations you MUST have a MERGED pull-request ➡️ you'll get your cut 😎

• Google pays you IF ONLY IF you pass the evaluations
• The mentors will evaluate you based on the performance during the project.

Timeline

The Full Program Timeline: https://summerofcode.withgoogle.com/how-it-works/#timeline

• Organization Applications Open - [ January 29, 2021 ] 🎬 ✔️
• Organization Application Deadline - [ February 19, 2021 ] 📅
• Organizations Announced - [ March 9, 2021 ] 🙏 -> WE ARE OUT :(
• Student Application Period - [ March 29, 2021 - April 13, 2021 ] 🧑‍🎓
• Application Review Period - [ April 13, 2021 - May 17, 2021 ] 📅
• Student Projects Announced - [ May 17, 2021 ] 📢
• Community Bonding - [ May 17, 2021 - June 7, 2021 ] 👥
• Coding - [ June 7, 2021 - August 16, 2021 ] 👩‍💻
• Evaluations - [ July 12 - 16, 2021 ]
• Students Submit Code and Final Evaluations [ August 16 - 23, 2021 ]
• Mentors Submit Final Evaluations [ August 23 - 30, 2021 ]
• Results Announced [ August 31, 2021 ] 📊