Add similar projects at the bottom of the project detail page. #5632

Aadesh-Baral · 2023-03-14T06:03:10Z

closes #5626
This PR adds similar projects at the bottom of the project detail page.

Implementation:

Similarity between the projects and given project will be calculated using cosine similarity and top 4 projects will be returned by default.
Similarity between projects is calculated using these fields. -> default_locale, difficulty, mapping_types, country, categories
All published projects except private ones are displayed on non logged in view while only projects in which user has permission to map are shown for a logged in user.
The function to create project matrix, which is the most significant part of calculating project similarity will be cached for a day.

How to test:

Install backend dependencies as new dependencies has been added in this PR.
Start frontend and backend locally.
Go to project detail page of any published projects
Check if Similar projects button is visible on the footer alongside question & comments, contributors ....
Click on similar projects button. You should be scrolled to Similar projects section.
On the similar projects section 4 other similar projects should be displayed.

ramyaragupathy · 2023-03-28T05:39:03Z

@Aadesh-Baral - can you also please add context on how the metadata fields were chosen to determine similarity between projects?

Aadesh-Baral · 2023-03-28T10:38:58Z

To find similar projects, we analyzed the correlation between different project metadata fields and iterated through them. For instance, we observed that organizations tend to create similar projects based on their country and category. However, considering the organization for the similarity score led to the display of projects from the same organization, which affected the diversity of our displayed projects. Hence, we excluded organization as a field. Furthermore, we found some fields that didn't significantly contribute to our results, so we discarded them to enhance performance.
Then the result was tested with some test users who found it to be satisfactory.

Aadesh-Baral · 2023-03-28T11:20:47Z

Adding @HelNershingThapa to review frontend changes and @d-rita to review backend changes.

HelNershingThapa

When you click the related project card, the page should scroll to the top so that users feel like they are viewing another project when they navigate to the related project. Use the useFetch hook to fetch linked projects as well, making the component look cleaner. Other than that, LGTM, I've pushed with the requested changes.

d-rita

@Aadesh-Baral will the limit of the projects returned always be 4? We have a limit parameter that we don't actually give the user a chance to use in the endpoint

backend/api/projects/resources.py

d-rita

Tested this, and it works as expected. The tests are good too. Overall, LGTM

Aadesh-Baral · 2023-04-11T06:11:38Z

Now the function to create project matrices is cached instead of the function that returns related projects. This change significantly improves our performance, as in cases where 'current_user' changes, the entire function would need to be re-run, causing a significant performance impact. By caching the 'create_project_matrix' function, we can optimize performance, as this function is a major component of calculating related projects and needs recalculation very rarely.

ramyaragupathy

Tested the basic functionality with @d-rita. Need to dig some more for edge cases @Aadesh-Baral

ramyaragupathy

Looks good to me @Aadesh-Baral

…ects

It is preferable to filter projects by completion when creating results rather than when creating project matrixes so that users can view related projects of completed projects and contribute to other projects that are similar to them.

Cache the function to generate project matrices instead of the function that returns related projects. This change significantly improves our performance, as in cases where 'current_user' changes, the entire function would need to be re-run, causing a significant performance impact. By caching the 'create_project_matrix' function, we can optimize performance, as this function is a major component of calculating related projects and needs recalculation very rarely.

When an error occurred while fetching related projects, add the message 'could not find related projects for this project.' This was done to ensure that when an error occurs in related projects, the entire project detail page is not impacted.

- Updated test_create_project_matrix_returns_project_matrix adding project_id on create_project_matrix to update cache - Mock get_similar_project_ids on test_get_related_projects_returns_related_projects - Remove test_only_non_completed_projects_are_returned as we no longer filter project by completion while fetching published projects see 178fdf1

--------------------------------------------------------------------- Since we are calculating project similarity rather than their relation, this change is done to align the wording with the logic that has been applied.

sonarcloud · 2023-06-01T10:26:20Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

github-actions bot added scope: backend scope: frontend labels Mar 14, 2023

Aadesh-Baral force-pushed the feature/5626-related_projects branch 4 times, most recently from 683ef26 to 31e1949 Compare March 22, 2023 08:45

github-actions bot added the scope: translations label Mar 22, 2023

Aadesh-Baral force-pushed the feature/5626-related_projects branch from 31e1949 to f6bb0a9 Compare March 22, 2023 08:47

Aadesh-Baral marked this pull request as ready for review March 28, 2023 10:24

Aadesh-Baral requested review from HelNershingThapa, ramyaragupathy and d-rita March 28, 2023 10:39

HelNershingThapa suggested changes Mar 29, 2023

View reviewed changes

HelNershingThapa approved these changes Mar 29, 2023

View reviewed changes

d-rita reviewed Mar 30, 2023

View reviewed changes

backend/api/projects/resources.py Show resolved Hide resolved

d-rita approved these changes Mar 30, 2023

View reviewed changes

Aadesh-Baral force-pushed the feature/5626-related_projects branch 4 times, most recently from 8c12815 to 7616213 Compare April 12, 2023 10:16

ramyaragupathy mentioned this pull request Apr 19, 2023

Improve logic to recommending projects to new users #5648

Open

Aadesh-Baral assigned ramyaragupathy Apr 26, 2023

ramyaragupathy reviewed Apr 27, 2023

View reviewed changes

Aadesh-Baral force-pushed the feature/5626-related_projects branch from 956ac34 to 8fa8975 Compare May 8, 2023 05:27

ramyaragupathy approved these changes May 31, 2023

View reviewed changes

Aadesh-Baral changed the title ~~Add related projects at the bottom of the project detail page.~~ Add similar projects at the bottom of the project detail page. May 31, 2023

Aadesh-Baral and others added 19 commits June 1, 2023 14:25

Add basic implementation of logic and endpoint to return related proj…

2537e77

…ects

Add realted projects component in project detail view

3d01810

Add token and limit to request

95044d0

Improve logic to calculate related projects

fe8ce37

Add scikit-learn and pandas in requirements.txt for related projects

5e38546

Cache function to calculate related projects

1459edd

Add test cases for functions to calculate related projects

7baee01

Add tests for endpoint to fetch related projects

615777a

Add missing limit query param in swagger-docs

37c752e

Use useFetch for fetching related projects

993f55f

Scroll project detail page to top on mount

205389d

fixed formatting

ec143f3

Set mapping type to [] if not set while creating a projet matrix

7613d93

Reword related projects to similar projects.

30336f2

--------------------------------------------------------------------- Since we are calculating project similarity rather than their relation, this change is done to align the wording with the logic that has been applied.

Prettify frontend code.

fe0029c

Aadesh-Baral force-pushed the feature/5626-related_projects branch from 8fa8975 to fe0029c Compare June 1, 2023 10:23

Aadesh-Baral merged commit 9a260e8 into develop Jun 1, 2023
8 checks passed

Aadesh-Baral deleted the feature/5626-related_projects branch June 1, 2023 10:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add similar projects at the bottom of the project detail page. #5632

Add similar projects at the bottom of the project detail page. #5632

Aadesh-Baral commented Mar 14, 2023 •

edited

ramyaragupathy commented Mar 28, 2023

Aadesh-Baral commented Mar 28, 2023

Aadesh-Baral commented Mar 28, 2023

HelNershingThapa left a comment •

edited

d-rita left a comment

d-rita left a comment

Aadesh-Baral commented Apr 11, 2023

ramyaragupathy left a comment

ramyaragupathy left a comment

sonarcloud bot commented Jun 1, 2023

Add similar projects at the bottom of the project detail page. #5632

Add similar projects at the bottom of the project detail page. #5632

Conversation

Aadesh-Baral commented Mar 14, 2023 • edited

ramyaragupathy commented Mar 28, 2023

Aadesh-Baral commented Mar 28, 2023

Aadesh-Baral commented Mar 28, 2023

HelNershingThapa left a comment • edited

Choose a reason for hiding this comment

d-rita left a comment

Choose a reason for hiding this comment

d-rita left a comment

Choose a reason for hiding this comment

Aadesh-Baral commented Apr 11, 2023

ramyaragupathy left a comment

Choose a reason for hiding this comment

ramyaragupathy left a comment

Choose a reason for hiding this comment

sonarcloud bot commented Jun 1, 2023

Aadesh-Baral commented Mar 14, 2023 •

edited

HelNershingThapa left a comment •

edited