Project List endpoint returns member email for some projects

[maxceem]

When we call the endpoint to get a list of projects, the endpoint shouldn't return email field for the members, but it does return them for some projects:

The URL is GET https://api.topcoder-dev.com/v5/projects/?fields=id,name,description,members,status,type,actualPrice,estimatedPrice,createdAt,updatedAt,createdBy,updatedBy,details,lastActivityAt,lastActivityUserId,version,templateId&sort=lastActivityAt+desc&perPage=20&page=1

User which is used to reproduce it is pshah_manager.

Project id on DEV: 8724.

There two possible desired behaviors:

• don't return emails for members to any users
• return emails for members, but only to admins (with administrator Topcoder role only, NOT to Connect Admin)

@vikasrohit which desired behavior you prefer?

[vikasrohit]

our desired behavior is return emails for members, but only to admins

[maxceem]

Issue background:

• when we add project members in ElasticSearch index we additionally populate them with user details like handle, firstName, lastName and email (see code). But our intention is only to use this additional data for searching purposes, so we can search projects in ES using members handle, name and email

• when getting the list of projects we explicitly define the list of fields to retrieve from ES index, and it doesn't contain handle, firstName, lastName and email for project members: https://github.com/topcoder-platform/tc-project-service/blob/develop/src/routes/projects/list.js#L544 but it looks like this list of field is ignored by ES

• We can manually define the list of fields as a query param to restrict the list of fields we want to get for projects, and it looks like it is complitely ignored at the moment, not only for email but for other fields also. We define the list of fields like this: fields=id,name,description,members,status,type,actualPrice,estimatedPrice,createdAt,updatedAt,createdBy,updatedBy,details,lastActivityAt,lastActivityUserId,version,templateId but we get all the fields:

  

TO DO

• We have to fix this issue in general
  1. we should be able to define the list of fields to return (note, that we can also define the list of fields for nested arrays, like for array of members), see how the method to parse fields works https://github.com/topcoder-platform/tc-project-service/blob/develop/src/util.js#L228-L247
  2. at the moment we have some predefined allowed list of fields (see code), I guess it is already works, but just in case we should make sure, that we can only list fields which are allowed, and not allowed fields would be ignored.
  3. additionally support fields firstName, lastName and handle to be returned for members (we return them now, but they should be only returned if we explicitly list them, but currently they are not in the available list)

Before making fixes make sure that can reproduce all of the issues

• As it's very important, we have to reproduce the issue with returning email locally: setup Project Service using Full docker-compose with ES index, import demo data and create a new project with a member, make sure that email is added to ES and make sure it's returned by the Project Service
• Create a unit test: should not return "email" for project members when "fields" query param is not defined (to non-admin users) (make sure it fails as email suppose to be returned by the current logic)
• Create a unit test: should not return "email" for project members even if it's defined in "fields" query param (to non-admin users) - also should fail
• Create a unit test: should not return "cancelReason" if it is not listed in "fields" query param (make sure it fails as currently, looks like logic is broken)

Create screenshots showing that all 4 issues above are reproduced.

For the next unit tests don't have to create screenshots, just implement them:

• Create a unit test: should not return "email" for project members when "fields" query param is not defined (to admin users) - also should fail
• Create a unit test: should return "email" for project members if it's defined in "fields" query param (to admin users) - it supposes to work as email is now returned to everyone

[maxceem]

@vikasrohit this issue has been fixed and HOTFIX PRs are created.

There are PRs to merge to both Project Service and Connect App:

• Project Service [HOTFIX] [PROD] Post release 2.1.1 #495
• Connect App [HOTIFX] [PROD] Post release 2.7.1 appirio-tech/connect-app#3729

The issues was that LIST and GET project endpoints didn't respect fields query param. So it has been fixed in general.

DEV patch has been already appliad to Connect App, as it's not-breaking for Connect App.
Here is a DEV HOTFIX for Project Service #494

[lakshmiathreya]

@drasticdpk