feat: script to clean up backend emails #1612

freaky4wrld · 2024-03-13T05:28:27Z

What changes did you make and why did you make them ?

Added scripts folder, and created a emailCleaner script to convert all the uppercase emails to lowercase
Identified the conflicts and merged them as told
This only works on the test-data.json also the script is saved in .mjs format to run it on node
test-data.json was the file I didn't included in the PR it consists of all the users information and can be created by making a GET request to http://localhost:4000/api/users/ with ThunderClient, then copying the contents to the a new file in the newly created scripts folder

Identified Uppercase emails

Uppercase emails

 [
  {
    name: { firstName: 'Dae', lastName: 'Kim' },
    accessLevel: 'user',
    skillsToMatch: [],
    projects: [],
    textingOk: false,
    managedProjects: [],
    isActive: true,
    _id: '5f28c25206f21000177e690a',
    email: 'Dannydaekim@gmail.com',
    currentRole: 'Software Developer',
    desiredRole: 'Frontend Developer',
    newMember: true,
    firstAttended: 'AUG 2020',
    createdDate: '2020-08-04T02:05:06.309Z',
    __v: 0
  },
  {
    name: { firstName: 'Scott', lastName: 'Larsen' },
    accessLevel: 'user',
    skillsToMatch: [],
    projects: [],
    textingOk: false,
    managedProjects: [],
    isActive: true,
    _id: '5e435ce7ef67e100175c1ee3',
    email: 'Scott@ScottLarsen.com',
    currentRole: 'Student',
    desiredRole: 'Python Developer',
    newMember: true,
    firstAttended: 'JAN 2020',
    createdDate: '2020-02-12T02:03:19.149Z',
    __v: 0,
    attendanceReason: 'Homelessness',
    currentProject: 'Food Oasis'
  },
  {
    name: { firstName: 'sflivnaldf', lastName: 'afbka' },
    accessLevel: 'user',
    skillsToMatch: [],
    projects: [],
    textingOk: false,
    managedProjects: [],
    isActive: true,
    _id: '6101db234815993498437084',
    email: 'tyWE@sharklasers.com',
    currentRole: 'af',
    desiredRole: 'argfg',
    newMember: true,
    firstAttended: 'JUL 2021',
    createdDate: '2021-07-28T22:33:07.120Z',
    __v: 0
  },
  {
    name: { firstName: 'JASON', lastName: 'YANG' },
    accessLevel: 'user',
    skillsToMatch: [],
    projects: [],
    textingOk: false,
    managedProjects: [],
    isActive: true,
    _id: '5e38d1568d52770017ae8a86',
    email: 'JASC68.JYANG@GMAIL.COM',
    currentRole: 'DATA SCIENTIST',
    desiredRole: 'DATA SCIENTIST',
    newMember: false,
    firstAttended: 'SEP 2019',
    createdDate: '2020-02-04T02:05:10.999Z',
    __v: 0
  },
  {
    name: { firstName: 'Jonathon', lastName: 'Dooley' },
    accessLevel: 'user',
    skillsToMatch: [],
    projects: [],
    textingOk: false,
    managedProjects: [],
    isActive: true,
    _id: '5f0540b24f31cf00174d7538',
    email: 'DooleyJM0902@student.laccd.edu',
    currentRole: 'Student',
    desiredRole: 'Developer',
    newMember: false,
    firstAttended: 'JUL 2020',
    createdDate: '2020-07-08T03:42:42.621Z',
    __v: 0
  },
  {
    name: { firstName: 'Julia', lastName: 'Fong' },
    accessLevel: 'user',
    skillsToMatch: [],
    projects: [],
    textingOk: false,
    managedProjects: [ '5ec326c7a973810017c0de0c', '619af6c86a8afa609cd5c419' ],
    isActive: true,
    _id: '5e435911ef67e100175c1ecb',
    email: 'Juliamzfong@gmail.com',
    currentRole: 'fellow',
    desiredRole: 'front-end developer',
    newMember: true,
    firstAttended: 'JAN 2020',
    createdDate: '2020-02-12T01:46:57.788Z',
    __v: 0
  },
  {
    name: { firstName: 'RYAN', lastName: 'COLLINS' },
    accessLevel: 'user',
    skillsToMatch: [],
    projects: [],
    textingOk: false,
    managedProjects: [],
    isActive: true,
    _id: '5e38d10a8d52770017ae8a81',
    email: 'RMCOLLINS95@GMAIL.COM',
    currentRole: 'NONE',
    desiredRole: 'DATA SCIENTIST',
    newMember: true,
    firstAttended: 'JAN 2020',
    createdDate: '2020-02-04T02:03:54.970Z',
    __v: 0
  },
  {
    name: { firstName: 'Chris', lastName: 'Schmitz' },
    accessLevel: 'user',
    skillsToMatch: [],
    projects: [],
    textingOk: false,
    managedProjects: [],
    isActive: true,
    _id: '5e30ee120b9d2300177d3b38',
    email: 'Christopher.A.Schmitz@gmail.com',
    currentRole: 'Mobile Engineering Manager',
    desiredRole: 'Mobile Engineering Manager',
    newMember: false,
    firstAttended: 'MAY 2017',
    createdDate: '2020-01-29T02:29:38.664Z',
    __v: 0,
    attendanceReason: 'Civic Engagement',
    currentProject: 'None'
  },
  {
    name: { firstName: 'akib', lastName: 'mohaimenur rahman' },
    accessLevel: 'user',
    skillsToMatch: [],
    projects: [],
    textingOk: false,
    managedProjects: [],
    isActive: true,
    _id: '60d6a4e82e675e4a90e9ca92',
    email: 'akib.rhASast@gmail.com',
    currentRole: 'asdb',
    desiredRole: 'asd',
    newMember: true,
    firstAttended: 'JUN 2021',
    createdDate: '2021-06-26T03:54:16.188Z',
    __v: 0
  },
  {
    name: { firstName: 'test', lastName: 'test' },
    accessLevel: 'user',
    skillsToMatch: [],
    projects: [],
    textingOk: false,
    managedProjects: [],
    isActive: true,
    _id: '6101d94c4815993498437083',
    email: 'deDRE@sharklasers.com',
    currentRole: 's',
    desiredRole: 's1',
    newMember: true,
    firstAttended: 'JUL 2021',
    createdDate: '2021-07-28T22:25:16.037Z',
    __v: 0
  },
  {
    name: { firstName: 'DAVID', lastName: 'PEASE' },
    accessLevel: 'user',
    skillsToMatch: [],
    projects: [],
    textingOk: false,
    managedProjects: [],
    isActive: true,
    _id: '5e1d2283316d2f00172ef05a',
    email: 'DPEASE@DATAMINR.COM',
    currentRole: 'DIRECTOR OF CUSTOMER SUCCESS',
    desiredRole: 'Project Manager',
    newMember: true,
    firstAttended: 'JAN 2020',
    createdDate: '2020-01-14T02:08:03.445Z',
    __v: 0
  },
  {
    name: { firstName: 'ELI', lastName: 'SELKIN' },
    accessLevel: 'user',
    skillsToMatch: [],
    projects: [],
    textingOk: false,
    managedProjects: [],
    isActive: true,
    _id: '5e7965101e29ad00179399bb',
    email: 'ELI.J.SELKIN@GMAIL.COM',
    currentRole: 'ENGAGE LEAD',
    desiredRole: 'ENGAGE LEAD',
    newMember: false,
    firstAttended: 'MAR 2017',
    createdDate: '2020-03-24T01:40:32.576Z',
    __v: 0
  },
  {
    name: { firstName: 'Trillium', lastName: 'Smith' },
    accessLevel: 'admin',
    skillsToMatch: [],
    projects: [],
    textingOk: false,
    managedProjects: [ '6407a1a8f97d2497a9f09dfa', '640500c62f66fc21b41289a5' ],
    isActive: false,
    _id: '5e965e554e2fc70017aa3970',
    email: 'Trillium@hatsfabulous.com',
    currentRole: 'Student',
    desiredRole: 'Full stack or front end development',
    newMember: true,
    firstAttended: 'APR 2020',
    createdDate: '2020-04-15T01:07:33.445Z',
    __v: 0
  },
  {
    name: { firstName: 'Kevin', lastName: 'Howley' },
    accessLevel: 'user',
    skillsToMatch: [],
    projects: [],
    textingOk: false,
    managedProjects: [],
    isActive: true,
    _id: '5e4c9b59b73a2a001732f487',
    email: 'Kphowley@gmail.com',
    currentRole: 'Project Manager',
    desiredRole: 'Project Manager',
    newMember: false,
    firstAttended: 'APR 2019',
    createdDate: '2020-02-19T02:20:09.712Z',
    __v: 0,
    attendanceReason: 'Transportation',
    currentProject: 'TDM Calculator'
  },
  {
    name: { firstName: 'Adam', lastName: 'Kendis' },
    accessLevel: 'user',
    skillsToMatch: [],
    projects: [],
    textingOk: false,
    managedProjects: [],
    isActive: true,
    _id: '5e27b1e54530cd0017eee431',
    email: 'ATKendis@gmail.com',
    currentRole: 'Front End Engineer',
    desiredRole: 'Front End Engineer',
    newMember: true,
    firstAttended: 'JAN 2020',
    createdDate: '2020-01-22T02:22:29.017Z',
    __v: 0
  },
  {
    name: { firstName: 'Jabari', lastName: 'Brown' },
    accessLevel: 'user',
    skillsToMatch: [],
    projects: [],
    textingOk: false,
    managedProjects: [],
    isActive: true,
    _id: '5e27abf74530cd0017eee417',
    email: 'Jbarib@gmail.com',
    currentRole: 'Project Lead',
    desiredRole: 'Project Lead',
    newMember: false,
    firstAttended: 'NOV 2019',
    createdDate: '2020-01-22T01:57:11.736Z',
    __v: 0
  },
  {
    name: { firstName: 'Aaron', lastName: 'Anderson' },
    accessLevel: 'user',
    skillsToMatch: [],
    projects: [],
    textingOk: false,
    managedProjects: [],
    isActive: false,
    _id: '5e4c995eb73a2a001732f47e',
    email: 'Acanderson358@gmail.com',
    currentRole: 'backhand',
    desiredRole: 'backhand',
    newMember: true,
    firstAttended: 'JAN 2020',
    createdDate: '2020-02-19T02:11:42.827Z',
    __v: 0
  }
]

github-actions · 2024-03-13T05:28:57Z

Want to review this pull request? Take a look at this documentation for a step by step guide!

From your project repository, check out a new branch and test the changes.

git checkout -b freaky4wrld-clean-backend-emails-1551 development
git pull https://github.com/freaky4wrld/VRMS.git clean-backend-emails-1551

freaky4wrld · 2024-03-13T05:31:32Z

@spiteless review the PR it only works on the test-data.json, provide your feedback and resources to execute the same in the database rather than the json file, while executing the script in .js format it was suggesting to convert to .mjs don't know why that happened.

jbubar

Nayan, we have one thing to consider. the User object is linked to checkin objects. and we want to make sure that if we delete a user than the userId in the checkin object is replaced also.. does that make sense? feel free to slack me if you want me to explain further

freaky4wrld · 2024-04-18T09:39:56Z

Can you give me some hints?? Or point me to some resource... I don't fully understand it!!

jbubar · 2024-05-07T01:19:38Z

I am so sorry for the crappy review above. the fear is that we may have an orphaned record, or zombie data.

definition of orphaned data

"Orphaned data refers to data that is no longer associated with a corresponding record or entity in a database, data storage or other information system. This situation typically arises when a record, file, or object is deleted, but the associated data remains in the system without a proper link to a parent entity. "

The checkin data references the users. and maybe we want to consolidate those checkins the same way you consolidated the rest of the data. We will bring this up in the meeting today and give you a better review.

Again, apologies about the review above

jbubar

Can you create a test-data.json file with dummy data in it?

freaky4wrld · 2024-05-11T03:42:32Z

Can you create a test-data.json file with dummy data in it?

@jbubar I've added the test-data.json file as required. Review the changes when you are free

JackHaeg · 2024-06-11T01:59:39Z

@jbubar Can you please review this PR when you have a chance?

JackHaeg · 2024-08-06T04:31:51Z

@jbubar Just checking in on this PR - do you think you will have a chance to review soon? If not, we can request a review from @trilliumsmith

jbubar · 2024-09-17T02:28:08Z

Nayan is no longer working on this PR. @ntrehan this might be interesting to read for you

freaky4wrld requested a review from trillium March 13, 2024 05:28

freaky4wrld changed the title ~~feat: added scripts to backend folder with json data~~ feat: script to clean up backend emails Mar 13, 2024

feat: added scripts to backend folder with json data

9c349e8

freaky4wrld force-pushed the clean-backend-emails-1551 branch from 4efedbd to 9c349e8 Compare March 13, 2024 16:44

JackHaeg mentioned this pull request Mar 26, 2024

VRMS: ALL: Monday Meeting Agenda 📝 #1280

Open

jbubar requested changes Apr 16, 2024

View reviewed changes

jbubar requested changes May 7, 2024

View reviewed changes

feat: added test-data.json file

fd33817

freaky4wrld requested a review from jbubar May 11, 2024 03:42

jbubar closed this Sep 17, 2024

JackHaeg mentioned this pull request Sep 26, 2024

Write a script for cleaning up backend emails #1551

Open

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: script to clean up backend emails #1612

feat: script to clean up backend emails #1612

freaky4wrld commented Mar 13, 2024 •

edited by jbubar

Loading

github-actions bot commented Mar 13, 2024

freaky4wrld commented Mar 13, 2024

jbubar left a comment

freaky4wrld commented Apr 18, 2024

jbubar commented May 7, 2024 •

edited

Loading

jbubar left a comment

freaky4wrld commented May 11, 2024

JackHaeg commented Jun 11, 2024

JackHaeg commented Aug 6, 2024

jbubar commented Sep 17, 2024

feat: script to clean up backend emails #1612

feat: script to clean up backend emails #1612

Conversation

freaky4wrld commented Mar 13, 2024 • edited by jbubar Loading

What changes did you make and why did you make them ?

Identified Uppercase emails

github-actions bot commented Mar 13, 2024

freaky4wrld commented Mar 13, 2024

jbubar left a comment

Choose a reason for hiding this comment

freaky4wrld commented Apr 18, 2024

jbubar commented May 7, 2024 • edited Loading

jbubar left a comment

Choose a reason for hiding this comment

freaky4wrld commented May 11, 2024

JackHaeg commented Jun 11, 2024

JackHaeg commented Aug 6, 2024

jbubar commented Sep 17, 2024

freaky4wrld commented Mar 13, 2024 •

edited by jbubar

Loading

jbubar commented May 7, 2024 •

edited

Loading