Skip to content

Add the new aggregate pipeline that group all the sections by the combination of course and professor #82

Description

@mikehquan19

Currently to address Trend's need, we've written 2 pipelines in /uploader/pipelines/trends_prof_sections & /uploader/pipelines/trends_course_sections. What do they do?

They insert the aggregated sections data directly to the database, so that Trends can query the sections without exceeding the number of API calls imposed by Vercel.

TrendsProfSectionsPipeline will aggregate data into format:

profFirst + profLast: {
   Section, 
   Section, 
   ...
}

TrendsCourseSectionsPipeline will aggregate data into format:

prefix + number: {
   Section, 
   Section, 
   ...
}

Now, in order to improve their search quality even further, I'm proposing another pipeline that can aggregate data into this format:

prefix + number + profFirst + profLast: {
   Section, 
   Section, 
   ...
}

Metadata

Metadata

Labels

Help WantedExtra attention is neededL2A task suitable for someone who is comfortable helping with implementing features.Type: Feature RequestNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions