Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
3d90b93
Fixed get_key docstring.
TryExceptElse Jun 1, 2018
e3d141c
Minor cleanup in server.py
TryExceptElse Jun 2, 2018
3ea2e63
Began implementing DataModel as a distinct class.
TryExceptElse Jun 2, 2018
4604b5d
Added initial tests for DataModel and reorganized test data.
TryExceptElse Jun 2, 2018
87301d5
Began implementing Departments + assorted fixes.
TryExceptElse Jun 3, 2018
6928cb5
Fixed owl infestation.
TryExceptElse Jun 3, 2018
3d329df
Added SectionQuarterView and other fixes.
TryExceptElse Jun 3, 2018
537e5e5
Implemented section properties and associated test methods.
TryExceptElse Jun 3, 2018
bde3669
Updated gitignore to ignore generated test files.
TryExceptElse Jun 3, 2018
9a2a529
Added more documentation to model.py
TryExceptElse Jun 3, 2018
d7d64de
Fixed odd occasional test issue by using absolute import.
TryExceptElse Jun 3, 2018
652f550
Minor debug message improvements, and minor additions.
TryExceptElse Jun 3, 2018
67d9412
Merge remote-tracking branch 'origin/data_model' into data_model
TryExceptElse Jun 3, 2018
ddbe327
Changed import order in model.py to be pep-8 compliant.
TryExceptElse Jun 3, 2018
f688570
Mending more minor model mistakes.
TryExceptElse Jun 4, 2018
d8c97ea
Began implementing request.py; handles validation + feedback of reque…
TryExceptElse Jun 4, 2018
b195404
Implemented Request class and associated classes and tests.
TryExceptElse Jun 6, 2018
f733a5d
Added more request tests + made additional bugfixes.
TryExceptElse Jun 6, 2018
33ab187
Added docstrings to Issue methods.
TryExceptElse Jun 6, 2018
42001fc
Added custom validators for Request.Fields
TryExceptElse Jun 7, 2018
ace3730
Minor request.py changes.
TryExceptElse Jun 7, 2018
7a52913
Requests now can store sub-requests.
TryExceptElse Jun 7, 2018
46df6ef
Added some more annotations to help pycharm's type annotation detection.
TryExceptElse Jun 10, 2018
aa3ce74
Condensed generate_test_data; slightly more readable now?
TryExceptElse Jun 10, 2018
388fdcd
Merge remote-tracking branch 'origin/data_model' into data_model
TryExceptElse Jun 10, 2018
0c1681d
Merge remote-tracking branch 'origin/data_model' into data_model
TryExceptElse Jun 10, 2018
2a195a0
Modified ClassDuration to store maya interval.
TryExceptElse Jun 10, 2018
b7347a1
Implemented input classes and schema.
TryExceptElse Jun 10, 2018
3525416
Implemented section filter.
TryExceptElse Jun 10, 2018
679d49f
Implemented data model Accessor class.
TryExceptElse Jun 10, 2018
63bf8d3
Outdated classes now skipped (should be changed/removed in the near f…
TryExceptElse Jun 10, 2018
52057d7
Removed tests for functions no longer implemented in server.py
TryExceptElse Jun 11, 2018
b299d18
Added test_access.py, reimplementing some tests from test_server.py
TryExceptElse Jun 11, 2018
0f784d9
Update to model and model accessor.
TryExceptElse Jun 11, 2018
c5ea03b
Refactoring mostly done. (bugs probably exist at this time)
TryExceptElse Jun 11, 2018
c2b2043
Added input tests, corrected various issues.
TryExceptElse Jun 11, 2018
7721458
Removed unused generate_url method.
TryExceptElse Jun 11, 2018
09cee4f
Cleanup of server.py imports and constants.
TryExceptElse Jun 11, 2018
d5a20f3
Code cleanup
TryExceptElse Jun 11, 2018
c94645f
added flask-inputs to pipfile
TryExceptElse Jun 11, 2018
0d4d3a9
Added jsonschema to dependencies.
TryExceptElse Jun 11, 2018
19c44f8
Removed unused test_server.py (may be restored in the future)
TryExceptElse Jun 11, 2018
ecc724f
Re-ordered imports in input.py to be pep-8 compliant.
TryExceptElse Jun 11, 2018
598e1e6
DataModel now accepts files with name form '123456_database.json'
TryExceptElse Jun 11, 2018
e00c23e
Fixed quarter db file path generation.
TryExceptElse Jun 11, 2018
69a6c22
Added some documentation.
TryExceptElse Jun 11, 2018
1932bdc
Added more documentation to access.py methods.
TryExceptElse Jun 11, 2018
fb2a5c8
Minor code cleanup and doc changes.
TryExceptElse Jun 11, 2018
8493396
Fixed awkward wording in DataError docstring.
TryExceptElse Jun 12, 2018
3d790b9
Added instructor and conflict filters to SectionFilter
TryExceptElse Jun 13, 2018
d1917c8
Added input tests and fixed schema + removed placeholders.
TryExceptElse Jun 13, 2018
dcfea37
completed unfinished get_urls method + supporting methods.
TryExceptElse Jun 13, 2018
aa373d1
Refactored model tests to use data in temporary directories.
TryExceptElse Jun 13, 2018
3714f29
Merge branch 'new_filters' into data_model
TryExceptElse Jun 13, 2018
3748667
Refactored Accessor tests to use temporary data directories.
TryExceptElse Jun 13, 2018
8ba767d
Began implementing quarter method caching.
TryExceptElse Jun 14, 2018
9de217b
Added documentation + minor fixes.
TryExceptElse Jun 14, 2018
6ecc9c6
Renamed NotRequest to PseudoRequest
TryExceptElse Jun 14, 2018
f322503
Added intersection tests for ClassDuration
TryExceptElse Jun 14, 2018
9e8a6c2
Added serialization module and quarter duration property.
TryExceptElse Jun 14, 2018
386de17
Improved deserialization hook.
TryExceptElse Jun 14, 2018
38a46fe
Added documentation for encoder default method.
TryExceptElse Jun 14, 2018
9cb81a0
Test speed of and ability to cache custom serializable objects.
TryExceptElse Jun 14, 2018
0602e48
Added documentation for _get_type_key in serial.py
TryExceptElse Jun 14, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,6 @@ frontend/static/README.md
.pytest_cache/
tests/test_db/data.py
!test_database.json

# Generated test files
test_resources/data_model_test_dir*
2 changes: 2 additions & 0 deletions Pipfile
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ maya = "*"
pytest = "*"
flask = "*"
pylint = "*"
flask-inputs = "*"
jsonschema = "*"


[dev-packages]
Expand Down
Empty file added owl/__init__.py
Empty file.
231 changes: 231 additions & 0 deletions owl/access.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,231 @@
"""
Intermediary module facilitating access of model.

Methods contained are distinct from those in owl.model because they
are intended primarily for the convenience of the user facing module
(server.py) and contain methods and classes that may result in user-
feedback (eg: AccessException's user_msg and similar).

Methods and classes are separate from server.py because they deal
primarily with access and retrieval of data, and not the form of the
response to the api user's request.
"""
import typing as ty

import owl.model
import owl.filter


# Access Argument keywords
ANY = 'any'
ALL = 'all'
LATEST = 'latest'


class AccessException(ValueError):
"""
Exception raised when requested data cannot be accessed, but not as
result of an internal error or bug, or badly a badly formed
request. This usually is because the requested data does not exist.
"""
def __init__(self, *args, user_msg='Bad Request'):
super().__init__(*args)
self.user_msg = user_msg


class ModelAccessor:
"""
Handles access of data from a specific data model.
"""
def __init__(self, model: owl.model.DataModel) -> None:
self.model = model

# This method may be removed in the future if the get_one
# method path is removed.
def get_one(
self,
school: str,
department: str,
course: str = ALL,
quarter: str = LATEST,
section_filter: owl.filter.SectionFilter = None
) -> ty.Union[owl.model.COURSE_DATA_T, owl.model.DEPT_DATA_T]:
"""
Retrieves data for a department or course, as identified by
the api user.
If no course is passed, all of the specified department's data
is returned.
:param school: str (ex: 'fh')
:param department: str (ex: 'CS')
:param course: str (ex: 1A) (optional)
:param quarter: str (ex: '201812')
:param section_filter: filter kwargs
:return: COURSE_DATA_T or DEPT_DATA_T
"""
if course == ALL:
return self.get_department_data(
school, department, quarter, section_filter)
return self.get_course_data(
school, department, course, quarter, section_filter)

def get_department_data(
self,
school: str,
department: str,
quarter: str = LATEST,
section_filter: owl.filter.SectionFilter = None
) -> owl.model.DEPT_DATA_T:
"""
Gets data for a specific department in a single quarter.
:param school: str (ex: 'fh')
:param department: str (ex: 'CS')
:param quarter: str (ex: '201812')
:param section_filter: filter kwargs
:return: DEPT_DATA_T
"""
# Find department view
department_view = self.get_department(school, department, quarter)

# get data from department view
return {
course_view.name: {
section_view.crn: section_view.data for
section_view in course_view.sections if
not section_filter or section_filter.check(section_view)
}
for course_view in department_view.courses
}

def get_course_data(
self,
school: str,
department: str,
course: str,
quarter: str = LATEST,
section_filter: owl.filter.SectionFilter = None
) -> owl.model.COURSE_DATA_T:
"""
Gets data for a specific course in a specific quarter.
:param school: str (ex: 'fh')
:param department: str (ex: 'CS')
:param course: str (ex: '1A')
:param quarter: str (ex: '201812')
:param section_filter: filter kwargs
:return: COURSE_DATA_T
"""
# Find department view
course_view = self.get_course(school, department, course, quarter)
return {
section_view.crn: section_view.data for
section_view in course_view.sections if
not section_filter or section_filter.check(section_view)
}

def get_quarter(
self,
school: str,
quarter: str = LATEST,
) -> 'owl.model.QuarterView':
"""
Gets QuarterView, throwing readable access errors in the event
that a passed value is not found.
:param school: str (ex: 'fh')
:param quarter: str (ex: '201812')
:return: QuarterView
"""
try:
school_view: owl.model.SchoolView = \
self.model.schools[school.upper()]
except KeyError as e:
raise AccessException(
f'No school found with identifier: {school}.') from e
try:
if quarter == LATEST:
quarter_view = school_view.latest_quarter
else:
quarter_view = school_view.quarters[quarter]
except KeyError as e:
raise AccessException(
f'No quarter in {school} with name: {quarter}.') from e
return quarter_view

def get_department(
self,
school: str,
department: str,
quarter: str = LATEST,
) -> owl.model.DepartmentQuarterView:
"""
Gets department view of department data for a specific quarter.
:param school: str (ex: 'fh')
:param department: str (ex: 'CS')
:param quarter: str (ex: '201812')
:return: DepartmentQuarterView
"""
quarter_view = self.get_quarter(school, quarter)
try:
department_view = quarter_view.departments[department]
except KeyError as e:
raise AccessException(
f'No department in {quarter} with name: {department}.') from e
return department_view

def get_course(
self,
school: str,
department: str,
course: str,
quarter: str = LATEST,
) -> owl.model.CourseQuarterView:
"""
Gets view of data for a specific course in a specific quarter.
:param school: str (ex: 'fh')
:param department: str (ex: 'CS')
:param course: str (ex: '1A')
:param quarter: str (ex: '201812')
:return: CourseQuarterView
"""
department_view = self.get_department(school, department, quarter)
try:
course_view = department_view.courses[course]
except KeyError as e:
raise AccessException(
f'No course in {department} with name: {course}.') from e
return course_view

def get_section(
self,
school: str,
department: str,
course: str,
section: str,
quarter: str = LATEST,
) -> owl.model.SectionQuarterView:
"""
Gets view of data for a specific section in a specific quarter.
:param school: str (ex: 'fh')
:param department: str (ex: 'CS')
:param course: str (ex: '1A')
:param quarter: str (ex: '201812')
:param section: str (ex: '12345')
:return: SectionQuarterView
"""
course_view = self.get_course(school, department, course, quarter)
try:
section_view = course_view.sections[section]
except KeyError as e:
raise AccessException(
f'No section in {course} with name: {section}.') from e
return section_view

def get_urls(
self, school: str, quarter: str = LATEST
) -> ty.Dict[str, ty.Dict[str, str]]:
"""
Helps list all courses, their names and departments.
:param school: str (ex: 'fh')
:param quarter: str (ex: '201812')
:return: Dict[str, Dict[str, str]]
"""
quarter_view = self.get_quarter(school, quarter)
return quarter_view.urls
109 changes: 109 additions & 0 deletions owl/filter.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
"""
Contains classes and methods for filtering data.
"""

import typing as ty
import maya

import owl.model

# Keys
MONDAY_KEY = 'M'
TUESDAY_KEY = 'T'
WEDNESDAY_KEY = 'W'
THURSDAY_KEY = 'Th'
FRIDAY_KEY = 'F'
SATURDAY_KEY = 'S'
SUNDAY_KEY = 'U'

# Filter Argument keywords
ANY = 'any'
ALL = 'all'


class SectionFilter:
"""
Filters courses based on passed parameters.
"""
def __init__(
self,
status: ty.Dict[str, int] = None,
types: ty.Dict[str, int] = None,
days: ty.Dict[str, int] = None,
time: ty.Dict[str, str] = None,
instructors: ty.Dict[str, int] = None,
conflict_sections: ty.Set[owl.model.SectionQuarterView] = None
):
self.status = status
self.types = types
self.days = days
self.time = time
self.instructors = instructors
self.conflict_sections = conflict_sections

def check(self, section: owl.model.SectionQuarterView) -> bool:
"""
Determines whether passed course passes filter
:param section: owl.model.SectionQuarterView
:return: bool (True if section passes filters)
"""

# Nested functions filter courses by taking a course key and
# returning a boolean indicating whether they should be included
# or excluded. True if they are to be included, False if excluded.

def status_filter() -> bool:
# {'open':0, 'waitlist':0, 'full':0}
if not self.status:
return True
# Create 'mask' of course statuses that are to be included.
status_mask = {k for (k, v) in self.status.items() if v}
# Return True only if course status is in mask.
return section.status in status_mask

def type_filter() -> bool:
# {'standard':1, 'online':1, 'hybrid':0}
if not self.types:
return True
# Get course section
mask = {k for (k, v) in self.types.items() if v}
return section.section_type in mask

def day_filter() -> bool:
# {'M':1, 'T':0, 'W':1, 'Th':0, 'F':0, 'S':0, 'U':0}
if not self.days:
return True
# create set of days that are allowed by passed filters
mask = {k for (k, v) in self.days.items() if v}
return section.days <= mask

def time_filter() -> bool:
# {'start':'8:30 AM', 'end':'9:40 PM'}
if not self.time:
return True
filter_range = maya.MayaInterval(
start=maya.when(self.time['start']),
end=maya.when(self.time['end']))
for duration in section.durations:
if not filter_range.contains(duration.interval):
return False
return True

def instructor_filter() -> bool:
if not self.instructors:
return True
return section.instructor_name in \
{k for k, v in self.instructors.items() if v}

def conflict_filter() -> bool:
if not self.conflict_sections:
return True
for conflict_section in self.conflict_sections:
if section.conflicts(conflict_section):
return False
return True

return all((
status_filter(), type_filter(), day_filter(), time_filter(),
instructor_filter(), conflict_filter()
))
Loading