Skip to content

Releases: UCSD-Historical-Enrollment-Data/webreg_scraper

[0.5.1] Dependency Updates & Minor Endpoint Update

Choose a tag to compare

@ewang2002 ewang2002 released this 13 Nov 02:39
4ebfb83

This release serves as the official v0.5.0 release (see rc1, rc2, rc3, rc4, and rc5 for full changes), since I forgot to make a release for that.

On top of what was released in v0.5.0, the following changes have been made to this project in v0.5.1:

  • Updates dependencies in all Rust binary and crate projects.
  • Fixes a minor issue where the /terms endpoint requires a term query parameter.

[0.5.0-rc.5] Add Course Text & Section Text Endpoint Support

Choose a tag to compare

@ewang2002 ewang2002 released this 04 Dec 02:22
fe9d7fa

See #30 for release notes.

v0.5.0-rc.4

v0.5.0-rc.4 Pre-release
Pre-release

Choose a tag to compare

@ewang2002 ewang2002 released this 03 Nov 23:57
01689d1
[0.5.0-rc.4] Adjusted Delay Time and Support for Hidden Terms (#29)

* feat: change login backoff time from sec -> min

* fix: hidden terms aren't being recognized by scraper

* feat: add constant delay for initial login after failure
- also updated version to rc.4

* feat: add extra delays between requests

* chore: update scraper dependencies

v0.5.0-rc.3

v0.5.0-rc.3 Pre-release
Pre-release

Choose a tag to compare

@ewang2002 ewang2002 released this 02 Oct 01:37
a7219fd
[0.5.0-rc.3] Login Stat Fix (#27)

* fix: return JSON obj instead of str for login script stats

* chore: bump version to rc.3

* style: run cargo fmt

0.5.0-rc2

0.5.0-rc2 Pre-release
Pre-release

Choose a tag to compare

@ewang2002 ewang2002 released this 23 Sep 07:42
7c80a29
[v0.5.0-rc.2] Single Cookie Scraper (#26)

* feat: automatically terminate cookie server if duo auth happens after init

* feat: use exponential backoff for determining delay

* refactor: fix notifier to use new API, 5 min cooldown, auth support

* docs: include link to nonexistent wiki

* chore: update version

* chore: update cargo.lock dependency versions

v0.5.0-rc1

v0.5.0-rc1 Pre-release
Pre-release

Choose a tag to compare

@ewang2002 ewang2002 released this 23 Sep 01:09
51219fa
[v0.5] Single Cookie Scraper (#25)

* feat: update wrapper and rewrite scraper for single login

* refactor: use tracing `log` and `error` instead of `println`

* refactor: add more logging to login script functions

* docs: update README with updated examples, docs

* refactor: use method to set stop flag directly

* chore: move scraper code to its own module

* feat: rewrote/integrate basic webreg server to scraper

* chore: remove `Extension` param for course_info endpoint

* chore: remove unnecessary client, close_after_request builder calls

* feat: add extra endpoints for cookie-related reqs

* refactor: update function accessibility level

* feat: add authorization support for api

* chore: update ci config

* chore: update dependencies to be more general + .gitignore

* chore: update `webweg` to `0.9`

* feat: add `description` column to sql table

* chore: move both helper scripts to `helpers` directory

* refactor: use workspace to separate webreg + authmanager crates

* refactor: move project dirs around, create basicauth lib for both bins

* feat: add more helper methods for basicauth lib

* test: add basic unit tests for basic auth

* style: run cargo fmt

* style: run cargo clippy

* refactor: update enum variant to be more clear

* feat: authmanager complete

* fix: compile script for webregautoin now uses curr path for proj

* chore: rename gh ci config to explicitly include bin file name

* style: run `cargo clippy` and `cargo fmt`

* docs: add plenty of new READMEs

* docs: update hyperlinks to project folders

* chore: move config.example.json to webreg dir

* docs: update docs to be more relevant to current project

* chore: minor word correction for no cookie found

* docs: modify documentation to be more clear, remove outdated docs

* chore: change version to rc.1

v0.4.7: Add Timeout to Requests

Choose a tag to compare

@ewang2002 ewang2002 released this 20 Apr 07:33

All requests will now have 25 seconds to complete or else be terminated.

v0.4.5: Bump WebWeg Library Version

Choose a tag to compare

@ewang2002 ewang2002 released this 31 Mar 07:37
1b3906a

This release is exactly the same as v0.4.4, but bumps the webweg library version to v0.7.4.

v0.4.4: Time Stat API Fix

Choose a tag to compare

@ewang2002 ewang2002 released this 19 Feb 01:46
bf460fd

This PR achieves the following:

  • fixed a bug where the time stat API would return the most recent requests as a string instead of an array
  • fixed a bug where calling the time stat API may require you to wait a while (up to the cooldown specified for a term) instead of nearly instantaneously.
  • increased number of recent requests from 100 to 1000.

v0.4.3: Add Time Stats to Scraper

Choose a tag to compare

@ewang2002 ewang2002 released this 17 Feb 09:46
4164b80

In this minor release, the scraper now keeps track of how long each request to WebReg takes. This might be useful if, for example, WebReg takes an abnormally long time to process a request.