Engineering Notebook: Aaron Van De Brook

Engineering Notebook: Aaron Van De Brook

4/27

• Made a PR to merge and merged the edit paper info branch into the main branch.

4/25

• Fixed a bug in the edit page where paper citations were not always showing up.

4/21

• Made a few style changes to make to reference page look a bit better with the text boxes

4/20

• Added button to update paper info in the reference info page
• Added keys (or tried) to the form tables so that it can be parsed on the back end
• Problems with key errors, not sure why. I think it's because there are two forms on the web page:
  • It is because there are no input fields in the form for the server to parse
  • Need to change some of the html (mess with that tonight)
  • Changed table entries to textareas; also had to give the form an id since there is more than one form on the page
  • New problem: return template rendered with new info; need to change the logic of the /papers/new endpoint:
    • Need to parse information, store in database
    • Template needs to be changed to populate fields from database entries instead of directly from the parsers
    • Database schema needs to be changed a bit

4/15

• Added database functionalities into the reference info page
  • Still need to mess with the parsers to speed up the response times
• Need to add some kind visual response to the edit page so the user knows they're are editing the page and that it can be edited

4/13

• Looking at Corrina's pull request
• Tried to get some work done search function and database integration

3/29

• Added manifest and setup files so that the project is easier to setup/install and run
• Setup so that running pip install -e . automatically installs all project dependencies (note: these files need to be updated to reflect any new dependencies added as the project grows)
• pip list | grep -i citation-graph (Linux) and pip list | find "citation-graph" (Windows, I think; I haven't tested it)
• Updated readme to reflect changes and to better explain how to set up the project, dependencies, and how to run
• PLEASE REVIEW MY PULL REQUEST, IT'S BEEN THERE SINCE THURSDAY

3/25

• Made some small changes (line breaks between input fields) to the input screen to make it more readable
• Removed plaintext server secret key from initialization file and moved all user configurable fields into a config.py file which is ignored by git
• Demo

3/23

• Set up server to convert papers into C, and run doxygen on them
• Current problem: when reading the doxygen files I get the error codec cant decode byte 0x89 in position 0: invalid start byte

3/22

• Integrated web pages (from Luis and Sam) into the server
• Set up endpoints to handle new papers and searches from users
  • Wrote a template to return with paper/bib data once the server processes the data
  • New paper endpoint handles POST requests from the browser, saves the files entered, processes them, and returns the data back to the user in a readable/edittable format
• Last steps: integrate doxygen, add a way to access doxygen web pages, integrate database (might be pushed off to next sprint), add search functionality
• Also, ran into a weird bug in the bibtex parser when testing the forms; when the bibtexparser library was parsing the document it ran into errors because it could not decode the byte 0x92 in utf-8. This is a 'smart quote' from windows, which doesn't exist in utf-8
  • Solved by switching the encoding/decoding scheme from utf-8 to cp1252 (in retrospect, using dos2unix might have worked too, but using a different encoding scheme is probably a better idea)

3/18

• Fixed an issue in the Latex parser where it wasn't saving all citations
• Refactored server to make static resources easier to use (all contained in the static folder now, use something like {{ url_for('static', filename='<file_or_path>') }} in templates to link the pages together easily
• Set up blueprint to use for the request and response form pages
• Any more progress for the server is at a roadblock until the parser(s), webpages, and (possibly) database are complete or closer to a point where they can be tested with other modules
  • Database will be useful for ensuring we're not having the parsers do extra work when they don't need to, improves efficiency and performance
  • Database is really easy to implement and integrate with flask, but need info on which to use because the syntax and initialization processes for all of them are different

3/12

• Started refactoring server and setting up Jinja templates for form pages (input paper, paper info like references, authors, publication date, etc.)

3/11

• Refactor endpoint function to use @after_this_request function to (hopefully) make the responses faster or at the very least easier to read
• Searched for fonts and messed around with linking external fonts in HTML documents as well as scripts

3/8

• Did some testing to make sure stylesheets linked correctly, turns out endpoints need to be set up for everything
  • Set up endpoints and error handling for stylesheets, scripts, images, and favicon(s)
  • Some browsers (Firefox) won't accept server responses if specific HTTP packet header properties are not set, so for all CSS documents the 'Content-Type' property is set to 'text/css' and for JS documents it is set to 'application/javascript'. These are also defined by standards which are referenced in the MDN documentation
• Should probably be setting response headers with the @after_this_request decorator, so this will be kept in mind as I refactor the code

3/4

• Got a basic server started with flask, serves placeholder pages at the moment, but these can be easily replaced/added when the front-end starts to come together
  • The index (home page), and 404 page are the only endpoints working at the moment
• Assembled a list resources and documentation for HTML, CSS, Python, and Flask (will update these more as we incorporate more things)

3/2

• Created and assigned tasks for sprint 2, started looking into flask as a means of setting up a back-end server for the system

2/22

• Finished the LaTeX parser (with the exception of the index terms). Grabs the abstract, citations, bibliographies, etc. from a LaTeX paper.
• Skipped flask and opted for just opening the generated doxygen html from a python script.

2/18

• Refining/creating code to grab important sections from the paper (have the abstract working so far, working on citations/contributions next).
• Started on manually generated graphs. Using example papers first to keep things simple and then expanding from there.
• Going to start messing with flask this weekend so we can have some semblance of a prototype for the demo.

2/16

• Finally created a system design (for the back-end at least). Still need to come up with ideas for the front end design.
  • Thinking of using Flask for ease of use between front and back ends
• Created/cleaned up some code to give us a starting point on the parser modules

2/10

• Figured out some details for the LaTeX parser, starting writing some code to find important sections (e.g. Abstract, Citations, etc.)
  • Need to convert to classes for modularity when using it in a 'back-end'
• Once the parsers are separated into classes, will starting working on creating the C-like functions for Doxygen to process, then front-end framework

2/2 - 2/4

• Got BibTeX parser 'working', wrote a few functions to grab paper information and convert them into C-like functions
• Uploaded some (or all) of the files that Liu emailed us

1/28

• Met with team in person to discuss initial design