Permalink
Switch branches/tags
Nothing to show
Commits on Sep 24, 2017
  1. Forgot what the extension was

    floft committed Sep 24, 2017
  2. Added picture of website

    floft committed Sep 24, 2017
  3. Note have to compile PoDoFo with C++11

    floft committed Sep 24, 2017
    Otherwise freetron won't extract the images. Also print the PoDoFo error
    messages. Better to have them then to have to recompile to get them.
    Also changed from -O3 to -O2 since apparently PoDoFo errors don't show
    up if we have -O3? Not sure why.
Commits on Jun 7, 2015
  1. Made README point to new links

    floft committed Jun 7, 2015
  2. Moved examples into repository

    floft committed Jun 7, 2015
Commits on Mar 23, 2015
  1. Keep sorted list of points we've been to in Outline

    floft committed Mar 23, 2015
    It is faster to detect if we've been to a point before if we have a
    sorted list, so now use std::set in addition to the ordered by
    last been to list in a std::vector. This uses more memory, but if the
    path grows to be very large (e.g. 100,000 points or so), this will make
    a considerable difference.
    
    Note: may not matter much here since we probably don't have that long of
    outlines in these black and white images, but it makes a huge difference
    when tracing blobs in photos.
  2. Fixed link to MIT license

    floft committed Mar 23, 2015
Commits on Nov 13, 2014
  1. Fixed HTML escaping of filename in website

    floft committed Nov 13, 2014
    Previously if you uploaded a file such as "<b> file.pdf", it would not
    escape the HTML.
Commits on Oct 19, 2014
  1. Reworded "What is This?" paragraph

    floft committed Oct 19, 2014
    Tried to make it less confusing to somebody who has no idea what a
    Scantron machine is.
  2. Output as a CSV file

    floft committed Oct 19, 2014
    This way from the website UI you can click an "export" link to download
    a format that is easier to work with. Also, from the CLI you can specify
    an option to generate this output instead of the default summary format.
Commits on Oct 5, 2014
  1. Added "Navigating the Code" to README

    floft committed Oct 5, 2014
    This may help if people want to extend or modify the program a bit.
  2. Fixed P4 off by 32 pixels bug

    floft committed Oct 5, 2014
    Previously even in the P4 1-bit black-and-white bitmap I was putting
    '255\n' in the header, which in the 1-bit version shows up as image data
    instead of specifying the maximum pixel value as in P5 and P6.
  3. Image extraction: handle ICCBased color space, handle 1-bit P4 bitmap

    floft committed Oct 5, 2014
    Previously ICCBased color spaces would result in an unknown colorspace.
    Now it'll look for the Alternate on the ICCBased color space.
    
    Previously a P4 bitmap (1-bit per pixel) would result in an invalid
    buffer size warning, but now it'll extract the image correctly.
Commits on Oct 2, 2014
  1. Working upload progress bar

    floft committed Oct 2, 2014
    I previously listened for the "progress" event, but it was the download
    progress event instead of the upload one. Thus, if we're sending data
    with the http() function in Javascript it'll listen to the upload
    progress event and otherwise the download one.
Commits on Sep 17, 2014
  1. Minor fixes

    floft committed Sep 17, 2014
    Changed --hmac-cbc to --cbc-file to account for the next version of
    cppcms_make_key included with CppCMS. Bug report:
    
    https://sourceforge.net/p/cppcms/bugs/129/
    
    Fixed bug in stdev where it wasn't actually using the result of
    std::accumulate. However, since this function currently isn't
    being used, this has no effect on the program.
    
    Fixed tabbing issues in Javascript code.
Commits on Aug 27, 2014
  1. Use dist^2 instead of sqrt(dist^2)

    floft committed Aug 27, 2014
    Instead of finding the actual distance, if we're just finding two
    farthest or closest points, we can calulate the distance squared using
    integers instead of doubles because the maximums and minimums of
    sqrt(x^2) and x^2 occur at the same x values, just differing in y
    values. This, however, doesn't appear to make any noticable speed up.
Commits on Aug 13, 2014
  1. make install only installs instead of configures

    floft committed Aug 13, 2014
    Using cmake you can generate the keys for the website, but with make
    install it will only install the files that are the same for all users.
    This is to ease packaging, for example on Arch Linux with a PKGBUILD.
    The install is the same for everybody but in a freetron.install file the
    keys are generated per install so nobody has the same keys.
  2. Server-side password salts and hashing

    floft committed Aug 13, 2014
    I read a bit more about how salting passwords should be done. Now I grab
    a new salt from /dev/urandom (via CppCMS) for every generated password
    and hash the password on the server as well as on the client. This way
    during a man in the middle attack the attacker doesn't have the password
    due to the Javascript hash (in case this user uses the same password on
    multiple sites). If the attacker gets the passwords hashes from the
    database, he has a bunch of different salts to look at, so he can't
    easily use reverse lookup tables.
    
    Good reference:
       https://crackstation.net/hashing-security.htm
    
    Also, make the database, key files, and uploads directory only readable
    by the owner during installation.
  3. Database locking, smart pointers, JS messages

    floft committed Aug 13, 2014
    Display "Loading..." while loading the forms. Display the "Uploading"
    message as soon as it starts uploading from the website since it may
    take a while for a progress message to be displayed (if ever). Forbid
    updating your account with a blank password in Javascript. There is
    nothing to stop you on the server from adding a blank password since it
    knows nothing about the salt or hash, but if the user wishes to go to
    the work to submit a blank password, it's their problem.
    
    Additionally, under heavy load I got an exception:
    
    terminate called after throwing an instance of 'cppdb::cppdb_error'
      what():  sqlite3:cannot start a transaction within a transaction
    
    So I added a lock for all database actions so only one thread can
    read/write to the database at a time. Then, replaced a manual allocation
    and deletion with a smart pointer in case Pixels throws an error.
    
    I added an ilDeleteImages when ilLoadL fails. This may or may not be
    needed, but if it isn't, it should just check to see if the name exists
    and not do anything if it doesn't exist.
Commits on Aug 12, 2014
  1. RPC as async application, limit status connections

    floft committed Aug 12, 2014
    Previously a new RPC thread would be created for each incoming
    connection since in the main loop this application which was supposed to
    be asynchronous was started as a synchronous application. Now it is
    started properly and explicitly loads the session since they are not
    loaded by default in an async app.
    
    Limit the number of connections allowed to monitor a specific form to
    one. Only one person should ever be waiting on a form, so quickly end
    the connection if one already exists. This way an attacker can't just
    create a few million requests wasting server memory and processing time.
    
    Also fixed a signal initialized issue and moved srand initialization to
    main instead of in each RPC thread.
Commits on Aug 8, 2014
  1. Statuses added to queue, RPC thread monitors queue

    floft committed Aug 8, 2014
    Instead of having the RPC website thread block until the form had
    another status update, now if the form isn't done, we save the request.
    The StatusThread in the RPC will wake up every time there is a change in
    the status queue in Processor and complete the request waiting on that
    form. This is the proper way to handle long request since the CppCMS
    documentation specifically states, "It is important to remember that
    every operation executed in the event loop should be very fast and never
    block." Now we're not blocking in the event loop but in a separate
    thread for each RPC application instance.
    
    This allows Ctrl+C to exit the website immediately and then just wait
    for the two thread pools to exit. Note that this may take a while since
    an item in the extraction thread queue extracts the entire PDF. I have
    yet to figure out a good way to cancel extractImages and parseImage.
  2. Only add image to queue if it was loaded

    floft committed Aug 8, 2014
    Previously if an "image" was found in the PDF that wasn't ever
    successfully loaded (e.g. the width or height wasn't specified in the
    PDF), the Pixels object be blank and still added to be processed, which
    obviously was unsuccessful and a waste of time. It errored giving not
    enough boxes detected, but didn't specify the filename. Now we just
    don't process an image if it isn't loaded.
  3. Faster Blobs algorithm

    floft committed Aug 8, 2014
    Instead of having a single main pass containing numerous partial passes
    replacing all the labels of a certain ID, now it'll use two passes: one
    to create the labels and mark which are equivalent and another to
    replace all labels with a representative of the equivalent labels.
    
    In order to do this I wrote a Disjunct Set (or Union Find) data
    structure. It requires its elements to be sortable since it's using
    std::set, but we're using it with int anyway. I used std::set since it
    has a O(log(n)) find function, and there are a lot of find's going on in
    this new algorithm.
    
    Everything was written from scratch, but the ideas were from the
    "Sequential algorithm" section of the Connected-Component Labeling
    Wikipedia article:
    https://en.wikipedia.org/wiki/Connected-component_labeling#Sequential_algorithm
    
    Timing Results:
    Removed 13 seconds from test suite of 56 forms, about 0.6 seconds for
    freetron_example2.pdf on 8 cores (1.6 if only running on one core), and
    basically no change for freetron_example.pdf.
Commits on Aug 7, 2014
  1. Moved includes, some messages only for CLI mode

    floft committed Aug 7, 2014
    I switched from putting all includes in the header files to putting only
    the needed ones there and the rest in the corresponding code file. This
    speeds up building a bit. Then, debug mode will only work when using
    this program from the CLI, i.e. not in website mode. And in website mode
    don't include thread IDs and filenames in error messages.
Commits on Aug 6, 2014
  1. Handle SIGHUP reloading the website config

    floft committed Aug 6, 2014
    Now you can change the config and reload the website without restarting
    the entire application. Note that this will display an error for people
    currently uploading or processing forms, however the processing ones
    will still be processed.
Commits on Aug 5, 2014
  1. Added systemd service file

    floft committed Aug 5, 2014
    It is assuming that you install with a /usr prefix, i.e. either:
      cd cmake; cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr . && make install
      make install PREFIX=/usr
    
    But, it is easily changed by overriding it in /etc/systemd/system.
  2. Generate keys on install

    floft committed Aug 5, 2014
    Now there is no example config, just the real one pointing to hmac.txt
    and cbc.txt that will contain the keys. Then on install generate these
    keys.
    
    Note: the command to generate the keys uses --hmac-cbc to output to the
    cbc file although the cppcms_make_key says it should be --cbc-file. I
    reported the bug on Sourceforge.
    
    https://sourceforge.net/p/cppcms/bugs/129/
  3. Put website files in /srv/freetron instead of /usr/share/freetron

    floft committed Aug 5, 2014
    Seems to be more fitting.
Commits on Aug 4, 2014
  1. Install config.js if it exists, otherwise config.js.example

    floft committed Aug 4, 2014
    And, have the same behavior with both CMake and make. Previously
    CMake would overwrite config.js if it existed.
  2. Added install targets for CMake and make

    floft committed Aug 4, 2014
    Now it'll copy the needed files. Eventually I'll add a systemd service
    file and maybe generate the keys automatically on install.
  3. Updated CMake to link to CppDB and CppCMS

    floft committed Aug 4, 2014
    Now both the make and cmake methods of building should work. There's no
    guarantee that the cmake version will work on any other systems though.
    Haven't tested it.
  4. Fallback POST upload of forms

    floft committed Aug 4, 2014
    Older devices and browsers don't necessarily support the newer
    Javascript file uploading.
  5. Minimized CSS and JS

    floft committed Aug 4, 2014