Skip to content
Permalink
Branch: master
Commits on Mar 20, 2019
  1. Add TODO for the next thing I want to do but can't because there's a …

    Notgnoshi committed Mar 20, 2019
    …bazillion homework assignments due next week
  2. Use literal converters when reading CSV file.

    Notgnoshi committed Mar 20, 2019
    Interpret lists as actual lists, instead of the string __repr__ of a
    list.
  3. Add lemmatized and stopword-less haikus to dataset

    Notgnoshi committed Mar 20, 2019
    This balloons the dataset size, but I believe it's worth it, because
    stopword removal and lemmatization is a theme I want to explore in
    multiple areas.
    
    Also note that doing the preprocessing *before* duplication removal
    removes 7 more duplicate haikus.
  4. Perform lemmatization in parallel.

    Notgnoshi committed Mar 20, 2019
    By and large, lemmatization is the most computational aspect of the data
    standardization. spaCy can parallelize its pipeline, so take advantage
    of that fact.
    
    However, this changes the interface - it's no longer possible to
    lemmatize an individual haiku - the haikus must be consumed from an
    iterable.
Commits on Mar 18, 2019
  1. Finish Zipf's law EDA.

    Notgnoshi committed Mar 18, 2019
  2. unit test tools library

    Notgnoshi committed Mar 18, 2019
Commits on Mar 17, 2019
Commits on Mar 16, 2019
  1. install sklearn

    Notgnoshi committed Mar 16, 2019
  2. Add line breaks at 100 cols

    Notgnoshi committed Mar 16, 2019
  3. Fix british spelling

    Notgnoshi committed Mar 16, 2019
  4. Add TODO

    Notgnoshi committed Mar 16, 2019
  5. Save CSV file in a way that preserves the order.

    Notgnoshi committed Mar 16, 2019
    This will make the git history more reasonable, as well as preserve
    topical clusterings observed to be present in the txt file.
  6. Move notebook

    Notgnoshi committed Mar 16, 2019
  7. Remove extra blank lines

    Notgnoshi committed Mar 16, 2019
  8. Remove more false positives

    Notgnoshi committed Mar 16, 2019
Older
You can’t perform that action at this time.