Research Exploration Group

What are we finding that’s interesting?

Words that we didn’t expect would be attested this early (e.g. “scenography”)

A surprising range of types of documents (for the physical movement cluster---conduct and dance manuals, travel narratives that involve “savages”, poetry, etc).

For the evolution search: confirmation that this is a successful tool. Doing some rough google books searches for “evolution” plus the authornames for the texts that had the highest number of unique keywords and interesting heatmap results, and all of them are identified in the literature as Darwin’s precursors. I wonder if the next step is to take these c18 passages and re-generate the keyword cluster to see if there are any differences, and then run the search on the texts that are outside of the literary and canonical set to see if anything interesting pops up. (Or look specifically at the literary ones, if you have those tagged accordingly.)

The heatmap itself, when combined with a sort by the total number of unique keywords that appear in the passage, seems to help indicate how many unique keywords are the right number to look for (unless this is just reflecting how the search algorithm counts the “signal”).

How are we manipulating the results? What tools are we using to explore?

Creating a column that says how many of your keywords appear in the passage, using the formula: =COUNTIF(I2:X2, ">0")

Create a heat map of keywords to visualize the relative frequencies of keywords to each other. Because there are so many zeros and the numbers are very unevenly distributed, the green-white color scale seems to work best. See screenshot: https://drive.google.com/file/d/1QqhqIvinqHRF9BhWNl1uCMN40mIsYmlf/view?usp=sharing

Question of whether it’s more interesting to be able to compare the raw counts of the word occurrences or to compare the count relative to the maximum frequency of that particular word in your results. (e.g. if word x appears very frequently in your passages, and word y is infrequent, do you want a heat map to show 1 occurrence of word y at the same heat level as 20 occurrences of the word x?) Or maybe ratio of word x to the total number of keywords in the passage (keywordsCount)

What’s NOT working?

OCR problems

It’s super clunky to look up the passages---demonstrating the value of building this search tool!

For now, though, would it be possible to include the 5-10 words of the passage so that you could just copy and paste that into ECCO to look it up?

ECCO’s keyword search! I keep trying to locate the original passages using phrase searching for bits where the OCR is the cleanest and it’s failed like half the time. I seem to be having more success in HathiTrust.

For the example “evolution” (Document ID: 819200301) I found passages with the term “child” occuring 43 times and all other terms all occuring 0 times. The passage seems to be about a family instructor rather than evolution. (Amelia: I also have a couple of questions: will this tool work for a corpus in a language other than English? )