WIP: Convert Library of Congress codes into a classification #3309

cclauss · 2020-04-03T21:03:59Z

Fixes #3396
Related to #3290

NOTE: See the new testing instructions below.

Run this experiment with the command scripts/lcc_to_genre_subgenre.py and then enter Library of Congress Classification codes to see their genre and subgenre. Problem codes will be written to the file lcc_to_genre_subgenre.py_debug.txt so that you can copy from that file and paste Into comments here to show us which codes were not able to generate genreandsubgenre`.

@BrittanyBunk @finnless Please try to run this and enter valid LCC codes from OpenLibrary to see if you get two entries (genre and subgenre) each time. Are they the classification codes that you expected for each work?

There are two functions in this code. The first one attempts to use the LCC letters only. If that results in both a genre and subgenre then we are done. If not, then we run the second function that uses both the letters and the numbers to attempt to get both a genre and subgenre. If you find codes that do not return both a genre and subgenre then please add comments to this PR.

NC248 uses only the first function
KLA940 needs to use the second function to obtain the subgenre
MLCM95 delivers a genre but no subgenre

% scripts/lcc_to_genre_subgenre.py
Please enter Library of Congress codes like: HB1951 .R64 1995...
Or leave blank to quit: NC248
NC248 ['Fine Arts', 'Drawing. Design. Illustration']
Or leave blank to quit: KLA940
Needed numbers: KLA940
KLA940 ['Law', 'Russia, Soviet Union']
Or leave blank to quit: MLCM95
Needed numbers: MLCM95
MLCM95 ['Music']
Or leave blank to quit:

Technical

Testing

Yes. Please. (I have another version with tons of doctests.)

Evidence

Stakeholders

BrittanyBunk · 2020-04-04T07:31:45Z

@cclauss it shows on this thread that the error checks failed. Should I still try it out? I will check out the code and see what's going on, as I don't have python on my computer to run it and my computer's been having problems lately, so I can't add it on - unless there's a different, easier approach to this.

BrittanyBunk · 2020-04-04T07:50:58Z

I looked at the code and it seems at the end it's cut short - there's no return like for the last section. scripts/lc_classifiers_letters_only.json does look complete, so this is good to know for reference to know what to do next.

BrittanyBunk · 2020-04-04T08:03:38Z

As I thought about MLCM95 ['Music'], you're right - sometimes there are misspellings and other issues that may require the number or other otpions. I'm assuming this this one that it was 'ML' ['Music', 'Literature on music'], but with extra lettering typos. So if we're using the 1st two letters, then it'll come out like I showed, but if we're truly worried about potential false info, it'll be good to leave it off like you did. I'm not sure.

cclauss · 2020-04-04T10:14:02Z

the error checks failed

Fixed. I have gotten out of the habit of writing Python 2 code but the linter caught me so now it is OK to test.

I don't have Python on my computer

What is your operating system? I can build a standalone app if I know the OS.

So, for MLCM95, my thought is do things in this order...

First try M to get the genre
Then try MLCM to deal with a DAW-type code if the table gets improved
Then try MLC again to deal with a DAW-type code
Finally, try ML which will give us the correct subgenre

If you get a chance to test then please try any valid letter combinations but especially:

DA vs. DAW
DJ vs. DJK
K combinations
M combinations

seabelis · 2020-04-04T11:59:03Z

Let's be careful to use the correct terminology. LC Classification is different from genre.

cclauss · 2020-04-04T12:25:23Z

OK...

Should we use lcc_class and lcc_subclass instead.
Are we in agreement that we only need two levels for now?

seabelis · 2020-04-04T13:16:05Z

Should we use lcc_class and lcc_subclass instead.

Yes, please.

scripts/lc_classifiers_letters_only.json

cclauss · 2020-04-04T16:08:33Z

If we look at https://openlibrary.org/books/OL1025841M/Multiregional_demography, we are categorizing it by seven different subjects. And if we follow the LC Control Number link to https://catalog.loc.gov/vwebv/search?searchCode=LCCN&searchArg=96100951&searchType=1 we see that the LoC is tracking it by four subjects. I doubt that the letters alone are going to provide rich enough data in the long run. Especially for Class/Subclasses K and P we are going to have to utilize the numbers as well as the letters.

seabelis · 2020-04-04T16:12:12Z

Subject headings are different from classification. So if you think of a physical library and a physical card catalog, the book is a single item, located in a single place - this is identified by the classification (call number). The card catalog, on the other hand, has index cards to represent the one physical item across multiple categories - those are the subject headings. And again those are separate from genre, which is what the book is versus what the book is about.

cclauss · 2020-04-04T16:14:00Z

Nice way to put in. I understand now. So in the end, we want just one Class and one Subclass for each book?

seabelis · 2020-04-04T16:17:11Z

Usually, yes, but I've seen some cases where a book has been assigned more than one LC number. Not so common and I don't think it's something to worry about.

BrittanyBunk · 2020-04-04T17:03:15Z

I don't have Python on my computer

What is your operating system? I can build a standalone app if I know the OS.

I've installed it before. I have Windows 8.1. I haven't done testing of code like this yet, so I'm still learning.

LC Classification is different from genre.

That is true. The LoC calls them 'classes' and 'subclasses'. I was assigning 'class' as 'genre' when it's posted on the OL UI and 'subclasses' as a 'subgenre' there too. I've seen them called 'categories' on other sites too. Thank you @seabelis for clarifying. You're right - I wasn't specific enough on that.

tfmorris · 2020-04-04T19:04:03Z

What's the relationship of this code to the linked issue? Instead of indexing LCC directly, as that issue describes, is the proposal now to convert them to text and index them as subjects instead?

tfmorris · 2020-04-04T19:07:23Z

scripts/lc_classifiers_letters_only.json

@@ -0,0 +1,242 @@
+{


What is the provenance of this file (and the other JSON file)?

We parse these .pdf files with a variant of #3290 (comment) (parsing code is MIT License) That is the linkage to that issue.

This is a WIP Draft because there are trade-offs to consider between looking something up in a dict vs. embedding it in every record.

tfmorris · 2020-04-04T19:08:24Z

scripts/lcc_to_class_subclass.py

+    >>> get_ol_book_info()  # doctest: +ELLIPSIS
+    {'olid:OL26617202M': ...
+    """
+    url = "https://openlibrary.org/api/books?jscmd=details&format=json&bibkeys=olid:"


It would be easier to read/update if the query parameters were passed in a dict, as supported by requests.

This is a WIP in Draft mode. Those two utility functions are present to allow testers to experiment if they are more comfortable entering OL codes rather than LCC codes. Given that we want get detailed data formatted as json from a fixed endpoint, these functions only need to modify a single parameter. Dict-driven URLs are verbose and by no means mandatory for fixed endpoints such as APIs.

tfmorris · 2020-04-04T19:08:59Z

scripts/lcc_to_genre_subgenre.py

+    >>> get_ol_book_info()  # doctest: +ELLIPSIS
+    {'olid:OL26617202M': ...
+    """
+    url = "https://openlibrary.org/api/books?jscmd=details&format=json&bibkeys=olid:"


Same as above

This file will be removed before we exit WIP/Draft staus.

cclauss · 2020-04-04T19:58:16Z

@BrittanyBunk

I have Windows

You should be able to run py --version and/or python --version to see what versions of Python are available to you.

I think of a classification as a hierarchy of classes like zoologists order living creatures by Kingdom, Phylum, Class, Order, Family, Genus, Species. For a creature, I render its classification listing out its membership at all of these levels.

I interpret LoC classification starting with a single letter class and then a hierarchy of subclasses underneath. Some top level classes are fairly simple and do not have many levels of nesting below while others are more complex and their subclasses can nest deeply.

In zoology classifications range over kingdom, phylum, class, order, family, genus, species while in LoC classifications range over class, subclass, subclass, subclass...

cclauss · 2020-04-04T20:14:51Z

NOTE: Python 3 has order-preserving dicts while Python 2 does not.
This means that the order of elements in a classification might be incorrect in legacy Python.

BrittanyBunk · 2020-04-04T22:21:07Z

@cclauss thanks. I couldn't find the documentation on it on the Testing page, so I talked with Mek and decided I will help out with the documentation on that. I'll install it.

BrittanyBunk · 2020-04-05T05:40:49Z

Ok. For some reason it seems like my Python library (version 3.8.2) doesn't work well with the code (I keep getting errors). Which Python version do you have (so I can run it on that)?

BrittanyBunk · 2020-04-05T15:32:50Z

I ran it in IDLE. One of them is:

cclauss · 2020-04-05T15:34:59Z

Try the Binder link above instead.

seabelis · 2020-04-05T15:53:12Z

MLC stands for Minimal Level Cataloging. More info here. https://www.loc.gov/rr/cmd/collservedcalm.html and here https://www.loc.gov/catdir/cpso/catmodes.pdf.

BrittanyBunk · 2020-04-05T16:18:33Z

*see newest comment

Here are the ones that were incorrect (LoC, what shows, Correct version. Source: https://www.loc.gov/catdir/cpso/lcco/):

DP402.C8 O46 1995
- 'History, General and Old World', 'History of Spain', 'Local history and description', 'Other cities, towns, etc., A-Z'
- 'World History and History of Europe, Asia, Africa, Australia, New Zealand, Etc.', 'History of Spain', 'Local history and description', 'Other cities, towns, etc., A-Z'
MLCM 95/14118 (P)
- Object is not iterable
- MLC number - see WIP: Convert Library of Congress codes into a classification #3309 (comment). Would be 'Language and Literature' (MLCs don't provide the subclass).
HQ755.8 .T63 1995
- 'Social sciences', 'The Family. Marriage. Women', 'The family. Marriage. Home', 'Eugenics'
- 'Social sciences', 'The Family. Marriage. Women', 'The family. Marriage. Home', Parents. Parenthood' (I'm not sure, but do we also include the rest?: 'Including parent and child, husbands, fathers, wives, mothers'. I think we should using a ':' to separate them)
KLA940 .K65 1990
- 'Law', 'Russia, Soviet Union'
- 'Law', 'Asia and Eurasia, Africa, Pacific Area, and Antarctica', 'Eurasia', 'Russia. Soviet Union'
KHA878 .G37 1996
- 'Law', 'Argentina'
- 'Law', 'South America (General)', 'Argentina'
KHH3003 .Q57 1995
- see KHA878 .G37 1996
HM216 .G44 1993
- 'Social sciences', 'Sociology', 'These are obsolete numbers no longer used'
- 'Social sciences', 'Sociology', 'These are obsolete numbers no longer used by the Library of Congress'
SD418 .A38 1990 (I made up this one)
- 'Agriculture', 'Forestry', 'Conservation and protection'
- 'Agriculture', 'Forestry', 'Conservation and protection: Including forest influences, damage by elements, fires, forest reserves'
KK2222 .L36 1993
- 'Law', 'Law of Germany', 'Commercial law', 'Commercial transactions', 'Banking. Stock exchange'
- 'Law', 'Europe', 'Germany', 'Germany and West Germany'
HM261 .H47 1993
- see HM216 .G44 1993
KF27 .A3 1992H
- 'Law', 'Law of the United States (Federal)', 'Congressional documents'
- 'Law', 'Law of the United States', 'Federal law. Common and collective state law: Individual states'
KF3613.4 .C34
- 'Law', 'Law of the United States (Federal)', 'Social legislation', 'Social insurance'
- 'Law', 'Law of the United States', 'Federal law. Common and collective state law: Individual states'
DR1313.3 .U54 1993, DS557.8.M9 B55 1992B, DR82 .G46 1993
- see DP402.C8 O46 1995
HN530.2.A85 I86 1992
- 'Social sciences', 'Social history and conditions. Social problems.', 'By region or country'
- 'Social sciences', 'Social history and conditions. Social problems. Social Reform', 'By region or country'
TK5105.5 .O653 1993
- 'Technology', 'Electrical engineering. Electronics. Nuclear', 'Telecommunication'
- 'Technology', 'Electrical engineering. Electronics. Nuclear engineering', 'Telecommunication: Including telegraphy, telephone, radio, radar, television'

I'm assuming some of these is due to not enough space, so it gets cut off. The others seem that the lines below the first line don't get read.

BrittanyBunk · 2020-04-05T16:25:13Z

Scratch what I said earlier. I do notice something about the 4 letter LCC's - there's possibly two LC's in them (all the ones with 4 letters have a "/" in them - I believe on purpose. As @seabelis said - some books have 2 LCC's).
So MLCM 95/14118 is both ML95 and CM14118. I didn't realize that until now. So with those, it would show 2.
Here: 'Music', 'Literature on music', 'Manuscript studies and manuscripts' and 'Auxiliary Sciences of History', (there's no CM).
Another MLCS 97/02275 so ML 97 and CS02275: 'Music', 'Literature on music', 'Manuscript studies and manuscripts' (there's no 97) and 'Auxiliary Sciences of History', 'Genealogy', 'By region or country' (there's no 2275)

Is this correct @seabelis?

seabelis · 2020-04-05T17:50:24Z

No. Not correct. MLC stands for Minimal Level Cataloging. I posted two links with more information above.

BrittanyBunk · 2020-04-05T18:25:07Z

@seabelis To make sure I understand everything: MLC12 3/4 (5)

size
https://www.loc.gov/aba/publications/FreeDCM/DCM_2017-03.pdf on pg 11:
Label | Meaning | Use for books that are
MLCS | Small | 22 cm or less
MLCM | Medium | 23-30 cm
MLCL | Large | 31-37 cm
MLCF | Folio | 38 cm or larger
Special categories
A – South Asia materials, Asian Division*
C – Chinese Section, Asian Division*
E – Southeast Asia materials, Asian Division*
H – Hebraic Section, African and Middle Eastern Division
J – Japanese Section, Asian Division*
K – Korean Section, Asian Division*
M – Mongolian Section, Asian Division*
N – Near East Section, African and Middle Eastern Division*
R – Rare Book and Special Collections Division
(Resources housed in the Rare Book and Special Collections Division do not
receive an indication of size; instead a blank is used: MLC R)
T – Tibetan materials, Asian Division
On page 12, it looks like they switched Add a setup.py #1 and catalog/onix/onix.py attempts to use a global variable in init(), but doesn't declare it global #2 for the last example
Year:
Before 2000 - 2-digit number with last two digits (like 1998 is 98)
After 2000 - all digits
unique 5 digit ID
https://www.loc.gov/aba/publications/FreeDCM/DCM_2009-01.pdf on page 23:
use the 1 letter for a category on http://www.loc.gov/catdir/cpso/lcco/lcco/

So MLCS 98/02371 (H) is a small book from 1998 in Social Sciences. For class, it'll just show 'Social Sciences'.

cclauss · 2020-04-06T09:22:31Z

Your help please on this one...

test_cases = {
    "DP402.C8 O46 1995": [
        "World History and History of Europe",
        "Asia, Africa, Australia, New Zealand, Etc.",
        "History of Spain",
        "Local history and description",
        "Other cities, towns, etc., A-Z",
    ],
}

Why would History of Spain be under Asia, Africa, Australia, New Zealand, Etc.?

seabelis · 2020-04-06T09:55:14Z

@cclauss > Why would History of Spain be under Asia, Africa, Australia, New Zealand, Etc.?

That appears to be an error. DP1-402 is History of Spain. Asia is DS. Africa is DT. Australia is under DU.

BrittanyBunk · 2020-04-06T13:53:53Z

@cclauss @seabelis There's an error here. You split up "World History and History of Europe" from "Asia, Africa, Australia, New Zealand, Etc.". If you look at what I wrote, it's one phrase. I'll show you what I mean:

cdrini

Nice use of binder! :) This looks great! Note MLC* are invalid for our purposes here; they don't denote any classes, just physical properties about the book (it would incorrectly list all such books as being about Music :P ). Once out of WIP, you might want to use utils.lcc.short_lcc_to_sortable_lcc from #3302, which will filter those out.

In terms of permanent home for this, a utils.lcc_classes.py might be a good place. The main "public" method we'd need to show these in the UI would be the lcc_to_classification function 👍

cdrini · 2020-04-06T14:01:43Z

scripts/lcc_to_class_subclass.py

+    return chars, 0
+
+
+def lcc_to_classification(lcc):


Maybe this name might be better?

Suggested change

def lcc_to_classification(lcc):

def get_lcc_classes(lcc):

@cdrini the MLCs do have classes in parentheses, but I could see leaving it out for now - as it's extra coding. They just don't have subclasses, but just classes is something that should be included (maybe one day).

The name change does look a lot cleaner and more readable :), but I'll leave it to the code writers to make the final decision.

cdrini · 2020-04-06T14:34:30Z

@BrittanyBunk That sounds perfect 👍 Long term, we should probably store these in a different field in openlibrary, not in the LCC field; e.g. maybe something like:

{
  "lc_classifications": ["S"],
  "mlc": ["MLCS 91-9437"],
}

But for now excluding them seems best 👍

BrittanyBunk · 2020-04-06T18:45:41Z

@cdrini maybe it'll even be named get_lcc_classes_MLC_only.
I just wanted to make a quick correction. The "S" is the size in your example, not the LCC. This is what it should look like, but with a different example:

`{
"lc_classifications": ["P"],
"mlc": ["MLCM 95/14118 (P)"],
}'

I think this MLC file would need to refer to the lc_classifiers_letters_only.json, but I'm thinking it could refer to a smaller one, as it'll use only the first letters. I created this: lc_classifiers_first_letter_only.json. There were some typos that I corrected from the original one, so if the MLCs get added in, they should be consistent. @cclauss should I create a corrected version (that capitalizes letters and such) of lc_classifiers_letters_only.json and place the link here?

BrittanyBunk · 2020-04-15T03:22:14Z

I made a mistake with which LoC page to use. I used the outline, but in reality I should've used the schedules (more specific and up-to-date). So I am going to redo my answers and add them to test_lcc_classifier.py :).

cclauss · 2020-04-15T03:34:27Z

Cool but... We are using https://github.com/thisismattmiller/lcc-pdf-to-json to download our translation table. This is using outline URLs so what "schedules" URLs do we want to use instead?

A Preface   General Works (PDF, 1 p., 14 KB)
A Outline   General Works (PDF, 1 p., 86 KB)
A Text      General Works (PDF, 77 p., 193 KB)

BrittanyBunk · 2020-04-15T03:57:40Z

You're using the correct ones, the schedule outlines. I accidentally used the outline outlines without realizing they weren't the same ones. So I will correct my answers. The LoC sure has ambiguous terminology lol, but I'll get this on track.

cclauss · 2020-04-15T05:29:50Z

OK. My sense is that the parser currently has difficulties with classifier text that is wrapped across two or more lines. In some cases these should be one long string and in others they should be two separate strings. Please add some of these to our test data to ensure that our testing properly covers these cases.

BrittanyBunk · 2020-04-15T18:52:31Z

I'm having confusion. Here's the thing:
Say it's H. http://www.loc.gov/aba/publications/FreeLCC/freelcc.html will show H as "Social Sciences", but its outline http://www.loc.gov/aba/publications/FreeLCC/LCC_H2019OUT.pdf shows "Social sciences (General)". I don't know which one is used for the classifications.

cclauss · 2020-04-17T15:09:07Z

@BrittanyBunk I too noticed the discrepancy on the class names and am happy that there is only a single source for subclass names. ;-) I will leave it to you and @seabelis, etc. to determine which class names we should be using.

I hope to have time this weekend to get this back into working order so that testing can resume.

If you have the bandwidth, please keep creating test cases here or here. Thanks much...

BrittanyBunk · 2020-04-17T15:27:44Z

@cclauss I noticed more issues when going through the whole lc_classifiers_letters_and_numbers.json:

not using multiple lines,
having cutoffs, but
shortening of words (like indivual instead of individual),
because the schedules keep updating, there are actually new subclasses (like the new one above Crete)!

My recommendations:

I will create a new json file (I might need help with it) that substitutes the lc_classifiers_letters_and_numbers.json, as it probably is more effective than writing exceptions codes (i.e. the ones you listed). This sounds like a 2 day job for me - it's a lot more work than I thought (catalog/onix/onix.py attempts to use a global variable in init(), but doesn't declare it global #2-Adds BeautifulSoup as dependency during install #4 are over 500 changes alone!), but I'll update you along the way.
The discrepancy on the class names is because you use a separate file for it: lc_classifiers_letters_only.json. Let's keep the formatting simple and consistent by removing it - as we don't need it and just use the lc_classifiers_letters_and_numbers.json only!

Does this work for everybody?

cclauss · 2020-04-17T18:23:47Z

OK.

I will drop ~~lc_classifiers_letters_only.json~~
we only use lc_classifiers_letters_and_numbers.json

I worry about you spending a ton of time manually correcting the .json doc that was auto-generated. Are the changes that you plan to make from reading newer .pdf files? If not, what other sources of data do you use? If so, we should try to regenerate the .json file using #3309 (comment) .

Please show me your corrections after you have done a few letters (A, B, C) and the I can tell you if we can automate some (much!) of this. Thx.

BrittanyBunk · 2020-04-17T22:37:58Z

@cclauss I worried about it too. The new pdf files I use are https://www.loc.gov/aba/cataloging/classification/. I put them into one pdf here. They are not the ones in the comment you linked (those are different - I already made that mistake :)).

I am able to generate an entire list of all the exceptions that need to be changed - that doesn't take long. Maybe you could make the .json file from them? I'll get started on that and message when that's finished.

BrittanyBunk · 2020-04-17T22:43:01Z

There were other issues, but we'll ignore them for now (just putting for reference, but we won't worry about it):

numbers with parentheses, like AE1-(90) Encyclopedias
some go past numbers and letters to be more specific, like: DF901.C78-C88 Crete <- after the period, there's .C78-C88. So DF901 is not Crete, but with the json's, it'll show up like that.

BrittanyBunk · 2020-04-18T00:39:35Z

@cclauss The numbers refer to the ones in this comment. Exceptions lists:
1 - https://drive.google.com/file/d/1nV79v775-vQqpSYKOuL6VjcxsrdAY0-E/view?usp=sharing
2-4 - https://drive.google.com/file/d/1grA01vxftK4pdwCbg0fJ-nSkXran_zyi/view?usp=sharing

These are all the exceptions lists you need to get started. #1 would need to have its lines combined. I didn't do that - but there are ways - as I've done something similar before.

Note:
1 was done manually and 2-4 was done using computations. So 1 might have mistakes, but 2-4 shouldn't.

BrittanyBunk · 2020-04-18T17:19:20Z

Using the schedules is harder than the LCC outline, as there's times that look like this:
KDZ-KH1 752 Trade regulaton. Control of trade practices. Consumer protection Primary production. Extractive industries
The Primary production. Extractive industries part is actually the subclass for the next line subclasses after that (KDZ-KH1 765 through 810), but is not separated to where it'll pop up for them. I'm not able to place it properly into the list without rearranging everything, so I'm just going to combine it normally (as matching every class perfectly is too difficult (i.e. too much work/too messy) to do).

It's a tradeoff - either:

use the schedules with some difficulties in getting every subclass exactly right (due to the outline not aligning with the LCCs) or
get them all right with the LCC outlines but with a shortened, incomplete version.

Since we're choosing the first option (a full outline), these consequences (misalignment) will emerge.

Fortunately, it's only a few places, so it shouldn't be a big deal.

BrittanyBunk · 2021-01-22T03:41:33Z

@cclauss how come this closed, is it complete now?

WIP: Convert Library of Congress codes to genre and subgenre

fd5c200

cclauss requested review from cdrini and seabelis April 3, 2020 21:03

cclauss added 2 commits April 3, 2020 23:16

from __future__ import print_function for legacy Python

7609508

Allow the entry of OLxxxM values

981451a

Remove type hints for legacy Python

0ad6add

seabelis reviewed Apr 4, 2020

View reviewed changes

scripts/lc_classifiers_letters_only.json Outdated Show resolved Hide resolved

seabelis reviewed Apr 4, 2020

View reviewed changes

scripts/lc_classifiers_letters_only.json Outdated Show resolved Hide resolved

cclauss added 2 commits April 4, 2020 16:55

Fix PQ and PT

ada7be4

Make lc_classifiers_letters_and_numbers.json readable

5a58cf7

Add lcc_to_class_subclass.py

b57d103

tfmorris reviewed Apr 4, 2020

View reviewed changes

cclauss changed the title ~~WIP: Convert Library of Congress codes to genre and subgenre~~ WIP: Convert Library of Congress codes into a classification Apr 5, 2020

cdrini approved these changes Apr 6, 2020

View reviewed changes

lcc_to_classification() --> get_lcc_classes()

9d2d839

cclauss closed this Jan 21, 2021

cclauss deleted the lcc_to_genre_subgenre branch January 21, 2021 11:21

LeadSongDog mentioned this pull request Mar 26, 2022

Book Page: Deduplicate work description / Remove work-level classifications #6266

Merged

WIP: Convert Library of Congress codes into a classification #3309

WIP: Convert Library of Congress codes into a classification #3309

Conversation

cclauss commented Apr 3, 2020 • edited

Technical

Testing

Evidence

Stakeholders

BrittanyBunk commented Apr 4, 2020

BrittanyBunk commented Apr 4, 2020 • edited

BrittanyBunk commented Apr 4, 2020

cclauss commented Apr 4, 2020 • edited

seabelis commented Apr 4, 2020

cclauss commented Apr 4, 2020 • edited

seabelis commented Apr 4, 2020

cclauss commented Apr 4, 2020

seabelis commented Apr 4, 2020 • edited

cclauss commented Apr 4, 2020

seabelis commented Apr 4, 2020

BrittanyBunk commented Apr 4, 2020 • edited

tfmorris commented Apr 4, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cclauss commented Apr 4, 2020 • edited

cclauss commented Apr 4, 2020

BrittanyBunk commented Apr 4, 2020 • edited

BrittanyBunk commented Apr 5, 2020 • edited

BrittanyBunk commented Apr 5, 2020

cclauss commented Apr 5, 2020

seabelis commented Apr 5, 2020 • edited

BrittanyBunk commented Apr 5, 2020 • edited

BrittanyBunk commented Apr 5, 2020 • edited

seabelis commented Apr 5, 2020

BrittanyBunk commented Apr 5, 2020 • edited

cclauss commented Apr 6, 2020

seabelis commented Apr 6, 2020

BrittanyBunk commented Apr 6, 2020

cdrini left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cdrini commented Apr 6, 2020

BrittanyBunk commented Apr 6, 2020 • edited

BrittanyBunk commented Apr 15, 2020

cclauss commented Apr 15, 2020 • edited

BrittanyBunk commented Apr 15, 2020

cclauss commented Apr 15, 2020

BrittanyBunk commented Apr 15, 2020 • edited

cclauss commented Apr 17, 2020 • edited

BrittanyBunk commented Apr 17, 2020 • edited by cclauss

cclauss commented Apr 17, 2020 • edited

BrittanyBunk commented Apr 17, 2020 • edited

BrittanyBunk commented Apr 17, 2020 • edited

BrittanyBunk commented Apr 18, 2020 • edited

BrittanyBunk commented Apr 18, 2020 • edited

BrittanyBunk commented Jan 22, 2021

cclauss commented Apr 3, 2020 •

edited

BrittanyBunk commented Apr 4, 2020 •

edited

cclauss commented Apr 4, 2020 •

edited

cclauss commented Apr 4, 2020 •

edited

seabelis commented Apr 4, 2020 •

edited

BrittanyBunk commented Apr 4, 2020 •

edited

cclauss commented Apr 4, 2020 •

edited

BrittanyBunk commented Apr 4, 2020 •

edited

BrittanyBunk commented Apr 5, 2020 •

edited

seabelis commented Apr 5, 2020 •

edited

BrittanyBunk commented Apr 5, 2020 •

edited

BrittanyBunk commented Apr 5, 2020 •

edited

BrittanyBunk commented Apr 5, 2020 •

edited

BrittanyBunk commented Apr 6, 2020 •

edited

cclauss commented Apr 15, 2020 •

edited

BrittanyBunk commented Apr 15, 2020 •

edited

cclauss commented Apr 17, 2020 •

edited

BrittanyBunk commented Apr 17, 2020 •

edited by cclauss

cclauss commented Apr 17, 2020 •

edited

BrittanyBunk commented Apr 17, 2020 •

edited

BrittanyBunk commented Apr 17, 2020 •

edited

BrittanyBunk commented Apr 18, 2020 •

edited

BrittanyBunk commented Apr 18, 2020 •

edited