I get the following error messages when I try to make it:

python3 scripts/externallinks_placeholder.py data/data-papers.yaml data/data-papers-routeviews.yaml
    loading data/data-papers.yaml
    loading data/data-papers-routeviews.yaml
    found 69 duplicates
unparseable "" in "Citadels in cyberspace"
unparseable "" in "CAIDA Macroscopic IP Topology Data Kit (ITDK) #0204 provided to the Network Modeling and Simulation (NMS) community under DARPA grant N66001-01-1-8909"
unparseable "" in "The Internet Under Crisis Conditions: Learning from September 11"
unparseable "" in "ISMA Winter 2000 Workshop - Final Report"
make[2]: *** No rule to make target `routerviews', needed by `run'.  Stop.
make[1]: *** [fast] Error 2
make: *** [readable] Error 2

I believe this is a typo in the Makefile. The routerviews target is no longer needed and has been removed. Should I also fix the "unparseable" errors above? They are a result of missing authors/information for these papers on routeviews.org where I scraped from to generate the routeviews yaml.

It looks like there's no way to fix the remaining unparseable errors without editing the data-papers-routeviews.yaml that gets generated by scripts/external_routeviews_parse.py.

For example, the first unparseable paper has authors Rawat; Madhur; Chakravarty; Sambuddho on the routeviews website. This paper looks like it has 4 authors, and there is no way to systematically recognize that it actually has two authors without searching online and manually changing the yaml. Similarly, another has author Unknown. However, the routeviews yaml should not be committed (#553). What should I do?

Make a list of papers that are broken. We will email routeviews and have them fix it on their end.

python3 scripts/externallinks_placeholder.py data/data-papers.yaml data/data-papers-routeviews.yaml
    loading data/data-papers.yaml
    loading data/data-papers-routeviews.yaml
    found 69 duplicates
unparseable "" in "Citadels in cyberspace"
unparseable "" in "CAIDA Macroscopic IP Topology Data Kit (ITDK) #0204 provided to the Network Modeling and Simulation (NMS) community under DARPA grant N66001-01-1-8909"
unparseable "" in "The Internet Under Crisis Conditions: Learning from September 11"
unparseable "" in "ISMA Winter 2000 Workshop - Final Report"

Of these unparseable papers, these two should be corrected if possible:
(1) "Citadels in cyberspace": badly formatted authors (see prev comment)
(2) "The Internet Under Crisis Conditions: Learning from September 11": author is "Unknown"

Both of the remaining unparseable papers has author "CAIDA". I fixed the issues with "CAIDA" author by fixing my script that generates the routeviews yaml. The external placeholder script is able to handle single-name authors just fine.

Happy Fourth of July!

The caida paper should match against papers generated from pubdb. You will need to change the Makefile, so that it generates the papers from pudb before your code is called. It will then need to check the files generated in sources/papers and not ignore duplicates.

What do you mean by not ignore duplicates? What should I do if there is a duplicate paper? Would this be similar to merging duplicates like we discussed before (take union of keys)?

Actually, just skip those papers for now.

Ok, the "CAIDA" author papers should be skipped now. There should only be two unparseable papers.

(Edit: it is not showing up here, but I did push to the 553 branch )

They are:

• CAIDA Macroscopic IP Topology Data Kit (ITDK)...
• ISMA Winter 2000 Workshop - Final Report

do you mean you don't know what to map them too?

• CAIDA Macroscopic IP Topology Data Kit (ITDK)...

• https://catalog.caida.org/dataset/ark_itdk

• ISMA Winter 2000 Workshop - Final Report

• https://catalog.caida.org/paper/2000_isma0012

I think I am misunderstanding your previous comments. This is how I interpreted our discussion:

• Initially, my script for generating routeviews yaml did not produce the correct author for the two papers mentioned in my previous comment which results in "unparseable..." for these papers
• I fixed this bug in an earlier commit (commit A)
• In a previous comment, you mentioned that we should skip these papers for now
• I made another change (commit B) to skip over these papers, they should no longer be in included in the generated yaml

With commit A: The two papers are included in the routeviews yaml. Then, they will be parsed correctly by the placeholder script and placeholder objects will be created for them.

With commit B: These papers are effectively ignored.

Is there something else missing that should done for this issue?

Please remove debugging error messages:

Matching up Papers with media/presentations with the same name
tag:slides
tag:slides
tag:slides
tag:slides
tag:slides

Resolve error messages:

python3 scripts/externallinks_placeholder.py data/data-papers.yaml data/data-papers-routeviews.yaml
    loading data/data-papers.yaml
    loading data/data-papers-routeviews.yaml
    found 69 duplicates
unparseable "" in "Citadels in cyberspace"
unparseable "" in "The Internet Under Crisis Conditions: Learning from September 11"