References persist after removal from source file

[ghost]

Hey, thanks for making this!
I just found this issue with 0.6.6. I can't easily upgrade to 0.6.7 right now, but thought I'd raise this in case it's still present

Issue

References are added to the output file even after removal from the source file

Steps to Reproduce

1. save example from README

# Introduction

The GAN was first introduced in [@gan].

# References

[@gan]: https://papers.nips.cc/paper/5423-generative-adversarial-nets

2. run pandoc

$ pandoc --filter=pandoc-url2cite --citeproc --csl ieee-with-url.csl minimal.md -o out.pdf

3. add a reference

# Introduction

The GAN was first introduced in [@gan].

Information about corn prices.[@cornprice]

# References

[@gan]: https://papers.nips.cc/paper/5423-generative-adversarial-nets
[@cornprice]: http://dx.doi.org/10.1093/ajae/aaq063

4. run pandoc again

5. remove a reference (markdown file is the same as step 1)

6. run pandoc again

The removed reference is still in the references list

Expected Behaviour

References that have been completely removed from the source file should not be written to the output file

Software

$ pandoc-url2cite --version
pandoc-url2cite v0.6.6
Don't run this directly, use it as a pandoc --filter=. Usage: https://github.com/phiresky/pandoc-url2cite
$ pandoc --version
pandoc 2.13
Compiled with pandoc-types 1.22, texmath 0.12.2, skylighting 0.10.5,
citeproc 0.3.0.9, ipynb 0.1
User data directory: /home/user/.local/share/pandoc
Copyright (C) 2006-2021 John MacFarlane. Web:  https://pandoc.org
This is free software; see the source for copying conditions. There is no
warranty, not even for merchantability or fitness for a particular purpose.
$ uname -r
5.10.27-gentoo

[phiresky]

I've noticed this as well. I'm not sure why this happens, if I remember correctly when I initially wrote this pandoc citeproc did not output unused citations, so pandoc-url2cite intentionally just outputs all citations it knows and relies on citeprocs filtering.

So this might be a regression in citeproc, especially since there's specific documentation about a nocite field in the documentation: https://pandoc.org/MANUAL.html#including-uncited-items-in-the-bibliography . Or it might be something different. Next step would probably be to see if this happens without url2cite with a minimal example with just pandoc --citeproc (input a .bib file and a .md file). You can try that or I'll try it when I have time.

0.6.7 doesn't change anything about this.

[phiresky]

I've opened an upstream issue: jgm/pandoc#7388

[ghost]

Tested and working in pandoc 2.14.0.2