flickr-commons-metadata

flickr-commons-metadata contains the raw metadata for Flickr Commons photos in JSON format pulled from the Flickr API using the following methods:

• flickr.photos.getInfo
• flickr.photos.getSizes
• flickr.photos.comments.getList
• flickr.photos.getAllContexts
• flickr.photos.getExif – if publicly available

For example:

$>ls -al data/83979593@N00/224/682/902/*
-rw-r--r--  1 asc  staff    53 Dec 20 15:49 83979593@N00/224/682/902/224682902-c.json
-rw-r--r--  1 asc  staff   255 Dec 20 15:49 83979593@N00/224/682/902/224682902-ctx.json
-rw-r--r--  1 asc  staff  7952 Dec 20 15:49 83979593@N00/224/682/902/224682902-e.json
-rw-r--r--  1 asc  staff  2273 Dec 20 15:49 83979593@N00/224/682/902/224682902-i.json
-rw-r--r--  1 asc  staff  2060 Dec 20 15:49 83979593@N00/224/682/902/224682902-s.json

As of mid-morning December 21 (2013) metadata files contain a flarchive:created attribute which is a Unix timestamp indicating when the file was last archived.

For example:

$> cat 6260452122-i.json | python -mjson.tool
{
    "flarchive:created": 1387636691,
    "photo": {
    	"comments": {
         "_content": "0"
    	},
    ... and so on
}

That means as of mid-morning December 21 (2013) a bunch of metadata files archived before this date are still missing the flarchive:created attribute.

Details

Individual institutions are archived using the py-flarchive library, like this:

$> python ./flarchive/__init__.py -v -a <APIKEY> -n <NSID> -d flickr-commons-metadata/data

This is a massive repository and it's not at all clear that the way the data has been organized is the best way to do things. This is a first pass just to get the data and once that's done some time spent re-thinking how and where the data is stored is in order.

Institutions

In process

The following institutions are currently being archived by one or more people

• nothing right now

If you are archiving one or more institutions below add them here and send me a pull request or open an issue and I will add it to the list.

Complete list

• Australian National Maritime Museum on The Commons - last archived 20131223
• Australian War Memorial collection – last archived 20131228
• Bergen Public Library – last archived 20131224
• bibliothequedetoulouse – last archived 20131228
• Biblioteca de Arte-Fundação Calouste Gulbenkian – last archived 20131222
• Brooklyn Museum – last archived 20131220
• California Historical Society Digital Collection – last archived 20131222
• Center for Jewish History, NYC – last archived 20140102
• Cornell University Library – last archived 20140113
• Costică Acsinte Archive – last archived 20131223
• DC Public Library Commons – last archived 201403
• DEXTRA Photo – last archived 20131228
• Deseronto Archives – last archived 20140108
• Dundas Museum and Archiveslast archived 20131228
• Fylkesarkivet i Sogn og Fjordane – last archived 20140102
• Galt Museum & Archives on The Commons – last archived 20131228
• George Eastman House – last archived 20131222
• Getty Research Institute – last archived 20131228
• Het Nieuwe Instituut - Architecture Collection – last archived 20131221
• IWM Collections – last archived 20140102
• JWA Commons – last archived 20131228
• Jewish Historical Society of the Upper Midwest – last archived 20131229
• Keene and Cheshire County (NH) Historical Photos – last archived 20140113
• LSE Library – last archived 20131223
• Law Society of Upper Canada Archives – last archived 20131223
• Ljósmyndasafn Reykjavíkur / Reykjavík Museum of – last archived 20131224
• LlGC ~ NLW – last archived 20140108
• Mennonite Church USA Archives – last archived 20140113
• Miami University Libraries - Digital Collections – last archived 20140311
• Mississippi Department of Archives and History – partially archived as of 20131223
• Musée McCord Museum – last archived 20140411
• Museum of Hartlepool – last archived 20131224
• Museum of Photographic Arts Collections – last archived 20140102
• NASA on The Commons – last archived 20131221
• Nationaal Archief – last archived 20131222
• National Archives of Estonia – last archived 20131222
• National Galleries of Scotland Commons – last archived 20131222
• National Library NZ on The Commons – last archived 20131220
• National Library of Australia Commons – last archived 20140102
• National Library of Ireland on The Commons – last archived 20131224
• National Library of Norway – last archived 20131223
• National Library of Scotland – last archived 20131222
• National Library of Sweden – last archived 20131223
• National Maritime Museum – last archived 20131222
• National Media Museum – last archived 20140102
• National Museum of Denmark – last archived 20131222
• New York Public Library – last archived 20131221
• nha.library – last archived 20131228
• Nova Scotia Archives – last archived 20131228
• OSU Special Collections & Archives : Commons – last archived 20140113
• Powerhouse Museum Collection – last archived 20131220
• Provincial Archives of Alberta – last archived 20140311
• Public Record Office of Northern Ireland – last archived 20131224
• The British Library - last archived 20140316
• Nationaal Archief – last archived 20131222
• bibliothequedetoulouse – last archived 20131228
• California Historical Society Digital Collection – last archived 20131222
• Law Society of Upper Canada Archives – last archived 20131223
• Vancouver Public Library Historical Photographs – last archived 20131223
• The Library of Virginia – last archived 20140102
• Riksarkivet (National Archives of Norway) – last archived 20131223
• Royal Australian Historical Society – last archived 20131228
• SMU Central University Libraries – last archived 20140113
• San Diego Air & Space Museum Archives – last archived 20140115
• Schlesinger Library, RIAS, Harvard University – last archived 20140102
• Smithsonian Institution – last archived 20131220
• State Library and Archives of Florida – last archived 20131222
• State Library of New South Wales collection - last archived 20140116
• State Library of Queensland, Australia – last archived 20140113
• Biblioteca de Arte-Fundação Calouste Gulbenkian – last archived 20131222
• The Library of Congress – last archived 20131221
• Royal Australian Historical Society – last archived 20131228
• DC Public Library Commons – last archived 20140308
• Fylkesarkivet i Sogn og Fjordane – last archived 20140102
• New York Public Library – last archived 20131221
• Riksarkivet (National Archives of Norway) – last archived 20131223
• Australian War Memorial collection – last archived 20131228
• State Records NSW – last archived 20140113
• Stockholm Transport Museum Commons – last archived 20131221
• Svenska litteratursällskapet i Finland – last archived 20131221
• Swedish National Heritage Board – last archived 20131223
• Texas State Archives – last archived 20140110
• The Field Museum Library – last archived 20140110
• The Library of Congress – last archived 20131221
• The Library of Virginia – last archived 20140102
• The National Archives UK – last archived 20131224
• The Royal Library, Denmark – last archived 20140411
• The U.S. National Archives – last archived 20131222
• Tyne & Wear Archives & Museums - last archived 20140116
• UA Archives | Upper Arlington History – last archived 20131228
• UL Digital Library – last archived 20131229
• UW Digital Collections – last archived 20140102
• Vancouver Public Library Historical Photographs – last archived 20131223
• Woodrow Wilson Presidential Library Archives – last archived 20140102
• Tasmanian Archive and Heritage Office – last archived 20140111
• Library Company of Philadelphia – last archived 20140318
• Finnish Museum of Photography – last archived 20140411

Contributors

• @straup
• @hugovk

See also

• http://inkdroid.org/journal/2014/04/07/glass-houses/

• http://inkdroid.org/journal/2014/04/08/where-brooklyn-at/

• http://inkdroid.org/journal/2014/04/11/flickr-commons-lams/

• py-flarchive