Update Science Museum DAG to use new API response format

[AetherUnbound]

Description

Sometime over the last few months, Science Museum changed their API response format in such a way that we're now no longer ingesting any records. This decline from ~100k records to 0 can be seen in our ingestion counts.

I captured one of the return records, it's encapsulated below:

Response body

[ins] In [6]: pprint(d)
{'attributes': {'@admin': {'added': 1699634014404,
                           'id': 'object-89774',
                           'processed': 1712780300687,
                           'sequence': 14504549,
                           'source': 'Mimsy XG',
                           'stream': 'collections-online',
                           'uid': 'co89774',
                           'uuid': '07d0045b-e62d-3c51-8c92-d12042a46f05'},
                'category': [{'museum': 'SCM',
                              'name': 'Classical & Medieval Medicine',
                              'type': 'category1',
                              'value': 'SCM - Classical & Medieval Medicine'}],
                'creation': {'@entity': 'lifecycle',
                             'date': [{'from': '151',
                                       'primary': True,
                                       'source': 'catalogue',
                                       'to': '126',
                                       'value': '151-300 CE'},
                                      {'from': '151',
                                       'role': [{'value': 'made'}],
                                       'to': '300',
                                       'value': '151-300 CE'}]},
                'cumulation': {'@entity': 'lifecycle',
                               'collector': [{'@entity': 'literal',
                                              '@link': {'source': 'catalogue'},
                                              'name': [{'value': 'Sir Henry '
                                                                 "Wellcome's "
                                                                 'Museum '
                                                                 'Collection'}],
                                              'summary': {'title': 'Sir Henry '
                                                                   "Wellcome's "
                                                                   'Museum '
                                                                   'Collection'}}]},
                'description': [{'primary': True,
                                 'type': 'catalogue description',
                                 'value': 'Glass bowl, Roman, 151 to 300 AD'},
                                {'type': 'accession register',
                                 'value': 'Glass bowl, Roman, 151 to 300 AD'}],
                'identifier': [{'primary': True,
                                'type': 'accession number',
                                'value': 'A628098'}],
                'language': [{'value': 'eng'}],
                'legal': {'credit': "Sotheby's"},
                'measurements': {'count': '1',
                                 'dimensions': [{'primary': True,
                                                 'type': 'catalogue dimensions',
                                                 'value': 'height 77 mm; '
                                                          'diameter 135 mm'},
                                                {'units': 'mm', 'value': '135'},
                                                {'units': 'mm',
                                                 'value': '77'}]},
                'multimedia': [{'@admin': {'id': 'media-365988',
                                           'source': 'smgi',
                                           'uid': 'i365988',
                                           'uuid': '6a144dfe-c861-320a-a8ed-ff86e70528af'},
                                '@entity': 'reference',
                                '@processed': {'large': {'@type': 'image',
                                                         'format': 'jpeg',
                                                         'location': '365/988/large_smg00234816.jpg',
                                                         'measurements': {'dimensions': [{'dimension': 'height',
                                                                                          'units': 'pixels',
                                                                                          'value': 1387},
                                                                                         {'dimension': 'width',
                                                                                          'units': 'pixels',
                                                                                          'value': 1536}],
                                                                          'filesize': {'units': 'bytes',
                                                                                       'value': 90100}},
                                                         'modified': 1697176882000,
                                                         'resizable': True},
                                               'large_thumbnail': {'@type': 'image',
                                                                   'format': 'jpeg',
                                                                   'location': 'https://coimages.sciencemuseumgroup.org.uk/365/988/large_thumbnail_smg00234816.jpg',
                                                                   'measurements': {'dimensions': [{'dimension': 'height',
                                                                                                    'units': 'pixels',
                                                                                                    'value': 289},
                                                                                                   {'dimension': 'width',
                                                                                                    'units': 'pixels',
                                                                                                    'value': 320}],
                                                                                    'filesize': {'units': 'bytes',
                                                                                                 'value': 12515}},
                                                                   'modified': 1697176882000,
                                                                   'resizable': True},
                                               'medium': {'@type': 'image',
                                                          'format': 'jpeg',
                                                          'location': '365/988/medium_smg00234816.jpg',
                                                          'measurements': {'dimensions': [{'dimension': 'height',
                                                                                           'units': 'pixels',
                                                                                           'value': 576},
                                                                                          {'dimension': 'width',
                                                                                           'units': 'pixels',
                                                                                           'value': 638}],
                                                                           'filesize': {'units': 'bytes',
                                                                                        'value': 25419}},
                                                          'modified': 1697176882000,
                                                          'resizable': True},
                                               'medium_thumbnail': {'@type': 'image',
                                                                    'format': 'jpeg',
                                                                    'location': '365/988/medium_thumbnail_smg00234816.jpg',
                                                                    'measurements': {'dimensions': [{'dimension': 'height',
                                                                                                     'units': 'pixels',
                                                                                                     'value': 173},
                                                                                                    {'dimension': 'width',
                                                                                                     'units': 'pixels',
                                                                                                     'value': 192}],
                                                                                     'filesize': {'units': 'bytes',
                                                                                                  'value': 9239}},
                                                                    'modified': 1697176882000,
                                                                    'resizable': True},
                                               'small_thumbnail': {'@type': 'image',
                                                                   'format': 'jpeg',
                                                                   'location': '365/988/small_thumbnail_smg00234816.jpg',
                                                                   'measurements': {'dimensions': [{'dimension': 'height',
                                                                                                    'units': 'pixels',
                                                                                                    'value': 48},
                                                                                                   {'dimension': 'width',
                                                                                                    'units': 'pixels',
                                                                                                    'value': 53}],
                                                                                    'filesize': {'units': 'bytes',
                                                                                                 'value': 7113}},
                                                                   'modified': 1697176882000,
                                                                   'resizable': True},
                                               'zoom': {'@type': 'image',
                                                        'format': 'pyramid '
                                                                  'tiff',
                                                        'location': '365/988/Glass_bowl__Roman__151_to_300_AD.ptif',
                                                        'measurements': {'dimensions': [{'dimension': 'height',
                                                                                         'units': 'pixels',
                                                                                         'value': 3851},
                                                                                        {'dimension': 'width',
                                                                                         'units': 'pixels',
                                                                                         'value': 4264}],
                                                                         'filesize': {'units': 'bytes',
                                                                                      'value': 32569396}},
                                                        'modified': 1697176882000,
                                                        'resizable': True}},
                                '@type': 'image',
                                'credit': {'value': 'Science Museum Group'},
                                'legal': {'rights': [{'copyright': '© The '
                                                                   'Board of '
                                                                   'Trustees '
                                                                   'of the '
                                                                   'Science '
                                                                   'Museum',
                                                      'licence': 'CC BY-NC-SA '
                                                                 '4.0'}]},
                                'title': [{'type': 'caption',
                                           'value': 'Glass bowl, Roman, 151 to '
                                                    '300 AD'}],
                                'upload_sort': '21/04/2019'}],
                'name': [{'primary': True,
                          'type': 'catalogue name',
                          'value': 'bowl - vessel'},
                         {'type': 'object name', 'value': 'bowl - vessel'}],
                'summary': {'title': 'Glass bowl, Roman, 151 to 300 AD (bowl - '
                                     'vessel)'},
                'title': [{'primary': True,
                           'type': 'catalogue title',
                           'value': 'Glass bowl, Roman, 151 to 300 AD'},
                          {'type': 'display title',
                           'value': 'Glass bowl, Roman, 151 to 300 AD'}]},
 'id': 'co89774',
 'links': {'self': 'https://collection.sciencemuseumgroup.org.uk/objects/co89774/glass-bowl-roman-151-to-300-ad-bowl-vessel'},
 'relationships': {},
 'type': 'objects'}

We'll need to update the script, and all of the testing resources as well.

Screenshots

Additional context

Found this while attempting to debug #4013