·
49 commits
to master
since this release
New sources
- NASA/ESA James Webb Space Telescope: https://webbtelescope.org & https://esawebb.org
- NASA ASTER (Advanced Spaceborne Thermal Emission and Reflection Radiometer instrument abord Terra satellite): https://asterweb.jpl.nasa.gov
- NASA JPL Photojournal: https://photojournal.jpl.nasa.gov
- NASA SDO (Solar Dynamics Observatory): https://sdo.gsfc.nasa.gov
- NOIRLab: https://noirlab.edu
- Individuals on Flickr: Judy Schmidt, Kevin Gill, Pierre Markuse, Andrea Luck, Harry Stranger
- New US military units; merged all of them in a single "US Space Force/Command" source
- Enable automatic upload for:
- US military (DVIDS & Flickr), DLR, KARI, James Webb Space Telescope (ESA & NASA): all files
- NASA, IAU, ESO, NoirLab, Hubble (ESA & NASA), Individuals : only files published after 2022 for now
- Enable manual upload of other sources
Major features
- Tweet bot activity on Mastodon and Twitter
- Add SDC (Structured Data on Commons) for uploaded files
- Start a complete calculation of perceptual hashes on the whole Commons database to detect duplicates (still ongoing as of May 2023...)
- Report exact duplicate files to Commons administrators by parsing Special:ListDuplicatedFiles (up to 190 files max to avoid spamming them too much)
- Translate non-English text using Google Translate
- Support WebP images files
- Blocklist of terms implying uninteresting content published by US military and NASA
- New remote capabilities to compute hashes on distant computers more powerful than toolforge pods
- Switch spring schedules to toolforge jobs framework and Cloud VPS cronjobs
- Support extracting information from Wikidata (ISS crew members, astronomical objects, telescopes, instruments...)
- Detect and ignore courtesy photos in media published by US military and NASA (from ULA, SpaceX, Lockheed Martin...) using a blocklist of terms in media description and a blocklist of photographers in EXIF metadata
- Look up for (NASA) images on Commons by their id to avoid upload not-exact duplicates and upload high-resolution version if needed
Minor features
- Allow to manually refresh a media by reassessing all its metadata
- Display a video/audio icons above preview images
- New REST endpoint to return commons last timestamp
- New REST enpoint to put a new hash association
- NASA: Extract metadata for ISS and Artemis images
- Initial support of upload in chunks for very large files exceeding memory on Wikimedia servers. Does not work for now :(
Behind the scene
- Update to Java 17 and Spring Boot 2.7
- Update to latest versions of Mediawiki
- Update to MariaDB 10.4: https://phabricator.wikimedia.org/T301949
- Update to breaking changes on hubblesite.org website
- Update to breaking change on DVIDS CDN
- Migrate from Phabricator Diffusion to Wikimedia GitLab
- Migrate from eqiad.wmflabs to wikimedia.cloud
- Hubble and Webb NASA websites handled as a single "STScI" repository
- IAU, ESO, NoirLab websites handled as "Djangoplicity" repositories
- Use JPEG plugin from twelve-monkeys in order to read more files
- Disabled video support on toolforge, requires too much memory
- As usual, lots of general performance/reliability improvements and dependencies upgrades
Full Changelog: spacemedia-0.4.0...spacemedia-0.4.1