Rework provisional standard media type loading (#53)

Instead of downloading, parsing, and saving in one pass, make this a three-pass operation. 1. Download and parse from the IANA media types registry and the provisional media types registry. 2. Merge the provisional media types into the appropriate media types. 3. Save the primary media type files. (Remove "examples", which had previously been skipped, and the provisional registry).
mime-types · Feb 18, 2023 · f439d00 · f439d00
1 parent b97a499
commit f439d00
Show file tree

Hide file tree

Showing 26 changed files with 1,736 additions and 346 deletions.
diff --git a/.standard.yml b/.standard.yml
@@ -1,4 +1,6 @@
 parallel: true
-ruby_version: 2.0
+ruby_version: 2.3
 ignore:
   - '*.gemspec'
+  - Rakefile:
+    - Layout/HeredocIndentation
diff --git a/Contributing.md b/Contributing.md
@@ -204,6 +204,7 @@ Thanks to everyone else who has contributed to mime-types:
 - Bradley Meck
 - Brandon Galbraith
 - Chris Gat
+- Chris Salzberg
 - David Genord
 - Eric Marden
 - Garret Alfert
@@ -222,6 +223,7 @@ Thanks to everyone else who has contributed to mime-types:
 - Lucia
 - Martin d'Allens
 - Mauricio Linhares
+- Mohammed Gad
 - Myk Klemme
 - nycvotes-dev
 - Postmodern

diff --git a/History.md b/History.md
@@ -2,29 +2,49 @@
 
 <!-- automatic-release -->
 
+## 3.2023.0220 / 2023-02-18
+
+- Updated the Apache and IANA media registry entries as of release date.
+
+- Mohammed Gad added the `jfif` file extension for `image/jpeg` text format.
+  [#52][]
+
+- Reworked the loading of IANA provisional media registries to merge them
+  into the top-level media-type registries instead of a standalone registry
+  file. [#53][] originally identified by Chris Salzberg in [#50][].
+
+  It is worth noting that this is an _imperfect_ solution as if a media type
+  is provisionally registered and withdrawn, it will linger in the registry
+  with no clean way of identifying them at the moment. See [#54][].
+
+  This release also fixes [ruby-mime-types#163][], where logs show "Type
+  application/netcdf is already registered as a variant of
+  application/netcdf".
+
 ## 3.2022.0105 / 2022-01-05
 
 - Updated the Apache and IANA media registry entries as of release date.
 
-- Fixed an incorrect definition of `image/bmp`, which had been marked obsolete
-  and later registered. Fixed [#48], found by William T. Nelson.
+- Fixed an incorrect definition of `image/bmp`, which had been marked
+  obsolete and later registered. Fixed [#48][], found by William T. Nelson.
 
 ## 3.2021.1115 / 2021-11-15
 
 - Updated the Apache and IANA media registry entries as of release date.
 
 - Added conversion utilities that support the `mini_mime` data format. These
-  have been ported from the [mini_mime] repository. [#47]
+  have been ported from the [mini_mime][] repository. [#47][]
 
-- Added IANA provisional media registries. Added some notes to Contributing.md
-  about the transient nature of the provisional registration data. This was
-  triggered in part by a pull request by Jon Sneyers. Thanks! [#45], [#43]
+- Added IANA provisional media registries. Added some notes to
+  Contributing.md about the transient nature of the provisional registration
+  data. This was triggered in part by a pull request by Jon Sneyers. Thanks!
+  [#45][], [#43][]
 
 ## 3.2021.0901 / 2021-09-01
 
 - Updated the Apache and IANA media registry entries as of release date.
 
-- Added file extension for WebVTT text format. [#46]
+- Added file extension for WebVTT text format. [#46][]
 
 ## 3.2021.0704 / 2021-07-04
 
@@ -34,82 +54,82 @@
 
 - Updated the Apache and IANA media registry entries as of release date.
 
-- Added file extension for AVIF video format. [#40]
+- Added file extension for AVIF video format. [#40][]
 
 ## 3.2021.0212 / 2021-02-12
 
 - Updated the IANA media registry entries as of release date.
 
-- Added a new rake task (`release:automatic`) that downloads and converts the
-  data from Apache and IANA; if there are changes detected, it updates the
-  release version, changelog, manifest, and gemspec and commits the changes
-  to git.
+- Added a new rake task (`release:automatic`) that downloads and converts
+  the data from Apache and IANA; if there are changes detected, it updates
+  the release version, changelog, manifest, and gemspec and commits the
+  changes to git.
 
 ## 3.2020.1104 / 2020-11-04
 
 - Updated the IANA media registry entries as of release date.
 
-- Added `application/x-zip-compressed`. [#36]
+- Added `application/x-zip-compressed`. [#36][]
 
 - Updated the contributing guide to include information about the release
-  process as described in [#18]
+  process as described in [#18][]
 
-- Corrected a misspelling of Yoran Brondsema’s name. Sorry, Yoran. [#35]
+- Corrected a misspelling of Yoran Brondsema’s name. Sorry, Yoran. [#35][]
 
 ## 3.2020.0512 / 2020-05-12
 
 - Updated the IANA media registry entries as of release date.
 
-- Added file extensions for HEIC image types. [#34]
+- Added file extensions for HEIC image types. [#34][]
 
 ## 3.2020.0425 / 2020-04-25
 
 - Updated the IANA media registry entries as of release date.
 
 - Added several RAW image types based on data from GNOME RAW Thumbnailer.
-  [#33] fixing [#32]
+  [#33][] fixing [#32][]
 
-- Added `audio/wav`. [#31]
+- Added `audio/wav`. [#31][]
 
-- Added a type for Smarttech notebook files. [#30]
+- Added a type for Smarttech notebook files. [#30][]
 
-- Added an alias for audio/m4a files. [#29]
+- Added an alias for audio/m4a files. [#29][]
 
-- Added application/x-ms-dos-executable. [#28]
+- Added application/x-ms-dos-executable. [#28][]
 
 ## 3.2019.1009 / 2019-10-09
 
 - Updated the IANA media registry entries as of release date.
 
-- Reordered the `.ai` extension so that it is not the preferred extension for
-  `application/pdf` [#24]
+- Reordered the `.ai` extension so that it is not the preferred extension
+  for `application/pdf` [#24][]
 
 ## 3.2019.0904 / 2019-09-04
 
 - Updated the IANA media registry entries as of release date.
 
-- Moved the `.ai` extension from `application/postscript` to `application/pdf`.
-  [#23] fixing [#22]
+- Moved the `.ai` extension from `application/postscript` to
+  `application/pdf`. [#23][] fixing [#22][]
 
 ## 3.2019.0331 / 2019-03-31
 
 - Updated the IANA media registry entries as of release date.
 
-- Added support for `application/wasm` with extension `.wasm`. [#21]
+- Added support for `application/wasm` with extension `.wasm`. [#21][]
 
-- Fixed `application/ecmascript` extensions. [#20]
+- Fixed `application/ecmascript` extensions. [#20][]
 
 ## 3.2018.0812 / 2018-08-12
 
-- Added `.xsd` extension to `text/xml`. [#10]
+- Added `.xsd` extension to `text/xml`. [#10][]
 
 - Added `.js` and `.mjs` extensions to `text/ecmascript` and
-  `text/javascript`. [#11]
+  `text/javascript`. [#11][]
 
-- Added `.ipa` extension to `application/octet-stream`. [#12]
+- Added `.ipa` extension to `application/octet-stream`. [#12][]
 
 - Moved extensions `.markdown` and `.md` and added `.mkd` extension to
-  `text/markdown`. [#13]
+  `text/markdown`. [#13][]
 
 - Because of a bug found with mime-types 3 before 3.2.1, this version
   requires mime-types 3.1 or later to manage data.
@@ -182,7 +202,8 @@
 
 - Extracted from [ruby-mime-types][rmt].
 - Added a [Code of Conduct].
-- The versioning has changed to be semantic on format plus date in two parts.
+- The versioning has changed to be semantic on format plus date in two
+  parts.
 
   - All registry formats have been updated to remove deprecated data.
   - The columnar format has been updated to store three boolean flags in a
@@ -224,8 +245,7 @@
 ## 2.6 / 2015-05-25
 
 - Steven Michael Thomas (@stevenmichaelthomas) added `woff2` as an extension
-  to application/font-woff,
-  [ruby-mime-types#99](https://github.com/mime-types/ruby-mime-types/pull/99).
+  to application/font-woff, [ruby-mime-types#99][].
 - Updated the IANA media registry entries as of release date:
   - Updated metadata for application/jose, application/jose+json,
     application/jwk+json, application/jwk-set+json, application/jwt to
@@ -497,6 +517,12 @@
 [#46]: https://github.com/mime-types/mime-types-data/pull/46
 [#47]: https://github.com/mime-types/mime-types-data/pull/47
 [#48]: https://github.com/mime-types/mime-types-data/issues/48
+[#50]: https://github.com/mime-types/mime-types-data/pull/50
+[#52]: https://github.com/mime-types/mime-types-data/pull/52
+[#53]: https://github.com/mime-types/mime-types-data/pull/53
+[#54]: https://github.com/mime-types/mime-types-data/issues/54
 [rmt]: https://github.com/mime-types/ruby-mime-types
 [code of conduct]: Code-of-Conduct.md
 [mini_mime]: https://github.com/discourse/mini_mime/issues/41
+[ruby-mime-types#99]: https://github.com/mime-types/ruby-mime-types/pull/99
+[ruby-mime-types#163]: https://github.com/mime-types/ruby-mime-types/issues/163
diff --git a/Rakefile b/Rakefile
@@ -23,7 +23,7 @@ Hoe.spec "mime-types-data" do
   extra_dev_deps << ["nokogiri", "~> 1.6"]
   extra_dev_deps << ["hoe-doofus", "~> 1.0"]
   extra_dev_deps << ["hoe-gemspec2", "~> 1.1"]
-  extra_dev_deps << ["hoe-git", "~> 1.6"]
+  extra_dev_deps << ["hoe-git2", "~> 1.7"]
   extra_dev_deps << ["hoe-rubygems", "~> 1.0"]
   extra_dev_deps << ["rake", ">= 10.0", "< 14"]
   extra_dev_deps << ["mime-types", ">= 3.4.0", "< 4"]
@@ -120,7 +120,7 @@ namespace :update do
   desc "Update the release version"
   task :version do
     file = IO.read("lib/mime/types/data.rb")
-    updated = file.sub(/VERSION = ['"][.0-9]+['"]/, %Q(VERSION = "#{new_version}"))
+    updated = file.sub(/VERSION = ['"][.0-9]+['"]/, %(VERSION = "#{new_version}"))
 
     IO.write("lib/mime/types/data.rb", updated)
   end

diff --git a/data/content_type_mime.db b/data/content_type_mime.db
@@ -787,6 +787,7 @@ ics         text/calendar
 css         text/css                                                                  8bit            
 csv         text/csv                                                                  8bit            
 html        text/html                                                                 8bit            
+js          text/javascript                                                           quoted-printable
 markdown    text/markdown                                                             quoted-printable
 n3          text/n3                                                                   quoted-printable
 txt         text/plain                                                                quoted-printable

diff --git a/data/ext_mime.db b/data/ext_mime.db
@@ -422,7 +422,7 @@ jpgm        image/jpm
 jpgv        video/JPEG                                                                base64          
 jpm         image/jpm                                                                 base64          
 jpx         image/jpx                                                                 base64          
-js          application/javascript                                                    8bit            
+js          text/javascript                                                           quoted-printable
 json        application/json                                                          8bit            
 jsonml      application/jsonml+json                                                   base64          
 k25         image/x-kodak-k25                                                         base64          
@@ -521,7 +521,7 @@ mj2         video/MJ2
 mjp2        video/MJ2                                                                 base64          
 mjpeg       video/x-motion-jpeg                                                       base64          
 mjpg        video/x-motion-jpeg                                                       base64          
-mjs         application/javascript                                                    8bit            
+mjs         text/javascript                                                           quoted-printable
 mk3d        video/x-matroska                                                          base64          
 mka         audio/x-matroska                                                          base64          
 mkd         text/markdown                                                             quoted-printable

diff --git a/data/mime-types.json b/data/mime-types.json