[WIP] [deliver][produce][spaceship] Update to use new App Store Conne…

…ct API endpoints (#16640)

* [deliver][produce][spaceship] Update to use new App Store Connect API endpoints (#16626)

* Initial commit

* Fixed ensuring version at start of deliver and selecting build at the end

* Added error message for submission not working yet

* Big fix for screenshots and fix for erroring out when not able to activate language

* Removal of unused things and fix for explicitly submitting build

* Fix for circular reference

* Fix for first screenshot

* content_rights_contains_third_party_content works for submission information

* Updated comment

* Quick fix

* This should fix all bad state issues

* Removed UI from spaceship

* fixed submit stuff

* Updating categories works holy cow

* I think all meta data is working

* Whoops. had submit for review commented out

* Allowed rejected apps to be edit and only send up whatsNew if live version

* rubocop is happy

* So many fixes

* Fixed validate doc and lint

* Fixed and commented out some tests

* Added in age rating

* ruocoped

* Adding back in IDFA because i dont know where it went

* Fix produce for app store connect

* Removed some pp

* One extra line

* Update to ConnectAPI usage

Adding authorization and retrieving list of apps / find specific app.

* [2.150.0][deliver] better error when uses non exempt encryption not set (#16646)

* [deliver] fix for better error on uses non exempt encryption

* New error messages and fixes of wrong things being set

* [2.150.0][deliver] fix category deleting when not specified (#16652)

* [deliver] fix category from deleting when not specified

* Warn users of deprecated mapped categories

* Updated some doc

* [deliver] warn about mapped/deprecated age rating values (#16655)

* [deliver] warn about mapped/deprecated age rating values

* Full in on https

* Lots of fixes whoops

Co-authored-by: Max Ott <max.ott@me.com>

* [deliver][produce] map language names and available countries on app (#16656)

* [deliver][produce] map language names and available countries on app

* Fix price match and remove comment

* [deliver] allow default language for metadata (#16657)

* [action] fix set_changelog (#16658)

* [deliver][spaceship] multi-thread delete screenshots and retry on 504 (#16667)

* [deliver][spaceship] multi-thread delete screenshots and retry on 504

* Improved error handling

* Fix tests

* [spaceship] add app preview (#16671)

* [spaceship] add app preview

* Added new preview devices

* [spaceship] increase timeout for web session connect api to 1200 (#16676)

* Version bump 2.150.0.rc2

* Fix to make produce create on ASC work again

* [spaceship] waiting_for_review is editable (ish) (#16680)

* [spaceship] waiting_for_review is editable (ish)

* Added for app store version too

* [spaceship] retry file uploader and poll app preview to set frame (#16684)

* [spaceship] retry file uploader and poll app preview to set frame

* Some doc cleanup

* Handle fail logic if patch retries on file upload and fix mapping of age category

* Raise error if all failed

* [spaceship][fastlane_core] run itmstransporter with xcrun if xcode 11 or up (#16689)

* [deliver][spaceship] fix 500 with appInfos and skip already uploaded screenshots with deliver (#16694)

* Version bump to 2.150.0.rc4

* [produce][spaceship] add users to app when created (#16695)

* [produce][spaceship] add users to app when created

* Also add company name

* Updated docs on how to use submission (#16706)

* [deliver][spaceship] wait for screenshots to be completed after processing and download properly formatted screenshots and error on processing error (#16709)

* [deliver][spaceship] choose highest edit version, retry on 500, submit without app version (#16713)

* 2.150.0.rc5 bump

* [deliver] give better error message when edit or live version is not found when downloading screenshots (#16714)

* 2.150.0.rc6 bump

* [deliver] fix first version logic now uses number of versions, reject_if_possible, and automatic_release_date uses ms again (#16715)

* [deliver] fix first version logic now uses number of versions

* Fix deliver reject and fix automatic release date to use time in ms

* 2.150.0.rc7 bump

* [spaceship] reorder app screenshots in AppScreenshotSet (#16718)

* [deliver] fix deliver download screenshot file extension (#16719)

* [deliver] fix deliver download screenshot file extension

* Updated how skip waiting for processing works

* [deliver] fix language detection on upload meta data (#16722)

* Reset version to 2.149.0

* Fix for broken review attachment

Co-authored-by: Max Ott <max.ott@me.com>

deliver/lib/deliver.rb

@@ -6,7 +6,6 @@
 require_relative 'deliver/upload_metadata'
 require_relative 'deliver/upload_screenshots'
 require_relative 'deliver/upload_price_tier'
-require_relative 'deliver/upload_assets'
 require_relative 'deliver/submit_for_review'
 require_relative 'deliver/app_screenshot'
 require_relative 'deliver/html_generator'

deliver/lib/deliver/app_screenshot.rb

@@ -1,6 +1,7 @@
 require 'fastimage'
 
 require_relative 'module'
+require 'spaceship/connect_api/models/app_screenshot_set'
 
 module Deliver
   # AppScreenshot represents one screenshots for one specific locale and
@@ -95,31 +96,31 @@ def initialize(path, language, screen_size = nil)
     # The iTC API requires a different notation for the device
     def device_type
       matching = {
-        ScreenSize::IOS_35 => "iphone35",
-        ScreenSize::IOS_40 => "iphone4",
-        ScreenSize::IOS_47 => "iphone6", # also 7 & 8
-        ScreenSize::IOS_55 => "iphone6Plus", # also 7 Plus & 8 Plus
-        ScreenSize::IOS_58 => "iphone58",
-        ScreenSize::IOS_65 => "iphone65",
-        ScreenSize::IOS_IPAD => "ipad",
-        ScreenSize::IOS_IPAD_10_5 => "ipad105",
-        ScreenSize::IOS_IPAD_11 => "ipadPro11",
-        ScreenSize::IOS_IPAD_PRO => "ipadPro",
-        ScreenSize::IOS_IPAD_PRO_12_9 => "ipadPro129",
-        ScreenSize::IOS_40_MESSAGES => "iphone4",
-        ScreenSize::IOS_47_MESSAGES => "iphone6", # also 7 & 8
-        ScreenSize::IOS_55_MESSAGES => "iphone6Plus", # also 7 Plus & 8 Plus
-        ScreenSize::IOS_58_MESSAGES => "iphone58",
-        ScreenSize::IOS_65_MESSAGES => "iphone65",
-        ScreenSize::IOS_IPAD_MESSAGES => "ipad",
-        ScreenSize::IOS_IPAD_PRO_MESSAGES => "ipadPro",
-        ScreenSize::IOS_IPAD_PRO_12_9_MESSAGES => "ipadPro129",
-        ScreenSize::IOS_IPAD_10_5_MESSAGES => "ipad105",
-        ScreenSize::IOS_IPAD_11_MESSAGES => "ipadPro11",
-        ScreenSize::MAC => "desktop",
-        ScreenSize::IOS_APPLE_WATCH => "watch",
-        ScreenSize::IOS_APPLE_WATCH_SERIES4 => "watchSeries4",
-        ScreenSize::APPLE_TV => "appleTV"
+        ScreenSize::IOS_35 => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::APP_IPHONE_35,
+        ScreenSize::IOS_40 => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::APP_IPHONE_40,
+        ScreenSize::IOS_47 => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::APP_IPHONE_47, # also 7 & 8
+        ScreenSize::IOS_55 => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::APP_IPHONE_55, # also 7 Plus & 8 Plus
+        ScreenSize::IOS_58 => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::APP_IPHONE_58,
+        ScreenSize::IOS_65 => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::APP_IPHONE_65,
+        ScreenSize::IOS_IPAD => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::APP_IPAD_97,
+        ScreenSize::IOS_IPAD_10_5 => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::APP_IPAD_105,
+        ScreenSize::IOS_IPAD_11 => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::APP_IPAD_PRO_3GEN_11,
+        ScreenSize::IOS_IPAD_PRO => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::APP_IPAD_PRO_129,
+        ScreenSize::IOS_IPAD_PRO_12_9 => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::APP_IPAD_PRO_3GEN_129,
+        ScreenSize::IOS_40_MESSAGES => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::IMESSAGE_APP_IPHONE_40,
+        ScreenSize::IOS_47_MESSAGES => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::IMESSAGE_APP_IPHONE_47, # also 7 & 8
+        ScreenSize::IOS_55_MESSAGES => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::IMESSAGE_APP_IPHONE_55, # also 7 Plus & 8 Plus
+        ScreenSize::IOS_58_MESSAGES => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::IMESSAGE_APP_IPHONE_58,
+        ScreenSize::IOS_65_MESSAGES => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::IMESSAGE_APP_IPHONE_65,
+        ScreenSize::IOS_IPAD_MESSAGES => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::IMESSAGE_APP_IPAD_97,
+        ScreenSize::IOS_IPAD_PRO_MESSAGES => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::APP_IPAD_PRO_129,
+        ScreenSize::IOS_IPAD_PRO_12_9_MESSAGES => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::APP_IPAD_PRO_3GEN_129,
+        ScreenSize::IOS_IPAD_10_5_MESSAGES => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::IMESSAGE_APP_IPAD_105,
+        ScreenSize::IOS_IPAD_11_MESSAGES => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::IMESSAGE_APP_IPAD_PRO_3GEN_11,
+        ScreenSize::MAC => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::APP_DESKTOP,
+        ScreenSize::IOS_APPLE_WATCH => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::APP_WATCH_SERIES_3,
+        ScreenSize::IOS_APPLE_WATCH_SERIES4 => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::APP_WATCH_SERIES_4,
+        # ScreenSize::APPLE_TV => Spaceship::ConnectAPI::AppScreenshotSet::DisplayType::
       }
       return matching[self.screen_size]
     end

deliver/lib/deliver/download_screenshots.rb

@@ -1,4 +1,5 @@
 require_relative 'module'
+require 'spaceship'
 require 'open-uri'
 
 module Deliver
@@ -13,35 +14,56 @@ def self.run(options, path)
     end
 
     def self.download(options, folder_path)
-      v = options[:use_live_version] ? options[:app].live_version(platform: options[:platform]) : options[:app].latest_version(platform: options[:platform])
-
-      v.screenshots.each do |language, screenshots|
-        screenshots.each do |screenshot|
-          file_name = [screenshot.sort_order, screenshot.device_type, screenshot.sort_order].join("_")
-          original_file_extension = File.basename(screenshot.original_file_name)
-          file_name += "." + original_file_extension
-
-          UI.message("Downloading existing screenshot '#{file_name}' for language '#{language}'")
-
-          # If the screen shot is for an appleTV we need to store it in a way that we'll know it's an appleTV
-          # screen shot later as the screen size is the same as an iPhone 6 Plus in landscape.
-          if screenshot.device_type == "appleTV"
-            containing_folder = File.join(folder_path, "appleTV", screenshot.language)
-          else
-            containing_folder = File.join(folder_path, screenshot.language)
-          end
+      legacy_app = options[:app]
+      app_id = legacy_app.apple_id
+      app = Spaceship::ConnectAPI::App.get(app_id: app_id)
 
-          if screenshot.is_imessage
-            containing_folder = File.join(folder_path, "iMessage", screenshot.language)
-          end
+      platform = Spaceship::ConnectAPI::Platform.map(options[:platform])
+      if options[:use_live_version]
+        version = app.get_live_app_store_version(platform: platform)
+        UI.user_error!("Could not find a live version on App Store Connect. Try using '--use_live_version false'") if version.nil?
+      else
+        version = app.get_edit_app_store_version(platform: platform)
+        UI.user_error!("Could not find an edit version on App Store Connect. Try using '--use_live_version true'") if version.nil?
+      end
+
+      localizations = version.get_app_store_version_localizations
+      localizations.each do |localization|
+        screenshot_sets = localization.get_app_screenshot_sets
+        screenshot_sets.each do |screenshot_set|
+          screenshot_set.app_screenshots.each_with_index do |screenshot, index|
+            file_name = [index, screenshot_set.screenshot_display_type, index].join("_")
+            original_file_extension = File.extname(screenshot.file_name).strip.downcase[1..-1]
+            file_name += "." + original_file_extension
+
+            url = screenshot.image_asset_url(type: original_file_extension)
+            next if url.nil?
+
+            language = localization.locale
+
+            UI.message("Downloading existing screenshot '#{file_name}' for language '#{language}'")
+
+            # If the screen shot is for an appleTV we need to store it in a way that we'll know it's an appleTV
+            # screen shot later as the screen size is the same as an iPhone 6 Plus in landscape.
+            if screenshot_set.apple_tv?
+              containing_folder = File.join(folder_path, "appleTV", language)
+            else
+              containing_folder = File.join(folder_path, language)
+            end
+
+            if screenshot_set.imessage?
+              containing_folder = File.join(folder_path, "iMessage", language)
+            end
+
+            begin
+              FileUtils.mkdir_p(containing_folder)
+            rescue
+              # if it's already there
+            end
 
-          begin
-            FileUtils.mkdir_p(containing_folder)
-          rescue
-            # if it's already there
+            path = File.join(containing_folder, file_name)
+            File.binwrite(path, open(url).read)
           end
-          path = File.join(containing_folder, file_name)
-          File.binwrite(path, open(screenshot.url).read)
         end
       end
     end

deliver/lib/deliver/options.rb

@@ -2,7 +2,6 @@
 require 'credentials_manager/appfile_config'
 
 require_relative 'module'
-require_relative 'upload_assets'
 
 module Deliver
   # rubocop:disable Metrics/ClassLength
@@ -206,6 +205,7 @@ def self.available_options
                                      short_option: "-b",
                                      description: "Extra information for the submission (e.g. compliance specifications, IDFA settings)",
                                      is_string: false,
+                                     type: Hash,
                                      optional: true),
 
         # affiliation
@@ -283,6 +283,7 @@ def self.available_options
         FastlaneCore::ConfigItem.new(key: :individual_metadata_items,
                                      env_name: "DELIVER_INDIVUDAL_METADATA_ITEMS",
                                      description: "An array of localized metadata items to upload individually by language so that errors can be identified. E.g. ['name', 'keywords', 'description']. Note: slow",
+                                     deprecated: "Removed after the migration to the new App Store Connect API in June 2020",
                                      is_string: false,
                                      type: Array,
                                      default_value: []),
@@ -291,21 +292,15 @@ def self.available_options
         FastlaneCore::ConfigItem.new(key: :app_icon,
                                      env_name: "DELIVER_APP_ICON_PATH",
                                      description: "Metadata: The path to the app icon",
+                                     deprecated: "Removed after the migration to the new App Store Connect API in June 2020",
                                      optional: true,
-                                     short_option: "-l",
-                                     verify_block: proc do |value|
-                                       UI.user_error!("Could not find png file at path '#{File.expand_path(value)}'") unless File.exist?(value)
-                                       UI.user_error!("'#{value}' doesn't seem to be one of the supported files. supported: #{Deliver::UploadAssets::SUPPORTED_ICON_EXTENSIONS.join(',')}") unless Deliver::UploadAssets::SUPPORTED_ICON_EXTENSIONS.include?(File.extname(value).downcase)
-                                     end),
+                                     short_option: "-l"),
         FastlaneCore::ConfigItem.new(key: :apple_watch_app_icon,
                                      env_name: "DELIVER_APPLE_WATCH_APP_ICON_PATH",
                                      description: "Metadata: The path to the Apple Watch app icon",
+                                     deprecated: "Removed after the migration to the new App Store Connect API in June 2020",
                                      optional: true,
-                                     short_option: "-q",
-                                     verify_block: proc do |value|
-                                       UI.user_error!("Could not find png file at path '#{File.expand_path(value)}'") unless File.exist?(value)
-                                       UI.user_error!("'#{value}' doesn't seem to be one of the supported files. supported: #{Deliver::UploadAssets::SUPPORTED_ICON_EXTENSIONS.join(',')}") unless Deliver::UploadAssets::SUPPORTED_ICON_EXTENSIONS.include?(File.extname(value).downcase)
-                                     end),
+                                     short_option: "-q"),
         FastlaneCore::ConfigItem.new(key: :copyright,
                                      env_name: "DELIVER_COPYRIGHT",
                                      description: "Metadata: The copyright notice",

deliver/lib/deliver/runner.rb

@@ -7,7 +7,6 @@
 require 'spaceship'
 require_relative 'html_generator'
 require_relative 'submit_for_review'
-require_relative 'upload_assets'
 require_relative 'upload_price_tier'
 require_relative 'upload_metadata'
 require_relative 'upload_screenshots'
@@ -87,9 +86,14 @@ def precheck_app
     # If not, the new version will automatically be created
     def verify_version
       app_version = options[:app_version]
-      UI.message("Making sure the latest version on App Store Connect matches '#{app_version}' from the ipa file...")
+      UI.message("Making sure the latest version on App Store Connect matches '#{app_version}'...")
 
-      changed = options[:app].ensure_version!(app_version, platform: options[:platform])
+      legacy_app = options[:app]
+      app_id = legacy_app.apple_id
+      app = Spaceship::ConnectAPI::App.get(app_id: app_id)
+
+      platform = Spaceship::ConnectAPI::Platform.map(options[:platform])
+      changed = app.ensure_version!(app_version, platform: platform)
 
       if changed
         UI.success("Successfully set the version to '#{app_version}'")
@@ -110,31 +114,13 @@ def upload_metadata
       # Assign "default" values to all languages
       upload_metadata.assign_defaults(options)
 
-      # Handle app icon / watch icon
-      prepare_app_icons(options)
-
       # Validate
       validate_html(screenshots)
 
       # Commit
       upload_metadata.upload(options)
       upload_screenshots.upload(options, screenshots)
       UploadPriceTier.new.upload(options)
-      UploadAssets.new.upload(options) # e.g. app icon
-    end
-
-    # If options[:app_icon]/options[:apple_watch_app_icon]
-    # is supplied value/path will be used.
-    # If it is unset files (app_icon/watch_icon) exists in
-    # the fastlane/metadata/ folder, those will be used
-    def prepare_app_icons(options = {})
-      return unless options[:metadata_path]
-
-      default_app_icon_path = Dir[File.join(options[:metadata_path], "app_icon.{png,jpg}")].first
-      options[:app_icon] ||= default_app_icon_path if default_app_icon_path && File.exist?(default_app_icon_path)
-
-      default_watch_icon_path = Dir[File.join(options[:metadata_path], "watch_icon.{png,jpg}")].first
-      options[:apple_watch_app_icon] ||= default_watch_icon_path if default_watch_icon_path && File.exist?(default_watch_icon_path)
     end
 
     # Upload the binary to App Store Connect
@@ -173,7 +159,10 @@ def upload_binary
     end
 
     def reject_version_if_possible
-      app = options[:app]
+      legacy_app = options[:app]
+      app_id = legacy_app.apple_id
+      app = Spaceship::ConnectAPI::App.get(app_id: app_id)
+
       if app.reject_version_if_possible!
         UI.success("Successfully rejected previous version!")
       end

deliver/lib/deliver/setup.rb

@@ -60,9 +60,9 @@ def generate_metadata_files(v, path)
       app_details = v.application.details
 
       # All the localised metadata
-      (UploadMetadata::LOCALISED_VERSION_VALUES + UploadMetadata::LOCALISED_APP_VALUES).each do |key|
+      (UploadMetadata::LOCALISED_VERSION_VALUES.keys + UploadMetadata::LOCALISED_APP_VALUES.keys).each do |key|
         v.description.languages.each do |language|
-          if UploadMetadata::LOCALISED_VERSION_VALUES.include?(key)
+          if UploadMetadata::LOCALISED_VERSION_VALUES.keys.include?(key)
             content = v.send(key)[language].to_s
           else
             content = app_details.send(key)[language].to_s
@@ -76,8 +76,8 @@ def generate_metadata_files(v, path)
       end
 
       # All non-localised metadata
-      (UploadMetadata::NON_LOCALISED_VERSION_VALUES + UploadMetadata::NON_LOCALISED_APP_VALUES).each do |key|
-        if UploadMetadata::NON_LOCALISED_VERSION_VALUES.include?(key)
+      (UploadMetadata::NON_LOCALISED_VERSION_VALUES.keys + UploadMetadata::NON_LOCALISED_APP_VALUES).each do |key|
+        if UploadMetadata::NON_LOCALISED_VERSION_VALUES.keys.include?(key)
           content = v.send(key).to_s
         else
           content = app_details.send(key).to_s
@@ -88,19 +88,8 @@ def generate_metadata_files(v, path)
         UI.message("Writing to '#{resulting_path}'")
       end
 
-      # Trade Representative Contact Information
-      UploadMetadata::TRADE_REPRESENTATIVE_CONTACT_INFORMATION_VALUES.each do |key, option_name|
-        content = v.send(key).to_s
-        content += "\n"
-        base_dir = File.join(path, UploadMetadata::TRADE_REPRESENTATIVE_CONTACT_INFORMATION_DIR)
-        FileUtils.mkdir_p(base_dir)
-        resulting_path = File.join(base_dir, "#{option_name}.txt")
-        File.write(resulting_path, content)
-        UI.message("Writing to '#{resulting_path}'")
-      end
-
       # Review information
-      UploadMetadata::REVIEW_INFORMATION_VALUES.each do |key, option_name|
+      UploadMetadata::REVIEW_INFORMATION_VALUES_LEGACY.each do |key, option_name|
         content = v.send(key).to_s
         content += "\n"
         base_dir = File.join(path, UploadMetadata::REVIEW_INFORMATION_DIR)
@@ -111,20 +100,6 @@ def generate_metadata_files(v, path)
       end
 
       UI.success("Successfully created new configuration files.")
-
-      # get App icon + watch icon
-      if v.large_app_icon.asset_token
-        app_icon_extension = File.extname(v.large_app_icon.url)
-        app_icon_path = File.join(path, "app_icon#{app_icon_extension}")
-        File.write(app_icon_path, open(v.large_app_icon.url).read)
-        UI.success("Successfully downloaded large app icon")
-      end
-      if v.watch_app_icon.asset_token
-        watch_app_icon_extension = File.extname(v.watch_app_icon.url)
-        watch_icon_path = File.join(path, "watch_icon#{watch_app_icon_extension}")
-        File.write(watch_icon_path, open(v.watch_app_icon.url).read)
-        UI.success("Successfully downloaded watch icon")
-      end
     end
 
     def download_screenshots(path, options)