Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix yaml manifest schema and parsing for ExpectedReturnCodes #1646

Merged
merged 5 commits into from
Nov 2, 2021

Conversation

yao-msft
Copy link
Contributor

@yao-msft yao-msft commented Oct 26, 2021

Change

  • Update InstallerLocale regex to be in sync with package locale regex
  • Main language in local regex is updated to allow 3 characters for less common languages
  • Remove the Moniker from additional locales files as Moniker is supposed to be 1:1 mapping to a package
  • Default expected return code should not override SuccessInstallerCodes, it's causing manifest validation error previously.

Validation

Validated all manifests in latest winget-pkgs repo passed

Microsoft Reviewers: Open in CodeFlow

@yao-msft yao-msft requested a review from a team as a code owner October 26, 2021 03:28
@github-actions
Copy link

github-actions bot commented Oct 26, 2021

@check-spelling-bot Report

Unrecognized words, please review:

  • overrided
Previously acknowledged words that are now absent activatable Globals mytool URIs Utils
To accept these unrecognized words as correct (and remove the previously acknowledged and now absent words), run the following commands

... in a clone of the git@github.com:yao-msft/winget-cli.git repository
on the fixschema branch:

update_files() {
perl -e '
my @expect_files=qw('".github/actions/spelling/expect.txt"');
@ARGV=@expect_files;
my @stale=qw('"$patch_remove"');
my $re=join "|", @stale;
my $suffix=".".time();
my $previous="";
sub maybe_unlink { unlink($_[0]) if $_[0]; }
while (<>) {
if ($ARGV ne $old_argv) { maybe_unlink($previous); $previous="$ARGV$suffix"; rename($ARGV, $previous); open(ARGV_OUT, ">$ARGV"); select(ARGV_OUT); $old_argv = $ARGV; }
next if /^(?:$re)(?:(?:\r|\n)*$| .*)/; print;
}; maybe_unlink($previous);'
perl -e '
my $new_expect_file=".github/actions/spelling/expect.txt";
use File::Path qw(make_path);
use File::Basename qw(dirname);
make_path (dirname($new_expect_file));
open FILE, q{<}, $new_expect_file; chomp(my @words = <FILE>); close FILE;
my @add=qw('"$patch_add"');
my %items; @items{@words} = @words x (1); @items{@add} = @add x (1);
@words = sort {lc($a)."-".$a cmp lc($b)."-".$b} keys %items;
open FILE, q{>}, $new_expect_file; for my $word (@words) { print FILE "$word\n" if $word =~ /\w/; };
close FILE;
system("git", "add", $new_expect_file);
'
}

comment_json=$(mktemp)
curl -L -s -S \
  --header "Content-Type: application/json" \
  "https://api.github.com/repos/microsoft/winget-cli/issues/comments/951545273" > "$comment_json"
comment_body=$(mktemp)
jq -r .body < "$comment_json" > $comment_body
rm $comment_json

patch_remove=$(perl -ne 'next unless s{^</summary>(.*)</details>$}{$1}; print' < "$comment_body")
  

patch_add=$(perl -e '$/=undef;
$_=<>;
s{<details>.*}{}s;
s{^#.*}{};
s{\n##.*}{};
s{(?:^|\n)\s*\*}{}g;
s{\s+}{ }g;
print' < "$comment_body")
  
update_files
rm $comment_body
git add -u

@yao-msft yao-msft merged commit 2f7f20d into microsoft:master Nov 2, 2021
@yao-msft yao-msft deleted the fixschema branch November 2, 2021 23:36
Trenly added a commit to Trenly/winget-pkgs that referenced this pull request Nov 2, 2021
denelon added a commit to microsoft/winget-pkgs that referenced this pull request Nov 3, 2021
* Install powershell-yaml

* Enforce ordering of keys

* Create function for adding list parameters

* Write Version Manifest using Yaml Parser

* Add To-Do Section

* Rebase on master and cleanup commit history

* Revert inadvertent changes from testing

* Implement logic for installers using YAML parser

* Fix statement in incorrect position

* Remove RAW writing

* Begin reading MultiManifests using Yaml parser

* Cleanup unneeded variables

* Bump Version

* Read parameters from singletons

* Add Product Code

* Abstract ReadInstallerManifest to sub-function

* Fix: Handle uninitialized variables

* Merge New/Update, add EditMetadata

* Fix: Change EditMetadata to work on singleton manifests

* Fix: Update Git commit messages for accuracy

* Remove Empty Manifest Folders

* Fix: Correct text

* Fix: Remove code for testing

* Feature: If old manifests exist, update in place to preserve extra keys

* Fix: Null Error when creating new

* Fix: Cast only when installer code

* Fix: Respect installer locale

* Fix: Use UTF8 Encoding

* Sort Yaml Keys

* Git error handling

* Fixed Moniker prompt for NewLocale

* minItems, maxItems, pattern, variable fix

* Implemented minItems from Schema
* Implemented maxItems from Schema
* Implemented pattern from Schema
* Implemented available Architecture's from Schema
* Implemented available InstallerType's from Schema
* Added $script: to some variables which needs to stay throughout the script

Only tested with one manifest as of now

Cleanup If/Else logic & Rebase on master

* Update method of getting Product Code from installer

* SignatureSha256 and PackageFamilyName (#5)

* More CommitTypes

* Fill in PR Parameters #21940 (#4)

* Detect PackageFamilyName for MSIX/APPX (#8)

* Update YamlCreate.ps1

* Update YamlCreate.ps1

* Exclude .validation (#9)

* Exclude .validation

* Added exclude to ExistingVersions

Co-authored-by: Levvie - she/her <11600822+ItzLevvie@users.noreply.github.com>

* Add new line for PackageFamilyName (#10)

* Standardize Keypress Menus (#11)

* Standardize Keypress Menus

* Fix: Use variable instead of pipeline

* Fix Help Texts & Spacing

* Cleanup Git Messages

* Filter by Yaml

* Remove debugging line

Co-authored-by: Esco <OfficialEsco@users.noreply.github.com>

Co-authored-by: Kaleb Luedtke <kaleb.carver.luedtke@jci.com>
Co-authored-by: Esco <OfficialEsco@users.noreply.github.com>

* Replaced PrBodyContentReply `n with array (#12)

* Menu reduction (#13)

* Simplify Code + Make Issues Entry Safer

* Check issue validity

* Fix unintended Revert

Co-authored-by: Esco <OfficialEsco@users.noreply.github.com>

* Fix unintended Revert

Co-authored-by: Esco <OfficialEsco@users.noreply.github.com>

* Fix unintended Revert

Co-authored-by: Esco <OfficialEsco@users.noreply.github.com>

Co-authored-by: Esco <OfficialEsco@users.noreply.github.com>

* More patterns (#15)

Added pattern for

* PackageIdentifier
* PackageVersion
* PackageFamilyName
* PackageLocale

* Readability (#16)

* Add Comparison Functions for Readability

* Simplify Validation Function

* Fix: Missing text

* Remove spaces from split function

* Fix: Resolve renamed variable

* Validate Installer Modes

* Validate File Extensions with Pattern and Length

* Add Custom Error Class (#18)

* Create function for adding list parameters

* Remove RAW writing

* Remove Empty Manifest Folders

* minItems, maxItems, pattern, variable fix

* Implemented minItems from Schema
* Implemented maxItems from Schema
* Implemented pattern from Schema
* Implemented available Architecture's from Schema
* Implemented available InstallerType's from Schema
* Added $script: to some variables which needs to stay throughout the script

* Add Custom Error Class

* Cleanup Class Constructor

* Additional error messages + cleanup

* Continue adding error messages

* Switched Local formatter to OTBS + Static Strings (Where applicable)

* Fix Spacing

* Remove unused variables from rebase

Co-authored-by: Esco <OfficialEsco@users.noreply.github.com>

* Fix: Check for package before do-until

* Finish Locale Errors

* Fix Spacing Issues

* Make Enums Case Sensitive

* Fix Unique Items + Incorrect Variables

Co-authored-by: Esco <OfficialEsco@users.noreply.github.com>

* Subfolder fix (#21)

* Fix: Subfolder Exists Erroring

* Return

* Write Unused Keys as Comments (#22)

* Write Unused Keys as Comments

* Exclude certain keys from appearing as comments

* Comment only Locale + Version

* Quick Update, Function Reduction, Detect Installer Types, Detect Installer Architectures, and Comments (#27)

* Update YamlCreate.ps1

* auto-detect installer architecture

* Fix Trenly#27 (comment)

* Fix auto-detection

* Apply suggestions from code review

Co-authored-by: Kaleb Luedtke <trenlymc@gmail.com>

* Move Test + Submit to main

* Update Tools/YamlCreate.ps1

* fix quick-update check

* Reduce input required for quick version update (#4)

* Reduce input required for quick version update

* Fix Product Code

* Don't add null product code if no product code key exists

* improve quick update

* Move prompt outside of Read-Info for clarity

* Move Update Prompt outside of Read-Info for clarity

* Update Messages

* if there are any errors due to this commit, i will revert changes

* Add Comments for code clarity

Co-authored-by: Kaleb Luedtke <trenlymc@gmail.com>

* Fix error from when functions were moved

* Handle Manifest Level Parameters (#34)

* Begin Handling Manifest Level Parameters

* Fix InstallModes

* Add delete manifest functionality (#39)

* Add functionality to manually delete manifests

* Add settings file and script documentation (#38)

* Do not default installer locale

* Add a script settings file with documentation

* Add setting to suppress quick update warning

* Remove accidental file inclusion

* Fix: Update SignatureSha256 in quick update mode (#41)

* Change Settings to allow for negative suppression (#46)

* Change Settings to allow for negative suppression

* Fix sample settings file

* Invalid SandboxTest variable

Co-authored-by: Vedant Mohan Goyal <83997633+vedantmgoyal2009@users.noreply.github.com>

Co-authored-by: Kaleb Luedtke <kaleb.carver.luedtke@jci.com>
Co-authored-by: Esco <OfficialEsco@users.noreply.github.com>
Co-authored-by: Vedant Mohan Goyal <83997633+vedantmgoyal2009@users.noreply.github.com>

* Cleaner references to linked issues (#49)

* Command Line Arguments (#50)

* Add switch for settings

* Allow PackageIdentifier and PackageVersion to be passed as optional parameters

* Add -help switch and documentation

* Settings File Location

* Fix spacing issue [Trenly#49] Trenly#51

Co-authored-by: Kaleb Luedtke <kaleb.carver.luedtke@jci.com>
Co-authored-by: Vedant Mohan Goyal <83997633+vedantmgoyal2009@users.noreply.github.com>

* Fix Parameter Reading and Condensing(#54)

* Fix: Allow InstallerSwitch keys to be split between installer level and manifest level on a per-key basis

* Fix: Save parameters to variables before removing them from manifest level

* Fix: Sandboxtest not working

* Fix: Prompt for PFN if file does not exist (#55)

* Auto Mode (#58)

* Auto Mode

* Fix for same version

* Keep ProductCodes for .exe files

* Keep ProductCodes for .exe files in Option-2

* Simplify function names, add debug info (#62)

* Fix: Allow autoupdate to update old package versions (#66)

* Make Simple Update automatically detect Sha256, SignatureSha256, and ProductCode without prompts (#61)

* Make Simple Update automatically detect Sha256, SignatureSha256, and ProductCode without prompts

* Add Parameter Mode (#64)

* Add Parameter Mode

* Update YamlCreate.ps1

* Update YamlCreate.md

Co-authored-by: Vedant Mohan Goyal <83997633+vedantmgoyal2009@users.noreply.github.com>

* Use Unique Branch Names (#67)

* Use Unique Branch Names

* Process Branch Names Safely

* Fix: Allow values other than en-US for default locale (#70)

* Made it so git config was only modified for the local repo. (#72)

* Only modify git config for local repo.

* Fixed 2214.

* Support for settings on Linux and macOS (#73)

* Support for settings on Linux and macOS

* Easton's suggestion

* Update YamlCreate.md

* Update Tools/YamlCreate.ps1

Co-authored-by: Kaleb Luedtke <trenlymc@gmail.com>

* Update YamlCreate.ps1

Co-authored-by: Kaleb Luedtke <trenlymc@gmail.com>

* Check for open PR's before submitting (#69)

* Check for open PR's before submitting

* Rebase on c82b39f

* Use API instead of CLI

* Exit when user-choice to terminate

Co-authored-by: Vedant Mohan Goyal <83997633+vedantmgoyal2009@users.noreply.github.com>

Co-authored-by: Vedant Mohan Goyal <83997633+vedantmgoyal2009@users.noreply.github.com>

* Throw when script error (#74)

* Fix: Add locale when converting from singleton (#75)

* Fix: Branch names when deleting (#79)

* Fix: Don't check for PRs on deletion (#80)

* Update PR Content when removing a manifest (#82)

* Update PR Content when removing a manifest

* More realistic character count limits

* Add is:pr to exclude issues from results (#88)

* Ensure file names are valid before saving (#85)

* Ensure file names are valid before saving

* Fix: Catch when content disposition doesn't exist

* Fix UserAgent not following redirects

* Additional Web Request Parameters

* Fix: Min descriptor length + Agent

* Fix: Escape Regex in variables (#91)

* Fix: Only add moniker to defaultLocale (#92)

* Chore: Add references to YamlCreate documentation (#93)

* Typo (#94)

* Chore: Add note to enable settings

Co-authored-by: denelon <denelon@microsoft.com>

* Chore: Remove To-do message

* Update YamlCreate.ps1

* Move integer validation (#99)

* Rename String.Validate -> Test-String (#101)

* Suppress Write-Host warnings from ScriptAnalyzer

* Fix missing constructor parameter

* Rename Write-Locale-Manifests -> Write-LocaleManifest

* Rename Write-Version-Manifest -> Write-VersionManifest

* Rename Write-Installer-Manifest -> Write-InstallerManifest

* Rename Read-WinGet-LocaleManifest -> Read-LocaleMetadata

* Rename Read-WinGet-InstallerManifest -> Read-InstallerMetadata

* Rename Read-Installer-Values-Minimal -> Read-QuickInstallerEntry

* Rename Read-Installer-Values -> Read-InstallerEntry

* Rename Enter-PR-Parameters -> Read-PRBody

* Rename Write-Colors -> Write-MulticolorLine

* Use Named Parameters for AddYamlParameter

* Use Named Parameters for AddYamlListParameter

* Use Named Parameters for PromptInstallerManifestValue

* Support ShouldProcess for removing manifests

* Move Downloading of installer to a separate function

* More function name updates

* Fix: Download Method (#107)

* Check for version when setting proxy (#109)

* Use defaultLocale due to microsoft/winget-cli#1646 resolving microsoft/winget-cli#1595

Co-authored-by: Esco <OfficialEsco@users.noreply.github.com>
Co-authored-by: Vedant Mohan Goyal <83997633+vedantmgoyal2009@users.noreply.github.com>
Co-authored-by: Levvie - she/her <11600822+ItzLevvie@users.noreply.github.com>
Co-authored-by: Kaleb Luedtke <kaleb.carver.luedtke@jci.com>
Co-authored-by: Easton Pillay <easton@planeteaston.com>
Co-authored-by: denelon <denelon@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants