Jl/nfdv proxy #73
Merged
Jl/nfdv proxy #73
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jordlay
added a commit
that referenced
this pull request
Sep 5, 2023
* added publisher scope to input file; changed to proxy nsdv * fixed get request on proxy nfdv (should be overwritten when autogenerated again) * small print bug fix --------- Co-authored-by: Jordan <jordan.layton@metaswitch.com>
jordlay
added a commit
that referenced
this pull request
Sep 5, 2023
Cyclam
added a commit
that referenced
this pull request
Oct 25, 2023
* removed deploy with sdk * added artifact list * push example cnf bicep * Add temporary build workflow for AOSM extension (#4) * Add temporary build workflow for AOSM extension * Add Releaser to maintain a release with the latest build in a consistent place * added output dir; fixing getchartmappingschema * Add empty init files to inner modules so setuptools recognises them as modules * Use latest RG model to remove unnecessary dependency on version * Use latest deployment model to remove unnecessary dependency on version * fixed parameter mappings + copied needed files to output folder * jinja2 and find value paths * fixed typos + self.aritfacts only has unique artifacts * use regex instead of string to find deploy params * delete accidentaly commited input.json * fixed deploy params; added indent to mapping file * Update readme with install/bug reporting instructions * Adjust headers so rendered readme looks nicer * renamed values.nondef + added cli errors * Split help command example lines * Clarify that bug process is internal * fixed parameter vals not updating * delete unuseed temoplate and update jinja2 templates * Update README.md * Update README.md * added docstrings; added basic error catching with file handling * small refactor of generate nfd * fixed j2 template captions; added create_nfd to cnf; edited error messages; added deleting existing folder to custom.py * fixed file not found erroe * Refactor commands to az aosm nfd|nsd build (#10) * Refactor commands to az aosm nfd|nsd build * setup.py version * Version was wrong * remove publish option from build * lint * wrong params * fixed most style errors; ran static checks and azdev style * pre merge commit * more style changes; renamed vnfbicepnfd to vnfnfd * Update readme with workflow * added configfile validation * added temp dir to vnf; renamed shared constants * sunny markups * removed create_nfd_folder; added schema prefix to constant * added deploymentparams to constants.py * added error catching for get chart name and version * added meaningful logs; prevent auto overwriting input.json * edited vnf schema to have accepted types * added templates to setup.py * added location from input file not resourceGroup().location * added path_to_mappings to input.json; added logic to take mappings from file outside of helm package * renamed non_def_values to mappings_path * Pk5/add nsd cli (#15) * First working version of the CLI NSD create * Fully working version of the NSD CLI * minor change to nsd_generate * Sunny's refactor * First round of cleanup * Secound Round of cleanup * fix the 2023 api NSDV * description updates * deleted comment * Fix SNS creation * Fix SNS creation try 2 * markups * delete unnecessary file * Testing markups * Fix the SNS attempt 3 * minor fixes * Fix config validation * fix CNF depends on * initial commit; ran python static checks fmt (except on vendored sdks), fixed styling on _configuration.py * Name uploaded VHD correctly * Self review markups * broken config generation code * fixed up configuration.py * fixed cnf generator styling * fixed styling for cnf generator * fixed up nfd generator base and deploy with arm * fixed styling for artifact.py * fixed styling for atrifact manifest .py * Code review markups * fixed more linting * first attempt at regex * Sunny/choose deploy parameters (#23) * choose-deploy-parameters * optioned deployParameters for CNF * lint * lint2 * docs * docs * lint * 9.82 score * Fix bugs * more useful debug logs * Fix bugs and logging * lint * markups * Fix for oras target string with erroneous spaces * fixed regex; tested on existing charts and sas charts; committing to not lose prints for testing * changed regex constants + tidied * fixed blankspace * markups * initial commit * tidied code and added comments * add multi nf config * Add style and lint check * added logic for handling deployparams within lists * one line fix from review * removing print statement * added new test file and one unit test * added workflow for unit tests in pipeline + set up rough structure of unit testing * instantiated cnf class; added fake invalid helm package; added new files to use for testing * Nsd for cnfs (#33) * NSD building for CNFs * linting * Add Publish command to the CNF azure CLI (#24) * Working publish * Fix the artifact upload * Working image copy * minor fix * Minor fixes * sunny merge add-aosm-extension into patryk's branch (#25) * Sunny/choose deploy parameters (#23) * choose-deploy-parameters * optioned deployParameters for CNF * lint * lint2 * docs * docs * lint * 9.82 score * Fix bugs * more useful debug logs * Fix bugs and logging * lint * markups * comment out breaking line * minor TODOs * deleted comment * fix bring your own parameters * Markups * Fix the helm upload * Minor markups * Change error message --------- Co-authored-by: sunnycarter <36891339+sunnycarter@users.noreply.github.com> Co-authored-by: Sunny Carter <sunny.carter@metaswitch.com> * Expose NFD version and managed identities (#34) * NFDV version exposed as a CGV on an SNS * Managed identities support on NFs * Fix identiy, fix API versions * history --------- Co-authored-by: Jamie Parsons <jamie.parsons@metaswitch.com> Co-authored-by: Sunny Carter <sunny.carter@metaswitch.com> * Add a first VNF test * achurchard/style fixes (#35) Fix style issues raised by `azdev style` * Add CNF UTs that will fail at the moment * Actually include all files. * Always run tests * Add another test that won't pass yet * remove github pipeline tests - they should run in Azure * Sundry fixes for CNF quickstart (#38) * Sundry fixes for CNF quickstart * merge add-aosm-ext in (#37) * markups * NSD UTs * Update read me. * Improve path handling, other small refactorings (#39) # Main changes ## Make methods relying on self._tmp_dir private - `self._tmp_dir` is only available in the context of calling `generate_nfd()`, so methods relying on `self._tmp_dir` should be private ## Use pathlib.Path rather than os file operations - Provides clearer and stronger typing than passing `str`s around - Adds some handy utility functions ## Variable renaming for clarity - E.g. consistently use 'directory' / 'dir' (rather than mix with 'folder') - Obvs somewhat subjective, but as someone new to most of this code, the changes made sense to me ## Add nfd_bicep_path as abstract property on NFDGenerator - We rely on it when calling the concrete implementations - Also use ABC rather than raise NotImplementedError ## Miscellaneous style updates to keep `azdev style aosm` happy - isort - black * Fix Artifact upload on Windows (#40) * Black * Mark ups * Sunny/fix artifact upload windows (#41) * Fix Artifact upload on Windows * mypy fixups * mypy fixes * linting * mypy * mypy for _configuration.py * mypy for vnf_nfd_generator.py * mypy appeasement * python-static-checks fmt * az style happy * lint * mypy cnf_nfd_generator * copyright * more lint * Remove windows oras workaround now 0.0.18 oras out * history --------- Co-authored-by: Jamie Parsons <jamie.parsons@metaswitch.com> * Fix VNFD build (#42) * Multiple instances of the same NF * fix vnf deploy schema types (#44) * Tidy up * Mypy and docs * history.rst * Update README with logging info (#43) * Update README with logging info * Update docs * self markups * Take oras 0.1.19 to fix Windows artifact upload (#46) * Rename a few things * DRY the deploy_nfd_from_bicep code * Make conditional statements clearer * black * Update custom.py to use new nfd deploy method * Black with text processing * Create new nfRET class * Mypy passing * UTs passing * Dry the deploy_nfd_from_bicep() code (#48) - Removed duplicated code in the deploy_vnfd_from_bicep() and deploy_cnfd_from_bicep() methods - Moved variables onto the class instance and stopped passing them around in the method signatures - Also made the deployer class a dataclass for cleaner __init__ - Made parameters a property of the class, and moved the logic for getting/generating them into that property method - Improved the typing - Included adding enums for SkipSteps and DeployableResourceTypes - Added the new skip step logic for image uploads - Added a new skip step type of IMAGE_UPLOAD Testing: - No MyPy errors - Live tests for VNFS+NSD, CNFs, including all --skip options * Unit tests added * Versions are wrong in NSD template comments * Self review markups * Remove wrong comments * Code review markups * Extra markups * Fix mypy * Pk5/add integration tests (#51) * Somewhat working example * Cleanups * Update recording * fix minor linting error * More markups * Update output in tests * Pk5/minor integration tests changes (#52) * Modify changelog and rename tests * Delete recording file * Update recordings * Update developer docs * Remove credentials * Fix linting * cheeky extra change * Remove credentials from the integration test recordings (#54) * Modify changelog and rename tests * Delete recording file * Fix the credentials issue in the integration tests * Update CHANGELOG * Markups * Fix typo * Minor markups * Make cleaning up resources more robust and remove the VHD external reference * Markups * Remove the integration test from the repo (#56) * prerequisite docker for cnf * Fix bicep render on Windows (#57) * Fixes for Windows (#58) * Fix bicep render on Windows * Fixes for Windows * python-static-checks ran * ADO-880627: Use docker-less ACR login For running in environments without docker, such as CI pipelines. This affects uploading Helm charts for CNFs - the ACR login now fetches an access token, which is used to login to the Helm registry and upload the chart. * docker is not required any more * Remove pre-release build and lint files from release branch (#63) * Release markups - release branch (#65) * Markups from release comments * history * markups and fixes * README changes * linting fixes * Fix integration tests (#67) * Fix integration tests * Update src/aosm/development.md Co-authored-by: Cyclam <95434717+Cyclam@users.noreply.github.com> * Update src/aosm/azext_aosm/tests/latest/test_aosm_cnf_publish_and_delete.py Co-authored-by: Cyclam <95434717+Cyclam@users.noreply.github.com> * Update src/aosm/azext_aosm/tests/latest/test_aosm_cnf_publish_and_delete.py Co-authored-by: Cyclam <95434717+Cyclam@users.noreply.github.com> * Markup changes --------- Co-authored-by: Cyclam <95434717+Cyclam@users.noreply.github.com> Co-authored-by: Sunny Carter <sunny.carter@metaswitch.com> * Add aosm to service_name.json * move artifact upload to before nfd/nsd deploy (#62) * move artifact upload to before nfd/nsd deploy * markups * check in new test recording * non-working attempt to use token creds * Check for Azure features on deploy and delete (#71) * WIP: Check for features on deploy and delete * linting * Anand has confirmed names of flags * Update test recording as now has calls to Features API * Update src/aosm/azext_aosm/custom.py Co-authored-by: jamiedparsons <111778988+jamiedparsons@users.noreply.github.com> * docstring markups --------- Co-authored-by: jamiedparsons <111778988+jamiedparsons@users.noreply.github.com> * code working * Fix generate-config * Fix tests * lint * linting * Jl/nfdv proxy (#73) * added publisher scope to input file; changed to proxy nsdv * fixed get request on proxy nfdv (should be overwritten when autogenerated again) * small print bug fix --------- Co-authored-by: Jordan <jordan.layton@metaswitch.com> * Code review mark ups * Linting and fix UTs * aosm codeowner * Update HISTORY.rst * Normalise aosm version to 1.0.0b1 for release * update hashes in recording * Temporary test revert of "aosm codeowner" This reverts commit 479ba54. * Fix license header formatting * Revert "Temporary test revert of "aosm codeowner"" This reverts commit 0e4d0e7. * Remove nonexistent entry from codeowners * Fix source_local_docker_image defaults to placeholder text (#79) * temp fix for bug * minor formatting * refactor + change validation ordering to mimic previous behaviour * add docstring --------- Co-authored-by: Jordan <jordan.layton@metaswitch.com> * renamed nsdg to nsd * Helptext + a little bit of validation * Fixed up validation * Return True if _all_ required artifact manifests exist, not just the last one checked. * Code markups + linting. * Docs: CNF publish options and permissions (#91) * Requirement for docker again * revert bad markup (#94) * Move live tests to swedensouth * Actually move to uaenorth * Don't log out ACR passwords when artifact commands fail (#97) * Don't log out passwords when artifact commands fail * Don't log out passwords when artifact commands fail part 2 * Comment to explain dropping the original exception * Fix cnf image take 2 (#101) * Fix CNF image copy to work cross subscription * Cross subscription works for image copy. Still test same subscription * lint * Error message * oops, code paste error * markups * appease mypy * Default RG and ACR Values with Publisher Name (#103) * added default rg and acr values for nf + nsd; added blob_url default for vnf * added bug fix to close Paulo's bug * change error message back --------- Co-authored-by: Jordan <jordan.layton@metaswitch.com> * Fix unauthorized error bug * Add spacing * fix unexpected symbol when parsing lists (#107) * Validation requirements on helm names (#109) * Validation requirements on helm names * lint * markups * lint * Fix error parsing * Lower case acr names (#112) * Lower case ACR names and fix interactive mode for lists * lint * remove duplicate else branch * always allow azureDeployLocation to be configurable * Add new optional VHD parameters * Make VNF publish and delete a live test only * Validate VHD parameters in build UT * Markups from the CLI team * mark-ups * remove azureDeployLocation as a configurable vhd parameter * add explanitory comments * Updates for 2023-09-01 API (#84) * Regen Python SDK from 2023-09-01 API - also uses latest AutoRest client - fix for HybridNetworkManagementClient init signature (swap order of subscription_id and credential parameters) * Update CLI extension code to use new SDK * added SAMI to publisher pre deploy * Update bicep templates to use 2023-09-01 * Update NF templates * Update metaschema * Add Allow-Publisher to required feature flags * Use secure objects for deployment parameters * Correctly get array item types in CNF deployment parameter schemas * Revert master .flake8 config * Fix double space in user output. Co-authored-by: Xing Zhou <Zhou.Xing@microsoft.com> * Remove delete message to be consistent with other CLI extensions * Move "Deleted <resource>" messages to logger.info(). * Remove Blob_SAS_URL Option from ArmTemplate in input file (#119) * moved blob url to vhd config only; untested * stopped error in post innit before validate * changed ordering of inputs so that blob and filepath are next to each other; helptext for filepath different for each option * Markups from sunny * appease mypy --------- Co-authored-by: Jordan <jordan.layton@metaswitch.com> Co-authored-by: Sunny Carter <sunny.carter@metaswitch.com> --------- Co-authored-by: Jordan <jordan.layton@metaswitch.com> Co-authored-by: Jacob Darby <jdarby@microsoft.com> Co-authored-by: Chaos <chaosultima@outlook.com> Co-authored-by: Chaos Chhapi <chaos.chhapi@metaswitch.com> Co-authored-by: jordlay <72226943+jordlay@users.noreply.github.com> Co-authored-by: patrykkulik-microsoft <116072282+patrykkulik-microsoft@users.noreply.github.com> Co-authored-by: Jacob <53872686+jddarby@users.noreply.github.com> Co-authored-by: Jamie Parsons <jamie.parsons@metaswitch.com> Co-authored-by: jamiedparsons <111778988+jamiedparsons@users.noreply.github.com> Co-authored-by: Andy Churchard <andy.churchard@metaswitch.com> Co-authored-by: Cyclam <95434717+Cyclam@users.noreply.github.com> Co-authored-by: William Bradley <wbradley@microsoft.com> Co-authored-by: Xing Zhou <Zhou.Xing@microsoft.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adding proxy call for NSDVs, used for creating NSDs.
Tested with 2 subscriptions (AIOpsUK and AIOps2), to test proxy GET.
Tested 4 scenarios:
Related command
az aosm nsd build --config-file proxy_input.json
az aosm nsd publish--config-file proxy_input.json
General Guidelines
azdev style <YOUR_EXT>locally? (pip install azdevrequired)python scripts/ci/test_index.py -qlocally?For new extensions:
About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your pull request is merged into main branch, a new pull request will be created to update
src/index.jsonautomatically.You only need to update the version information in file setup.py and historical information in file HISTORY.rst in your PR but do not modify
src/index.json.