v2v in gaprindashvili

[himdel]

Creating an issue just to track all the relevant bits.

Cc @martinpovolny @dclarizio @ohadlevy @simaishi

---

WIP work visible in:

• v2v
• ui-classic
• api (just missing backports)
• core (just missing backports)

Random TODO:

• possibly move mount_react_component to v2v, as only v2v uses it in gaprindashvili (plus, ensures we don't break it with the new one :))
  * [ ] i18n strings -- somebody else is dealing with this
• v2v:
  • remove dead (and wrong) app/controllers/miq_v2v_ui/application_controller.rb
  • ensure both rails controllers use the standard set of filters - at least :check_privileges
  • make v2v use the standard layouts instead of baking its own and failing on differences
  • handle page title
• patternfly version differences (Remove unused npm deps manageiq-v2v#336 (comment)) (hopefully done)
• ensure NODE_ENV=production during real build - miq_v2v_ui/migration.js is 2.2M now, and only loaded on v2v screens
• missing /api/transformation_mapping in gapri - Transformation Mappings Read and Create manageiq-api#313
  • but missing TransformationMapping - Introduce model changes for v2v manageiq#16787
  • but getting 403 on /api/transformation_mappings?expand=resources&attributes=transformation_mapping_items: Use of the read action is forbidden - Add product features needed for v2v Transformation Mappings API manageiq#16947
  • but still getting 403 "Use of Action create is forbidden" for post - Fix typo in transformation mapping product features manageiq#16988
• missing /api/clusters/*?attributes=lans in gapri - Add lans collection with read and show manageiq-api#325
  • but "Invalid attributes specified: lans" - Add lans as a virtual relationship to ems_cluster manageiq#17019
• fix production webpack build failing because csv-generate is using es6
• fix BootstrapSelect not seeing $().selectpicker (may need to rewrite require('jquery') to window.$ during compile, or backport the whole bower2npm thing)
• fix v2v modal appearing over error modal - Error modal - ensure visibility over other modals #3988
• seeing a breadcrumb - make sure it is supposed to be there - it is
• when the post fails, the "Error Creating Transformation Mapping" message is intdented weirdly to the right - wizard-pf-main - update margin for patternfly 3.25 compatibility manageiq-v2v#354
• use transformation label for everyhting related
• backport http changes too (from http helpers for react, API to new js #3662 + http - send cookie #3978, for Use http, not API for querying rails controller endpoints manageiq-v2v#340) - [GAPRINDASHVILI] API transformRequest support + http service #4005
• drop typescript & angular2 from gaprindashvili branch
• manageiq-appliance/appliance-build - make sure we're using the correct invocation to webpack:paths + yarn + webpack - Use update:ui task, instead of update:bower manageiq-appliance-build#220 (comment)
• backport loader changes, the rest of webpack stuff - [GAPRINDASHVILI] Backport Webpack related changes #4059
• backport rake task changes - [GAPRINDASHVILI] Rake task changes for update:ui #4071
• regenerate yarn.lock in the end
• update yarn.lock in v2v for gaprindashvili branch (already there, just not up to date after everything)
• make manageiq use v2v - [GAPRINDASHVILI] v2v plugin for GAPRINDASHVILI manageiq#17529

---

So far, I'm thinking we can backport (reasonably quickly) all the changes except for the ones related to:

• updating all rails engines
• handling duplicate JS dependencies
• moving dependencies from bower to npm
• possibly patternfly?

There, current gaprindashvili might or might not install any v2v npm dependencies, might or might not pick up the plugin during webpack (will if config/webpack/paths.json is right), will include -common packs (if it picks up the plugin), but may include them in the wrong order. There is no vendor pack in gaprindashvili.

[himdel]

Menu changes work with just #3347, #3683 and #3726 backported (including active item when accessing /migration).

EDIT: third PR added

[himdel]

Layout changes so that v2v uses ops ui layouts:

#3965
#3970
ManageIQ/manageiq-v2v#331

[himdel]

Webpacker loader changes:

backport #3506 and #3828 (TODO: prepare gapri PRs - not backportable)
and remove ts* support (no ts in gapri), and add react support (preset)

[AllenBW]

Just gonna tack observations here, this'll be the first of a few...

you'll notice, bottom of the modal the 5 per page looks a lil odd, a divergence from master, caused by align-items: baseline on class .content-view-pf-pagination .form-group of uicomponents

same thing happens here:

[himdel]

API & http changes - gaprindashvili specific PR

#4005

[AllenBW]

An issue with downloading logs 😢

throw "error";
$("#center_div").html("\u003cdiv id='content' style='overflow: scroll; height: 100%; padding-left: 50px;'\u003e\n\u003ch1 style='color: #c90813'\u003e\nUnexpected error encountered\n\u003c/h1\u003e\n\u003cbr\u003e\n\u003ch2\u003e\nErrors in Management Engine can be caused by:\n\u003c/h2\u003e\n\u003cbr\u003e\n\u003cfieldset\u003e\n\u003cfont style='font-size: 116%;color: black'\u003e\u0026#x2460;\u003c/font\u003e\nAccessing Management Engine from multiple tabs or windows of the same browser on a single machine.\nClose any duplicate browser sessions, then select an option from the menu.\n\u003c/fieldset\u003e\n\u003cfieldset\u003e\n\u003cfont style='font-size: 116%;color: black'\u003e\u0026#x2461;\u003c/font\u003e\nPressing the back button during a session.\n\u003cbr\u003e\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\nClose any duplicate browser sessions, then select an option from the menu.\n\u003c/fieldset\u003e\n\u003cfieldset\u003e\n\u003cfont style='font-size: 116%;color: black'\u003e\u0026#x2462;\u003c/font\u003e\nAn internal system error.\n\u003cbr\u003e\n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\nPlease contact your administrator for assistance.\n\u003c/fieldset\u003e\n\u003cbr\u003e\n\u003cbr\u003e\n\u003ch2\u003e\nError text:\n\u003c/h2\u003e\n\u003cbr\u003e\n\u003ch3\u003e\nundefined method `transformation_log_queue\u0026#39; for #\u0026lt;ServiceTemplateTransformationPlanTask:0x00007fa72d8541f8\u0026gt; [migration_log/download_migration_log]\n\u003c/h3\u003e\n\u003c/div\u003e\n");
miqSparkle(false);
if (miqDomElementExists('adv_searchbox_div')) $('#adv_searchbox_div').hide();

[AllenBW]

Here's another oddish one...

Create and execute a migration plan, it fails cuz, well thats not the point... (Locally they will all fail, 's all notional)

Batters up, create another one, same infrastructure mapping, same Choose from a list of VMs discovered in the selected infrastructure mapping setting and I get this error...

{"error":{"kind":"internal_server_error","message":"uninitialized constant ServiceResource::STATUS_COMPLETED","klass":"NameError"}}

Now here's the doozy, if we create a new migration, identical, works just fine, but attempt to duplicate that new infrastructure mapping? Same as above occurs.

[himdel]

@AllenBW So.. the last one - ServiceResource::STATUS_COMPLETED - we need ManageIQ/manageiq#17027 + ManageIQ/manageiq#17256 .. added to the branch.

[himdel]

@AllenBW the one before that - undefined method transformation_log_queue - we need ManageIQ/manageiq#17333, added to the branch :)

[himdel]

The issue before that - misaligned "per page" is still TODO...

The style was introduced in ManageIQ/ui-components#103 and never changed since - so it is shared between gaprindashvili and master.

So this is a patternfly difference ...

https://github.com/patternfly/patternfly/blob/0e2b545335a3761da334de7cf963facba48ebb73/src/sass/converted/patternfly/_pagination.scss#L91-L99

is the style that fixes this in patternfly 3.46...

That comes from patternfly/patternfly#910.

I'm thinking adding this style to the gaprindashvili version, with an extra v2v clas so that this doesn't affect any other dropdowns.

[AllenBW]

Welll we got that outer most div with id="reactRoot" could be a good differentiator, work in both the modal table and the details page table thing

[himdel]

Except that any dialogs are created outside of that root :( Oh well, they still exist inside %body :)

=> ManageIQ/manageiq-v2v#362

[AllenBW]

Got a new one for the list, though this one is a little unfair because it was JUST merged into v2v... but for the record...

{"error":{"kind":"bad_request","message":"Unsupported Action delete for the transformation_mappings resource specified","klass":"Api::BadRequestError"}}

when POST {"action": "delete"} tohttp://localhost:3000/api/transformation_mappings/1

Deleting transformation mappings...

[himdel]

@AllenBW yup, looks like from now on, you must be using my g-v2v branch even in v2v.

According to @AparnaKarve that change does not go in gaprindashvili.

[AllenBW]

OH SNAP! @himdel is that feature cut off listed anywhere?

[AparnaKarve]

Wait, is this about deleting the transformation mappings feature?

I do not know atm if this needs to be included in G or not.
(tbh, I don't see why not)

[AllenBW]

☝️ @AparnaKarve yeah it is, do yah know if we have a feature cut off for what is to be backported?

[himdel]

OK, if it turns out it should be, we just need to add gaprindashvili/yes to ManageIQ/manageiq-v2v#356 and ManageIQ/manageiq-api#383 :)

[AparnaKarve]

@himdel @AllenBW I think it would be preferable if we kept master and gaprindashvili identical to each other in terms of functionalities until 5.9.4 is released.

[himdel]

Agreed :)

So.. g-v2v branch update,

• my manageiq g-v2v branch is no longer necessary, everything we depended on was backported (as far as I know at least) - rebased, but use the upstream gaprindashvili branch

• the manageiq-api branch is still needed because of Allow ordering of service templates resource manageiq-api#316, rebased

• the manageiq-ui-classic branch is still needed because of [GAPRINDASHVILI] API transformRequest support + http service #4005 and build changes, rebased

• the miq_v2v_ui_plugin branch is no longer necessary either - rebased but use the upstream gaprindashvili branch

EDIT:

Thanks to Satoe, nobody should need any of my branches now.
I will continue to make changes in my ui-classic branch but those should only affect the build.

[simaishi]

@himdel I can get ManageIQ/manageiq-api#316 backported as soon as I know what to do with ManageIQ/manageiq#17023 (comment)!!

[himdel]

Thanks, just replied there, sorry, bad ordering :)

[simaishi]

@himdel I got everything backported/merged that had gaprindashvili/yes and transformation, with exception of #3564. Please confirm this is also needed, as this wasn't on your list.

[himdel]

@simaishi As far as I'm aware, #3564 can't possibly affect v2v the plugin - the PR deals with reports, and v2v is not using our GTL code in any way.

I'm also not seeing any :hidden actually used in any col_options under product/views in manageiq, so I'm not sure the feature ended up being used.

Maybe @gtanzillo knows more?

(I don't think it would break anything though.)

[himdel]

Also, thanks for backporting everything!

As far as I'm aware, that means the only remaining issues are build issues, working on that today.

[simaishi]

@bzwei ping, since you added transformation label to #3564 😄

[AparnaKarve]

@simaishi I can confirm that we do need #3564 for v2v
(it's required for the CSV step in the Plan Wizard)

[simaishi]

@AparnaKarve ok, thank you!

[AllenBW]

Running the g-v2v for cui and g for both v2v and api, hitting this error when bin/update

[Webpacker] Compilation Failed
yarn run v1.6.0
$ /Users/awight/Desktop/Projects/manageiq/plugins/manageiq-ui-classic/node_modules/.bin/webpack --config /Users/awight/Desktop/Projects/manageiq/plugins/manageiq-ui-classic/config/webpack/development.js
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

** override_gem: manageiq-ui-classic, [{:path=>"/Users/awight/Desktop/Projects/manageiq/plugins/manageiq-ui-classic"}], caller: /Users/awight/Desktop/Projects/manageiq/bundler.d/Gemfile.dev.rb
** override_gem: manageiq-api, [{:path=>"/Users/awight/Desktop/Projects/manageiq/plugins/manageiq-api"}], caller: /Users/awight/Desktop/Projects/manageiq/bundler.d/Gemfile.dev.rb
warning From Yarn 1.0 onwards, scripts don't require "--" for options to be forwarded. In a future version, any explicit "--" will be forwarded as-is to the scripts.
/Users/awight/Desktop/Projects/manageiq/plugins/manageiq-ui-classic/config/webpack/loaders/babel.js:4
  query: {
  ^^^^^
SyntaxError: Unexpected identifier
    at Object.exports.runInThisContext (vm.js:76:16)
    at Module._compile (module.js:542:28)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)
    at Module.require (module.js:497:17)
    at require (internal/module.js:20:19)
    at module.exports.module.rules.sync.map.loader (/Users/awight/Desktop/Projects/manageiq/plugins/manageiq-ui-classic/config/webpack/shared.js:42:57)
    at Array.map (native)
error Command failed with exit code 1.


== Command ["bundle exec rake update:ui"] failed in /Users/awight/Desktop/Projects/manageiq ==
awight-osx:manageiq awight$ ** Using session_store: ActionDispatch::Session::MemCacheStore
rails aborted!
SupportsFeatureMixin::UnknownFeatureError: Feature ':reconfigure_disksize' is unknown to SupportsFeatureMixin.
/Users/awight/Desktop/Projects/manageiq/app/models/mixins/supports_feature_mixin.rb:156:in `guard_queryable_feature'
/Users/awight/Desktop/Projects/manageiq/app/models/mixins/supports_feature_mixin.rb:200:in `supports'
/Users/awight/Desktop/Projects/manageiq/lib/extensions/require_nested.rb:11:in `require_nested'
/Users/awight/Desktop/Projects/manageiq/lib/extensions/descendant_loader.rb:235:in `load_subclasses'
/Users/awight/Desktop/Projects/manageiq/lib/extensions/descendant_loader.rb:255:in `descendants'
/Users/awight/Desktop/Projects/manageiq/app/models/blacklisted_event.rb:26:in `seed'
/Users/awight/Desktop/Projects/manageiq/lib/evm_database.rb:80:in `block (2 levels) in seed'
/Users/awight/Desktop/Projects/manageiq/lib/evm_database.rb:69:in `each'
/Users/awight/Desktop/Projects/manageiq/lib/evm_database.rb:69:in `block in seed'
/Users/awight/Desktop/Projects/manageiq/lib/extensions/ar_table_lock.rb:21:in `block (2 levels) in with_lock'
/Users/awight/Desktop/Projects/manageiq/lib/extensions/ar_table_lock.rb:21:in `block in with_lock'
/Users/awight/Desktop/Projects/manageiq/lib/extensions/ar_table_lock.rb:15:in `with_lock'
/Users/awight/Desktop/Projects/manageiq/lib/evm_database.rb:68:in `seed'
/Users/awight/Desktop/Projects/manageiq/db/seeds.rb:8:in `<top (required)>'
bin/rails:4:in `require'
bin/rails:4:in `<main>'
Tasks: TOP => db:seed

Working on figuring it out presently...

[himdel]

Sorry about that @AllenBW ... the first is a missing comma, fixed..

As for reconfigure_disksize that looks like you updated your repos in the middle of a backport or something like that... try if it disappears too now - rebased :) If not, try updating manageiq?

[AllenBW]

Thanks @himdel !! comma solved... as for the other... @d-m-u pointed me to https://gitter.im/ManageIQ/manageiq?at=5a8c99f0e4ff28713aa4ba40

Looks like we might need to backport ManageIQ/manageiq#16700

[himdel]

So... closing this, I believe we have everything needed to build gaprindashvili with v2v.

Feel free to reopen for bugs :)

[himdel]

Just a resume of the changes:

The Gaprindashvili JS build process is essentially the same as on master now - webpacker:compile need to run in ui-classic, webpack:paths in Rails.root.

Caveats for gaprindashvili:

• we lost typescript support (mostly because it was not used and would take a few more PRs to backport, it can still be done if needed)
• not all developer-only changes were backported (webpack-dev-server may not behave exactly as in master, not sure), no jest either
• patternfly is fixed at 3.23.*, so anything that v2v needs from newer patternfly has to be added in the v2v gapridnashvili branch (with appropriate wrapping - the .miq-v2v class is guaranteed to be on body when in v2v code in gaprindashvili - [GAPRINDASHVILI] Fix "<select> per page" styling in patternfly 3.25 manageiq-v2v#362 - so that this doesn't break other parts of the UI)
• technically, this means gaprindashvili supports multiple plugins, but the catch is - when multiple plugins depend on the same package in different versions, directly or indirectly, a random version will get used (this is not a problem now since the ui itself only depends on 3 packages via npm, so v2v is mostly free to do whatever, but would become a problem with more plugins)

[martinpovolny]

Thanks, @himdel !

[AllenBW]

Can confirm, dev server does noooooooooooot function as master 😢 's ok though 🤗