adding "udmi validate" tool

[grafnu]

Adds "udmi validate" tool (alpha), and some minor improvements to validator output for increased usability.

[johnrandolph]

Honestly all this source'ing makes me nervous. Is there some other way we can accomplish this?

If we just want to put configuration data into shell scripts that's one thing, and carefully pull the values out. But at the point that we are building multiple source (with sourceing them) shell scripts I think we might need a different approach.

WDYT?

[grafnu]

I don't know how else to do it other than with "source" -- if you have a specific suggestion on "some other way" that's more appropriate then I'm all ears... but -- if you want/need to do something like define "common functions" I'm not sure how else to do it...

[johnrandolph]

Yes, I think if you are sourcing shell scripts you need to move to a different language.

[johnrandolph]

Is there some reason these aren't named same_as_the_others instead of using - ?

[grafnu]

I was basing it off of the git extension format, e.g. the file "git-utility" would map to the command-line "git utility " -- as a way to provide transparent extensions. So, "udmi-validate" maps to "udmi validate". The snake_case ones are generally things then that should be run directly (bin/test_sequencer). So, that was the reason (consistency with git extension naming)...

[noursaidi]

Needs a bug/task for getting this documented, but worked nicely in my testing

[grafnu]

How is source functionally different from #include in C or import in Python? I'm just trying to understand the underlying concern since to me they all provide roughly the same functionality, so I don't see how just using a "different language" would help anything...?

…

On Fri, Jan 27, 2023 at 7:13 AM John R ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In bin/udmi-locate <#561 (comment)>: > @@ -1,10 +1,6 @@ #!/bin/bash -e -UDMI_ROOT=$(realpath $(dirname $(readlink -f $0))/..) - -source $UDMI_ROOT/etc/shell_common.sh - -source $UDMI_ROOT/etc/find_udmi_profile.sh +source $(realpath $(dirname $(readlink -f $0))/..)/etc/udmi_preamble.sh Yes, I think if you are sourcing shell scripts you need to move to a different language. — Reply to this email directly, view it on GitHub <#561 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIEPD7VNYF32LXAFP7LMQTWUPQZDANCNFSM6AAAAAAT2272GE> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[johnrandolph]

As our needs for cli tooling have increased, my suggestion is that we reduce the points of individual R/W operations against the local filesystem, assuming files are in certain places, etc, not expand them. We are spending a lot of time basically reading json files with jq or chaining configs around which could be probably one python(or go, which I don't personally know but am ok with for this use case) class. It has been my experience that if you're doing more than one script worth of automation in shell, you're going to run into trouble, which we do already experience. The actual technical differences between shell source, C #include and Python import are probably secondary concerns here?...

…

On Fri, Jan 27, 2023 at 10:48 AM Trevor ***@***.***> wrote: How is source functionally different from #include in C or import in Python? I'm just trying to understand the underlying concern since to me they all provide roughly the same functionality, so I don't see how just using a "different language" would help anything...? On Fri, Jan 27, 2023 at 7:13 AM John R ***@***.***> wrote: > ***@***.**** commented on this pull request. > ------------------------------ > > In bin/udmi-locate > <#561 (comment)>: > > > @@ -1,10 +1,6 @@ > #!/bin/bash -e > > -UDMI_ROOT=$(realpath $(dirname $(readlink -f $0))/..) > - > -source $UDMI_ROOT/etc/shell_common.sh > - > -source $UDMI_ROOT/etc/find_udmi_profile.sh > +source $(realpath $(dirname $(readlink -f $0))/..)/etc/udmi_preamble.sh > > Yes, I think if you are sourcing shell scripts you need to move to a > different language. > > — > Reply to this email directly, view it on GitHub > <#561 (comment)>, or > unsubscribe > < https://github.com/notifications/unsubscribe-auth/AAIEPD7VNYF32LXAFP7LMQTWUPQZDANCNFSM6AAAAAAT2272GE > > . > You are receiving this because you modified the open/close state.Message > ID: ***@***.***> > — Reply to this email directly, view it on GitHub <#561 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALMRW5LKOGK7YLVGM4SGZ3WUPU5HANCNFSM6AAAAAAT2272GE> . You are receiving this because your review was requested.Message ID: ***@***.***>

-- John Randolph -- Google NYC

[grafnu]

Sounds like your core observation is "python is better than bash for CLI utilities" -- no inherent argument from me there, I would agree that in general it's true. A couple of specific points: "... that we reduce the points of individual R/W operations against the local filesystem," ==> That seems totally orthogonal to me. How would switching to python change anything? "which could be probably one python class." ==> I wouldn't bet on one class... there's actual differences between the various bits so it would likely be more than one. You could argue that inheritance etc... help out, but I wouldn't expect a monolith. "It has been my experience that if you're doing more than one script worth of automation in shell, you're going to run into trouble, which we do already experience." ==> Maybe, but also most of the "trouble" that I've actually seen are *not* this, so I'm not sure how to parse that argument. The bottom line is that although I don't see any inherent problem with using python for things, at this stage I don't have the bandwidth to convert things over and/or build out that infrastructure. There's bigger pieces that need to get in place, and most of this seems largely inconsequential to me (maybe it's fragile to changes, but I haven't seen it as the source of any significant problems). So, if somebody wants to pick up the joust and take on the windmills that's fine, but not something I personally would sign up for.

…

On Fri, Jan 27, 2023 at 8:01 AM John R ***@***.***> wrote: As our needs for cli tooling have increased, my suggestion is that we reduce the points of individual R/W operations against the local filesystem, assuming files are in certain places, etc, not expand them. We are spending a lot of time basically reading json files with jq or chaining configs around which could be probably one python(or go, which I don't personally know but am ok with for this use case) class. It has been my experience that if you're doing more than one script worth of automation in shell, you're going to run into trouble, which we do already experience. The actual technical differences between shell source, C #include and Python import are probably secondary concerns here?... On Fri, Jan 27, 2023 at 10:48 AM Trevor ***@***.***> wrote: > How is source functionally different from #include in C or import in > Python? I'm just trying to understand the underlying concern since to me > they all provide roughly the same functionality, so I don't see how just > using a "different language" would help anything...? > > > On Fri, Jan 27, 2023 at 7:13 AM John R ***@***.***> wrote: > > > ***@***.**** commented on this pull request. > > ------------------------------ > > > > In bin/udmi-locate > > <#561 (comment)>: > > > > > @@ -1,10 +1,6 @@ > > #!/bin/bash -e > > > > -UDMI_ROOT=$(realpath $(dirname $(readlink -f $0))/..) > > - > > -source $UDMI_ROOT/etc/shell_common.sh > > - > > -source $UDMI_ROOT/etc/find_udmi_profile.sh > > +source $(realpath $(dirname $(readlink -f $0))/..)/etc/udmi_preamble.sh > > > > Yes, I think if you are sourcing shell scripts you need to move to a > > different language. > > > > — > > Reply to this email directly, view it on GitHub > > <#561 (comment)>, or > > unsubscribe > > < > https://github.com/notifications/unsubscribe-auth/AAIEPD7VNYF32LXAFP7LMQTWUPQZDANCNFSM6AAAAAAT2272GE > > > > . > > You are receiving this because you modified the open/close state.Message > > ID: ***@***.***> > > > > — > Reply to this email directly, view it on GitHub > <#561 (comment)>, or > unsubscribe > < https://github.com/notifications/unsubscribe-auth/AALMRW5LKOGK7YLVGM4SGZ3WUPU5HANCNFSM6AAAAAAT2272GE > > . > You are receiving this because your review was requested.Message ID: > ***@***.***> > -- John Randolph -- Google NYC — Reply to this email directly, view it on GitHub <#561 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIEPDZGIIELQXVMXIWQRILWUPWPFANCNFSM6AAAAAAT2272GE> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>