refactor: noports core #507

XavierChanth · 2023-09-28T23:48:23Z

- What I did

CI:

Added the ability to test using --legacy-daemon in the end2end test suite
- Set the reverse and sshnpd installer tests to legacy daemon
Added tag verification composite actions for both noports_core and sshnoports
- reimplemented these checks where they were previously hardcoded
- Also added unit tests to check these tags as well

Docker

Changed the context for all images to be the root of the repo
- Allows building from the local copy of noports_core when necessary

noports_core:

Moved all of lib/ into lib/src/ so that the code is protected.
- Reexported all of the code into separate libraries:
  - sshnp
  - sshnp_params - this has a lot of addons, so only the bare minimum is included in sshnp
  - sshnpd
  - sshrv
  - sshrvd
  - utils
Wrapped all default args as static const vars in DefaultArgs and all SSHNP only ones in DefaultSSHNPArgs
Added 2 new layers of abstraction to SSHNP:
- followed the appropriate code paths to ensure that all of the same logic runs in the same order as before.

classDiagram
    SSHNP <|-- SSHNPImpl
    SSHNPImpl <|-- SSHNPForwardDirection
    SSHNPImpl <|-- SSHNPReverseDirection
    SSHNPForwardDirection <|-- SSHNPForwardDartImpl
    SSHNPForwardDirection <|-- SSHNPForwardExecImpl
    SSHNPReverseDirection <|-- SSHNPReverseImpl
    SSHNPReverseDirection <|-- SSHNPLegacyImpl
    class SSHNP{
        <<abstract interface>>
      Exports public API
    }
    class SSHNPImpl{
        <<abstract>>
      Implements common code between all 4 implementations 
    }
    class SSHNPForwardDirection{
      <<mixin>>
      Implements common forward code
    }
    class SSHNPReverseDirection{
      <<mixin>>
      Implements common reverse code
    }
    class SSHNPForwardDartImpl {
        forward pure-dart client
    }
    class SSHNPForwardExecImpl {
        forward ssh exec client
    }
    class SSHNPReverseImpl {
        reverse ssh client for non-sshrvd hosts
    }
    class SSHNPLegacyImpl {
        reverse ssh client for <4.0.0 daemons
    }

Instead of using boolean for checking if initialization has occurred, there is now a completer which completes when initialization is done. By default, SSHNP automatically schedules initialization to begin in the constructor.
- An internal boolean is still stored, which prevents successive init calls from running.
Args:
- Added --help to print usage
- Moved createArgParser into the SSHNPArg class as a static function
- Made mandatory arguments in SSHNPParams to be final String instead of final String?
Rewrote SSHNPResult

classDiagram
    SSHNPResult <|-- SSHNPSuccess
    SSHNPResult <|-- SSHNPFailure
    SSHNPResult <|-- SSHNPConnectionBean
    SSHNPFailure <|-- SSHNPError
    Exception <|-- SSHNPError
    SSHNPSuccess <|-- SSHNPNoOpSuccess
    SSHNPSuccess <|-- SSHNPCommand
    SSHNPConnectionBean <|-- SSHNPCommand
    class SSHNPConnectionBean {
        <<mixin>>
    }
    class SSHNPNoOpSuccess {
      Needed for pure dart since it doesn't output a command
    }

noports_core now has it's own version.dart (this is the version.dart that will be used to report version to the daemon)

- How I did it

- How to verify it
N.B. the trunk tests are expected to fail because the branch doesn't have the updated code to build sshnoports with the local noports_core dependency yet (this PR will fix that)

- Description for the changelog
refactor: noports core

… re-expose under libraries

… the public APIs

XavierChanth · 2023-09-29T00:20:17Z

Running the e2e tests against 55a5ad2

Which uses the already published noports_core 4.0.0-dev.2 ~ the end2end tests aren't equipped to handle building and using noports_core from local yet.

…d build for packaging

…run in a dart container

XavierChanth · 2023-09-29T19:20:05Z

6407b66 - I was being over strict with SDK constraints, it doesn't need to be enforced there, I just did it because we had no reason not to... until now...

XavierChanth · 2023-10-03T00:12:48Z

packages/sshnp_gui/lib/src/presentation/widgets/profile_form/profile_form.dart

@@ -102,7 +102,7 @@ class _ProfileFormState extends ConsumerState<ProfileForm> {
                    crossAxisAlignment: CrossAxisAlignment.start,
                    children: [
                      CustomTextFormField(
-                        initialValue: oldConfig.sshnpdAtSign ?? '',
+                        initialValue: oldConfig.sshnpdAtSign,


these were changed from String? to String

XavierChanth · 2023-10-03T00:13:53Z

tests/end2end_tests/contexts/_init_/setup-sshnp-entrypoint.sh

+    legacy_sub='s/legacy/--no-legacy-daemon/g'
+fi
+eval "$prefix" "$legacy_sub" ../sshnp/entrypoint.sh


Here is where we insert the --legacy-daemon flag into the command.

…ility

XavierChanth · 2023-10-03T00:16:01Z

tests/end2end_tests/entrypoints/sshnp_entrypoint.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 echo "SSHNP START ENTRY"
-SSHNP_COMMAND="$HOME/.local/bin/sshnp -f @sshnpatsign -t @sshnpdatsign -d deviceName -h @sshrvdatsign -s id_ed25519.pub -v > sshnp.log"
+SSHNP_COMMAND="$HOME/.local/bin/sshnp -f @sshnpatsign -t @sshnpdatsign -d deviceName -h @sshrvdatsign -s id_ed25519.pub -v legacy > sshnp.log"


legacy is the placeholder where the --legacy-daemon flag might be inserted

XavierChanth · 2023-10-03T00:19:01Z

tests/end2end_tests/image/Dockerfile

-  dart compile exe ${REPO_DIR}/packages/sshnoports/bin/sshrvd.dart -o ${OUTPUT_DIR}/sshrvd ; \
-  dart compile exe ${REPO_DIR}/packages/sshnoports/bin/activate_cli.dart -o ${OUTPUT_DIR}/at_activate ;
+  dart pub get; \
+  dart run melos bootstrap --scope="noports_core" --scope="sshnoports"; \


This step is responsible for doing local linking of sshnoports and noports_core

XavierChanth · 2023-10-03T00:19:18Z

tests/end2end_tests/image/Dockerfile

-  dart compile exe ${REPO_DIR}/packages/sshnoports/bin/activate_cli.dart -o ${OUTPUT_DIR}/at_activate ;
+  dart pub get; \
+  dart run melos bootstrap --scope="noports_core" --scope="sshnoports"; \
+  dart pub get -C ${PACKAGE_DIR}; \


Then once linked, doing pub get will cause sshnoports to pull from noports_core locally

XavierChanth · 2023-10-03T00:19:28Z

tests/end2end_tests/image/Dockerfile

-  dart compile exe ${REPO_DIR}/packages/sshnoports/bin/activate_cli.dart -o ${OUTPUT_DIR}/at_activate ;
+  cd ${REPO_DIR}; \
+  dart pub get; \
+  dart run melos bootstrap --scope="noports_core" --scope="sshnoports"; \


Same as above

XavierChanth · 2023-10-03T00:20:25Z

tools/package-macos-arm64.sh

+}
+
+mv "$SRC_DIR/pubspec_overrides.yaml" "$SRC_DIR/pubspec_overrides.back.yaml"
+eval "$DART pub get -C $SRC_DIR" || restore_backup_and_exit 1


This prevents accidentally packaging for apple-silicon using the local noports_core instead of the published one if you've run melos bootstrap on your system.

gkc

@XavierChanth Thanks for this! There's a lot happening here... it all looks good to me but I note that the e2e tests did not run (for the most recent push at least) ... what's the plan for testing?

XavierChanth · 2023-10-04T12:50:01Z

@XavierChanth Thanks for this! There's a lot happening here... it all looks good to me but I note that the e2e tests did not run (for the most recent push at least) ... what's the plan for testing?

Tests should run now, there was a merge conflict in the pubspec.lock

Now that we have some code isolation we can start unit testing, I have a second PR coming soon which will enable mobile app support and might make some changes to the work I've done here

gkc

e2e tests failing

XavierChanth · 2023-10-05T13:32:31Z

e2e tests failing

trunk tests will fail until this is merged, because trunk doesn't know how to build noports_core (but noports_core is already merged)

gkc · 2023-10-05T13:35:48Z

Oh I see - ok

XavierChanth added 16 commits September 27, 2023 10:49

chore: rename SSHCommand to SSHNPSuccess

2f4ec28

chore: rename SSHNPCommandResult to SSHNPSuccess

58b4f7e

chore: remove unused code

44a619a

chore: reduce scope pollution of SSHNP's argParser

e6372cd

chore: separate params class, defaults

3606586

chore: remove unused imports

44ba240

chore: remove part statements from sshnp

1687d9f

chore: move literally everything into a src folder to protect it, and…

a503f47

… re-expose under libraries

chore: rename params to sshnp_params

1c21f15

refactor: remove the rest of the part definitions and properly expose…

15faa63

… the public APIs

refactor: split sshnp_impl by code path

e3f6d58

Merge remote-tracking branch 'origin/trunk' into refactor/noports_core

53a5280

chore: unify connectionBean on SSHNPSuccess class

2c59810

feat: add killConnectionBean to SSHNPSuccess

3346a58

feat: increment package version of noports core

29f33a1

chore: run end2end tests against noports_core 4.0.0-dev.2

55a5ad2

XavierChanth changed the title ~~refactor: noports core~~ [DRAFT] refactor: noports core Sep 29, 2023

XavierChanth added 2 commits September 29, 2023 08:51

ci: add better version tag checker

58ff1a2

chore: use local build for end2end tests where appropriate, use hoste…

e2a5081

…d build for packaging

XavierChanth changed the title ~~[DRAFT] refactor: noports core~~ refactor(DRAFT): noports core Sep 29, 2023

chore: remove flutter constraints from melos workspace, so it can be …

6407b66

…run in a dart container

XavierChanth added 7 commits September 29, 2023 15:23

fix(end2end tests): only bootstrap what is needed

4daa177

fix: add back initialization of publicKeyFileName

2548c1c

fix: ensure sshrvPort is always set when it should be

629b6a8

chore: remove unused code

db66f65

chore: checkin changes

9a66a8d

refactor: separate forward and reverse mixins to separate file

77dfdba

style: remove whitespace

e5fb53b

XavierChanth commented Oct 3, 2023

View reviewed changes

fix: no arg when not using legacy daemon for backward client compatab…

465c3b7

…ility

XavierChanth commented Oct 3, 2023

View reviewed changes

ci: use legacy placeholder

c9e411c

XavierChanth commented Oct 3, 2023

View reviewed changes

XavierChanth requested review from JeremyTubongbanua, gkc and cpswan October 3, 2023 00:20

XavierChanth self-assigned this Oct 3, 2023

XavierChanth changed the title ~~refactor(DRAFT): noports core~~ refactor: noports core Oct 3, 2023

XavierChanth added 3 commits October 2, 2023 20:28

ci: default legacy_daemon false

ae95c7b

chore: hardcode the legacy daemon conditions for installer

8d97b4c

chore: fix legacy check

b29498b

XavierChanth marked this pull request as ready for review October 3, 2023 01:04

XavierChanth added 3 commits October 3, 2023 09:39

chore: rename asciimatcher to avoid scope pollution

940ae1b

chore: final cleanup

afa2157

fix: double list devices timeout

8a1019a

gkc reviewed Oct 4, 2023

View reviewed changes

Merge branch 'trunk' into refactor/noports_core

0fb4db9

XavierChanth mentioned this pull request Oct 4, 2023

refactor: Remove file system bindings #516

Merged

gkc reviewed Oct 5, 2023

View reviewed changes

gkc approved these changes Oct 5, 2023

View reviewed changes

XavierChanth merged commit 4cb9e04 into trunk Oct 5, 2023
12 of 14 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: noports core #507

refactor: noports core #507

XavierChanth commented Sep 28, 2023 •

edited

XavierChanth commented Sep 29, 2023

XavierChanth commented Sep 29, 2023

XavierChanth Oct 3, 2023

XavierChanth Oct 3, 2023

XavierChanth Oct 3, 2023

XavierChanth Oct 3, 2023

XavierChanth Oct 3, 2023

XavierChanth Oct 3, 2023

XavierChanth Oct 3, 2023 •

edited

gkc left a comment

XavierChanth commented Oct 4, 2023 •

edited

gkc left a comment

XavierChanth commented Oct 5, 2023

gkc commented Oct 5, 2023

refactor: noports core #507

refactor: noports core #507

Conversation

XavierChanth commented Sep 28, 2023 • edited

CI:

Docker

noports_core:

XavierChanth commented Sep 29, 2023

XavierChanth commented Sep 29, 2023

XavierChanth Oct 3, 2023

Choose a reason for hiding this comment

XavierChanth Oct 3, 2023

Choose a reason for hiding this comment

XavierChanth Oct 3, 2023

Choose a reason for hiding this comment

XavierChanth Oct 3, 2023

Choose a reason for hiding this comment

XavierChanth Oct 3, 2023

Choose a reason for hiding this comment

XavierChanth Oct 3, 2023

Choose a reason for hiding this comment

XavierChanth Oct 3, 2023 • edited

Choose a reason for hiding this comment

gkc left a comment

Choose a reason for hiding this comment

XavierChanth commented Oct 4, 2023 • edited

gkc left a comment

Choose a reason for hiding this comment

XavierChanth commented Oct 5, 2023

gkc commented Oct 5, 2023

XavierChanth commented Sep 28, 2023 •

edited

XavierChanth Oct 3, 2023 •

edited

XavierChanth commented Oct 4, 2023 •

edited