Deciding How to implement runtimetool

[YJDoc2]

In the implementation of integration tests, I have started implementing tests which rely on test_inside_container functionality. I had some issues while understanding it originally, but as @Furisto helped in #582 (comment), now I understand that we need a standalone binary which will run inside the container and test the properties from inside.

In Go tests, this is implemented as a separate Go program which is compiled and then passed to the tests and test containers.

The issue I wish to discuss is how should we implement this in our tests.

Currently I see two ways to do this :

• Implement runtimetool as a separate crate in the workspace
  Advantages :

  • We have a dedicated crate to develop the runtime tool tests in.
  • As it is part of the workspace, there won't be any dependency issues / bloat, as cargo will use same compiled libs for all
  • It will be easier to maintain (in my opinion)
    Disadvantages :
  • The tests will now be a three crate thing ( framework + integration tests + runtimetool) instead of two crate ( framework + integration tests)
  • we will need additional cmd arg to get the location of runtime tools binary

• Implement runtimetool as part of integration tests crate
  Advantages :

  • We won't need extra arg for location of runtime tool, as arg0 will give us the location of integration tests
  • The tests will remain a two crate thing
  • We won't need to maintain extra cargo toml file, as deps will be shared by integration tests and runtimetool, as they are the same binary
    Disadvantages :
  • Development will get complicated (in my opinion)
  • we will need additional cmd arg to check what to run the binary as integration test or runtimetool
  • the binary size will be large, which will need to be copied to the /tmp container rootfs

Personally I'm in favor of first option, develop runtimetools as a separate crate, as it will be easier to develop separately, and I'm not sure how good the advantages of second option are to take the second option.

As this is a decision that will impact rest of the development of integration tests, I would like to know your opinion before going forward.
@utam0k @Furisto @yihuaf @tsturzl @tommady

Thank you :)

[YJDoc2]

Closed in #582

[utam0k]

@YJDoc2
Oh, I missed this issue😭 But I feel that the first option looks good to me.