minknow_api onremote machines implemented?

[neuropathbasel]

Hello,
Many thanks for forwarding me here.
First step on the local linux machine worked fine and prompted:

~/minknow_api$ python ./python/examples/list_sequencing_positions.py --host localhost --port 9501
Available sequencing positions on localhost:9501:
MN32638: running
secure: 8001
insecure: 8000

However, issues started with the attempt to obtain a similar readout form an identical second machine in the same subnet running under 192.18.01.18.
On this machine MinKNOW GUI
Installed version: 4.1.22 is running and a Minion is attached to it via USB 3. This unit is “detected” by the local MinKNOW GUI.

Running on the first linux machine
~/minknow_api$ python ./python/examples/list_sequencing_positions.py --host 192.168.1.18 --port 9501

yields:
Traceback (most recent call last):
File "./python/examples/list_sequencing_positions.py", line 41, in
main()
File "./python/examples/list_sequencing_positions.py", line 23, in main
manager = Manager(host=args.host, port=args.port, use_tls=False)
File "/home/user/minknow_api/gRPC/lib/python3.7/site-packages/minknow_api/manager.py", line 108, in init
minknow_api.manager_service.GetVersionInfoRequest()
File "/home/user/minknow_api/gRPC/lib/python3.7/site-packages/grpc/_channel.py", line 946, in call
return _end_unary_response_blocking(state, call, False, None)
File "/home/user/minknow_api/gRPC/lib/python3.7/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses"
debug_error_string = "{"created":"@1618939002.979941782","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":5419,"referenced_errors":[{"created":"@1618939002.979938522","description":"failed to connect to all addresses","file":"src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc","file_line":397,"grpc_status":14}]}"

ping to 192.168.1.18 yields an immediate reply.

Different attempts and the content of manager.py and manager_service.py imply to me that remote hosts are (so far ?) not supported.

Could you please let me know if gRPC calls to remote machine via the minknow_api are supported?
Many thanks
Claus

[0x55555555]

Hello @neuropathbasel,

When communicating with remote hosts, you must use tls (the examples dont support that in order to be concise).

You will also need to use port 9502 for the manager.

For example:

import argparse
from minknow_api.manager import Manager

def main():
    """Main entrypoint for list_sequencing_devices example"""
    parser = argparse.ArgumentParser(
        description="List sequencing positions connected to a host."
    )
    parser.add_argument(
        "--host", default="localhost", help="Specify which host to connect to."
    )
    parser.add_argument(
        "--port", default=None, help="Specify which porer to connect to."
    )

    args = parser.parse_args()

    # Construct a manager using the host + port provided.
    manager = Manager(host=args.host, port=args.port, use_tls=True) # Enable TLS

    # Find a list of currently available sequencing positions.
    positions = manager.flow_cell_positions()

    # Print out available positions.
    print("Available sequencing positions on %s:%s:" % (args.host, args.port))
    for pos in positions:
        print("%s: %s" % (pos.name, pos.state))

        if pos.running:
            print("  secure: %s" % pos.description.rpc_ports.secure)
            print("  insecure: %s" % pos.description.rpc_ports.insecure)

            # User could call {pos.connect()} here to connect to the running MinKNOW instance.


if __name__ == "__main__":
    main()

[JensUweUlrich]

Thanks for the information. I just wonder if every MinKNOW instance has its own certificate or if the certificate is the same on every remote machine?

Thanks
Jens

[0x55555555]

The certificate is the same for the moment, this is something we are considering changing in the future.

Thanks,

• George

[neuropathbasel]

hello jorj1988,
Many thanks for your prompt reply.
I did specify port 9502 for the remote machine.
However without any success - I am pretty sure I am missing something.
On machine 1 (with IP 192.168.0.15) I ran:
(gRPC) user@meqneuropat15:~/minknow_api$ python ./python/examples/list_sequencing_positions.py --host 192.168.1.18 --port 9502

But instead of a device listing I did obtain:
Traceback (most recent call last):
File "./python/examples/list_sequencing_positions.py", line 41, in
main()
File "./python/examples/list_sequencing_positions.py", line 23, in main
manager = Manager(host=args.host, port=args.port, use_tls=False)
File "/home/user/minknow_api/gRPC/lib/python3.7/site-packages/minknow_api/manager.py", line 108, in init
minknow_api.manager_service.GetVersionInfoRequest()
File "/home/user/minknow_api/gRPC/lib/python3.7/site-packages/grpc/_channel.py", line 946, in call
return _end_unary_response_blocking(state, call, False, None)
File "/home/user/minknow_api/gRPC/lib/python3.7/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses"
debug_error_string = "{"created":"@1618995241.639709258","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":5419,"referenced_errors":[{"created":"@1618995241.639706702","description":"failed to connect to all addresses","file":"src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc","file_line":397,"grpc_status":14}]}"

Any suggestion?
best
Claus

[0x55555555]

Hi @neuropathbasel,

Did you modify the example to use tls - as in the snippet above?

    manager = Manager(host=args.host, port=args.port, use_tls=True) # Enable TLS

[neuropathbasel]

Hi George,
I did modify the the example, but the error stayed the same until I started a new terminal.
Now I do need SSL certificates.
(gRPC) user@meqneuropat15:~/minknow_api$ python ./python/examples/list_sequencing_positions.py --host 192.168.1.18 --port 9502
Traceback (most recent call last):
File "/home/user/minknow_api/gRPC/lib/python3.7/site-packages/minknow_api/init.py", line 190, in grpc_credentials
return grpc_credentials.cached_credentials
AttributeError: 'function' object has no attribute 'cached_credentials'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "./python/examples/list_sequencing_positions.py", line 41, in
main()
File "./python/examples/list_sequencing_positions.py", line 23, in main
manager = Manager(host=args.host, port=args.port, use_tls=True)#set use_TLS=True for emote hosts Wed 2021-04-21
File "/home/user/minknow_api/gRPC/lib/python3.7/site-packages/minknow_api/manager.py", line 103, in init
use_tls=use_tls,
File "/home/user/minknow_api/gRPC/lib/python3.7/site-packages/minknow_api/manager.py", line 37, in init
minknow_api.grpc_credentials(),
File "/home/user/minknow_api/gRPC/lib/python3.7/site-packages/minknow_api/init.py", line 196, in grpc_credentials
"Couldn't find a valid path to MinKNOW's CA SSL certificate to initiate a secure connection"
minknow_api.MissingMinknowSSlCertError: Couldn't find a valid path to MinKNOW's CA SSL certificate to initiate a secure connection

will go through this repo to find a HowTo
cheers

Claus

[neuropathbasel]

@george
I understood your first reply that I have to work on port 9502 - a secure chanel.
So far I did not progress
standard error is "Couldn't find a valid path to MinKNOW's CA SSL certificate to initiate a secure connection"

Currently I would prefer for a fist trial to use a insecure channel on port 9501 - at least for a first attempt.
How can I get this to work?
best
Claus

[JensUweUlrich]

The certificate is the same for the moment, this is something we are considering changing in the future.

Thanks,

• George

Thanks George!

Please also consider a way to pull the certificate from the MinKNOW instance via some API calls when using different certificates for different instances of MinKNOW. Otherwise it would be pain in the ass to manually transfer and store each certificate from the remote machines to a local one.

Cheers
Jens

[0x55555555]

@neuropathbasel you can change how minknow binds insecure ports in the user_conf file (in <minknow install dir>/ conf / user_conf:

the local_connection_only setting should be set to "minknow_default" currently, if you set it to all_open minknow will bind all insecure ports to the network. This is not recommended from a security POV, but I agree is simpler to test things work as expected.

[neuropathbasel]

Thanks for the rapid help.
I introduced on both machines the changes you suggested and restarted the minknow service with sudo restart minknow service.
however, so far, without any luck.
for reasons unclear to me there is still the need for defining the path to a to MinKNOW's CA SSL certificate. I assumed to use an insecure connection.
Entering
python list_sequencing_positions_remote.py --host 192.168.1.18 --port 9501

prompts:
Traceback (most recent call last):
File "/home/user/minknow_api/gRPC/lib/python3.7/site-packages/minknow_api/init.py", line 190, in grpc_credentials
return grpc_credentials.cached_credentials
AttributeError: 'function' object has no attribute 'cached_credentials'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "list_sequencing_positions_remote.py", line 55, in
main()
File "list_sequencing_positions_remote.py", line 26, in main
manager = Manager(host=args.host, port=args.port, use_tls=True)
File "/home/user/minknow_api/gRPC/lib/python3.7/site-packages/minknow_api/manager.py", line 103, in init
use_tls=use_tls,
File "/home/user/minknow_api/gRPC/lib/python3.7/site-packages/minknow_api/manager.py", line 37, in init
minknow_api.grpc_credentials(),
File "/home/user/minknow_api/gRPC/lib/python3.7/site-packages/minknow_api/init.py", line 196, in grpc_credentials
"Couldn't find a valid path to MinKNOW's CA SSL certificate to initiate a secure connection"
minknow_api.MissingMinknowSSlCertError: Couldn't find a valid path to MinKNOW's CA SSL certificate to initiate a secure connection.

I did find the https://github.com/nanoporetech/minknow_api/blob/2fad019ee71546c00f534bdd0ca37d885baf073a/README.md
the section "How do I connect to a "secure" port?", but I am not where to implement the given code.

Any suggestion?
Best
Claus

[0x55555555]

Hi @neuropathbasel ,

The code above implies you are using tls still:

 manager = Manager(host=args.host, port=args.port, use_tls=True)

If you are using port 9501 you will need to not use tls.

If you use tls you need to ensure you supply the certificate from the minknow sequencing device on the local machine.

Thanks,

• George

[neuropathbasel]

thanks for your rapid help. I did correct the code, but did not try to start the code from another terminal. On this headless system, I am accessing via VNC I feel that something (an editor?) is blocking some functions. I have not seen this before.

Is there an example for supplying the certificate with the simple list_sequencing devices?

In my hand the link to the API description files (https://github.com/nanoporetech/minknow_api/blob/2fad019ee71546c00f534bdd0ca37d885baf073a/protos/minknow_api) is not found.
many thanks Claus

[0x55555555]

Please see this other issue where we discussed how to link CA certificates for minknow:

#16