Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Display "Published ports" information when output format is JSON #887

Merged
merged 14 commits into from Aug 1, 2022
5 changes: 5 additions & 0 deletions clab/file.go
Expand Up @@ -35,6 +35,11 @@ type TopoFile struct {
name string // file name without extension
}

// GetDir returns the path of a directory that contains topology file
func (tf *TopoFile) GetDir() string {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just curious why you wanted this function, instead of accessing the field?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the issue is, that we put applogic in the cmd package ... what we should do is put all the app logic in the clab package and just do parameter parsing and population in the cmd package... that is not too clean of a structure. As a result of this, from the cmd package we are unable to access the lower case dir variable... instead of opting for making the var public (CamelCase) I introduced this getter.
We could come up with a seperate PR and seperate app logic from parameter paring and initialization if you like, I'd be happy to do so.

return tf.dir
}

// GetTopology parses the topology file into c.Conf structure
// as well as populates the TopoFile structure with the topology file related information
func (c *CLab) GetTopology(topo, varsFile string) error {
Expand Down
4 changes: 1 addition & 3 deletions cmd/deploy.go
Expand Up @@ -286,9 +286,7 @@ func deployFn(_ *cobra.Command, _ []string) error {
newVerNotification(vCh)

// print table summary
printContainerInspect(c, containers, format)

return nil
return printContainerInspect(c, containers, format)
}

func setFlags(conf *clab.Config) {
Expand Down
186 changes: 143 additions & 43 deletions cmd/inspect.go
Expand Up @@ -11,14 +11,17 @@ import (
"os"
"path/filepath"
"sort"
"strconv"
"strings"

"github.com/olekukonko/tablewriter"
log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"github.com/srl-labs/containerlab/clab"
"github.com/srl-labs/containerlab/mysocketio"
"github.com/srl-labs/containerlab/nodes"
"github.com/srl-labs/containerlab/runtime"
"github.com/srl-labs/containerlab/types"
"github.com/srl-labs/containerlab/utils"
)

var format string
Expand Down Expand Up @@ -76,10 +79,6 @@ var inspectCmd = &cobra.Command{
return fmt.Errorf("failed to list containers: %s", err)
}

if len(containers) == 0 {
log.Println("no containers found")
return nil
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to find a way of not printing an empty table if no containers are found

right now it prints an empty one, and before we used "no containers found"

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue was, that this was printed even in the json case ... which is bad when json output is expected.
What exactly is the issue with the empty table?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it just doesn't make a lot of sense to me printing an empty table. I haven't seen any other tool doing that. Gives an impression that something

we need to make sure it is not printed when format is set to json and printed when default output. I can have a look at it if you have more interesting things to have a crack on

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd personally prefere a stable output method. But nevermind.
Check out the new commit.
bcc6a2d

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why revert the test based on json?
lets keep both?! Just to make sure json output is also fine in the zero container case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

valid point. need to get back

if details {
b, err := json.MarshalIndent(containers, "", " ")
if err != nil {
Expand Down Expand Up @@ -117,11 +116,17 @@ func toTableData(det []types.ContainerDetails) [][]string {
}

func printContainerInspect(c *clab.CLab, containers []types.GenericContainer, format string) error {

if len(containers) == 0 && format == "table" {
fmt.Println("no containers found")
return nil
}

contDetails := make([]types.ContainerDetails, 0, len(containers))
// do not print published ports unless mysocketio kind is found
printMysocket := false
var mysocketCID string

// Gather details of each container
for i := range containers {
cont := &containers[i]
// get topo file path relative of the cwd
Expand All @@ -145,7 +150,6 @@ func printContainerInspect(c *clab.CLab, containers []types.GenericContainer, fo
cdet.Kind = kind
if kind == "mysocketio" {
printMysocket = true
mysocketCID = cont.ID
}
}
if group, ok := cont.Labels["clab-node-group"]; ok {
Expand All @@ -161,55 +165,151 @@ func printContainerInspect(c *clab.CLab, containers []types.GenericContainer, fo
return contDetails[i].LabName < contDetails[j].LabName
})

if format == "json" {
b, err := json.MarshalIndent(contDetails, "", " ")
resultJson := &types.LabData{Containers: contDetails, MySocketIo: []*types.MySocketIoEntry{}}
var socketdata []*types.MySocketIoEntry
var tokenFile string
var err error

// fetch mysocketio data if mysocketio node is detected to present in a list of nodes and nodes are not empty
// nodes are not populated when `inspect --all` is used, since we don't read topology files
if printMysocket && len(c.Nodes) != 0 {
// get mysocketio token file path by fetching it from the mysocketio node' binds section
tokenFile, err = mySocketIoTokenFileFromBindMounts(c.Nodes, c.TopoFile.GetDir())
if err != nil {
return err
}
// retrieve the MySocketIO Data
socketdata, err = getMySocketIoData(tokenFile)
if err != nil {
return fmt.Errorf("error when processing mysocketio data: %v", err)
}
resultJson.MySocketIo = socketdata
}

switch format {
case "json":
b, err := json.MarshalIndent(resultJson, "", " ")
if err != nil {
return fmt.Errorf("failed to marshal container details: %v", err)
}
fmt.Println(string(b))
return nil
}
tabData := toTableData(contDetails)
table := tablewriter.NewWriter(os.Stdout)
header := []string{
"Lab Name",
"Name",
"Container ID",
"Image",
"Kind",
"State",
"IPv4 Address",
"IPv6 Address",
}
if all {
table.SetHeader(append([]string{"#", "Topo Path"}, header...))
} else {
table.SetHeader(append([]string{"#"}, header[1:]...))
}
table.SetAutoFormatHeaders(false)
table.SetAutoWrapText(false)
// merge cells with lab name and topo file path
table.SetAutoMergeCellsByColumnIndex([]int{1, 2})
table.AppendBulk(tabData)
table.Render()

if !printMysocket {

case "table":
tabData := toTableData(contDetails)
table := tablewriter.NewWriter(os.Stdout)
header := []string{
"Lab Name",
"Name",
"Container ID",
"Image",
"Kind",
"State",
"IPv4 Address",
"IPv6 Address",
}
if all {
table.SetHeader(append([]string{"#", "Topo Path"}, header...))
} else {
table.SetHeader(append([]string{"#"}, header[1:]...))
}
table.SetAutoFormatHeaders(false)
table.SetAutoWrapText(false)
// merge cells with lab name and topo file path
table.SetAutoMergeCellsByColumnIndex([]int{1, 2})
table.AppendBulk(tabData)
table.Render()

// do not print mysocket data if printMysocket is false or we don't have nodes populated
// nodes are not populated when `inspect --all` is used, since we don't read topology files
if !printMysocket || len(c.Nodes) == 0 {
return nil
}

// prepare data for table
var tabDataMySocketIo [][]string
for _, entry := range socketdata {
var portstrarr []string
for _, port := range entry.Ports {
portstrarr = append(portstrarr, strconv.Itoa(port))
}
tabDataMySocketIo = append(tabDataMySocketIo, []string{*entry.SocketId, *entry.DnsName, strings.Join(portstrarr, ", "), *entry.Type, strconv.FormatBool(entry.CloudAuth), *entry.Name})
}
tableMySocketIo := tablewriter.NewWriter(os.Stdout)
headerMySocketIo := []string{
"Socket ID",
"DNS Name",
"Ports",
"Type",
"Cloud Auth",
"Name",
}
// configure table output
tableMySocketIo.SetHeader(headerMySocketIo)
tableMySocketIo.SetAutoFormatHeaders(false)
tableMySocketIo.SetAutoWrapText(false)
tableMySocketIo.AppendBulk(tabDataMySocketIo)
fmt.Println("Published ports:")
tableMySocketIo.Render()

return nil
}
return nil
}

runtime := c.GlobalRuntime()
// getMySocketioData uses the mysocketio.http client to retrieve the socket data
func getMySocketIoData(tokenfile string) ([]*types.MySocketIoEntry, error) {
result := []*types.MySocketIoEntry{}

stdout, stderr, err := runtime.Exec(context.Background(), mysocketCID, []string{"mysocketctl", "socket", "ls"})
client, err := mysocketio.NewClient(tokenfile)
if err != nil {
return fmt.Errorf("failed to execute cmd: %v", err)
return nil, err
}

sockets := []mysocketio.Socket{}
err = client.Request("GET", "connect", &sockets, nil)
if err != nil {
return nil, err
}
if len(stderr) > 0 {
log.Infof("errors during listing mysocketio sockets: %s", string(stderr))

for i := range sockets {
newentry := &types.MySocketIoEntry{
SocketId: &sockets[i].SocketID,
DnsName: &sockets[i].Dnsname,
Ports: sockets[i].SocketTcpPorts,
Type: &sockets[i].SocketType,
CloudAuth: sockets[i].CloudAuthEnabled,
Name: &sockets[i].Name,
}
result = append(result, newentry)
}
return result, nil
}

fmt.Println("Published ports:")
fmt.Println(string(stdout))
// mySocketIoTokenFileFromBindMounts finds a node of kind mysocketio.
// if that is found, the bindmounts are searched for ".mysocketio_token" and the path is being converted into an
// absolute path and returned.
func mySocketIoTokenFileFromBindMounts(_nodes map[string]nodes.Node, configPath string) (string, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @steiler
there is one little thing with this approach which I am ok to let slip

when you do clab ins -a this func will not work since we don't have c.Nodes populated as we don't read topology files.

One workaround for that is to use GenericContainer type where i added mounts information in this commit. We can rework this function to work on a slice of container info, rather than c.Nodes, and get the binds info from it. But then the problem is what to do if you have >1 mysocketio node, with different tokens...

So for the time being I just made a shunt so that mysocketdata is not fetched if we use inspect --all

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh ok, I see.
What if we then simply print multiple tables indexed with either the containername or maybe better with the lab name, availabel via the container labels?!
In the json case, that would require a change to the output format as well ... so we eiter make the highlevel mysocketio entry a simple list, also carrying the labname as on item of the enclosed entries or a map with the labname as index...
Thought?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I think adding lab name as a field for mysocket data is a valid approach. I would go with a list where lab name is a k:v pair inside the list

when inspect --all is used and the output is in the table format, then I think it is safe to have a single table that lists set up sockets with a field listing a lab for which these sockets were created.
we would just need to make sure we list through all containers which have label[kind]=mysocketio, get their respective source mount path and fetch the sockets.

Quite some work. Maybe better to do that in a separate PR?

// if not mysocketio kind then continue
var mysocketNode nodes.Node
var ok bool

return nil
if mysocketNode, ok = _nodes["mysocketio"]; !ok {
return "", fmt.Errorf("no mysocketio node found")
}
// if "mysocketio" kind then iterate through bind mounts
for _, bind := range mysocketNode.Config().Binds {
// look for ".mysocketio_token"
if strings.Contains(bind, ".mysocketio_token") {
// split the bindmount and resolve the path to an absolute path
deduced_absfilepath := utils.ResolvePath(strings.Split(bind, ":")[0], configPath)
// check file existence before returning
if !utils.FileExists(deduced_absfilepath) {
return "", fmt.Errorf(".mysocketio_token resolved to %s, but that file doesn't exist", deduced_absfilepath)
}
return deduced_absfilepath, nil
}
}

return "", fmt.Errorf("unable to find \".mysocketio_token\"")
}
111 changes: 111 additions & 0 deletions mysocketio/mysocketio.go
@@ -0,0 +1,111 @@
package mysocketio

import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io/ioutil"
h "net/http"
"os"
"strings"
)

type Client struct {
token string
}

func GetApiUrl() string {
if os.Getenv("MYSOCKET_API") != "" {
return os.Getenv("MYSOCKET_API")
} else {
return "https://api.mysocket.io"
}
}

func NewClient(tokenfile string) (*Client, error) {
token, err := GetToken(tokenfile)
if err != nil {
return nil, err
}

c := &Client{token: token}

return c, nil
}

func (c *Client) Request(method, url string, target, data interface{}) error {
jv, _ := json.Marshal(data)
body := bytes.NewBuffer(jv)

req, _ := h.NewRequest(method, fmt.Sprintf("%s/%s", GetApiUrl(), url), body)
req.Header.Add("x-access-token", c.token)
req.Header.Set("Content-Type", "application/json")
client := &h.Client{}
resp, err := client.Do(req)
if err != nil {
return err
}

defer resp.Body.Close()

if resp.StatusCode == 401 {
return errors.New("no valid token, Please login")
}

if resp.StatusCode < 200 || resp.StatusCode > 204 {
responseData, _ := ioutil.ReadAll(resp.Body)
return fmt.Errorf("failed to create object (%d) %v", resp.StatusCode, string(responseData))
}

if resp.StatusCode == 204 {
return nil
}

err = json.NewDecoder(resp.Body).Decode(target)
if err != nil {
return errors.New("failed to decode data")
}

return nil
}

func GetToken(tokenfile string) (string, error) {
if _, err := os.Stat(tokenfile); os.IsNotExist(err) {
return "", errors.New("please login first (no token found)")
}
content, err := ioutil.ReadFile(tokenfile)
if err != nil {
return "", err
}

tokenString := strings.TrimRight(string(content), "\n")
return tokenString, nil
}

type Socket struct {
Tunnels []Tunnel `json:"tunnels,omitempty"`
Username string `json:"user_name,omitempty"`
SocketID string `json:"socket_id,omitempty"`
SocketTcpPorts []int `json:"socket_tcp_ports,omitempty"`
Dnsname string `json:"dnsname,omitempty"`
Name string `json:"name,omitempty"`
SocketType string `json:"socket_type,omitempty"`
ProtectedSocket bool `json:"protected_socket"`
ProtectedUsername string `json:"protected_username"`
ProtectedPassword string `json:"protected_password"`
CloudAuthEnabled bool `json:"cloud_authentication_enabled,omitempty"`
AllowedEmailAddresses []string `json:"cloud_authentication_email_allowed_addressses,omitempty"`
AllowedEmailDomains []string `json:"cloud_authentication_email_allowed_domains,omitempty"`
SSHCa string `json:"ssh_ca,omitempty"`
UpstreamUsername string `json:"upstream_username,omitempty"`
UpstreamPassword string `json:"upstream_password,omitempty"`
UpstreamHttpHostname string `json:"upstream_http_hostname,omitempty"`
UpstreamType string `json:"upstream_type,omitempty"`
}

type Tunnel struct {
TunnelID string `json:"tunnel_id,omitempty"`
LocalPort int `json:"local_port,omitempty"`
TunnelServer string `json:"tunnel_server,omitempty"`
}
9 changes: 9 additions & 0 deletions runtime/docker/docker.go
Expand Up @@ -644,6 +644,15 @@ func (d *DockerRuntime) produceGenericContainerList(inputContainers []dockerType
ctr.NetworkSettings.IPv6addr = ifcfg.GlobalIPv6Address
ctr.NetworkSettings.IPv6pLen = ifcfg.GlobalIPv6PrefixLen
}

// populating mounts information
var mount types.ContainerMount
for _, m := range i.Mounts {
mount.Source = m.Source
mount.Destination = m.Destination
}
ctr.Mounts = append(ctr.Mounts, mount)

result = append(result, ctr)
}

Expand Down