L for general logging view instead of victorialogs

[polo871209]

I update my L keymap to directly to gcp logging provider with the plugin feature with shell. One of my most use feature.

proposal:
Detect the context, base on context cluster, direct the log to different cloud provider base on the regex
If want to direct to other log platform, config logging feature to other platform with url, support default like victorialogs, kibana, grafana, datadog, dynatrace or other popular platform, when you direct to it has default filter for the resource.

[nklmilojevic]

Thanks for this, I like the idea! Before we lock the scope, a few questions:

• Can you share the plugin you're using now? The TOML and the GCP Logging URL/filter you build would help a lot.
• How would you expect L to behave for a namespace or a deployment, not just a pod? Most log UIs need a different filter there.
• Do you want L to open the browser, or do you expect logs inside sofka like the VictoriaLogs view?
• Do you work over SSH or anywhere without a browser? Then we need a fallback, f.e. show the URL and copy it.

What I think so far:

• Routing per context already works. [providers.logs] follows the per-cluster and per-context override files (clusters/<cluster>/<context>/config.toml), so each cluster can point L at its own backend. I'd rather not add a second regex router next to that.
• The new part is a link provider - L fills a URL template ({context}, {cluster}, {namespace}, {kind}, {name}, {pod}, {container}, {lookback}, URL-encoded) and opens it in the browser. That covers GCP, Grafana, Kibana, Datadog, Dynatrace and anything else with a URL.
• Platform presets would live in the docs as example templates, not in code. Vendors change their URL formats and I don't want sofka to chase five of them.
• Querying GCP, Datadog, Elastic etc. inside sofka is out of scope for now - each one is a separate API and auth story. A native Loki backend could be its own discussion later.

Does that cover what you do with the plugin today?

[polo871209]

hi, here is how I use the plugin, simply map it to a shell script, source is here

behavior:
I would prefer it to open browser like below, with the filter already set, the purpose is to quickly jump to the log view that we use daily instead of built in with the TUI

for different set of the filter we can use the the cluster name in current context, for example
it is important not to use context name, for me I always custom my context name

contexts:
  - context:
        cluster: gke_{project}_asia-southeast1_{cluster}

for config for popular providers I think it would be nice to have default detection and other popular platform filter unless overwrite in config, in terns of implementation either in code or a default config both fine for me

and yes I do not need/prefer sofka query log from the remote source, since this is very heavy

[nklmilojevic]

Thanks, the script helps a lot! The selector handling is the part a plain URL template can't do, so here's the scope I'd lock:

• L opens the log UI in the browser with the filter already set. sofka won't query GCP, Datadog, Elastic etc. itself.

• A gcp log provider type is built in. With no other log provider configured, it turns on by itself for clusters named gke_<project>_<location>_<cluster>. It uses the kubeconfig cluster name, never the context name, so your renamed contexts don't matter.

• It builds the same filters your script does:

  • pods, namespaces and nodes match on their own fields
  • CronJobs match on the logging.gke.io/top_level_controller_* labels
  • Deployments, StatefulSets, DaemonSets, ReplicaSets, Jobs and Services turn their pod selector into labels."k8s-pod/..." terms, with In, NotIn, Exists and DoesNotExist all handled

  sofka already has the object in memory, so it doesn't need a kubectl call for this.

• EKS and AKS aren't detected for now. Neither ships container logs to a fixed place by default, so sofka can't build a working link without config. That can be its own discussion once someone can test it on a real account.

• A generic link type takes a URL template: {context}, {cluster}, {namespace}, {kind}, {name}, {pod}, {container}, {lookback}, all URL-encoded. Grafana, Kibana, Datadog and Dynatrace get example templates in the docs. A platform gets a built-in type like gcp only if someone needs selector support there and can test it on a real account.

• You pick the provider in the existing [providers.logs] table with type = "gcp" or type = "link". The per-cluster override files (clusters/<cluster>/config.toml) are keyed by kubeconfig cluster name too, so you can set a different provider per cluster without a new routing setting.

• If no browser opener is available (SSH, no open/xdg-open), sofka shows the URL in the status bar and copies it to the clipboard.

• VictoriaLogs stays the default. With nothing configured and no GKE match, L works the same as it does now.

Once that's in, you can drop the plugin and the provider_logs = [] override from your config.

Sounds good? I'll start working on it :)

[polo871209]

sounds great to me as starting point, this is all I need for built-in feature and can extend if other people want similar support

[nklmilojevic]

it is already out in v0.31.4 :)