Wally is a static analysis tool for attack surface mapping. It automates the initial stages of threat modelling by mapping RPC and HTTP routes in Go code.
demo_ui.mov
Read about this graph and how to explore it in the Exploring the graph with wally server section
Because Wally is a cartographer, I like Monkey Island, and I wanted it to be called that :).
So you are analyzing a Go-based application and you need to find all HTTP and RPC routes. You can run grep or ripgrep to find specific patterns that'd point you to routes in the code but:
- You'd need to parse through a lot of unnecessary strings.
- You may end up with functions that are similar to those you are targeting but have nothing to do with HTTP or RPC.
- Grep won't solve constant values that indicate methods and route paths.
Wally currently supports the following features:
- Discover HTTP client calls and route listeners in your code by looking at each function name, signature, and package to make sure it finds the functions that you actually care about.
- Wally solves the value of compile-time constant values that may be used in the functions of interest. Wally does a pretty good job at finding constants and global variables and resolving their values for you so you don't have to chase those manually in code.
- Wally will report the enclosing function where the function of interest is called.
- Wally will also give you all possible call paths to your functions of interest. This can be useful when analyzing monorepos where service A calls service B via a client function declared in service B's packages. This feature requires that the target code base is buildable.
- Wally will output a nice PNG graph of the call stacks for the different routes it finds.
You are conducting an analysis of a monorepo containing multiple microservices. Often, these sorts of projects rely heavily on gRPC, which generates code for setting up gRPC routes via functions that call Invoke
. Other services can then use these functions to call each other.
One of the built-in indicators in wally
will allow it to find functions that call Invoke
for gRPC routes, so you can get a nice list of all gRPC method calls for all your microservices. Further, with --ssa
you can also map the chains of methods gRPC calls necessary to reach any given gRPC route. With wally
, you can then answer:
- Can users reach service
Y
hosted internally via serviceA
hosted externally? - Which service would I have to initialize a call to send user input to service
X
? - What functions are there between service
A
and serviceY
that might sanitize or modify the input set to serviceA
?
Wally needs a bit of hand-holding. Though it can also do a pretty good job at guessing paths, it helps a lot if you tell it the packages and functions to look for, along with the parameters that you are hoping to discover and map. So, to help Wally do the job, you can specify a configuration file in YAML that defines a set of indicators.
Wally runs a number of indicators
which are basically clues as to whether a function in code may be related to a gRPC or HTTP route. By default, wally
has a number of built-in indicators
which check for common ways to set up and call HTTP and RPC methods using standard and popular libraries. However, sometimes a codebase may have custom methods for setting up HTTP routes or for calling HTTP and RPC services. For instance, when reviewing Nomad, you can give Wally the following configuration file with Nomad-specific indicators:
indicators:
- id: nomad-1
package: "github.com/hashicorp/nomad/command/agent"
type: ""
function: "forward"
indicatorType: 1
params:
- name: "method"
- id: nomad-2
package: "github.com/hashicorp/nomad/nomad"
type: ""
function: "RPC"
indicatorType: 1
params:
- name: "method"
- id: nomad-3
package: "github.com/hashicorp/nomad/api"
type: "s"
function: "query"
indicatorType: 1
params:
- name: "endpoint"
pos: 0
Note that you can specify the parameter that you want Wally to attempt to solve the value to. If you don't know the name of the parameter (per the function signature), you can give it the position in the signature. You can then use the --config
or -c
flag along with the path to the configuration file.
A good test project to run it against is nomad because it has a lot of routes set up and called all over the place. I suggest the following:
- Clone this project.
- In a separate directory, clone nomad.
- Build this project by running
go build
. - Navigate to the root of the directory where you cloned nomad (
path/to/nomad
). - Create a configuration file named
.wally.yaml
with the content shown in the previous section of this README, and save it to the root of the nomad directory. - Run the following command from the nomad root:
$ <path/to/wally/wally> map -p ./... -vvv
Wally can be easily run using Docker. Follow these steps:
-
Clone this project.
-
In a separate directory, clone nomad.
-
Build the Docker Image:
docker build -t go-wally .
-
Run an interactive shell inside the Docker container
docker run -it go-wally /bin/sh
-
Run Wally with Docker, specifying the necessary parameters, such as the project path, configuration file, etc.:
docker run -w /<PROJECT>/ -v $(pwd):/<PROJECT> go-wally map /<PROJECT>/... -vvv
Adjust the flags (-p, -vvv, etc.) as needed for your use case.
-
If you have a specific configuration file (e.g., .wally.yaml), you can mount it into the container:
docker run -w </PROJECT> -v $(pwd):</PROJECT> -v </PATH/TO/.wally.yaml>:</PROJECT>/.wally.yaml go-wally map -c .wally.yaml -p ./... -vvv
This will run Wally within a Docker container, analyzing your Go code for HTTP and RPC routes based on the specified indicators and configurations.
-
Optionally, if you encountered any issues during the Docker build, you can revisit the interactive shell inside the container for further debugging.
-
After running Wally, you can check the results and the generated PNG or XDOT graph output, as explained in the README.
Wally should work even if you are not able to build the project you want to run it against. However, if you can build the project without any issues, you can run Wally using the --ssa
flag, at which point Wally will be able to do the following:
- Solve the enclosing function more effectively using SSA.
- Output all possible call paths to the functions where the routes are defined and/or called.
When using the --ssa
flag you can expect output like this:
===========MATCH===============
Package: net/http
Function: Handle
Params:
pattern: "/v1/client/metadata"
Enclosed by: agent.registerHandlers
Position /Users/hex0punk/Tests/nomad/command/agent/http.go:444
Possible Paths: 6
Path 1:
n105973:(*github.com/hashicorp/nomad/command/agent.Command).Run --->
n24048:(*github.com/hashicorp/nomad/command/agent.Command).setupAgent --->
n24050:github.com/hashicorp/nomad/command/agent.NewHTTPServers --->
n47976:(*github.com/hashicorp/nomad/command/agent.HTTPServer).registerHandlers --->
Path 2:
n104203:github.com/hashicorp/nomad/command/agent.NewTestAgent --->
n92695:(*github.com/hashicorp/nomad/command/agent.TestAgent).Start --->
n32861:(*github.com/hashicorp/nomad/command/agent.TestAgent).start --->
n24050:github.com/hashicorp/nomad/command/agent.NewHTTPServers --->
n47976:(*github.com/hashicorp/nomad/command/agent.HTTPServer).registerHandlers --->
Path 3:
n105973:(*github.com/hashicorp/nomad/command/agent.Command).Run --->
n117415:(*github.com/hashicorp/nomad/command/agent.Command).handleSignals --->
n79534:(*github.com/hashicorp/nomad/command/agent.Command).handleReload --->
n79544:(*github.com/hashicorp/nomad/command/agent.Command).reloadHTTPServer --->
n24050:github.com/hashicorp/nomad/command/agent.NewHTTPServers --->
n47976:(*github.com/hashicorp/nomad/command/agent.HTTPServer).registerHandlers --->
Tip
When running Wally in SSA mode against large codebases wally might run get lost in external libraries used by the target code. In most cases, you'd want to filter analysis to only the module you want to target. For instance, when using wally to find HTTP and gRPC routes in nomad, you'd want to type the command below.
$ wally map -p ./... --ssa -vvv -f "github.com/hashicorp/nomad/"
Where -f
defines a filter for the call stack search function. If you don't do this, wally may end up getting stuck in some loop as it encounters recursive calls or very lengthy paths in scary dependency forests.
Important
If using -f
is not enough, and you are seeing Wally taking a very long time in the "solving call paths" step, Wally may have encountered some sort of recursive call. In that case, you can use -l
and an integer to limit the number of recursive calls Wally makes when mapping call paths. This will limit the paths you see in the output, but using a high enough number should still return helpful paths. Experiment with -l
, -f
, or both to get the results you need or expect.
To make visualization of callpaths easier, wally can lunch a server on localhost when via a couple methods:
After an analysis by passing the --server
flag to the map
command. For instance:
$ wally map -p ./... -c .wally.yaml --ssa -f "github.com/hashicorp/nomad" --server
Or, using the server
subcommand and passing a wally json file:
$ wally server -p ./nomad-wally.json -P 1984
Next, open a browser and head to the address in the output.
Graphs are generated using the cosmograph library. Each node represents a function call in code. The colors are not random. Each color has a a different purpose to help you make good use of the graph.
Finding node. This is a node discovered via wally indicators. Every finding node is the end of a path
This node is the root of a path to a finding node.
Intermediate node between a root and a finding node.
This node servers both as the root node to a path and an intermediary node for one or more paths
Clicking on any node will highlight all possible paths to that node. Click anywhere other than a node to exist the path selection view.
Clicking on any finding node will populate the section on the left with information about the finding.
Start typing on the search bar on the left to find a node by name. This feature is currently very experimental.
When using the --ssa
flag, you can also use -g
or --graph
to indicate a path for a PNG or XDOT containing a Graphviz-based graph of the call stacks. For example, running:
$ wally map -p ./... --ssa -vvv -f "github.com/hashicorp/nomad/" -g ./mygraph.png
From nomad/command/agent will output this graph:
Specifying a filename with a .xdot
extension will create an xdot file instead.
In the future, Wally will be able to make educated guesses for potential HTTP or RPC routes with no additional indicators. For now, you can define indicators with a wildcard package ("*"
) if you are not able (or don't want) to tell Wally which package each function may be coming from.
At its core, Wally is, essentially, a function mapper. You can define functions in configuration files that have nothing to do with HTTP or RPC routes to obtain the same information that is described here.
You can add logging statements as needed during development in any function with a Navigator
receiver like this: n.Logger.Debug("your message", "a key", "a value")
.
At the moment, wally will often give you duplicate stack paths, where you'd notice a path of, say, A->B->C is repeated a couple of times or more. Based on my testing and debugging this is a drawback of the cha
algorithm from Go's callgraph
package, which wally uses for the call stack path functionality. I am experimenting with other available algorithms in go/callgraph/
to determine what the best option to minimize such issues (while getting accurate call stacks) could be and will update wally's code accordingly. In the case that we stick to the cha
algorithm, I will write code to filter duplicates.
This is often caused by issues in the target code base. Make sure you are able to build the target codebase. You may want to run go build
and fix any issues reported by the compiler. Then, run wally again against it.
See the section on Filtering call path analysis
Viewing the description of each command
$ wally map --help
$ wally map --help
Feel free to open issues and send PRs. Please.