Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds a Serialize function #9

Merged
merged 2 commits into from Mar 27, 2018
Merged

Conversation

@nbareil
Copy link
Contributor

nbareil commented Mar 22, 2018

As referenced in #6, this PR adds a function to serialize memory structures into the Yara rule format.

I do not like adding this function in the data package but I don't see where it would fit better (and I don't want to create an util package), please feel free to move it at the right place.

@Northern-Lights

This comment has been minimized.

Copy link
Owner

Northern-Lights commented Mar 23, 2018

Thanks for getting this started. I will have a few comments for you.

Copy link
Owner

Northern-Lights left a comment

Looking good. Thanks for doing this. A couple changes should be made before we merge this in.

@@ -18,6 +24,78 @@ type Rule struct {
Condition string `json:"condition"`
}

// Serialize builds the Yara rule
func (rule *Rule) Serialize(output io.Writer) {

This comment has been minimized.

Copy link
@Northern-Lights

Northern-Lights Mar 23, 2018

Owner

I think ultimately, these serialization functions can go in a utils.go file. They will still be in the same package (directory); just a different file. We can modify it later, though, and it won't affect the library, which is great.

Also, I think maybe what we can do later on is use TextMarshaler on each of the types so that a ruleset can recursively be serialized. I need to do more research on it, though; I'm not 100% sure it's the best tool for the job.

if len(rule.Meta) > 0 {
fmt.Fprintf(output, " meta:\n")
for _, meta := range rule.Meta {
if _, ok := meta.Val.(string); ok {

This comment has been minimized.

Copy link
@Northern-Lights

Northern-Lights Mar 23, 2018

Owner

I recommend using a type switch here, something like:

switch t:= meta.Val.(type); t {
case string:
    fmt.Fprintf(output, "    %s = \"%s\"\n", meta.Key, meta.Val)

case int64, bool:
    fmt.Fprintf(output, "    %s = %v\n", meta.Key, meta.Val)

default:
    err := fmt.Errorf("Unsupported meta type %T\n", t)
    panic(err)
}
fmt.Fprintf(output, " fullword")
}

if s.Modifiers.I {

This comment has been minimized.

Copy link
@Northern-Lights

Northern-Lights Mar 23, 2018

Owner

The i and s modifiers should go directly after the regex string, for example:

rule regex_modifiers {
strings:
    $r1 = /regex with modifiers/is
condition:
    any of them
}
@Northern-Lights

This comment has been minimized.

Copy link
Owner

Northern-Lights commented Mar 24, 2018

What do you think about this:

8f44e38

I was going for recursive serialization, so that no matter which struct you start from (data.Ruleset, data.Rule, data.String, etc.), everything contained within the struct you are using will also get serialized.

If you feel like passing in an io.Writer would be better for the serialization methods, I'm thinking maybe the signature can look something like:

type YARASerializer interface {
    Serialize(io.Writer) error
}

That is, each struct type implements that method. Doing it this way might even help to bubble errors upward. If you would like to go with this or build upon the file I pushed to the branch, feel free. I will get around to it over the next week, otherwise.

@Northern-Lights

This comment has been minimized.

Copy link
Owner

Northern-Lights commented Mar 26, 2018

I put together some serialization methods here in a new branch: https://github.com/Northern-Lights/yara-parser/blob/serialization/data/serialize.go

They implement the following (hypothetical) interface:

type YARASerializer interface {
    Serialize() (string, error)
}

The error value is surfaced to the caller if something goes wrong during serialization. This helps us identify in which subcomponent something went wrong, and in the future it will help with verification (e.g. "is this rule name a valid YARA rule name?")

The string is the YARA text for whatever component is being serialized.

As an example of the serialization, if you use ruleset.Rules[0].Strings.Serialize(), the text will be something like:

strings:
  $s1 = "string 1"

If you use ruleset.Rules[0].Strings[0].Serialize(), then it will simply be:

$s1 = "string 1"

And it will work with any of the structs, recursively creating each of the subcomponents.

The serialization for the condition is kind of ugly because I'm assuming an indent of 2 spaces. Actually, all of the indenting and newlining may not be what most people are used to. I'm open to suggestions on how to better handle and/or standardize the formatting of reserialized rules.

@nbareil

This comment has been minimized.

Copy link
Contributor Author

nbareil commented Mar 26, 2018

Sorry I was offline this weekend, couldn't update this pull request in time.

Your code is much cleaner and better, thanks!

@Northern-Lights Northern-Lights changed the base branch from master to serialize Mar 27, 2018
@Northern-Lights Northern-Lights merged commit f5689d5 into Northern-Lights:serialize Mar 27, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.