Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds a Serialize function #9

Merged
merged 2 commits into from Mar 27, 2018
Merged

Conversation

nbareil
Copy link
Contributor

@nbareil nbareil commented Mar 22, 2018

As referenced in #6, this PR adds a function to serialize memory structures into the Yara rule format.

I do not like adding this function in the data package but I don't see where it would fit better (and I don't want to create an util package), please feel free to move it at the right place.

@Northern-Lights
Copy link
Owner

@Northern-Lights Northern-Lights commented Mar 23, 2018

Thanks for getting this started. I will have a few comments for you.

Copy link
Owner

@Northern-Lights Northern-Lights left a comment

Looking good. Thanks for doing this. A couple changes should be made before we merge this in.

@@ -18,6 +24,78 @@ type Rule struct {
Condition string `json:"condition"`
}

// Serialize builds the Yara rule
func (rule *Rule) Serialize(output io.Writer) {
Copy link
Owner

@Northern-Lights Northern-Lights Mar 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think ultimately, these serialization functions can go in a utils.go file. They will still be in the same package (directory); just a different file. We can modify it later, though, and it won't affect the library, which is great.

Also, I think maybe what we can do later on is use TextMarshaler on each of the types so that a ruleset can recursively be serialized. I need to do more research on it, though; I'm not 100% sure it's the best tool for the job.

if len(rule.Meta) > 0 {
fmt.Fprintf(output, " meta:\n")
for _, meta := range rule.Meta {
if _, ok := meta.Val.(string); ok {
Copy link
Owner

@Northern-Lights Northern-Lights Mar 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend using a type switch here, something like:

switch t:= meta.Val.(type); t {
case string:
    fmt.Fprintf(output, "    %s = \"%s\"\n", meta.Key, meta.Val)

case int64, bool:
    fmt.Fprintf(output, "    %s = %v\n", meta.Key, meta.Val)

default:
    err := fmt.Errorf("Unsupported meta type %T\n", t)
    panic(err)
}

fmt.Fprintf(output, " fullword")
}

if s.Modifiers.I {
Copy link
Owner

@Northern-Lights Northern-Lights Mar 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The i and s modifiers should go directly after the regex string, for example:

rule regex_modifiers {
strings:
    $r1 = /regex with modifiers/is
condition:
    any of them
}

@Northern-Lights
Copy link
Owner

@Northern-Lights Northern-Lights commented Mar 24, 2018

What do you think about this:

8f44e38

I was going for recursive serialization, so that no matter which struct you start from (data.Ruleset, data.Rule, data.String, etc.), everything contained within the struct you are using will also get serialized.

If you feel like passing in an io.Writer would be better for the serialization methods, I'm thinking maybe the signature can look something like:

type YARASerializer interface {
    Serialize(io.Writer) error
}

That is, each struct type implements that method. Doing it this way might even help to bubble errors upward. If you would like to go with this or build upon the file I pushed to the branch, feel free. I will get around to it over the next week, otherwise.

@Northern-Lights
Copy link
Owner

@Northern-Lights Northern-Lights commented Mar 26, 2018

I put together some serialization methods here in a new branch: https://github.com/Northern-Lights/yara-parser/blob/serialization/data/serialize.go

They implement the following (hypothetical) interface:

type YARASerializer interface {
    Serialize() (string, error)
}

The error value is surfaced to the caller if something goes wrong during serialization. This helps us identify in which subcomponent something went wrong, and in the future it will help with verification (e.g. "is this rule name a valid YARA rule name?")

The string is the YARA text for whatever component is being serialized.

As an example of the serialization, if you use ruleset.Rules[0].Strings.Serialize(), the text will be something like:

strings:
  $s1 = "string 1"

If you use ruleset.Rules[0].Strings[0].Serialize(), then it will simply be:

$s1 = "string 1"

And it will work with any of the structs, recursively creating each of the subcomponents.

The serialization for the condition is kind of ugly because I'm assuming an indent of 2 spaces. Actually, all of the indenting and newlining may not be what most people are used to. I'm open to suggestions on how to better handle and/or standardize the formatting of reserialized rules.

@nbareil
Copy link
Contributor Author

@nbareil nbareil commented Mar 26, 2018

Sorry I was offline this weekend, couldn't update this pull request in time.

Your code is much cleaner and better, thanks!

@Northern-Lights Northern-Lights changed the base branch from master to serialize Mar 27, 2018
@Northern-Lights Northern-Lights merged commit f5689d5 into Northern-Lights:serialize Mar 27, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants