Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ftp/eve: Convert to JsonBuilder #5030

Closed
wants to merge 2 commits into from
Closed

Conversation

jlucovsky
Copy link
Contributor

Link to redmine ticket: 3714

Describe changes:

  • Convert to JsonBuilder
  • Remove unused helper function (no longer needed)

#suricata-verify-pr:
#suricata-verify-repo:
#suricata-verify-branch:
#suricata-update-pr:
#suricata-update-repo:
#suricata-update-branch:
#libhtp-pr:
#libhtp-repo:
#libhtp-branch:

This commit converts the FTP logging mechanisms to use JsonBuilder.
This commit removes an unused helper function no longer required/used
after conversion to JsonBuilder.
@jlucovsky jlucovsky requested a review from jasonish June 6, 2020 18:08
@jlucovsky jlucovsky requested a review from a team as a code owner June 6, 2020 18:08
json_object_set_new(cjs, "command_data",
JsonAddStringN((const char *)tx->request + min_length,
tx->request_length - min_length));
char *s = BytesToString(tx->request + min_length,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jasonish iirc we talked about a direct jb_set_bytes or something to avoid an expensive operation like this?

Copy link
Contributor Author

@jlucovsky jlucovsky Jun 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see jb_set_string_from_bytes ... That requires a null-terminated string which I don't have.

An interface that accepted a count and worked with a non-null-terminated string would be needed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jb_set_string_from_bytes should work, just looks like it doesn't have an extern "C" wrapper yet (haven't needed it from C yet would be the reason).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See this PR: #5040

Should be able to:

jb_append_string_from_bytes(js_respcode_list, where, 3);

JsonAddStringN((const char *)where, 3));
char *s = BytesToString(where, 3);
if (s != NULL) {
jb_append_string(js_respcode_list, s);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the goal of the jsonbuilder transition is to avoid creating temporary objects like this, but instead "stream" the whole object into a single jb.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I modeled my implementation after that in output-json-email-common.c.

Note that I'm building 2 arrays as the response is processed and 2 arrays will be created during that. I'm not sure how the new interfaces support this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There isn't in some cases. In DNS, to avoid looping throught the responses 2x, I had to use intermediate objects as well in dns_log_json_answer.

break;
case FTP_COMMAND_RETR:
json_object_set_new(ftpd, "command", json_string("RETR"));
jb_set_string(jb, "command", "RETR");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jasonish I still think we can optimize these constant cases with some preproc magic to become a single string that we append

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, need to look at this again.

@jlucovsky
Copy link
Contributor Author

Continued in #5049

@jlucovsky jlucovsky closed this Jun 9, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
3 participants