Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Websockets 2695 v9 #10104

Closed
wants to merge 10 commits into from
Closed

Conversation

catenacyber
Copy link
Contributor

Link to redmine ticket:
https://redmine.openinfosecfoundation.org/issues/2695
https://redmine.openinfosecfoundation.org/issues/6647
https://redmine.openinfosecfoundation.org/issues/6645
https://redmine.openinfosecfoundation.org/issues/6648

Describe changes:

  • app-layer: websockets protocol support
  • rust : derive for protocol enumerations strings
  • enip: register on default port 44818 also for TCP (as is done on UDP)
  • http2: add newer settings, including the one for websocket over HTTP/2 cf https://www.rfc-editor.org/rfc/rfc8441#section-3
  • detect: integer keywords now accept bitmasks
SV_BRANCH=pr/1571

OISF/suricata-verify#1571

#10093 with

  • commit detect: integer keywords now accept bitmasks
  • keyword websocket.fin is now websocket.flags (and can use bit mask functionality)
  • protocol detection : fixed and split up in multiple own commits, tricky part is allow not port-based probing parsers, then run expected probing parser (which may improve detection for already known protocols)

I think this is good enough for a first version even if there may be improvements (that can happen in later tickets) :

  • payload logging : waiting for feedback wether this is only for alerts, or also for websocket event
  • using latest HTTP/1 transaction in a single rule, instead of splitting rule and using flowbits
  • support websockets over HTTP/2 : need pcap
    This is a big one as websockets over HTTP/2 only use a single HTTP/2 stream and not the whole TCP connection which keeps having newer regular HTTP/2 streams

#10089 and #9870 should likely precede this

catenacyber and others added 9 commits December 22, 2023 16:03
if no config option is found,
as is done for udp

Ticket: 6304
Ticket: 6647

Allows keywords using integers to use strings in signature
parsing based on a rust enumeration with a derive.
Including the one for websocket over HTTP/2
port is used in AppLayerProtoDetectProbingParserPort
and not in AppLayerProtoDetectProbingParserElement
As for WebSocket which is detected only by protocol change.
When there is a protocol change, and a specific protocol is
expected, like WebSeocket, always run it, no matter the port.
So that we can write enip.revision: 0x203

Ticket: 6645
Ticket: 6648

Like &0x40=0x40 to test for a specific bit set
@catenacyber
Copy link
Contributor Author

The check rust failure is due to the use of a new version of clippy and will fail master (jsonbuilder.rs)

Copy link

codecov bot commented Jan 2, 2024

Codecov Report

Attention: 82 lines in your changes are missing coverage. Please review.

Comparison is base (5cc872f) 82.19% compared to head (563d596) 81.96%.
Report is 2 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #10104      +/-   ##
==========================================
- Coverage   82.19%   81.96%   -0.23%     
==========================================
  Files         975      980       +5     
  Lines      271940   272486     +546     
==========================================
- Hits       223523   223352     -171     
- Misses      48417    49134     +717     
Flag Coverage Δ
fuzzcorpus 62.31% <51.40%> (-0.62%) ⬇️
suricata-verify 61.47% <77.11%> (+0.04%) ⬆️
unittests 62.76% <27.52%> (-0.08%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@suricata-qa
Copy link

WARNING:

field baseline test %
SURI_TLPR1_stats_chk
.tcp.pseudo 2810 3014 107.26%

Pipeline 17317

@suricata-qa
Copy link

WARNING:

field baseline test %
SURI_TLPR1_stats_chk
.tcp.pseudo 2810 3014 107.26%

Pipeline 17320

Comment on lines +61 to +73
#[no_mangle]
pub unsafe extern "C" fn SCWebSocketParseOpcode(
ustr: *const std::os::raw::c_char,
) -> *mut DetectUintData<u8> {
let ft_name: &CStr = CStr::from_ptr(ustr); //unsafe
if let Ok(s) = ft_name.to_str() {
if let Some(ctx) = WebSocketOpcode::to_detect_ctx(s) {
let boxed = Box::new(ctx);
return Box::into_raw(boxed) as *mut _;
}
}
return std::ptr::null_mut();
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we return DetectUintData<u8>, it would be cleaner if we parsed that I think, like:

	if let Ok(ctx) = DetectUintData::<u8>::from_str(s) {
	    let boxed = Box::new(ctx);
	    return Box::into_raw(boxed) as *mut _;
	}

would require this impl though:

impl<T: DetectIntType> FromStr for DetectUintData<T> {
    type Err = ();

    fn from_str(s: &str) -> Result<Self, Self::Err> {
	if let Ok((_, v)) = detect_parse_uint::<T>(s) {
	    Ok(v)
	} else {
	    Err(())
	}
    }
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider this patch to keep things generic: https://gist.github.com/jasonish/43f6672af27fc541352285853cd64edc

Or lets give it a better name. This is an enum type that is tightly couple to integer types and detectuintdata..

So maybe DetectIntEnum. This fits with where this trait is currently implemented, in the detect crate, and now that the naming is clear, I think its more fair to put arbitrary code in that that is less generic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does the patch wok for websocket.opcode: 1 ? (numeric value instead of string value)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does the patch wok for websocket.opcode: 1 ? (numeric value instead of string value)

Oh, the tests must not test for that as S-V did pass.

I do have the trait there so you can do:

let ctx = DetectUintData::from_str(s)

as well. So you would try one that the other. It just moves the conversions to be a long side their datatype, rather than somewhere that seems a little random.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It just moves the conversions to be a long side their datatype, rather than somewhere that seems a little random.

Seems right, see d5f1bf2 for next iteration

@catenacyber catenacyber marked this pull request as draft January 4, 2024 10:09
@catenacyber
Copy link
Contributor Author

Draft until #10110 is merged and this is rebased on it

@catenacyber catenacyber mentioned this pull request Jan 5, 2024
@catenacyber
Copy link
Contributor Author

Continues in #10119

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
3 participants