New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Elastic Stack Release Candidate 1 #1179

Closed
dougburks opened this Issue Dec 8, 2017 · 10 comments

Comments

Projects
None yet
4 participants
@dougburks
Contributor

dougburks commented Dec 8, 2017

@RANGERBEE

This comment has been minimized.

Show comment
Hide comment
@RANGERBEE

RANGERBEE Dec 8, 2017

RANGERBEE commented Dec 8, 2017

@RANGERBEE

This comment has been minimized.

Show comment
Hide comment
@RANGERBEE

RANGERBEE Dec 14, 2017

To add: for consideration:

Kibana: Home page::

 - Counter for sensor reads ZERO regardless of how many sensors are active or how many ossec agents live/active.
 - Devices counter shows sensors and the master server just fine. 
 - Clarity on the "localhost" within the devices counter list. If this is the host Ubuntu operating system to the docker containers, understood. How can it be renamed ?

RANGERBEE commented Dec 14, 2017

To add: for consideration:

Kibana: Home page::

 - Counter for sensor reads ZERO regardless of how many sensors are active or how many ossec agents live/active.
 - Devices counter shows sensors and the master server just fine. 
 - Clarity on the "localhost" within the devices counter list. If this is the host Ubuntu operating system to the docker containers, understood. How can it be renamed ?
@weslambert

This comment has been minimized.

Show comment
Hide comment
@weslambert

weslambert Dec 14, 2017

Collaborator

To address your questions:

  • Counter for sensor reads ZERO regardless of how many sensors are active or how many ossec agents live/active.
  • Devices counter shows sensors and the master server just fine.

Are you sure you have the latest updates? If you click to edit the visualization and check the panel options, you should get the correct results with the index set as *:logstash-* (which should be correct with the latest updates).

  • Clarity on the "localhost" within the devices counter list. If this is the host Ubuntu operating system to the docker containers, understood. How can it be renamed ?

This is the host from which the syslog was delivered. This visualization gets the count value by determining the number of unique values for the syslog-host_from field. In the future, for clarification, one possibility may be to enrich this value with a hostname during processing if syslog-host_from is equal to localhost.

Another option would be to update /etc/syslog-ng/syslog-ng.conf with:

options {   
        keep-hostname(yes);   
};  

so that the hostname will be written out as the machine's hostname, instead of "localhost".

Please make sure to pose any other questions or feedback to the mailing list:
https://github.com/Security-Onion-Solutions/security-onion/wiki/MailingLists#mailing-lists

Thanks,
Wes

Collaborator

weslambert commented Dec 14, 2017

To address your questions:

  • Counter for sensor reads ZERO regardless of how many sensors are active or how many ossec agents live/active.
  • Devices counter shows sensors and the master server just fine.

Are you sure you have the latest updates? If you click to edit the visualization and check the panel options, you should get the correct results with the index set as *:logstash-* (which should be correct with the latest updates).

  • Clarity on the "localhost" within the devices counter list. If this is the host Ubuntu operating system to the docker containers, understood. How can it be renamed ?

This is the host from which the syslog was delivered. This visualization gets the count value by determining the number of unique values for the syslog-host_from field. In the future, for clarification, one possibility may be to enrich this value with a hostname during processing if syslog-host_from is equal to localhost.

Another option would be to update /etc/syslog-ng/syslog-ng.conf with:

options {   
        keep-hostname(yes);   
};  

so that the hostname will be written out as the machine's hostname, instead of "localhost".

Please make sure to pose any other questions or feedback to the mailing list:
https://github.com/Security-Onion-Solutions/security-onion/wiki/MailingLists#mailing-lists

Thanks,
Wes

@RANGERBEE

This comment has been minimized.

Show comment
Hide comment
@RANGERBEE

RANGERBEE Dec 14, 2017

I have your "pass-thru-cache" enabled for the docker registry and updated all an hour ago.
Found that the index for the visualization reads " :logstash- " time now, after updates to master and sensors.

( I see in the Viz editor that the counter reads correct. Though not on home screen )

A; Found it. The builder index pattern did not match the counter visualization index pattern.

Thx Wes!

RANGERBEE commented Dec 14, 2017

I have your "pass-thru-cache" enabled for the docker registry and updated all an hour ago.
Found that the index for the visualization reads " :logstash- " time now, after updates to master and sensors.

( I see in the Viz editor that the counter reads correct. Though not on home screen )

A; Found it. The builder index pattern did not match the counter visualization index pattern.

Thx Wes!

@r32rtb

This comment has been minimized.

Show comment
Hide comment
@r32rtb

r32rtb Dec 15, 2017

With Elastic 6.x the mapping type will no longer work, all the reference to type will need to be adjusted if you intend to upgrade to Elastic 6. The use of type:bro_conn and type:bro_dns within the same index will not work.

Multiple mapping types are not supported in indices created in 6.0
The ability to have multiple mapping types per index has been removed in 6.0. New indices will be restricted to a single type. This is the first step in the plan to remove mapping types altogether. Indices created in 5.x will continue to support multiple mapping types.

r32rtb commented Dec 15, 2017

With Elastic 6.x the mapping type will no longer work, all the reference to type will need to be adjusted if you intend to upgrade to Elastic 6. The use of type:bro_conn and type:bro_dns within the same index will not work.

Multiple mapping types are not supported in indices created in 6.0
The ability to have multiple mapping types per index has been removed in 6.0. New indices will be restricted to a single type. This is the first step in the plan to remove mapping types altogether. Indices created in 5.x will continue to support multiple mapping types.

@dougburks

This comment has been minimized.

Show comment
Hide comment
@dougburks

dougburks Dec 15, 2017

Contributor

Hi @r32rtb ,

Yes, we're aware of the changes in Elastic 6. We've already changed type: to event_type: as part of the upcoming Beta 3 release:
#1172

Contributor

dougburks commented Dec 15, 2017

Hi @r32rtb ,

Yes, we're aware of the changes in Elastic 6. We've already changed type: to event_type: as part of the upcoming Beta 3 release:
#1172

@dougburks

This comment has been minimized.

Show comment
Hide comment
Contributor

dougburks commented Jan 26, 2018

@dougburks

This comment has been minimized.

Show comment
Hide comment

@dougburks dougburks closed this Jan 29, 2018

@r32rtb

This comment has been minimized.

Show comment
Hide comment
@r32rtb

r32rtb Feb 4, 2018

Great work guys! Have you thought about scaling this out when you have say 10 SO sensors and say 20K docs per second between all the sensors? I've had to move to a dedicated ES cluster with dedicate logstash servers. Do you think the ssh tunnel to the server will be able to handle?

r32rtb commented Feb 4, 2018

Great work guys! Have you thought about scaling this out when you have say 10 SO sensors and say 20K docs per second between all the sensors? I've had to move to a dedicated ES cluster with dedicate logstash servers. Do you think the ssh tunnel to the server will be able to handle?

@dougburks

This comment has been minimized.

Show comment
Hide comment
@dougburks

dougburks Feb 4, 2018

Contributor

Thanks @r32rtb !

Please see:
https://groups.google.com/d/topic/security-onion/EhYIfbwLZRU/discussion

If you have further questions or comments, please use the mailing list for discussion:
https://securityonion.net/wiki/MailingLists

Thanks!

Contributor

dougburks commented Feb 4, 2018

Thanks @r32rtb !

Please see:
https://groups.google.com/d/topic/security-onion/EhYIfbwLZRU/discussion

If you have further questions or comments, please use the mailing list for discussion:
https://securityonion.net/wiki/MailingLists

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment