Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rebase-3-1-0-from-release-3-0-0 #1132

Closed
wants to merge 23 commits into from
Closed

rebase-3-1-0-from-release-3-0-0 #1132

wants to merge 23 commits into from

Conversation

jack1902
Copy link
Contributor

to: @ryandeivert @chunyong-lin
cc: @airbnb/streamalert-maintainers
related to:
resolves:

Background

Keeping release-3-1-0 current with work carried out against release-3-0-0 so that i can update the documentation accordingly

Changes

  • Carried out git checkout release-3-1-0 && git pull Ensure i have the latest branchs on my machine then git checkout -b rebase_from/release-3-0-0 && git fetch upstream && git rebase upstream/release-3-0-0 This ensured i rebased a branch against the latest on the remote

  • Updated the style of rules.rst when the dynamic_outputs feature was added

Testing

  • Ran ./tests/scripts/test_the_docs.sh to ensure the documentation still built
  • Ran ./tests/scripts/pylint.sh Ensured pylint will pass
  • Ran ./tests/scripts/unit_tests.sh to ensure unit tests passed

Notes to merger

  • Add my remote git remote add jack git@github.com:jack1902/streamalert.git && git fetch jack

  • Run git merge -X theirs jack/rebase_from/release-3-0-0 to use my branch as the new release-3-1-0 otherwise you'll run into all the merge conflicts i resolved when i did the rebase

jack1902 and others added 23 commits January 9, 2020 21:19
* Updated terraform version and git branch

Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>

* removed step no longer required, as the choices are dynamically created based on the @StreamAlertOutput class decorator

Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>

* reset to stable and changed note

Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
#1081)

Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
…tring (#1085)

see issue: #1084 for more information

Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
[testing] Added trendmicro schema and rule test
Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
[testing] fixed test for rules_engine assertion

Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
This in-turn re-introduced #1047. I fixed this by ensuring that the
cleanup function removes the metric_filters.tf.json file, otherwise
terraform reads this in as part of its deployment.

Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
Updated the optional_top_level_keys for cloudtrail:events

Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
…eriods from bucket names (#1114)

* updating secrets bucket name to remove periods

* updating s3-logging bucket name to remove periods

* updating athena-results bucket name to remove periods

* updating streamalerts bucket name to remove periods

* updating terraform-state bucket name to remove periods

* updating streamalert-data bucket name to remove periods

* fixing misc places that were missed regarding periods in bucket names

* making data and alerts bucket names configurable

* restructuring docs to allow for highlighting global settings

* doc updates for alerts_table config

* updating documentation, round 1

* moving clusters and global docs to new file

* trailing space removal

* Update to commands for consistency

* misc formatting fixes, migrating rule-staging config to global

* updating cluster config docs

* massive updates to docs

* adding other changes related to doc updates and config changes

* adding prefix validation for periods

* removing prefix setting trash

* updates to remove need for setting terraform config

* removing nonsense for athena bucket configuration

* addressing PR comments

* removing kinesis region setting since it would break things

* addressing chunyong PR feedback

* adding new streamalert images and updating doc references
* [core] Initial Microsoft Teams output code commit, looking for feedback

Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>

* [testing] added TeamsOutput Testing (used slack tests as template), ammended list for output_base aswell

Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>

* [docs] added Microsoft Teams to output documentation
[setup] added pymsteams to reuirements-top-level and added sample-webhook to outputs.json

Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>

* [core] Moved pymsteams to package.py
[docs] Corrected docstring for teams and added teams to outputs
[core] Added Alert section to card (didn't have the alert_id which made
it confusing previously)
[testing] re-wrote the tests

Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
* Now possible to pass dynamic_outputs to the @rule decorator and have
outputs be dynamically configured based on information in the record.
For example, you could use lookup_tables to map an account_id to an
owner which maps to an output

[testing] Updated unit tests and added additional tests for new
dynamic_outputs

[docs] Added dynamic_outputs documentation

Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
[testing] added aws-ses output tests
[docs] updated docs/source/outputs.rst to include aws ses
[terraform] Added ses:SendRawEmail to tf_alert_processor_iam

Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
* threat intel downloader terraform module now uses tf_lambda

* small cleanups make for happier linters

* fixed some stale references in the threat_intel_downloader terraform module
* [setup] fixing missing awscli

Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>

* [setup] Natively pass AWS credentials through ssh

Updated the sshd_config file to allow AWS_* to be passed through when
using vagrant ssh. This allows you to not hardcode credentials inside of
the vm

Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
…ery events (#1127)

* [rules] Add community rule to alert on ssh login activity based on osquery detection

* address comments
@Ryxias
Copy link
Contributor

Ryxias commented Feb 19, 2020

We will need this ASAP since we have many changes in flight. Looks good so far, but there's 2 conflicts leftover in the documentation files

I believe we're going to feature freeze release-3-0-0 soon so we shouldn't have to do this anymore going forward, hopefully

@jack1902
Copy link
Contributor Author

I don't know how to remove these merge conflicts hence the git merge -X theirs to set release3-1-0 commit history to look like the branch I did the rebase on and resolved these conflicts

@@ -0,0 +1,129 @@
Dynamic Outputs
Copy link
Contributor

@ryandeivert ryandeivert Feb 19, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this format is now outdated and should be updated for consistency (either here or in another PR)

titles should now be:

###############
Dynamic Outputs
###############

and the page should then follow the hierarchy laid out here: https://documentation-style-guide-sphinx.readthedocs.io/en/latest/style-guide.html#headings

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will update and push this (I'll squash the commit into the one that introduced this)

@Ryxias
Copy link
Contributor

Ryxias commented Feb 19, 2020

This PR has been superseded by #1136

@Ryxias Ryxias closed this Feb 19, 2020
@jack1902 jack1902 deleted the rebase_from/release-3-0-0 branch February 23, 2020 10:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants