add mbon vm to IMaRS puppetmaster?

[7yl4r]

We could add the mbon server as a node on IMaRS's puppet configuration. I have a very small configuration I'm adding to all our nodes that would add:

1. monitoring via telegraf to push resource usage & security info to our graphite server
2. a few security features like a fail2ban setup

Honestly, I haven't developed any love for puppet in my few months using it and am happy to leave management of this server in your capable hands, @bbest . But I thought I would offer in case this arrangement might help lighten your load. What do you think?

[bbest]

Hi @7yl4r, I'm unfamiliar with puppet, telegraf, graphite, fail2ban; ie all the technologies mentioned. Feel free to provide links or give me a ring to chat. Happy to defer to you on management of server loads - would be great to have a more user friendly view of CPU and memory. I can ask more server tech savvy friends for recs too.

[7yl4r]

• puppet is configuration management, so (in theory) it lets me manage the software and configs from within a code repo so configuration is better tracked and reproduced. This will allow me to install all the others (and more in the future) with a few lines of code.
• telegraf is a client-side metric collection script. It collects together numbers on the server and then sends them off to my database.
• graphite is the database telegraf is talking to
• fail2ban helps prevent ssh brute-force attacks

Here's a screenshot from graphite for mbon's hypervisor:

and one from my security dashboard:

I guess the core of what I'm asking is: would you like for me to dig into the mbon server a bit more to add features I'm already pushing to the other servers, or would you prefer to avoid the potential risk and complication in the mbon setup?

[bbest]

Hi @7yl4r,

Yeah, these look like good features to have on the server. Please feel free to proceed.

By the way, a tech startup friend recommends the free version of New Relic.

[bbest]

Hi @7yl4r,

I can't seem to ssh ben@mbon.marine.usf.edu and all web services are down. Can you try to bring up the server please?

[7yl4r]

There was a power outage last night and one of our switches died. I'm working to bring everything back up now.

…

On Thu, Jul 20, 2017 at 8:16 AM, Ben Best ***@***.***> wrote: Hi @7yl4r <https://github.com/7yl4r>, I can't seem to ssh ***@***.*** and all web services are down. Can you try to bring up the server please? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#21 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABAK_oMzbE_xcOTfYBb0jVjWyPi_Luhyks5sP0UqgaJpZM4OHCA8> .

[bbest]

Ok, thanks Tylar!

…

On Thu, Jul 20, 2017 at 6:31 AM, Tylar ***@***.***> wrote: There was a power outage last night and one of our switches died. I'm working to bring everything back up now. On Thu, Jul 20, 2017 at 8:16 AM, Ben Best ***@***.***> wrote: > Hi @7yl4r <https://github.com/7yl4r>, > > I can't seem to ssh ***@***.*** and all web services are > down. Can you try to bring up the server please? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#21 (comment)>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/ABAK_oMzbE_ xcOTfYBb0jVjWyPi_Luhyks5sP0UqgaJpZM4OHCA8> > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#21 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACtLCTqnkOvRiHxWVfS_qwEIQu2kUlWfks5sP1bFgaJpZM4OHCA8> .

[7yl4r]

puppet has been added to the mbon vm and it has been connected to our puppetmaster. telegraf is not working (probably a permissions error), so no advanced logging for now. Next time I can come back around to it I will push metrics into this dashboard.

In the meantime the only two things that may be of use to you are fail2ban & etckeeper:

# to see fail2ban at work:
tylarmurray@mbon:/etc$ sudo fail2ban-client status sshd
Status for the jail: sshd
|- Filter
|  |- Currently failed:	1
|  |- Total failed:	195
|  `- File list:	/var/log/auth.log
`- Actions
   |- Currently banned:	4
   |- Total banned:	4
   `- Banned IP list:	116.31.116.6 59.45.175.29 221.194.47.236 221.194.47.233

# etckeeper auto-tracks all your /etc/ configs in a git repo
tylarmurray@mbon:/etc$ sudo git status
On branch master
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   telegraf/telegraf.conf

no changes added to commit (use "git add" and/or "git commit -a")

You might consider adding your IP range to the fail2ban ignoreip list so you don't accidentally lock yourself out if you mistype your password several times in a row.:

tylarmurray@mbon:/etc$ sudo fail2ban-client set sshd addignoreip 131.247.0.0/16
These IP addresses/networks are ignored:
|- 127.0.0.1/8
|- 131.247.0.0/16
`- 192.168.0.0/16

[7yl4r]

monitoring is up: http://graphite.marine.usf.edu/dashboard/#mbon