Skip to content

Upgrading NATS to a supported version#2391

Merged
rkoster merged 3 commits intomainfrom
nats-supported
Aug 22, 2022
Merged

Upgrading NATS to a supported version#2391
rkoster merged 3 commits intomainfrom
nats-supported

Conversation

@danielfor
Copy link
Copy Markdown
Contributor

@danielfor danielfor commented Jul 18, 2022

  • Updated NATS to a supported version: 2.8.3

  • NATS 2.8.3 does not support wildcards in the users authorization
    section in the nats.cfg file. Wildcards were used previously to
    allow any bosh agents to publish/subscribe to NATS.
    Because of the new wildcards restrictions we need to add all the
    users explicitly in the configuration and that needs to be done
    continuously, as more VMs/Agents are added.
    To execute this continuous synchronization, we created the ruby
    gem 'bosh-nats-sync' which keeps the list of active agents up to
    date.

  • Added the monit infrastructure to start the bosh-nats-sync gem.

What tests have you run against this PR?

The BOSH Director unit tests.

How should this change be described in bosh release notes?

This is an update of NATS. This component used by the director was in an unsupported version.

Does this PR introduce a breaking change?

No, this won't affect users using BOSH with a standard NATS installation.
Warning: For BOSH installations with custom NATS configurations (e.g custom NATS server running outside of the director VM) the bosh-nats-sync component needs to be configured and run according to the customizations.

Tag your pair, your PM, and/or team!

Co-authored-by: Brian Upton bupton@vmware.com
Co-authored-by: Seth Boyles sboyles@vmware.com
Co-authored-by: Brian Cunnie bcunnie@vmware.com
Co-authored-by: Manuel Alba albam@vmware.com
Co-authored-by: Long Nguyen nguyenlo@vmware.com
Co-authored-by: Daniel Felipe Ochoa danielfelipo@vmware.com

@lnguyen lnguyen requested review from beyhan and jpalermo July 20, 2022 14:57
@ramonskie ramonskie requested a review from rkoster July 21, 2022 14:44
@lnguyen
Copy link
Copy Markdown
Member

lnguyen commented Aug 1, 2022

@beyhan @jpalermo @rkoster could ya'll take a look at this at some point?

jpalermo
jpalermo suggested changes Aug 6, 2022
View reviewed changes
Copy link
Copy Markdown
Contributor

@jpalermo jpalermo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did we do load testing on this yet? Feels like we'd want to make sure this scales fine up to ~2000 VMs or so.

Comment thread config/blobs.yml
Comment thread jobs/nats/monit
Comment thread jobs/nats/spec Outdated
Comment thread jobs/nats/spec
Comment thread jobs/nats/spec
@linux-foundation-easycla
Copy link
Copy Markdown

linux-foundation-easycla Bot commented Aug 9, 2022 •
edited
Loading

CLA Signed

The committers listed above are authorized under a signed CLA.

Long Nguyen and others added 2 commits August 16, 2022 18:43
@ystros @sethboyles
@cunnie @malba @danielfor
Upgrading NATS to a supported version
60f51b6 
- Updated NATS to a supported version: 2.8.3

- NATS 2.8.3 does not support wildcards in the users authorization
  section in the `nats.cfg` file. Wildcards were used previously to
  allow any bosh agents to publish/subscribe to NATS.
  Because of the new wildcards restrictions we need to add all the
  users explicitly in the configuration and that needs to be done
  continuously, as more VMs/Agents are added.
  To execute this continuous synchronization, we created the ruby
  gem 'bosh-nats-sync' which keeps the list of active agents up to
  date.

- Added the monit infrastructure to start the bosh-nats-sync gem.

[#182098993] Upgrade NATS to 2.0

Co-authored-by: Brian Upton <bupton@vmware.com>
Co-authored-by: Seth Boyles <sboyles@vmware.com>
Co-authored-by: Brian Cunnie <bcunnie@vmware.com>
Co-authored-by: Manuel Alba <albam@vmware.com>
Co-authored-by: Long Nguyen <nguyenlo@vmware.com>
Co-authored-by: Daniel Felipe Ochoa <danielfelipo@vmware.com>
@danielfor
Update nats to use new ruby format
95961b9
@lnguyen lnguyen requested a review from ramonskie August 17, 2022 17:05
@rkoster rkoster requested review from mvach and removed request for beyhan August 17, 2022 17:30
@klakin-pivotal
Remove old gnatsd blob
c6338ee 
No packages use the blob so it can be removed.

[#182098993]

Signed-off-by: Joseph Palermo <jpalermo@pivotal.io>
@jpalermo
Copy link
Copy Markdown
Contributor

jpalermo commented Aug 17, 2022

@lnguyen could you note the performance tests you did here? How many "VMs" and how long it was taking to render?

rkoster
rkoster requested changes Aug 18, 2022
View reviewed changes
Copy link
Copy Markdown
Contributor

@rkoster rkoster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, just a small question with regards to the director and hm users (see comments above).

Comment thread jobs/nats/spec
Comment thread jobs/nats/templates/nats.cfg.erb
@lnguyen
Copy link
Copy Markdown
Member

lnguyen commented Aug 22, 2022 •
edited
Loading

@lnguyen could you note the performance tests you did here? How many "VMs" and how long it was taking to render?

I ran 500 vms on BOSH-lite and it was taking about 1 min per vm to come up

@rkoster rkoster merged commit d3d40ac into main Aug 22, 2022
@rkoster rkoster deleted the nats-supported branch August 22, 2022 17:16
ystros added a commit that referenced this pull request Nov 8, 2024
@ystros
Use nats-server blob instead of downloading gnatsd
82ebf4b 
The nats-server binary replaced the legacy forked gnatsd binary in 2022:
#2391

However, the integration specs were still using an old version of gnatsd
from an S3 bucket.

Now, the integration specs use the current nats-server blob directly.
This required that the nats-sync process also be launched during
integration tests in order to allow agents to connect. Additionally,
there was a race condition between an agent being launched by the Dummy
CPI and the nats-sync process adding the agent to the nats-server
config.
ystros added a commit that referenced this pull request Nov 11, 2024
@ystros
Use nats-server blob instead of downloading gnatsd
50c3f20 
The nats-server binary replaced the legacy forked gnatsd binary in 2022:
#2391

However, the integration specs were still using an old version of gnatsd
from an S3 bucket.

Now, the integration specs use the current nats-server blob directly.
This required that the nats-sync process also be launched during
integration tests in order to allow agents to connect. Additionally,
there was a race condition between an agent being launched by the Dummy
CPI and the nats-sync process adding the agent to the nats-server
config.
ystros added a commit that referenced this pull request Nov 11, 2024
@ystros
Use nats-server blob instead of downloading gnatsd
2c2369e 
The nats-server binary replaced the legacy forked gnatsd binary in 2022:
#2391

However, the integration specs were still using an old version of gnatsd
from an S3 bucket.

Now, the integration specs use the current nats-server blob directly.
This required that the nats-sync process also be launched during
integration tests in order to allow agents to connect. Additionally,
there was a race condition between an agent being launched by the Dummy
CPI and the nats-sync process adding the agent to the nats-server
config.
ystros added a commit that referenced this pull request Nov 12, 2024
@ystros
Use nats-server blob instead of downloading gnatsd
e381e7d 
The nats-server binary replaced the legacy forked gnatsd binary in 2022:
#2391

However, the integration specs were still using an old version of gnatsd
from an S3 bucket.

Now, the integration specs use the current nats-server blob directly.
This required that the nats-sync process also be launched during
integration tests in order to allow agents to connect. Additionally,
there was a race condition between an agent being launched by the Dummy
CPI and the nats-sync process adding the agent to the nats-server
config.
aramprice pushed a commit that referenced this pull request Nov 15, 2024
@ystros @aramprice
Use nats-server blob instead of downloading gnatsd
3e5a2bd 
The nats-server binary replaced the legacy forked gnatsd binary in 2022:
#2391

However, the integration specs were still using an old version of gnatsd
from an S3 bucket.

Now, the integration specs use the current nats-server blob directly.
This required that the nats-sync process also be launched during
integration tests in order to allow agents to connect. Additionally,
there was a race condition between an agent being launched by the Dummy
CPI and the nats-sync process adding the agent to the nats-server
config.
aramprice pushed a commit that referenced this pull request Nov 16, 2024
@ystros @aramprice
Use nats-server blob instead of downloading gnatsd
ec3716b 
The nats-server binary replaced the legacy forked gnatsd binary in 2022:
#2391

However, the integration specs were still using an old version of gnatsd
from an S3 bucket.

Now, the integration specs use the current nats-server blob directly.
This required that the nats-sync process also be launched during
integration tests in order to allow agents to connect. Additionally,
there was a race condition between an agent being launched by the Dummy
CPI and the nats-sync process adding the agent to the nats-server
config.
ystros added a commit that referenced this pull request Nov 18, 2024
@ystros
Use nats-server blob instead of downloading gnatsd
d54d127 
The nats-server binary replaced the legacy forked gnatsd binary in 2022:
#2391

However, the integration specs were still using an old version of gnatsd
from an S3 bucket.

Now, the integration specs use the current nats-server blob directly.
This required that the nats-sync process also be launched during
integration tests in order to allow agents to connect. Additionally,
there was a race condition between an agent being launched by the Dummy
CPI and the nats-sync process adding the agent to the nats-server
config.
ystros added a commit that referenced this pull request Nov 20, 2024
@ystros
Use nats-server blob instead of downloading gnatsd
a5c6d7b 
The nats-server binary replaced the legacy forked gnatsd binary in 2022:
#2391

However, the integration specs were still using an old version of gnatsd
from an S3 bucket.

Now, the integration specs use the current nats-server blob directly.
This required that the nats-sync process also be launched during
integration tests in order to allow agents to connect. Additionally,
there was a race condition between an agent being launched by the Dummy
CPI and the nats-sync process adding the agent to the nats-server
config.
ystros added a commit that referenced this pull request Nov 20, 2024
@ystros
Use nats-server blob instead of downloading gnatsd
290d765 
The nats-server binary replaced the legacy forked gnatsd binary in 2022:
#2391

However, the integration specs were still using an old version of gnatsd
from an S3 bucket.

Now, the integration specs use the current nats-server blob directly.
This required that the nats-sync process also be launched during
integration tests in order to allow agents to connect. Additionally,
there was a race condition between an agent being launched by the Dummy
CPI and the nats-sync process adding the agent to the nats-server
config.
ystros added a commit that referenced this pull request Nov 21, 2024
@ystros
Use nats-server blob instead of downloading gnatsd
25b1a94 
The nats-server binary replaced the legacy forked gnatsd binary in 2022:
#2391

However, the integration specs were still using an old version of gnatsd
from an S3 bucket.

Now, the integration specs use the current nats-server blob directly.
This required that the nats-sync process also be launched during
integration tests in order to allow agents to connect. Additionally,
there was a race condition between an agent being launched by the Dummy
CPI and the nats-sync process adding the agent to the nats-server
config.
ystros added a commit that referenced this pull request Nov 21, 2024
@ystros
Use nats-server blob instead of downloading gnatsd
a297fa5 
The nats-server binary replaced the legacy forked gnatsd binary in 2022:
#2391

However, the integration specs were still using an old version of gnatsd
from an S3 bucket.

Now, the integration specs use the current nats-server blob directly.
This required allowing all users connecting to the NATS server to
subscribe and publish to any channel, because the integration specs
require an admin-level user but the nats-sync process does not allow
custom users (only the Director, Health Monitor, and individual agents).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rkoster rkoster rkoster approved these changes

@ramonskie ramonskie Awaiting requested review from ramonskie

@mvach mvach Awaiting requested review from mvach

+1 more reviewer

@jpalermo jpalermo jpalermo approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

Foundational Infrastructure Working Group
Archived in project
Roadmap
Status: No status

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@danielfor @lnguyen @jpalermo @rkoster @klakin-pivotal