secure config for thredds

[fmigneault]

No description provided.

[crim-jenkins-bot]

Can one of the admins verify this patch?

[tlvu]

LGTM. Can you add brief description of the new component to README https://github.com/bird-house/birdhouse-deploy/blob/master/birdhouse/optional-components/README.md (follow existing format) and also add to

birdhouse-deploy/birdhouse/env.local.example

Lines 72 to 80 in 754d2d1

	#export EXTRA_CONF_DIRS="./optional-components/canarie-api-full-monitoring
	# ./optional-components/emu
	# ./optional-components/testthredds
	# ./optional-components/generic_bird
	# ./optional-components/all-public-access
	# ./components/scheduler
	# ./components/monitoring
	# /path/to/private-config-repo"

[tlvu]

In the README, make clear to user this will make all of Thredds publicly readable, except those 2 secure folders. Once enabled, to disable, user have to login to Magpie and perform manual changes, diactivating the component is not enough. Similar to https://github.com/bird-house/birdhouse-deploy/tree/master/birdhouse/optional-components#give-public-access-to-all-resources-for-testing-purposes

[tlvu]

On second though, do you even need this section? Just let user decide what they want to expose themselves. You just want to ensure those 2 folders are secured.

[fmigneault]

Acutally, coming back on my previous comment.
Yes, since the user can decide or not to add both the public-access + secure, no need to add the public one again here.

[tlvu]

I think it safer to not force public Thredds content on the user. So yeah, change that and update README and env.local also, thanks.

I guess we will have to test this together with the magpie upgrade? This can not be tested alone.

[fmigneault]

@tlvu
The feature makes sense only when combined with public at an higher level, otherwise the directories/files are blocked by default and there is not much point to add the explicit permission on the "secure" directories.

[dbyrns]

Maybe there is no point into adding permissions for thredds after all. Because, even if it would make sense to add permission to the secure directory, from the birdhouse-deploy repo perspective, we should not even know that this folder exists. For wps permissions it can make sense because the API is known, but otherwise it is a per-node config that really should only exist in the private config.

[fmigneault]

Maybe there is no point into adding permissions for thredds after all. Because, even if it would make sense to add permission to the secure directory, from the birdhouse-deploy repo perspective, we should not even know that this folder exists. For wps permissions it can make sense because the API is known, but otherwise it is a per-node config that really should only exist in the private config.

Indeed, most of these kind on configs should be only on private server side.
I see that entry more for a documentation example.

The listed directories are the only ones I could find that made sense to protect.
They are generated via https://github.com/bird-house/birdhouse-deploy/blob/master/birdhouse/scripts/bootstrap-testdata, so should be present on instances deployed for testing minimally.

[tlvu]

Maybe there is no point into adding permissions for thredds after all. Because, even if it would make sense to add permission to the secure directory, from the birdhouse-deploy repo perspective, we should not even know that this folder exists. For wps permissions it can make sense because the API is known, but otherwise it is a per-node config that really should only exist in the private config.

I think DavidB have a point here. Thredds permissions will be different for each organization, can not have a one size fit all config like this.

You should just move the 2 directives about /birdhouse/testdata/secure into optional-components/all-public-access in the other bump magpie PR #102 so it can be tested together and just abandon this PR.

[fmigneault]

I feel adding something that explicitly removes access into a config named all-public-access is more counter-intuitive.
Note the config is still optional, so it doesn't have to be included if the organization deems it unnecessary.

[tlvu]

I feel adding something that explicitly removes access into a config named all-public-access is more counter-intuitive.

In the README https://github.com/bird-house/birdhouse-deploy/tree/master/birdhouse/optional-components#give-public-access-to-all-resources-for-testing-purposes we clearly state all-public-access is for testing.

And we will only revoke access for /birdhouse/testdata/secure, not everything. In all-public-access we actually open up everything, like the original state of this PR.

Note the config is still optional, so it doesn't have to be included if the organization deems it unnecessary.

Agreed but why try to block /secure when we do not even know if the org will have it or not. As for blocking /birdhouse/testdata/secure it's really for testing so might as well move it to all-public-access which is meant for testing.

Like @dbyrns, I think a secure-thredds component similar to this one does have it's use but it will belong to each org private config repo, not in this generic repo.

[fmigneault]

I personally prefer to preserve the granularity of the two files, even if the configs are only for testing.
This way, if I want to boot a test instance that is fully open, I can do so, and if I want to boot another one fully open except the secure directory, I can also do it.

I agree with the root /secure. It is not needed here.

[tlvu]

I personally prefer to preserve the granularity of the two files, even if the configs are only for testing.
This way, if I want to boot a test instance that is fully open, I can do so, and if I want to boot another one fully open except the secure directory, I can also do it.

OK I see your point.

Can you then please add to the optional-components README https://github.com/bird-house/birdhouse-deploy/tree/master/birdhouse/optional-components#give-public-access-to-all-resources-for-testing-purposes for your new secure-thredds component, following the same pattern as all-public-access and mention in the primary README https://github.com/bird-house/birdhouse-deploy/tree/master/birdhouse#optional-prepare-instance-to-run-automated-end-to-end-test-suite that user also need to enable the secure-thredds for the Jenkins test suite.

[tlvu]

Almost there :D

[tlvu]

+1, merge when ready.