Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mattermost SMTP port cannot be configured by the module #289

Closed
smortex opened this issue Dec 4, 2018 · 8 comments
Closed

Mattermost SMTP port cannot be configured by the module #289

smortex opened this issue Dec 4, 2018 · 8 comments

Comments

@smortex
Copy link
Member

smortex commented Dec 4, 2018

Affected Puppet, Ruby, OS and module versions/distributions

  • Puppet: Not applicable
  • Ruby: Not applicable
  • Distribution: Not applicable
  • Module version: 2.x+

How to reproduce (e.g Puppet code you use)

Configure e-mail in mattermost:

class { 'gitlab':
  # ...
  mattermost => {
    env => {
      'MM_EMAILSETTINGS_SENDEMAILNOTIFICATIONS' => 'true',  # lint:ignore:quoted_booleans
      'MM_EMAILSETTINGS_SMTPPORT'               => '25',
      'MM_EMAILSETTINGS_SMTPSERVER'             => 'localhost',
    },
  },
}

Apply the catalog.

What are you seeing

Catalog fails to apply: /etc/gitlab/gitlab.rb is updated according to the change, but gitlab_reconfigure fails with "had an error: Chef::Exceptions::ValidationFailed: Property content must be one of: String, nil!
You passed 25."

The /etc/gitlab/gitlab.rb file indeed has the value as an Integer and not a String as expected:

mattermost['env'] = { [...], "MM_EMAILSETTINGS_SMTPPORT"=>25, [...] }

Quoting the number and triggering /usr/bin/gitlab-ctl reconfigure manually does what is expected.

What behavior did you expect instead

Since I put a string ('25') in my configuration, I expect the module to output a string into /etc/gitlab/gitlab.rb and not turn it into a number.

The numify function is responsible for this, but the name makes me think that removing this feature will break things:

def numify(obj)
if obj.is_a?(String)
Integer(obj) rescue Float(obj) rescue obj
elsif obj.is_a?(Array)
obj.map { |item| numify(item) }
elsif obj.is_a?(Hash)
sort_hash_by_key(obj.merge(obj) { |_, v| numify(v) })
else
obj
end
end

Output log

[...]
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns: Recipe: mattermost::enable
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:   * account[Mattermost user and group] action create
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     * group[Mattermost user and group] action create (up to date)
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     * linux_user[Mattermost user and group] action create (up to date)
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:      (up to date)
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:   * directory[/var/opt/gitlab/mattermost] action create (up to date)
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:   * directory[/var/log/gitlab/mattermost/] action create (up to date)
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:   * directory[/var/opt/gitlab/mattermost/data] action create (up to date)
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:   * directory[/var/opt/gitlab/mattermost/plugins] action create (up to date)
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:   * directory[/var/opt/gitlab/mattermost/client-plugins] action create (up to date)
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:   * file[/var/log/gitlab/mattermost/mattermost.log] action create (up to date)
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:   * postgresql_user[gitlab_mattermost] action create
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     * execute[create gitlab_mattermost postgresql user] action run (skipped due to not_if)
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:      (up to date)      
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:   * execute[create mattermost_production database] action run (skipped due to not_if)
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:   * ruby_block[populate mattermost configuration options] action run
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     - execute the ruby block populate mattermost configuration options
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:   * remote_file[/var/opt/gitlab/mattermost/config.json] action create_if_missing (up to date)
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:   * env_dir[/var/opt/gitlab/mattermost/env] action create
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     ================================================================================
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     Error executing action `create` on resource 'env_dir[/var/opt/gitlab/mattermost/env]'
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     ================================================================================
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     Chef::Exceptions::ValidationFailed
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     ----------------------------------
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     Property content must be one of: String, nil!  You passed 25.
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     Cookbook Trace:
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     ---------------
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/env_dir.rb:42:in `block (3 levels) in class_from_file'
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/env_dir.rb:41:in `block (2 levels) in class_from_file'
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/env_dir.rb:40:in `each'
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/env_dir.rb:40:in `block in class_from_file'
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     Resource Declaration:
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     ---------------------
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     # In /opt/gitlab/embedded/cookbooks/cache/cookbooks/mattermost/recipes/enable.rb
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     124: env_dir File.join(mattermost_home, 'env') do
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     125:   variables lazy { MattermostHelper.get_env_variables(node).merge(node['mattermost']['env']) }
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     126:   notifies :restart, "service[mattermost]"
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     127: end
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     128:
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     Compiled Resource:
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     ------------------
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     # Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/mattermost/recipes/enable.rb:124:in `from_file'
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     env_dir("/var/opt/gitlab/mattermost/env") do
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:       action [:create]
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:       default_guard_interpreter :default
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:       declared_type :env_dir
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:       cookbook_name "mattermost"
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:       recipe_name "enable"
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:       variables {"MM_SERVICESETTINGS_SITEURL"=>"https://mattermost.example.com", "MM_SERVICESETTINGS_LISTENADDRESS"=>"127.0.0.1:8065", "MM_SERVICESETTINGS_ALLOWEDUNTRUSTEDINTERNALCONNECTIONS"=>" gitlab.example.com", "MM_SERVICESETTINGS_ENABLEAPITEAMDELETION"=>"true", "MM_TEAMSETTINGS_SITENAME"=>"GitLab Mattermost", "MM_SQLSETTINGS_DRIVERNAME"=>"postgres", "MM_SQLSETTINGS_DATASOURCE"=>"user=gitlab_mattermost host=/var/opt/gitlab/postgresql port=5432 dbname=mattermost_production", "MM_SQLSETTINGS_ATRESTENCRYPTKEY"=>"xxx", "MM_LOGSETTINGS_FILELOCATION"=>"/var/log/gitlab/mattermost/", "MM_FILESETTINGS_DIRECTORY"=>"/var/opt/gitlab/mattermost/data", "MM_GITLABSETTINGS_ENABLE"=>"true", "MM_GITLABSETTINGS_SECRET"=>"xxx", "MM_GITLABSETTINGS_ID"=>"xxx", "MM_GITLABSETTINGS_SCOPE"=>"", "MM_GITLABSETTINGS_AUTHENDPOINT"=>"https://gitlab.example.com/oauth/authorize", "MM_GITLABSETTINGS_TOKENENDPOINT"=>"https://gitlab.example.com/oauth/token", "MM_GITLABSETTINGS_USERAPIENDPOINT"=>"https://gitlab.example.com/api/v4/user", "MM_PLUGINSETTINGS_DIRECTORY"=>"/var/opt/gitlab/mattermost/plugins", "MM_PLUGINSETTINGS_CLIENTDIRECTORY"=>"/var/opt/gitlab/mattermost/client-plugins", "SSL_CERT_DIR"=>"/opt/gitlab/embedded/ssl/certs/", "MM_EMAILSETTINGS_ENABLESIGNINWITHEMAIL"=>"false", "MM_EMAILSETTINGS_ENABLESIGNUPWITHEMAIL"=>"false", "MM_EMAILSETTINGS_SENDEMAILNOTIFICATIONS"=>"true", "MM_EMAILSETTINGS_SMTPPORT"=>25, "MM_EMAILSETTINGS_SMTPSERVER"=>"localhost", "MM_SERVICESETTINGS_ENABLEINCOMINGWEBHOOKS"=>"true", "MM_SERVICESETTINGS_ENABLEOUTGOINGWEBHOOKS"=>"true", "MM_SUPPORTSETTINGS_SUPPORTEMAIL"=>"contact@example.com"}
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     end
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     System Info:
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     ------------
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     chef_version=13.6.4
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     platform=debian
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     platform_version=9.6
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     ruby=ruby 2.4.5p335 (2018-10-18 revision 65137) [x86_64-linux]    
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     program_name=/opt/gitlab/embedded/bin/chef-client
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:     executable=/opt/gitlab/embedded/bin/chef-client      
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns: Running handlers:
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns: There was an error running gitlab-ctl reconfigure:
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns: env_dir[/var/opt/gitlab/mattermost/env] (mattermost::enable line 124) had an error: Chef::Exceptions::ValidationFailed: Property content must be one of: String, nil!
 You passed 25.
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns:
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns: Running handlers complete
Notice: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]/returns: Chef Client failed. 2 resources updated in 05 seconds
Error: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]: Failed to call refresh: '/usr/bin/gitlab-ctl reconfigure' returned 1 instead of one of [0]
Error: /Stage[main]/Gitlab::Config/Exec[gitlab_reconfigure]: '/usr/bin/gitlab-ctl reconfigure' returned 1 instead of one of [0]
Info: Class[Gitlab::Config]: Unscheduling all events on Class[Gitlab::Config]
Notice: /Stage[main]/Gitlab::Service/Service[gitlab-runsvdir]: Dependency Exec[gitlab_reconfigure] has failures: true
Warning: /Stage[main]/Gitlab::Service/Service[gitlab-runsvdir]: Skipping because of failed dependencies
Notice: Applied catalog in 38.75 seconds

Any additional information you'd like to impart

smortex added a commit to smortex/puppet-gitlab that referenced this issue Dec 4, 2018
Explicitely setting a value as a String does not produce the String in the
gitlab.rb file.
smortex added a commit to smortex/puppet-gitlab that referenced this issue Dec 4, 2018
Do not automagically convert Strings that look like Integer or Float to those
types.  If the user want an Integer or a Float, just let them put such Integer
/ Float.  If they want a String, let them put a String.
smortex added a commit to smortex/puppet-gitlab that referenced this issue Dec 4, 2018
Explicitely setting a value as a String does not produce the String in the
gitlab.rb file.
smortex added a commit to smortex/puppet-gitlab that referenced this issue Dec 4, 2018
Do not automagically convert Strings that look like Integer or Float to those
types.  If the user want an Integer or a Float, just let them put such Integer
/ Float.  If they want a String, let them put a String.
@juniorsysadmin juniorsysadmin added enhancement New feature or request bug Something isn't working and removed enhancement New feature or request labels Dec 5, 2018
smortex added a commit to smortex/puppet-gitlab that referenced this issue Dec 13, 2018
Do not automagically convert Strings that look like Integer or Float to those
types.  If the user want an Integer or a Float, just let them put such Integer
/ Float.  If they want a String, let them put a String.
smortex added a commit to smortex/puppet-gitlab that referenced this issue Dec 15, 2018
Explicitely setting a value as a String does not produce the String in the
gitlab.rb file.
smortex added a commit to smortex/puppet-gitlab that referenced this issue Dec 15, 2018
Do not automagically convert Strings that look like Integer or Float to those
types.  If the user want an Integer or a Float, just let them put such Integer
/ Float.  If they want a String, let them put a String.
@LongLiveCHIEF
Copy link
Contributor

We've found many times that deviating from valid puppet types with this decorator will cause no end of issues. Ultimately, this is a problem with the config.rb itself, and needs to be fixed in the upstream omnibus code.

@LongLiveCHIEF
Copy link
Contributor

@smortex what version of gitlab omnibus are you installing?

@smortex
Copy link
Member Author

smortex commented Dec 17, 2018

@LongLiveCHIEF we have updated to the latest code: puppet-gitlab at 3.0.2, and gitlab-ce package is at 6.5.3 11.5.4.

@LongLiveCHIEF
Copy link
Contributor

@smortex this module is not designed to support a version of gitlab so far out of date. That version of Gitlab was EoL in late 2014, and the gitlab.rb today is much different than the one from those days.

Gitlab only supports the last 3 months of releases, and this module typically is only able to support the most current major release and usually the one previous (at the moment that means gitlab-omnibus >= v10`)

@smortex
Copy link
Member Author

smortex commented Dec 17, 2018

Woops, looks-like I copy-pasted the wrong version number! We are running gitlab-ce 11.5.4 (i.e. latest GitLab)

@LongLiveCHIEF LongLiveCHIEF reopened this Dec 18, 2018
@LongLiveCHIEF
Copy link
Contributor

Honestly, despite the fact that this means there's likely a bug upstream that could cause us some big headaches with this module, I still feel way better knowing you're not using a 4 year old version of gitlab 😃

@smortex
Copy link
Member Author

smortex commented Dec 18, 2018

Hehe, sorry about that.

In the meantime, I opened a Merge Request upstream:
https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/2923

maxlazio pushed a commit to gitlabhq/omnibus-gitlab that referenced this issue Dec 19, 2018
Passing an integer to some configuration parameters prevent gitlab omnibus from
configuring the service correctly, e.g.

```ruby
mattermost['env'] = {
  "MM_EMAILSETTINGS_SMTPPORT" => 25
}
```

cause an exception to be raised:

```
================================================================================
Error executing action `create` on resource 'env_dir[/var/opt/gitlab/mattermost/env]'
================================================================================

Chef::Exceptions::ValidationFailed
----------------------------------
Property content must be one of: String, nil!  You passed 25.
```

The problem was initialy detected in the puppet-gitlab module, and the
maintainer suggested that the problem would rather be fixed here:
voxpupuli/puppet-gitlab#289
@smortex
Copy link
Member Author

smortex commented Dec 20, 2018

FWIW, the Merge Request was merged upstream. It should make it to 11.7.0 due Jan 22nd, 2019.

@LongLiveCHIEF LongLiveCHIEF removed the bug Something isn't working label Dec 20, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants