You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In redis-k8s-operator we run an integration test deploying and relating discourse-k8s to Redis. On some CI runs I experienced issues with the discourse charm because it can't write to the Redis database.
The reason for this is that the redis-client tries to connect to the first advertised host it can find in the relation data (here). But since a bug was fixed on redis-k8s-operator side, the relation data now includes all hosts there, not just the one of the Sentinel Master anymore. This was fixed with this PR, which means you now have to consider all advertised hosts, not just anyone anymore.
One more notice: This behaviour only happens if the Sentinel Master is not the unit /0. If, by accident, redis-k8s/0 is the Sentinel Master, or in setups with just one Redis unit, the integration works fine.
from `juju debug-log`:
unit-discourse-k8s-0: 11:35:26 ERROR unit.discourse-k8s/0.juju-log redis:11: Uncaught exception whilein charm code:
Traceback (most recent call last):
File "./src/charm.py", line 805, in<module>
main(DiscourseCharm, use_juju_for_storage=True)
File "/var/lib/juju/agents/unit-discourse-k8s-0/charm/venv/ops/main.py", line 544, in main
manager.run()
File "/var/lib/juju/agents/unit-discourse-k8s-0/charm/venv/ops/main.py", line 520, in run
self._emit()
File "/var/lib/juju/agents/unit-discourse-k8s-0/charm/venv/ops/main.py", line 509, in _emit
_emit_charm_event(self.charm, self.dispatcher.event_name)
File "/var/lib/juju/agents/unit-discourse-k8s-0/charm/venv/ops/main.py", line 143, in _emit_charm_event
event_to_emit.emit(*args, **kwargs)
File "/var/lib/juju/agents/unit-discourse-k8s-0/charm/venv/ops/framework.py", line 352, in emit
framework._emit(event)
File "/var/lib/juju/agents/unit-discourse-k8s-0/charm/venv/ops/framework.py", line 851, in _emit
self._reemit(event_path)
File "/var/lib/juju/agents/unit-discourse-k8s-0/charm/venv/ops/framework.py", line 941, in _reemit
custom_handler(event)
File "/var/lib/juju/agents/unit-discourse-k8s-0/charm/lib/charms/redis_k8s/v0/redis.py", line 87, in _on_relation_changed
self.charm.on.redis_relation_updated.emit()
File "/var/lib/juju/agents/unit-discourse-k8s-0/charm/venv/ops/framework.py", line 352, in emit
framework._emit(event)
File "/var/lib/juju/agents/unit-discourse-k8s-0/charm/venv/ops/framework.py", line 851, in _emit
self._reemit(event_path)
File "/var/lib/juju/agents/unit-discourse-k8s-0/charm/venv/ops/framework.py", line 941, in _reemit
custom_handler(event)
File "./src/charm.py", line 168, in _redis_relation_changed
self._setup_and_activate()
File "./src/charm.py", line 221, in _setup_and_activate
self._configure_pod()
File "./src/charm.py", line 691, in _configure_pod
self._config_force_https()
File "./src/charm.py", line 752, in _config_force_https
process.wait_output()
File "/var/lib/juju/agents/unit-discourse-k8s-0/charm/venv/ops/pebble.py", line 1559, in wait_output
raise ExecError[AnyStr](self._command, exit_code, out_value, err_value)
ops.pebble.ExecError: non-zero exit code 1 executing ['/srv/discourse/app/bin/rails', 'runner', 'SiteSetting.force_https=false'], stdout='', stderr="/srv/discourse/app/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:246:in `call_pipelined': READONLY You can't write against a read only replica. (Redis::CommandError)\n\tfrom /srv/discourse/app/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:200:in `block in call_pipeline'\n\tfrom /srv/discourse/app/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:344:in `with_reconnect'\n\tfrom /srv/discourse/app/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis/client.rb:198:in `call_pipeline'\n\tfrom /srv/discourse/app/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:177:in `block in pipelined'\n\tfrom /srv/discourse/app/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:265:in `block in synchronize'\n\tfrom /srv/discourse/app/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:265:in `synchronize'\n\tfrom /srv/discourse/app/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:265:in `synchronize'\n\tfrom /srv/discourse/app/vendor/bundle/ruby/3.2.0/gems/redis-4.8.1/lib/redis.rb:17" [truncated]
unit-discourse-k8s-0: 11:35:26 ERROR juju.worker.uniter.operation hook "redis-relation-changed" (via hook dispatching script: dispatch) failed: exit status 1
### Additional context
_No response_
The text was updated successfully, but these errors were encountered:
reneradoi
added a commit
to canonical/redis-k8s-operator
that referenced
this issue
Jul 2, 2024
* use redis-image:5 in release.yaml
* remove integration test for relating redis to the `discourse-charmers-discourse-k8s` charm as this has been discontinued
* disable integration test `test-redis-relation` with multiple units because of canonical/discourse-k8s-operator#268
Bug Description
Hi Team!
In
redis-k8s-operator
we run an integration test deploying and relatingdiscourse-k8s
to Redis. On some CI runs I experienced issues with the discourse charm because it can't write to the Redis database.The reason for this is that the
redis-client
tries to connect to the first advertised host it can find in the relation data (here). But since a bug was fixed onredis-k8s-operator
side, the relation data now includes all hosts there, not just the one of the Sentinel Master anymore. This was fixed with this PR, which means you now have to consider all advertised hosts, not just anyone anymore.One more notice: This behaviour only happens if the Sentinel Master is not the unit
/0
. If, by accident,redis-k8s/0
is the Sentinel Master, or in setups with just one Redis unit, the integration works fine.To Reproduce
juju deploy redis-k8s --channel edge -n 3
juju deploy postgresql-k8s --channel 14/stable
juju deploy discourse-k8s --channel latest/stable
juju integrate postgresql-k8s:database discourse-k8s
juju integrate redis-k8s discourse-k8s
Environment
see above.
Example for failed CI run: https://github.com/canonical/redis-k8s-operator/actions/runs/9760370147/job/26939124902
Relevant log output
The text was updated successfully, but these errors were encountered: