users' profile photos are not showing on the dockerized installation

[abdellani]

Describe the bug
After following the installation with docker guide and accessing the locally hosted application, users' profile photos weren't showing up.

web_1        | Cannot render console from 172.20.0.1! Allowed networks: 127.0.0.1, ::1, 127.0.0.0/127.255.255.255
web_1        | GET /uploads/user/profile_image/6/7a1aba89-1d79-486f-8c82-ab7a8f8a58c2.png completed with 404 Not Found in 0.677355ms
web_1        | Cannot render console from 172.20.0.1! Allowed networks: 127.0.0.1, ::1, 127.0.0.0/127.255.255.255
web_1        | GET /serviceworker.js completed with 304 Not Modified in 15.124716ms

To Reproduce

1- Follow the installation with docker guide.
2- Access 127.0.0.1:3000

Expected behavior

Screenshots

1-

2-

3-

Desktop (please complete the following information):

• OS: Ubuntu 18.04
• Browser: firefox
• Version: 70

Smartphone (please complete the following information):

• Device:
• OS:
• Browser:
• Version:

Additional context

[DarkSmile92]

I am currently looking into this issue.
So far I found out that example posts with users are generated and the physical files to the profile images referenced are not created and not included anywhere in the repo to copy from.
An example of a file also contained in a test:
app/javascript/listings/tests/singleListing.test.jsx

author: {
    name: 'Mrs. Yoko Christiansen',
    username: 'mrschristiansenyoko',
    profile_image_90:
      '/uploads/user/profile_image/7/4b1c980a-beb0-4a5f-b3f2-acc91adc503c.png',
  },

The folder uploads is empty. Creating the path manually, copying a PNG there with this exact name does not display it somehow.

[DanCharousek]

Hi, @cmgorton,

I'd be happy to participate on this issue as part of the dev.to bug smash. Is it still available?
If so, I've noticed that the docker installation guide link returns 404, so I assume it's safe to use this one instead.

Thanks for the reply,
Dan

[cmgorton]

Hi, @cmgorton,

I'd be happy to participate on this issue as part of the dev.to bug smash. Is it still available?
If so, I've noticed that the docker installation guide link returns 404, so I assume it's safe to use this one instead.

Thanks for the reply,
Dan

Thanks for the help here @DanCharousek . I will assign this issue to you. And yes you can use that link! I believe that is the updated link to the docs.

[DanCharousek]

Hi, @cmgorton,
I'd be happy to participate on this issue as part of the dev.to bug smash. Is it still available?
If so, I've noticed that the docker installation guide link returns 404, so I assume it's safe to use this one instead.
Thanks for the reply,
Dan

Thanks for the help here @DanCharousek . I will assign this issue to you. And yes you can use that link! I believe that is the updated link to the docs.

So I've started working on the issue and got stuck at the very beginning of installing the project via the containers.
I used the default .env_samle as it seems no changes to the ENV variables are required (or at least that's what I understood from the installation guide). And after all containers spin up I keep getting 400 Bad Request.

This is the snippet of the console log.

seed_1       | 2021/05/05 16:01:36 Received 400 from http://rails:3000. Sleeping 20s
sidekiq_1    | 2021/05/05 16:01:36 Received 400 from http://rails:3000. Sleeping 20s
rails_1      | Started GET "/" for 172.24.0.8 at 2021-05-05 16:01:56 +0000
rails_1      | Started GET "/" for 172.24.0.9 at 2021-05-05 16:01:56 +0000
rails_1      | Cannot render console from 172.24.0.8! Allowed networks: 172.24.0.1, 127.0.0.0/127.255.255.255, ::1
rails_1      | Cannot render console from 172.24.0.9! Allowed networks: 172.24.0.1, 127.0.0.0/127.255.255.255, ::1
rails_1      | Processing by StoriesController#index as HTML
rails_1      |   Parameters: {"locale"=>nil}
rails_1      | Processing by StoriesController#index as HTML
rails_1      |   Parameters: {"locale"=>nil}
rails_1      |   Rendering layout layouts/application.html.erb
rails_1      |   Rendering layout layouts/application.html.erb
rails_1      |   Rendering devise/registrations/new.html.erb within layouts/application
rails_1      |   Rendering devise/registrations/new.html.erb within layouts/application
rails_1      |   Flipper::Adapters::ActiveRecord::Gate Load (0.7ms)  SELECT "flipper_gates".* FROM "flipper_gates" WHERE "flipper_gates"."feature_key" = $1  [["feature_key", "creator_onboarding"]]
rails_1      |   Flipper::Adapters::ActiveRecord::Gate Load (0.7ms)  SELECT "flipper_gates".* FROM "flipper_gates" WHERE "flipper_gates"."feature_key" = $1  [["feature_key", "creator_onboarding"]]
rails_1      |   ↳ app/services/feature_flag.rb:3:in `enabled?'
rails_1      |   ↳ app/services/feature_flag.rb:3:in `enabled?'
rails_1      |   Rendered shared/authentication/_email_registration_form.html.erb (Duration: 2.1ms | Allocations: 3336)
rails_1      |   Rendered shared/authentication/_email_registration_form.html.erb (Duration: 2.2ms | Allocations: 3336)
rails_1      |   Rendered shared/authentication/_initial_account_wizard.html.erb (Duration: 3.7ms | Allocations: 3580)
rails_1      |   Rendered shared/authentication/_initial_account_wizard.html.erb (Duration: 3.7ms | Allocations: 3580)
rails_1      |   Rendered devise/registrations/new.html.erb within layouts/application (Duration: 12.2ms | Allocations: 5477)
rails_1      |   Rendered devise/registrations/new.html.erb within layouts/application (Duration: 12.3ms | Allocations: 5477)
rails_1      |   Rendered layout layouts/application.html.erb (Duration: 13.2ms | Allocations: 5606)
rails_1      |   Rendered layout layouts/application.html.erb (Duration: 13.2ms | Allocations: 5606)
rails_1      | Completed 400 Bad Request in 18ms (Views: 0.2ms | ActiveRecord: 0.7ms | Allocations: 6482)
rails_1      | 
rails_1      | 
rails_1      | Completed 400 Bad Request in 18ms (Views: 0.3ms | ActiveRecord: 0.7ms | Allocations: 6482)
rails_1      | 
rails_1      | 
sidekiq_1    | 2021/05/05 16:01:56 Received 400 from http://rails:3000. Sleeping 20s
seed_1       | 2021/05/05 16:01:56 Received 400 from http://rails:3000. Sleeping 20s

@cmgorton
Any idea who could point me in the right direction here? If needed, I can provide more details on my setup.

Thank you,
Dan

[cmgorton]

@joshpuetz @citizen428 Could either of you help debug this or point to someone on the team more familiar with Docker? Thanks.

[citizen428]

@cmgorton I haven't used or worked with our container setup in a long time, @jdoss is the go-to person for all things container-related.

[DanCharousek]

Hi @jdoss, do you think you could point me where to look for an issue with the dockerized setup?

I followed the instruction guide and kept getting error 400 after the containers spin up. I also tried to tear everything down and start over. I did not do any changes in the .env. Is it possible that I am missing some important env variable here?

Thanks in advance,
Dan

[djuber]

Hi Dan,

Just wanted to chime in since I've been using the docker images to do unit testing and have seen similar issues (the ProfileImageUploader does not create the image file for the user and in testing the user creation fails in the factory setting the profile_image). I'm still working to isolate this (I think I'm close to an answer of what's happening, I still don't understand why it's happening).

I tried to setup a docker based environment from scratch following the same process - clone repo - copy .env_sample to .env (no changes), call bin/container-setup - it looks like I had problems logging in when I accessed as http://localhost:3000/ :

forem_rails | Started POST "/users/sign_in" for 172.18.0.1 at 2021-05-12 14:55:06 +0000
forem_rails |    (0.5ms)            SELECT ff.key AS feature_key, fg.key, fg.value           FROM flipper_features ff           LEFT JOIN flipper_gates fg ON ff.key = fg.feature_key 
forem_rails |   ↳ app/middlewares/set_cookie_domain.rb:13:in `call'
forem_rails | Processing by Devise::SessionsController#create as HTML
forem_rails |   Parameters: {"utf8"=>"✓", "authenticity_token"=>"[FILTERED]", "user"=>{"email"=>"admin@forem.local", "password"=>"[FILTERED]", "remember_me"=>"1"}, "commit"=>"Continue"}
forem_rails | HTTP Origin header (http://localhost:3000) didn't match request.base_url (http://localhost:3000)
forem_rails | Completed 200 OK in 2ms (ActiveRecord: 0.0ms | Allocations: 2936)

I was able to work around this by adding rails to the localhost line in /etc/hosts (I believe this works on MacOS as well as on Linux, I'm not sure what the equivalent would be for windows).

127.0.0.1	localhost rails

I saw the "start your forem journey" first user setup page when I first connected, however, users were already seeded in the database during container startup, so it's safe to fill in anything there if you saw the same thing and use the default admin account.

Logging in to http://rails:3000/ as admin@forem.local using the default password worked - I would get a 200 response and no redirect on posts to localhost:3000 but get redirected to the home page after login when I changed the url hostname to rails - this might have shown as a 400 error for you (not sure).

I went to the admin user's profile, saw a seeded image, and set successfully a new image - I confirmed I can see the changed file saved - and displayed on the top bar when I visit other pages. I'm unable to write a post because some of the js packs appear to give a 404 (I get to rails:3000/new but the controls

I'm not sure if the original reported issue is still an active problem (the issue was opened in 2019 and we've made some recent changes to the container setup to support self-hosting Forems). However, I'm interested in the problem you are seeing with the 400 error - can you let me know whether that's something you're seeing from the docker-compose console logs or a request you're making with a browser that's failing?

[DanCharousek]

Hi Dan,

Just wanted to chime in since I've been using the docker images to do unit testing and have seen similar issues (the ProfileImageUploader does not create the image file for the user and in testing the user creation fails in the factory setting the profile_image). I'm still working to isolate this (I think I'm close to an answer of what's happening, I still don't understand why it's happening).

I tried to setup a docker based environment from scratch following the same process - clone repo - copy .env_sample to .env (no changes), call bin/container-setup - it looks like I had problems logging in when I accessed as http://localhost:3000/ :

forem_rails | Started POST "/users/sign_in" for 172.18.0.1 at 2021-05-12 14:55:06 +0000
forem_rails |    (0.5ms)            SELECT ff.key AS feature_key, fg.key, fg.value           FROM flipper_features ff           LEFT JOIN flipper_gates fg ON ff.key = fg.feature_key 
forem_rails |   ↳ app/middlewares/set_cookie_domain.rb:13:in `call'
forem_rails | Processing by Devise::SessionsController#create as HTML
forem_rails |   Parameters: {"utf8"=>"✓", "authenticity_token"=>"[FILTERED]", "user"=>{"email"=>"admin@forem.local", "password"=>"[FILTERED]", "remember_me"=>"1"}, "commit"=>"Continue"}
forem_rails | HTTP Origin header (http://localhost:3000) didn't match request.base_url (http://localhost:3000)
forem_rails | Completed 200 OK in 2ms (ActiveRecord: 0.0ms | Allocations: 2936)

I was able to work around this by adding rails to the localhost line in /etc/hosts (I believe this works on MacOS as well as on Linux, I'm not sure what the equivalent would be for windows).

127.0.0.1	localhost rails

I saw the "start your forem journey" first user setup page when I first connected, however, users were already seeded in the database during container startup, so it's safe to fill in anything there if you saw the same thing and use the default admin account.

Logging in to http://rails:3000/ as admin@forem.local using the default password worked - I would get a 200 response and no redirect on posts to localhost:3000 but get redirected to the home page after login when I changed the url hostname to rails - this might have shown as a 400 error for you (not sure).

I went to the admin user's profile, saw a seeded image, and set successfully a new image - I confirmed I can see the changed file saved - and displayed on the top bar when I visit other pages. I'm unable to write a post because some of the js packs appear to give a 404 (I get to rails:3000/new but the controls

I'm not sure if the original reported issue is still an active problem (the issue was opened in 2019 and we've made some recent changes to the container setup to support self-hosting Forems). However, I'm interested in the problem you are seeing with the 400 error - can you let me know whether that's something you're seeing from the docker-compose console logs or a request you're making with a browser that's failing?

Hi Daniel, thank you for your help. Unfortunately I do not get past the point where you reach the http://localhost:3000 as it returns the mentioned error 400.

My previous comment displays the output of my terminal when running ./bin/container-setup. The sidekiq_1 and sidekiq_1 containers keep receiving error 400. The same happens when I access the URL directly (see the screenshot below).

Do you have any idea how I can approach the debugging from here? I do not have experience with rails, but I have experties in other languages, so I might be able to debug with little bit of help. I am also very confident in dealing with the docker.

Thank you for your time.

Dan

[djuber]

Hi Daniel, thank you for your help. Unfortunately I do not get past the point where you reach the http://localhost:3000 as it returns the mentioned error 400.

My previous comment displays the output of my terminal when running ./bin/container-setup. The sidekiq_1 and sidekiq_1 containers keep receiving error 400. The same happens when I access the URL directly (see the screenshot below).

Do you have any idea how I can approach the debugging from here? I do not have experience with rails, but I have experties in other languages, so I might be able to debug with little bit of help. I am also very confident in dealing with the docker.

Thank you for your time.

Dan

This makes some sense - I think we saw this in an error handler when we were testing a self-hosted (container) install last week.

1. try to pull the latest version of the image (docker pull quay.io/forem/forem:latest or the equivalent)

2. if that fails - I think there's a rescue_from in the stories controller that's returning the bad request and hiding the underlying issue
   

   forem/app/controllers/stories_controller.rb

   Line 23 in bb9780f

   rescue_from ArgumentError, with: :bad_request

if you comment that out you might see the actual error message (I believe this was tied to #13673 and should be fixed in the latest image version)

For the "rails" explanation - the rescue_from causes a raised error (inside a view or controller action) to silently become a response (400 bad request in this case) - it sure makes troubleshooting hard when you can't see the error since it was handled without any logging or hints.

Sidekiq showing the 400 error is because it's waiting for rails to become available (it relies on a lot of the same steps as the webserver process so waiting for rails to come online is easier than checking the other requirements like redis and bundler). It's the same problem in either case.

[DanCharousek]

Hi Daniel, thank you for your help. Unfortunately I do not get past the point where you reach the http://localhost:3000 as it returns the mentioned error 400.
My previous comment displays the output of my terminal when running ./bin/container-setup. The sidekiq_1 and sidekiq_1 containers keep receiving error 400. The same happens when I access the URL directly (see the screenshot below).

Do you have any idea how I can approach the debugging from here? I do not have experience with rails, but I have experties in other languages, so I might be able to debug with little bit of help. I am also very confident in dealing with the docker.
Thank you for your time.
Dan

This makes some sense - I think we saw this in an error handler when we were testing a self-hosted (container) install last week.

1. try to pull the latest version of the image (`docker pull quay.io/forem/forem:latest` or the equivalent)

2. if that fails - I think there's a `rescue_from` in the stories controller that's returning the bad request and hiding the underlying issue
   https://github.com/forem/forem/blob/bb9780ff3d0179b03dde2d8550765bb44922be6c/app/controllers/stories_controller.rb#L23

if you comment that out you might see the actual error message (I believe this was tied to #13673 and should be fixed in the latest image version)

For the "rails" explanation - the rescue_from causes a raised error (inside a view or controller action) to silently become a response (400 bad request in this case) - it sure makes troubleshooting hard when you can't see the error since it was handled without any logging or hints.

Sidekiq showing the 400 error is because it's waiting for rails to become available (it relies on a lot of the same steps as the webserver process so waiting for rails to come online is easier than checking the other requirements like redis and bundler). It's the same problem in either case.

With latest changes from the main and latest image version the issue is gone. The app is now up and running. As you mentioned in the previous comment, it seems that the original issue has been resolved as I see users' profile pictures. Thank you for your help and explenation.

I also tried to login in as admin@forem.local, but I am not sure what the default password is. I tried password. After that, the only thing I see is white screen without any obvious issue (besides the white screen). Server response status code is 200 but does contain an empty body (see attached screenshot). But I belive this is getting little bit off-topic in context of this issue.

[djuber]

Hi Dan,

It looks like the "HTTP Origin header did not match" error in your logs screenshot is probably related to the problem you're seeing.

I think the underlying issue there is that the app is being selective about which Host: header it considers valid, and the containers need to use a name other than localhost to permit inter-container traffic (for example, sidekiq requesting http://rails:3000/ to know the app is alive).

I think I was able to bypass this with the following changes to docker-compose.yml

 modified   docker-compose.yml
@@ -20,7 +20,7 @@ services:
       RACK_TIMEOUT_WAIT_TIMEOUT: 10000
       RACK_TIMEOUT_SERVICE_TIMEOUT: 10000
       STATEMENT_TIMEOUT: 10000
-      APP_DOMAIN: rails
+      APP_DOMAIN: localhost:3000
     volumes:
       - .:/opt/apps/forem:delegated
     entrypoint: ["dockerize", "-wait", "tcp://db:5432", "-wait", "tcp://redis:6379", "-wait", "file:///opt/apps/forem/vendor/bundle/.bundle_finished", "-timeout", "2700s", "-wait-retry-interval", "10s"]
@@ -89,7 +89,7 @@ services:
       DATABASE_URL: postgresql://forem:forem@db:5432/PracticalDeveloper_development
     volumes:
       - .:/opt/apps/forem:delegated
-    entrypoint: ["dockerize", "-wait", "tcp://db:5432", "-wait", "tcp://redis:6379", "-wait", "http://rails:3000", "-timeout", "2700s", "-wait-retry-interval", "20s"]
+    entrypoint: ["dockerize", "-wait", "tcp://db:5432", "-wait", "tcp://redis:6379", "-wait", "tcp://rails:3000", "-timeout", "2700s", "-wait-retry-interval", "20s"]
     command: ["bundle", "exec", "rake","db:seed"]
 
   sidekiq:
@@ -108,7 +108,7 @@ services:
       DATABASE_URL: postgresql://forem:forem@db:5432/PracticalDeveloper_development
     volumes:
       - .:/opt/apps/forem:delegated
-    entrypoint: ["dockerize", "-wait", "tcp://db:5432", "-wait", "tcp://redis:6379", "-wait", "http://rails:3000", "-timeout", "2700s", "-wait-retry-interval", "20s"]
+    entrypoint: ["dockerize", "-wait", "tcp://db:5432", "-wait", "tcp://redis:6379", "-wait", "tcp://rails:3000", "-timeout", "2700s", "-wait-retry-interval", "20s"]
     command: ["bundle", "exec", "sidekiq","-c","2"]
 
   db:

In the rails container, set the env var APP_DOMAIN to localhost:3000 - this will match what you as a local user will be using ( I think this fixes the origin header did not match issue you're running into). Also, in the two containers which wait for the http service to startup (sidekiq and the db seed process) - change the request from http to tcp - this now will proceed when the service starts, and won't require an HTTP 200 response (you initially were seeing a 400 bad request when the sidekiq container tried to load the home page, and I was seeing a 403 when I used a name rails different from the APP_DOMAIN).

https://docs.forem.com/getting-started/db/#default-admin-user the seeded password (if that user exists) should be password, so that part looks right - but it relies on the seed having run successfully (and it may not have) - but I think the issue is the origin header checks for login are preventing login.

I'll see if I can't get that change above into a PR today, if you can confirm whether that addresses your problem I'd be grateful for the feedback.

I did see some hiccups with javascript bundles giving 404's - and I ended up clearing the root owned files from tmp/ and this seems to have "healed".

I appreciate your patience here! I think the docker setup for local development isn't used enough by the core team and ends up with a few rough edges that we're not seeing immediately.

[DanCharousek]

Hi Dan,

It looks like the "HTTP Origin header did not match" error in your logs screenshot is probably related to the problem you're seeing.

I think the underlying issue there is that the app is being selective about which Host: header it considers valid, and the containers need to use a name other than localhost to permit inter-container traffic (for example, sidekiq requesting http://rails:3000/ to know the app is alive).

I think I was able to bypass this with the following changes to docker-compose.yml

 modified   docker-compose.yml
@@ -20,7 +20,7 @@ services:
       RACK_TIMEOUT_WAIT_TIMEOUT: 10000
       RACK_TIMEOUT_SERVICE_TIMEOUT: 10000
       STATEMENT_TIMEOUT: 10000
-      APP_DOMAIN: rails
+      APP_DOMAIN: localhost:3000
     volumes:
       - .:/opt/apps/forem:delegated
     entrypoint: ["dockerize", "-wait", "tcp://db:5432", "-wait", "tcp://redis:6379", "-wait", "file:///opt/apps/forem/vendor/bundle/.bundle_finished", "-timeout", "2700s", "-wait-retry-interval", "10s"]
@@ -89,7 +89,7 @@ services:
       DATABASE_URL: postgresql://forem:forem@db:5432/PracticalDeveloper_development
     volumes:
       - .:/opt/apps/forem:delegated
-    entrypoint: ["dockerize", "-wait", "tcp://db:5432", "-wait", "tcp://redis:6379", "-wait", "http://rails:3000", "-timeout", "2700s", "-wait-retry-interval", "20s"]
+    entrypoint: ["dockerize", "-wait", "tcp://db:5432", "-wait", "tcp://redis:6379", "-wait", "tcp://rails:3000", "-timeout", "2700s", "-wait-retry-interval", "20s"]
     command: ["bundle", "exec", "rake","db:seed"]
 
   sidekiq:
@@ -108,7 +108,7 @@ services:
       DATABASE_URL: postgresql://forem:forem@db:5432/PracticalDeveloper_development
     volumes:
       - .:/opt/apps/forem:delegated
-    entrypoint: ["dockerize", "-wait", "tcp://db:5432", "-wait", "tcp://redis:6379", "-wait", "http://rails:3000", "-timeout", "2700s", "-wait-retry-interval", "20s"]
+    entrypoint: ["dockerize", "-wait", "tcp://db:5432", "-wait", "tcp://redis:6379", "-wait", "tcp://rails:3000", "-timeout", "2700s", "-wait-retry-interval", "20s"]
     command: ["bundle", "exec", "sidekiq","-c","2"]
 
   db:

In the rails container, set the env var APP_DOMAIN to localhost:3000 - this will match what you as a local user will be using ( I think this fixes the origin header did not match issue you're running into). Also, in the two containers which wait for the http service to startup (sidekiq and the db seed process) - change the request from http to tcp - this now will proceed when the service starts, and won't require an HTTP 200 response (you initially were seeing a 400 bad request when the sidekiq container tried to load the home page, and I was seeing a 403 when I used a name rails different from the APP_DOMAIN).

https://docs.forem.com/getting-started/db/#default-admin-user the seeded password (if that user exists) should be password, so that part looks right - but it relies on the seed having run successfully (and it may not have) - but I think the issue is the origin header checks for login are preventing login.

I'll see if I can't get that change above into a PR today, if you can confirm whether that addresses your problem I'd be grateful for the feedback.

I did see some hiccups with javascript bundles giving 404's - and I ended up clearing the root owned files from tmp/ and this seems to have "healed".

I appreciate your patience here! I think the docker setup for local development isn't used enough by the core team and ends up with a few rough edges that we're not seeing immediately.

Hi Daniel,

it seems the changes you suggested helped to resolve the issue. I was able to successfully login as an admin. Although for some reason I do not see posts when logged in (not sure if this is correct behaviour - see screenshot bellow).

Anyway, it seems a little bit OT with regards to this issue. Would you suggest to continue the potentional discussion (if it does not end here :)) somewhere else? If so, where is the else? As I mentioned earlier, I am not a rails person or ruby for that matter, but I'd be happy to offer my help in any other areas I possibly might be helpful at (my main devstack consists of PHP-Laravel/JS/TS-Vue.js/Docker)

Thank you in advance,
Dan

[djuber]

Hi Dan,

I wonder if https://forem.dev is the right venue for this kind of ongoing thread no longer tied to a specific issue? Actually - the first issue looks like it's described in https://forem.dev/manuel/forem-blank-page-after-login-using-docker-221d - so that's not wholly out of line as a discussion board to cover this. I don't think we have a great way to pull an issue's comment thread over.

I also don't know for sure why there wouldn't be any posts visible (they're all from the seed) and whether that's the wrong behavior (it looks like it) - and unless you see a bright red JS error message in the browser console, the absence of posts is almost surely in the rails backend (under the stories controller I think).

[maestromac]

Since we now have new setup options for local setup, highlighted in this post , this issue is now stale.