Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Docker. Squid is already running #19

Closed
rivmar opened this issue Mar 12, 2018 · 5 comments
Closed

Docker. Squid is already running #19

rivmar opened this issue Mar 12, 2018 · 5 comments

Comments

@rivmar
Copy link

rivmar commented Mar 12, 2018

Hi.
I am trying to use haipproxy with docker:

~/$ docker version
Client:
 Version:	18.02.0-ce
 API version:	1.36
 Go version:	go1.9.3
 Git commit:	fc4de44
 Built:	Wed Feb  7 21:16:33 2018
 OS/Arch:	linux/amd64
 Experimental:	false
 Orchestrator:	swarm

Server:
 Engine:
  Version:	18.02.0-ce
  API version:	1.36 (minimum version 1.12)
  Go version:	go1.9.3
  Git commit:	fc4de44
  Built:	Wed Feb  7 21:15:05 2018
  OS/Arch:	linux/amd64
  Experimental:	false

on Ubuntu 16.04.3 LTS

And I am constantly getting error:

haipproxy_1 | 2018/03/12 14:46:39| Squid is already running! Process ID 12

I added to docker-compose:

    tty: true
    restart: always

but without too much success:

haipproxy_1 | 2018/03/12 14:53:01| Set Current Directory to /var/spool/squid
haipproxy_1 | 2018/03/12 14:53:01| Starting Squid Cache version 3.5.12 for x86_64-pc-linux-gnu...
haipproxy_1 | 2018/03/12 14:53:01| Service Name: squid
haipproxy_1 | 2018/03/12 14:53:01| Process ID 12
haipproxy_1 | 2018/03/12 14:53:01| Process Roles: master worker
haipproxy_1 | 2018/03/12 14:53:01| With 1048576 file descriptors available
haipproxy_1 | 2018/03/12 14:53:01| Initializing IP Cache...
haipproxy_1 | 2018/03/12 14:53:01| DNS Socket created at [::], FD 8
haipproxy_1 | 2018/03/12 14:53:01| DNS Socket created at 0.0.0.0, FD 9
haipproxy_1 | 2018/03/12 14:53:01| Adding nameserver 127.0.0.11 from /etc/resolv.conf
haipproxy_1 | 2018/03/12 14:53:01| Adding ndots 1 from /etc/resolv.conf
haipproxy_1 | 2018/03/12 14:53:01| Logfile: opening log daemon:/var/log/squid/access.log
haipproxy_1 | 2018/03/12 14:53:01| Logfile Daemon: opening log /var/log/squid/access.log
haipproxy_1 | 2018/03/12 14:53:01| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec
haipproxy_1 | 2018/03/12 14:53:01| Store logging disabled
haipproxy_1 | 2018/03/12 14:53:01| Swap maxSize 0 + 262144 KB, estimated 20164 objects
haipproxy_1 | 2018/03/12 14:53:01| Target number of buckets: 1008
haipproxy_1 | 2018/03/12 14:53:01| Using 8192 Store buckets
haipproxy_1 | 2018/03/12 14:53:01| Max Mem size: 262144 KB
haipproxy_1 | 2018/03/12 14:53:01| Max Swap size: 0 KB
haipproxy_1 | 2018/03/12 14:53:01| Using Least Load store dir selection
haipproxy_1 | 2018/03/12 14:53:01| Set Current Directory to /var/spool/squid
haipproxy_1 | 2018/03/12 14:53:01| Finished loading MIME types and icons.
haipproxy_1 | 2018/03/12 14:53:01| HTCP Disabled.
haipproxy_1 | 2018/03/12 14:53:01| commBind: Cannot bind socket FD 13 to [::1]: (99) Cannot assign requested address
haipproxy_1 | 2018/03/12 14:53:01| commBind: Cannot bind socket FD 14 to [::1]: (99) Cannot assign requested address
haipproxy_1 | 2018/03/12 14:53:01| ERROR: Failed to create helper child read FD: UDP[::1]
haipproxy_1 | 2018/03/12 14:53:01| Squid plugin modules loaded: 0
haipproxy_1 | 2018/03/12 14:53:01| Adaptation support is off.
haipproxy_1 | 2018/03/12 14:53:01| Accepting HTTP Socket connections at local=[::]:3128 remote=[::] FD 12 flags=9
haipproxy_1 | 2018/03/12 14:53:02| storeLateRelease: released 0 objects
haipproxy_1 | 2018/03/12 14:53:04| Reconfiguring Squid Cache (version 3.5.12)...
haipproxy_1 | 2018/03/12 14:53:04| Closing HTTP port [::]:3128
haipproxy_1 | 2018/03/12 14:53:04| Logfile: closing log daemon:/var/log/squid/access.log
haipproxy_1 | 2018/03/12 14:53:04| Logfile Daemon: closing log daemon:/var/log/squid/access.log
haipproxy_1 | 2018/03/12 14:53:04| Startup: Initializing Authentication Schemes ...
haipproxy_1 | 2018/03/12 14:53:04| Startup: Initialized Authentication Scheme 'basic'
haipproxy_1 | 2018/03/12 14:53:04| Startup: Initialized Authentication Scheme 'digest'
haipproxy_1 | 2018/03/12 14:53:04| Startup: Initialized Authentication Scheme 'negotiate'
haipproxy_1 | 2018/03/12 14:53:04| Startup: Initialized Authentication Scheme 'ntlm'
haipproxy_1 | 2018/03/12 14:53:04| Startup: Initialized Authentication.
haipproxy_1 | 2018/03/12 14:53:04| Processing Configuration File: /etc/squid/squid.conf (depth 0)
haipproxy_1 | 2018/03/12 14:53:04| Logfile: opening log daemon:/var/log/squid/access.log
haipproxy_1 | 2018/03/12 14:53:04| Logfile Daemon: opening log /var/log/squid/access.log
haipproxy_1 | 2018/03/12 14:53:04| Squid plugin modules loaded: 0
haipproxy_1 | 2018/03/12 14:53:04| Adaptation support is off.
haipproxy_1 | 2018/03/12 14:53:04| Store logging disabled
haipproxy_1 | 2018/03/12 14:53:04| DNS Socket created at [::], FD 10
haipproxy_1 | 2018/03/12 14:53:04| DNS Socket created at 0.0.0.0, FD 11
haipproxy_1 | 2018/03/12 14:53:04| Adding nameserver 127.0.0.11 from /etc/resolv.conf
haipproxy_1 | 2018/03/12 14:53:04| Adding ndots 1 from /etc/resolv.conf
haipproxy_1 | 2018/03/12 14:53:04| HTCP Disabled.
haipproxy_1 | 2018/03/12 14:53:04| commBind: Cannot bind socket FD 13 to [::1]: (99) Cannot assign requested address
haipproxy_1 | 2018/03/12 14:53:04| commBind: Cannot bind socket FD 14 to [::1]: (99) Cannot assign requested address
haipproxy_1 | 2018/03/12 14:53:04| ERROR: Failed to create helper child read FD: UDP[::1]
haipproxy_1 | 2018/03/12 14:53:04| Finished loading MIME types and icons.
haipproxy_1 | 2018/03/12 14:53:04| Accepting HTTP Socket connections at local=[::]:3128 remote=[::] FD 12 flags=9
haipproxy_1 | 2018/03/12 14:56:04| Reconfiguring Squid Cache (version 3.5.12)...
haipproxy_1 | 2018/03/12 14:56:04| Closing HTTP port [::]:3128
haipproxy_1 | 2018/03/12 14:56:04| Logfile: closing log daemon:/var/log/squid/access.log
haipproxy_1 | 2018/03/12 14:56:04| Logfile Daemon: closing log daemon:/var/log/squid/access.log
haipproxy_1 | 2018/03/12 14:56:04| Startup: Initializing Authentication Schemes ...
haipproxy_1 | 2018/03/12 14:56:04| Startup: Initialized Authentication Scheme 'basic'
haipproxy_1 | 2018/03/12 14:56:04| Startup: Initialized Authentication Scheme 'digest'
haipproxy_1 | 2018/03/12 14:56:04| Startup: Initialized Authentication Scheme 'negotiate'
haipproxy_1 | 2018/03/12 14:56:04| Startup: Initialized Authentication Scheme 'ntlm'
haipproxy_1 | 2018/03/12 14:56:04| Startup: Initialized Authentication.
haipproxy_1 | 2018/03/12 14:56:04| Processing Configuration File: /etc/squid/squid.conf (depth 0)
haipproxy_1 | 2018/03/12 14:56:05| Logfile: opening log daemon:/var/log/squid/access.log
haipproxy_1 | 2018/03/12 14:56:05| Logfile Daemon: opening log /var/log/squid/access.log
haipproxy_1 | 2018/03/12 14:56:05| Squid plugin modules loaded: 0
haipproxy_1 | 2018/03/12 14:56:05| Adaptation support is off.
haipproxy_1 | 2018/03/12 14:56:05| Store logging disabled
haipproxy_1 | 2018/03/12 14:56:05| DNS Socket created at [::], FD 10
haipproxy_1 | 2018/03/12 14:56:05| DNS Socket created at 0.0.0.0, FD 11
haipproxy_1 | 2018/03/12 14:56:05| Adding nameserver 127.0.0.11 from /etc/resolv.conf
haipproxy_1 | 2018/03/12 14:56:05| Adding ndots 1 from /etc/resolv.conf
haipproxy_1 | 2018/03/12 14:56:05| HTCP Disabled.
haipproxy_1 | 2018/03/12 14:56:05| commBind: Cannot bind socket FD 13 to [::1]: (99) Cannot assign requested address
haipproxy_1 | 2018/03/12 14:56:05| commBind: Cannot bind socket FD 14 to [::1]: (99) Cannot assign requested address
haipproxy_1 | 2018/03/12 14:56:05| ERROR: Failed to create helper child read FD: UDP[::1]
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 89.236.17.108/3128/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 61.220.26.97/80/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 159.89.195.153/8118/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 35.196.26.166/3128/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 34.231.147.235/3128/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 216.177.233.181/8080/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 103.74.246.124/65205/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 153.126.201.100/3128/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 121.129.127.209/80/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 109.105.40.39/53281/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 179.106.37.39/8080/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 144.76.176.72/8080/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 181.49.24.126/8080/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 191.210.166.52/8080/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 137.74.254.242/3128/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 192.116.142.153/8080/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 137.74.168.174/8080/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 117.6.161.118/53281/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 128.199.138.67/8118/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 186.227.8.21/3128/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 82.196.12.94/3128/0
haipproxy_1 | 2018/03/12 14:56:05| Finished loading MIME types and icons.
haipproxy_1 | 2018/03/12 14:56:05| Accepting HTTP Socket connections at local=[::]:3128 remote=[::] FD 12 flags=9
haipproxy_1 | 2018/03/12 14:56:31| netdbExchangeHandleReply: corrupt data, aborting
haipproxy_1 | 2018/03/12 14:56:39| netdbExchangeHandleReply: corrupt data, aborting
haipproxy_1 | 2018/03/12 15:02:03| Squid is already running! Process ID 12
haipproxy_1 | 2018/03/12 15:02:04| Squid is already running! Process ID 12
haipproxy_1 | 2018/03/12 15:02:05| Squid is already running! Process ID 12
haipproxy_1 | 2018/03/12 15:02:06| Squid is already running! Process ID 12
haipproxy_1 | 2018/03/12 15:02:07| Squid is already running! Process ID 12
haipproxy_1 | 2018/03/12 15:02:09| Squid is already running! Process ID 12
haipproxy_1 | 2018/03/12 15:02:13| Squid is already running! Process ID 12
haipproxy_1 | 2018/03/12 15:02:20| Squid is already running! Process ID 12
haipproxy_1 | 2018/03/12 15:02:33| Squid is already running! Process ID 12
haipproxy_haipproxy_1 exited with code 0

Is it normal? Or maybe I should change some other settings?
@ResolveWang
Copy link
Member 

haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 191.210.166.52/8080/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 137.74.254.242/3128/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 192.116.142.153/8080/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 137.74.168.174/8080/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 117.6.161.118/53281/0
haipproxy_1 | 2018/03/12 14:56:05| Configuring Parent 128.199.138.67/8118/0

It seems that you have just sent requests with squid successfully.If you can send requests with squid, just leave the messages alone.They are logs of squid.

@rivmar
Copy link
Author

rivmar commented Mar 13, 2018 •
edited

The problem is that when error "Squid is already running! " appears I can not sent request anymore...
Here is another log:

haipproxy_1 | 2018/03/13 07:27:58| Configuring Parent 181.30.101.242/3128/0
haipproxy_1 | 2018/03/13 07:27:58| Configuring Parent 118.178.227.171/80/0
haipproxy_1 | 2018/03/13 07:27:58| Configuring Parent 27.123.1.46/65103/0
haipproxy_1 | 2018/03/13 07:27:58| Configuring Parent 52.207.79.142/80/0
haipproxy_1 | 2018/03/13 07:27:58| Configuring Parent 190.24.131.250/3128/0
haipproxy_1 | 2018/03/13 07:27:58| Configuring Parent 185.119.56.8/53281/0
haipproxy_1 | 2018/03/13 07:27:58| Configuring Parent 137.74.254.242/3128/0
haipproxy_1 | 2018/03/13 07:27:58| Finished loading MIME types and icons.
haipproxy_1 | 2018/03/13 07:27:58| Accepting HTTP Socket connections at local=[::]:3128 remote=[::] FD 16 flags=9
haipproxy_1 | 2018/03/13 07:27:58| Closing HTTP port [::]:3128
haipproxy_1 | 2018/03/13 07:27:58| storeDirWriteCleanLogs: Starting...
haipproxy_1 | 2018/03/13 07:27:58| Finished. Wrote 0 entries.
haipproxy_1 | 2018/03/13 07:27:58| Took 0.00 seconds ( 0.00 entries/sec).
haipproxy_1 | Aborted (core dumped)
haipproxy_1 | 2018/03/13 07:28:00| Squid is already running! Process ID 12
haipproxy_1 | 2018/03/13 07:28:01| Squid is already running! Process ID 12
haipproxy_1 | 2018/03/13 07:28:02| Squid is already running! Process ID 12

I have to make docker stop and docker rm to delete container and up it again to make it work.

@ResolveWang
Copy link
Member

ResolveWang commented Mar 13, 2018 •
edited

I'm sorry that i can't reproduce this problem, may be this answer is useful to you.You can try to deploy haipproxy without docker, and see whether this problem will happen

BTW, if you are familiar with python, you can use py_cli

zingdle added a commit to zingdle/haipproxy that referenced this issue Apr 8, 2018
@zingdle
Update run.sh
0149032 
Fix issue SpiderClub#19 Docker. Squid is already running SpiderClub#19
zingdle added a commit to zingdle/haipproxy that referenced this issue Apr 8, 2018
@zingdle
Merge pull request #1 from zingdle/zingdle-patch-1
82ddb5f 
Fix issue SpiderClub#19
zingdle added a commit to zingdle/haipproxy that referenced this issue Apr 8, 2018
@zingdle
Update run.sh
23af84a 
Fix SpiderClub#19
@zingdle zingdle mentioned this issue Apr 8, 2018
Merged
@teadrinker2015
Copy link

teadrinker2015 commented Mar 27, 2023 •
edited

This information may be useful to you: I was encountered the same error message, because I set the port for Redis on host to 6380, due to conflict with another existing Redis on 6379. The haipproxy container does not immediately fail, instead had been running normally for several hours, and after a computer sleep-wake cycle, completely refuse to restart.

The problem docker-compose:

version: '2.0'
services:
  redis:
    image: redis
    ports:
      - '6380:6379' # I fixed 6380 to 6379 later
    command: >
      --requirepass 123456
  splash:
    image: scrapinghub/splash
    ports:
      - '8050:8050'
  haipproxy:
    build: .
    command: sh run.sh
    volumes:
      - .:/haipproxy
    ports:
      - '3128:3128'
    links:
      - redis
      - splash

I also modified the dockerfile: ( because the original one cannot build properly the time I was using )

FROM ubuntu:18.04

MAINTAINER ResolveWang <resolvewang@foxmail.com>

ENV LC_ALL C.UTF-8
ENV LANG C.UTF-8
RUN apt update
RUN apt install squid -yq
RUN sed -i 's/http_access deny all/http_access allow all/g' /etc/squid/squid.conf
RUN cp /etc/squid/squid.conf /etc/squid/squid.conf.backup
RUN apt install python3 python3-pip -yq
RUN which python3|xargs -i ln -s {} /usr/bin/python
RUN which pip3|xargs -i ln -s {} /usr/bin/pip
COPY . /haipproxy
WORKDIR /haipproxy
RUN python -m pip install --upgrade pip
RUN pip install -i https://pypi.douban.com/simple/ -r requirements.txt
CMD ['python', 'crawler_booter.py', '--usage', 'crawler', 'common']

@teadrinker2015
Copy link

I was wrong, the haipproxy container still refuse to start due to Squid is already running after a host reboot (perhaps include a docker compose stop and start cycle). While it can continue longer than before.

And I tried the solution in answer:

may be this answer is useful to you.

The pid_file option seems to be the default in my current version of squid.conf. I will report after next reboot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ResolveWang @rivmar @teadrinker2015