Issue with connecting to Elasticsearch #281
Labels
question
Further information is requested
Comments
|
It is "normal" to see connection errors when using the docker-compose file like that. Due to the slow startup of the elasticsearch container, and the really fast startup of py-image-dedup, the latter tries to connect to the db before it is up and running. To work around this py-image-dedup has a retry machanism, so waiting a couple of seconds usually lets the processing continue as normal. The real fix would be to have the elasticsearch instance running before launching the py-image-dedup container, or setup some kind of dependency between the two inside of the Since you do get logs about the "analyze" phase, I guess the order of the log output is just mixed up? |
This is exactly what my docker-compose addition does: healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9200"]
interval: 30s
timeout: 30s
retries: 10 depends_on:
elasticsearch:
condition: service_healthyMy issue is that after the initial phase elasticsearch becomes unresponsive: |
|
I cannot reproduce this. I don't use the docker-compose file in production, I use individual containers managed within a kubernetes cluster. Do you get the same behavior with the docker-compose file in this repo? |
|
Thanks for getting back to me! I'm currently running But after some time i get the same issue as before: It is definitely an Elasticsearch issue: I'll investigate some more. |
|
I will close this issue due to inactivity. Feel free to reopen if needed. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
I'm having issue with connections to my Elasticsearch instance.
The following is an example of the behaviour I'm experiences. I'm not sure how to handle it or if it is an issue with Elasticsearch or
py-image-dedup.issue
(venv) fer@SynHome:~/py-image-dedup$ docker-compose up Creating network "py-image-dedup_docker-elk" with driver "bridge" Creating py-image-dedup_elasticsearch_1 ... done Creating py-image-dedup_py-image-dedup_1 ... done Attaching to py-image-dedup_elasticsearch_1, py-image-dedup_py-image-dedup_1 elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:26:45,833Z", "level": "WARN", "component": "o.e.b.JNANatives", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "unable to install syscall filter: ", elasticsearch_1 | "stacktrace": ["java.lang.UnsupportedOperationException: seccomp unavailable: CONFIG_SECCOMP not compiled into kernel, CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER are needed", elasticsearch_1 | "at org.elasticsearch.bootstrap.SystemCallFilter.linuxImpl(SystemCallFilter.java:342) ~[elasticsearch-7.9.2.jar:7.9.2]", elasticsearch_1 | "at org.elasticsearch.bootstrap.SystemCallFilter.init(SystemCallFilter.java:617) ~[elasticsearch-7.9.2.jar:7.9.2]", elasticsearch_1 | "at org.elasticsearch.bootstrap.JNANatives.tryInstallSystemCallFilter(JNANatives.java:260) [elasticsearch-7.9.2.jar:7.9.2]", elasticsearch_1 | "at org.elasticsearch.bootstrap.Natives.tryInstallSystemCallFilter(Natives.java:113) [elasticsearch-7.9.2.jar:7.9.2]", elasticsearch_1 | "at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:116) [elasticsearch-7.9.2.jar:7.9.2]", elasticsearch_1 | "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:178) [elasticsearch-7.9.2.jar:7.9.2]", elasticsearch_1 | "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393) [elasticsearch-7.9.2.jar:7.9.2]", elasticsearch_1 | "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) [elasticsearch-7.9.2.jar:7.9.2]", elasticsearch_1 | "at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) [elasticsearch-7.9.2.jar:7.9.2]", elasticsearch_1 | "at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-7.9.2.jar:7.9.2]", elasticsearch_1 | "at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127) [elasticsearch-cli-7.9.2.jar:7.9.2]", elasticsearch_1 | "at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-7.9.2.jar:7.9.2]", elasticsearch_1 | "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) [elasticsearch-7.9.2.jar:7.9.2]", elasticsearch_1 | "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-7.9.2.jar:7.9.2]"] } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:26:47,794Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "version[7.9.2], pid[8], build[default/docker/d34da0ea4a966c4e49417f2da2f244e3e97b4e6e/2020-09-23T00:45:33.626720Z], OS[Linux/3.10.108/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/15/15+36]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:26:47,824Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "JVM home [/usr/share/elasticsearch/jdk]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:26:47,827Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/elasticsearch-16856439618775791360, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Des.cgroups.hierarchy.override=/, -Xms1g, -Xmx1g, -XX:MaxDirectMemorySize=536870912, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,767Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [aggs-matrix-stats]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,771Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [analysis-common]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,786Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [constant-keyword]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,790Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [flattened]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,791Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [frozen-indices]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,799Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [ingest-common]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,803Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [ingest-geoip]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,805Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [ingest-user-agent]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,826Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [kibana]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,836Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [lang-expression]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,837Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [lang-mustache]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,838Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [lang-painless]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,839Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [mapper-extras]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,840Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [parent-join]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,841Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [percolator]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,851Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [rank-eval]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,865Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [reindex]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,868Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [repository-url]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,869Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [search-business-rules]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,872Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [searchable-snapshots]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,874Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [spatial]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,877Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [tasks]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,885Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [transform]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,894Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [transport-netty4]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,895Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [vectors]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,896Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [wildcard]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,897Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-analytics]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,897Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-async]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,898Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-async-search]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,899Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-autoscaling]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,899Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-ccr]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,900Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-core]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,901Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-data-streams]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,902Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-deprecation]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,903Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-enrich]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,903Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-eql]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,904Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-graph]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,905Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-identity-provider]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,906Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-ilm]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,906Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-logstash]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,907Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-ml]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,908Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-monitoring]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,909Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-ql]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,910Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-rollup]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,910Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-security]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,912Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-sql]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,913Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-stack]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,913Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-voting-only-node]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,914Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "loaded module [x-pack-watcher]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:04,916Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "no plugins loaded" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:05,408Z", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/mapper/cachedev_1)]], net usable_space [54.8gb], net total_space [888.9gb], types [btrfs]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:05,417Z", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "heap size [1gb], compressed ordinary object pointers [true]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:06,025Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "node name [elasticsearch], node ID [Rnp6dtg9QsmDG0INGrREvg], cluster name [docker-cluster]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:35,273Z", "level": "INFO", "component": "o.e.x.m.p.l.CppLogMessageHandler", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "[controller/221] [Main.cc@114] controller (64 bit): Version 7.9.2 (Build 6a60f0cf2dd5a5) Copyright (c) 2020 Elasticsearch BV" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:38,174Z", "level": "INFO", "component": "o.e.x.s.a.s.FileRolesStore", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "parsed [0] roles from file [/usr/share/elasticsearch/config/roles.yml]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:45,386Z", "level": "INFO", "component": "o.e.t.NettyAllocator", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "creating NettyAllocator with the following configs: [name=unpooled, factors={es.unsafe.use_unpooled_allocator=false, g1gc_enabled=true, g1gc_region_size=1mb, heap_size=1gb}]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:46,063Z", "level": "INFO", "component": "o.e.d.DiscoveryModule", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "using discovery type [single-node] and seed hosts providers [settings]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:50,394Z", "level": "WARN", "component": "o.e.g.DanglingIndicesState", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:53,830Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "initialized" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:53,834Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "starting ..." } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:54,729Z", "level": "INFO", "component": "o.e.t.TransportService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "publish_address {192.168.16.2:9300}, bound_addresses {0.0.0.0:9300}" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:56,713Z", "level": "WARN", "component": "o.e.b.BootstrapChecks", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:56,715Z", "level": "WARN", "component": "o.e.b.BootstrapChecks", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:56,721Z", "level": "INFO", "component": "o.e.c.c.Coordinator", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "cluster UUID [VrXadE3iRXC2MvquiVfAtQ]" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:57,084Z", "level": "INFO", "component": "o.e.c.s.MasterService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "elected-as-master ([1] nodes joined)[{elasticsearch}{Rnp6dtg9QsmDG0INGrREvg}{j5fgjBsMS7aLU13iOZiOOw}{192.168.16.2}{192.168.16.2:9300}{dilmrt}{ml.machine_memory=8287412224, xpack.installed=true, transform.node=true, ml.max_open_jobs=20} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 42, version: 210, delta: master node changed {previous [], current [{elasticsearch}{Rnp6dtg9QsmDG0INGrREvg}{j5fgjBsMS7aLU13iOZiOOw}{192.168.16.2}{192.168.16.2:9300}{dilmrt}{ml.machine_memory=8287412224, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}]}" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:57,475Z", "level": "INFO", "component": "o.e.c.s.ClusterApplierService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "master node changed {previous [], current [{elasticsearch}{Rnp6dtg9QsmDG0INGrREvg}{j5fgjBsMS7aLU13iOZiOOw}{192.168.16.2}{192.168.16.2:9300}{dilmrt}{ml.machine_memory=8287412224, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}]}, term: 42, version: 210, reason: Publication{term=42, version=210}" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:57,728Z", "level": "INFO", "component": "o.e.h.AbstractHttpServerTransport", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "publish_address {192.168.16.2:9200}, bound_addresses {0.0.0.0:9200}", "cluster.uuid": "VrXadE3iRXC2MvquiVfAtQ", "node.id": "Rnp6dtg9QsmDG0INGrREvg" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:57,743Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "started", "cluster.uuid": "VrXadE3iRXC2MvquiVfAtQ", "node.id": "Rnp6dtg9QsmDG0INGrREvg" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:59,605Z", "level": "INFO", "component": "o.e.l.LicenseService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "license [8a370969-b86b-4e90-8722-29ba38b2975b] mode [basic] - valid", "cluster.uuid": "VrXadE3iRXC2MvquiVfAtQ", "node.id": "Rnp6dtg9QsmDG0INGrREvg" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:59,612Z", "level": "INFO", "component": "o.e.x.s.s.SecurityStatusChangeListener", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "Active license is now [BASIC]; Security is disabled", "cluster.uuid": "VrXadE3iRXC2MvquiVfAtQ", "node.id": "Rnp6dtg9QsmDG0INGrREvg" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:27:59,689Z", "level": "INFO", "component": "o.e.g.GatewayService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "recovered [1] indices into cluster_state", "cluster.uuid": "VrXadE3iRXC2MvquiVfAtQ", "node.id": "Rnp6dtg9QsmDG0INGrREvg" } elasticsearch_1 | {"type": "server", "timestamp": "2022-07-23T18:28:03,268Z", "level": "INFO", "component": "o.e.c.r.a.AllocationService", "cluster.name": "docker-cluster", "node.name": "elasticsearch", "message": "Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[images][0]]]).", "cluster.uuid": "VrXadE3iRXC2MvquiVfAtQ", "node.id": "Rnp6dtg9QsmDG0INGrREvg" } py-image-dedup_1 | DEBUG:py_image_dedup.util:Starting daemon... py-image-dedup_1 | Starting daemon... py-image-dedup_1 | DEBUG:py_image_dedup.util:Starting prometheus reporter... py-image-dedup_1 | Starting prometheus reporter... py-image-dedup_1 | DEBUG:py_image_dedup.util:==> DRY RUN! No files or folders will actually be deleted! <== py-image-dedup_1 | ==> DRY RUN! No files or folders will actually be deleted! <== py-image-dedup_1 | DEBUG:py_image_dedup.util:Phase 1/6: Cleaning up database ... py-image-dedup_1 | Phase 1/6: Cleaning up database ... py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 0% (1/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 1% (23/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 2% (46/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 3% (69/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 4% (92/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 5% (115/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 6% (138/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 7% (161/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 8% (184/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 9% (207/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 10% (229/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 11% (252/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 12% (275/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 13% (298/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 14% (321/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 15% (344/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 16% (367/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 17% (390/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 18% (413/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 19% (435/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 20% (458/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 21% (481/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 22% (504/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 23% (527/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 24% (550/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 25% (573/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 26% (596/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 27% (619/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 28% (641/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 29% (664/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 30% (687/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 31% (710/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 32% (733/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 33% (756/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 34% (779/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 35% (802/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 36% (825/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 37% (847/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 38% (870/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 39% (893/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 40% (916/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 41% (939/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 42% (962/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 43% (985/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 44% (1008/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 45% (1031/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 46% (1053/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 47% (1076/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 48% (1099/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 49% (1122/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 50% (1145/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 51% (1168/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 52% (1191/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 53% (1214/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 54% (1237/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 55% (1259/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 56% (1282/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 57% (1305/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 58% (1328/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 59% (1351/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 60% (1374/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 61% (1397/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 62% (1420/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 63% (1443/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 64% (1465/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 65% (1488/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 66% (1511/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 67% (1534/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 68% (1557/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 69% (1580/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 70% (1603/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 71% (1626/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 72% (1649/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 73% (1671/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 74% (1694/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 75% (1717/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 76% (1740/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 77% (1763/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 78% (1786/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 79% (1809/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 80% (1832/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 81% (1855/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 82% (1877/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 83% (1900/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 84% (1923/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 85% (1946/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 86% (1969/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 87% (1992/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 88% (2015/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 89% (2038/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 90% (2061/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 91% (2083/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 92% (2106/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 93% (2129/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 94% (2152/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 95% (2175/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 96% (2198/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 97% (2221/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 98% (2244/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 99% (2267/2289) py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Cleanup database: 100% (2289/2289) py-image-dedup_1 | DEBUG:py_image_dedup.util:Phase 2/6: Counting files ... py-image-dedup_1 | Phase 2/6: Counting files ... py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Counting files: 100% (1/1) py-image-dedup_1 | DEBUG:py_image_dedup.util:Phase 3/6: Analyzing files py-image-dedup_1 | Phase 3/6: Analyzing files py-image-dedup_1 | INFO:py_image_dedup.library.progress_manager:Analyzing files in '/mnt/source': 0% (1/179325) py-image-dedup_1 | WARNING:elasticsearch:POST http://elasticsearch:9200/images/_doc?refresh=false [status:N/A request:0.021s] py-image-dedup_1 | Traceback (most recent call last): py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn py-image-dedup_1 | conn = connection.create_connection( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py", line 95, in create_connection py-image-dedup_1 | raise err py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection py-image-dedup_1 | sock.connect(sa) py-image-dedup_1 | ConnectionRefusedError: [Errno 111] Connection refused py-image-dedup_1 | py-image-dedup_1 | During handling of the above exception, another exception occurred: py-image-dedup_1 | py-image-dedup_1 | Traceback (most recent call last): py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/elasticsearch/connection/http_urllib3.py", line 232, in perform_request py-image-dedup_1 | response = self.pool.urlopen( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen py-image-dedup_1 | retries = retries.increment( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 525, in increment py-image-dedup_1 | raise six.reraise(type(error), error, _stacktrace) py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/packages/six.py", line 770, in reraise py-image-dedup_1 | raise value py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen py-image-dedup_1 | httplib_response = self._make_request( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 398, in _make_request py-image-dedup_1 | conn.request(method, url, **httplib_request_kw) py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 239, in request py-image-dedup_1 | super(HTTPConnection, self).request(method, url, body=body, headers=headers) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 1282, in request py-image-dedup_1 | self._send_request(method, url, body, headers, encode_chunked) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 1328, in _send_request py-image-dedup_1 | self.endheaders(body, encode_chunked=encode_chunked) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 1277, in endheaders py-image-dedup_1 | self._send_output(message_body, encode_chunked=encode_chunked) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 1037, in _send_output py-image-dedup_1 | self.send(msg) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 975, in send py-image-dedup_1 | self.connect() py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 205, in connect py-image-dedup_1 | conn = self._new_conn() py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 186, in _new_conn py-image-dedup_1 | raise NewConnectionError( py-image-dedup_1 | urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 111] Connection refused py-image-dedup_1 | WARNING:elasticsearch:POST http://elasticsearch:9200/images/_doc?refresh=false [status:N/A request:0.005s] py-image-dedup_1 | Traceback (most recent call last): py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn py-image-dedup_1 | conn = connection.create_connection( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py", line 95, in create_connection py-image-dedup_1 | raise err py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection py-image-dedup_1 | sock.connect(sa) py-image-dedup_1 | ConnectionRefusedError: [Errno 111] Connection refused py-image-dedup_1 | py-image-dedup_1 | During handling of the above exception, another exception occurred: py-image-dedup_1 | py-image-dedup_1 | Traceback (most recent call last): py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/elasticsearch/connection/http_urllib3.py", line 232, in perform_request py-image-dedup_1 | response = self.pool.urlopen( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen py-image-dedup_1 | retries = retries.increment( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 525, in increment py-image-dedup_1 | raise six.reraise(type(error), error, _stacktrace) py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/packages/six.py", line 770, in reraise py-image-dedup_1 | raise value py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen py-image-dedup_1 | httplib_response = self._make_request( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 398, in _make_request py-image-dedup_1 | conn.request(method, url, **httplib_request_kw) py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 239, in request py-image-dedup_1 | super(HTTPConnection, self).request(method, url, body=body, headers=headers) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 1282, in request py-image-dedup_1 | self._send_request(method, url, body, headers, encode_chunked) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 1328, in _send_request py-image-dedup_1 | self.endheaders(body, encode_chunked=encode_chunked) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 1277, in endheaders py-image-dedup_1 | self._send_output(message_body, encode_chunked=encode_chunked) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 1037, in _send_output py-image-dedup_1 | self.send(msg) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 975, in send py-image-dedup_1 | self.connect() py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 205, in connect py-image-dedup_1 | conn = self._new_conn() py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 186, in _new_conn py-image-dedup_1 | raise NewConnectionError( py-image-dedup_1 | urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 111] Connection refused py-image-dedup_1 | WARNING:elasticsearch:POST http://elasticsearch:9200/images/_doc?refresh=false [status:N/A request:0.015s] py-image-dedup_1 | Traceback (most recent call last): py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn py-image-dedup_1 | conn = connection.create_connection( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py", line 95, in create_connection py-image-dedup_1 | raise err py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection py-image-dedup_1 | sock.connect(sa) py-image-dedup_1 | ConnectionRefusedError: [Errno 111] Connection refused py-image-dedup_1 | py-image-dedup_1 | During handling of the above exception, another exception occurred: py-image-dedup_1 | py-image-dedup_1 | Traceback (most recent call last): py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/elasticsearch/connection/http_urllib3.py", line 232, in perform_request py-image-dedup_1 | response = self.pool.urlopen( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen py-image-dedup_1 | retries = retries.increment( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 525, in increment py-image-dedup_1 | raise six.reraise(type(error), error, _stacktrace) py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/packages/six.py", line 770, in reraise py-image-dedup_1 | raise value py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen py-image-dedup_1 | httplib_response = self._make_request( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 398, in _make_request py-image-dedup_1 | conn.request(method, url, **httplib_request_kw) py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 239, in request py-image-dedup_1 | super(HTTPConnection, self).request(method, url, body=body, headers=headers) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 1282, in request py-image-dedup_1 | self._send_request(method, url, body, headers, encode_chunked) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 1328, in _send_request py-image-dedup_1 | self.endheaders(body, encode_chunked=encode_chunked) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 1277, in endheaders py-image-dedup_1 | self._send_output(message_body, encode_chunked=encode_chunked) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 1037, in _send_output py-image-dedup_1 | self.send(msg) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 975, in send py-image-dedup_1 | self.connect() py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 205, in connect py-image-dedup_1 | conn = self._new_conn() py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 186, in _new_conn py-image-dedup_1 | raise NewConnectionError( py-image-dedup_1 | urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 111] Connection refused py-image-dedup_1 | WARNING:elasticsearch:POST http://elasticsearch:9200/images/_doc?refresh=false [status:N/A request:0.006s] py-image-dedup_1 | Traceback (most recent call last): py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn py-image-dedup_1 | conn = connection.create_connection( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py", line 95, in create_connection py-image-dedup_1 | raise err py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection py-image-dedup_1 | sock.connect(sa) py-image-dedup_1 | ConnectionRefusedError: [Errno 111] Connection refused py-image-dedup_1 | py-image-dedup_1 | During handling of the above exception, another exception occurred: py-image-dedup_1 | py-image-dedup_1 | Traceback (most recent call last): py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/elasticsearch/connection/http_urllib3.py", line 232, in perform_request py-image-dedup_1 | response = self.pool.urlopen( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen py-image-dedup_1 | retries = retries.increment( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 525, in increment py-image-dedup_1 | raise six.reraise(type(error), error, _stacktrace) py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/packages/six.py", line 770, in reraise py-image-dedup_1 | raise value py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen py-image-dedup_1 | httplib_response = self._make_request( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 398, in _make_request py-image-dedup_1 | conn.request(method, url, **httplib_request_kw) py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 239, in request py-image-dedup_1 | super(HTTPConnection, self).request(method, url, body=body, headers=headers) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 1282, in request py-image-dedup_1 | self._send_request(method, url, body, headers, encode_chunked) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 1328, in _send_request py-image-dedup_1 | self.endheaders(body, encode_chunked=encode_chunked) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 1277, in endheaders py-image-dedup_1 | self._send_output(message_body, encode_chunked=encode_chunked) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 1037, in _send_output py-image-dedup_1 | self.send(msg) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 975, in send py-image-dedup_1 | self.connect() py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 205, in connect py-image-dedup_1 | conn = self._new_conn() py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 186, in _new_conn py-image-dedup_1 | raise NewConnectionError( py-image-dedup_1 | urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 111] Connection refused py-image-dedup_1 | WARNING:elasticsearch:POST http://elasticsearch:9200/images/_doc?refresh=false [status:N/A request:0.004s] py-image-dedup_1 | Traceback (most recent call last): py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn py-image-dedup_1 | conn = connection.create_connection( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py", line 95, in create_connection py-image-dedup_1 | raise err py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection py-image-dedup_1 | sock.connect(sa) py-image-dedup_1 | ConnectionRefusedError: [Errno 111] Connection refused py-image-dedup_1 | py-image-dedup_1 | During handling of the above exception, another exception occurred: py-image-dedup_1 | py-image-dedup_1 | Traceback (most recent call last): py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/elasticsearch/connection/http_urllib3.py", line 232, in perform_request py-image-dedup_1 | response = self.pool.urlopen( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen py-image-dedup_1 | retries = retries.increment( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 525, in increment py-image-dedup_1 | raise six.reraise(type(error), error, _stacktrace) py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/packages/six.py", line 770, in reraise py-image-dedup_1 | raise value py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen py-image-dedup_1 | httplib_response = self._make_request( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 398, in _make_request py-image-dedup_1 | conn.request(method, url, **httplib_request_kw) py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 239, in request py-image-dedup_1 | super(HTTPConnection, self).request(method, url, body=body, headers=headers) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 1282, in request py-image-dedup_1 | self._send_request(method, url, body, headers, encode_chunked) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 1328, in _send_request py-image-dedup_1 | self.endheaders(body, encode_chunked=encode_chunked) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 1277, in endheaders py-image-dedup_1 | self._send_output(message_body, encode_chunked=encode_chunked) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 1037, in _send_output py-image-dedup_1 | self.send(msg) py-image-dedup_1 | File "/usr/local/lib/python3.10/http/client.py", line 975, in send py-image-dedup_1 | self.connect() py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 205, in connect py-image-dedup_1 | conn = self._new_conn() py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 186, in _new_conn py-image-dedup_1 | raise NewConnectionError( py-image-dedup_1 | urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 111] Connection refused py-image-dedup_1 | WARNING:elasticsearch:POST http://elasticsearch:9200/images/_doc?refresh=false [status:N/A request:0.003s] py-image-dedup_1 | Traceback (most recent call last): py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn py-image-dedup_1 | conn = connection.create_connection( py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py", line 95, in create_connection py-image-dedup_1 | raise err py-image-dedup_1 | File "/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection py-image-dedup_1 | sock.connect(sa) py-image-dedup_1 | ConnectionRefusedError: [Errno 111] Connection refused py-image-dedup_1 |To Reproduce
Running with the following
version: '3.7' services: elasticsearch: image: docker.elastic.co/elasticsearch/elasticsearch:7.9.2 ports: - "9200:9200" - "9300:9300" environment: - discovery.type=single-node - cluster.routing.allocation.disk.threshold_enabled=true - cluster.routing.allocation.disk.watermark.flood_stage=200mb - cluster.routing.allocation.disk.watermark.low=500mb - cluster.routing.allocation.disk.watermark.high=300mb - node.name=elasticsearch - "ES_JAVA_OPTS=-Xms1g -Xmx1g" networks: - docker-elk restart: on-failure volumes: - ./esdata1:/usr/share/elasticsearch/data healthcheck: test: ["CMD", "curl", "-f", "http://localhost:9200"] interval: 30s timeout: 30s retries: 10 py-image-dedup: image: markusressel/py-image-dedup:latest environment: - PUID=0 - PGID=100 # change configuration to your liking - PY_IMAGE_DEDUP_DRY_RUN=True - PY_IMAGE_DEDUP_ANALYSIS_SOURCE_DIRECTORIES=/mnt/source/ - PY_IMAGE_DEDUP_ANALYSIS_RECURSIVE=True - PY_IMAGE_DEDUP_ANALYSIS_ACROSS_DIRS=True - PY_IMAGE_DEDUP_ANALYSIS_FILE_EXTENSIONS=.png,.jpg,.jpeg - PY_IMAGE_DEDUP_ANALYSIS_THREADS=4 - PY_IMAGE_DEDUP_ANALYSIS_USE_EXIF_DATA=True - PY_IMAGE_DEDUP_DEDUPLICATION_DUPLICATES_TARGET_DIRECTORY=/mnt/duplicates/ - PY_IMAGE_DEDUP_ELASTICSEARCH_HOST=elasticsearch - PY_IMAGE_DEDUP_ELASTICSEARCH_PORT=9200 - PY_IMAGE_DEDUP_ELASTICSEARCH_INDEX=images - PY_IMAGE_DEDUP_ELASTICSEARCH_AUTO_CREATE_INDEX=True - PY_IMAGE_DEDUP_ELASTICSEARCH_MAX_DISTANCE=0.1 - PY_IMAGE_DEDUP_REMOVE_EMPTY_FOLDERS=False - PY_IMAGE_DEDUP_STATS_ENABLED=True - PY_IMAGE_DEDUP_STATS_PORT=8000 volumes: # optionally mount a YAML configuration file # into /app/py_image_dedup.yaml instead of using environment: # - /mnt/data3/py_image_dedup.yaml:/app/py_image_dedup.yaml # change this to your local source directory: - /volume2/pictures/gto:/mnt/source:rw # change this to your local duplicates directory: - /volume2/Media/photo-dupes:/mnt/duplicates/ links: - elasticsearch networks: - docker-elk ports: - "8000:8000" depends_on: elasticsearch: condition: service_healthy restart: on-failure networks: docker-elk: driver: bridgeThe text was updated successfully, but these errors were encountered: