-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bugfix: Error: Search Backend only searching default admin search fields #654
Comments
@alsokpisz The error message describes a mis configured dns in the docker compose setup. If the search backend can't be queried the search will only occur in the url and title, the admin fields. |
@alsokpisz as @jdcaballerov mentioned this is likely a DNS resolving issue inside your docker-compose network. Docker on macOS is infamous for having container DNS issues, so I wouldn't be surprised if Docker on Windows is plagued by similar bugs. First please make sure you have Sonic's # these linux commands may be different on Windows, sorry I don't know the equivalents for batch/powershell
mkdir -p ./etc/sonic
cd ./etc/sonic
curl -O https://raw.githubusercontent.com/ArchiveBox/ArchiveBox/master/etc/sonic/config.cfg Then confirm that Sonic is up and running and accessible from the archivebox container, can you run these python commands manually and report back what output you get: docker-compose run archivebox /usr/local/bin/python3
>>> import socket
>>> HOST = 'sonic'
>>> PORT = 1491
>>> with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
... s.connect((HOST, PORT))
... s.sendall(b'Hello, world')
... data = s.recv(1024)
...
>>> print('Received', repr(data))
Received b'CONNECTED <sonic-server v1.3.0>\r\n' # this line means everything is working, if your output is different then something is wrong
>>> |
Several errors in the terminal (see screenshot 1). EDIT:
EDIT 2 (one hot cup of coffee later):
|
Inside of docker is always linux, so having a Windows path in this docker command doesn't make sense: Run it verbatim as I posted above, and paste in the script line by line (don't make a file): docker-compose run archivebox /usr/local/bin/python3
>>> ... paste in lines above here |
try |
Result: |
Great! That means both the inter-container DNS and the TCP socket to the sonic container are working. Try this next in >>> from sonic import SearchClient
>>> from archivebox.config import SEARCH_BACKEND_HOST_NAME, SEARCH_BACKEND_PORT, SEARCH_BACKEND_PASSWORD, SONIC_BUCKET, SONIC_COLLECTION
>>> with SearchClient(SEARCH_BACKEND_HOST_NAME, SEARCH_BACKEND_PORT, SEARCH_BACKEND_PASSWORD) as querycl:
>>> print(querycl.query(SONIC_COLLECTION, SONIC_BUCKET, 'test')) |
Results: EDIT: While troubleshooting why mass import links never seem to get the Chrome headless stuff to capture (pdf, scrnshot, dom) I essentially re-imported all of my links. Two new folders appeared in |
Ok, getting closer, sound like Sonic is working and connected but it's not getting text to index. Can you try running this to force a re-index: docker-compose run archivebox update --index-only Then you can test full-text search from the CLI like so: archivebox list --filter-type=search example If it works from the Admin and the CLI then we can try and track down why the public index isn't working. If it's broken on the CLI then there's still an issue with the Sonic backend we have figure out. Thanks for bearing with me here! |
Seems like any page which is a .pdf, or .jpg causes this error during the index command:
And then it hangs. Sometimes I'd get just the one error. I think this happened after I got rid of the .pdf links. I jotted it down but didn't write any context with it.
After removing all the .pdf/.jpg links, there are no errors in the terminal when I run the re-index command, but it will still spend ages on random pages. Notably stuff with 'weirder' components like live webcam feeds or something. I removed those one by one until it managed to get through the 80ish bookmarks in less than 10 minutes. The terminal results were the same ones as the admin search, but the public search still didn't work. |
@alsokpisz the public search view is not connected to the search backend for security and performance reasons. |
Well that's that then I suppose. |
Ok this should be somewhat improved in f67a5a2. It will be out with the next v0.6 release soon. Comment back here if you're still having issues with indexing failures/hanging and I'll reopen the issue. |
This is what I get when I follow the troubleshooting: Anyone know what I'm doing incorrectly? |
Looks like you messed up the indentation, make sure to copy paste that whole block together above, or remove the extra newline before that print to be doubly sure. @jdqw210 |
Describe the bug
Bug:
The bug occurs when I attempt to search any query. An error message appears saying:
"Error from the search backend, only showing results from default admin search fields -Error:[Errno -3] Temporary failure in name resolution."
If the search query is a word in the title of a website, it will return results with that word in it.
If it is only in the wget snapshot of the item, it will not return that item.
Context:
I am running ArchiveBox using on Windows 10 with docker-compose and have launched the web UI which I am successfully accessing at
http://127.0.0.1:8000
. As far as I can tell, all the snapshots are functional and there are no pending links. The output directory is on an external hard drive, but there have been no issues reading/writing from this drive (except for speed, though I can't tell if that's just how the Django Web UI is or not).Relevant Info:
Bug seems similar to @jdcaballerov comment when search enabled but backend failed in his testing (see screenshot 4).
Steps to reproduce
mkdir archivebox && cd archivebox
curl -O https://raw.githubusercontent.com/ArchiveBox/ArchiveBox/master/docker-compose.yml
Edit the docker-compose yml's
volumes
section to read:(unsure if external drive-specific setup needed to reproduce, so wanted to include)
Open a Windows Terminal in administrator mode, navigate to
D:/archivebox/
, open a Git Bash tab and run the following:> docker-compose up -d
> docker-compose run archivebox init
> docker-compose run archivebox manage createsuperuser
> docker-compose run archivebox add 'https://www.dailydot.com/parsec/fandom/dieselpunk-steampunk-beginners-guide/'
Navigate to
http://127.0.0.1:8000
and search "beginners' (see screenshot 1). Because it is in the title, it will show up. The error message will also show up.Search "biopunk" (see screenshot 2). Even though it is in the wget file, it will not show up (see screenshot 3). The error message will show up. I have not done extensive testing on whether different filetype snapshots will get searched or not, but I don't think it picks any of them up if they are not in title.
Screenshots or log output
Screenshot 1:
![image](https://user-images.githubusercontent.com/36479341/107894062-40e03880-6ee3-11eb-9720-2ee36c32cc9a.png)
Screenshot 2:
![image](https://user-images.githubusercontent.com/36479341/107894100-640ae800-6ee3-11eb-8e23-dea3c8034ec8.png)
Screenshot 3:
![image](https://user-images.githubusercontent.com/36479341/107894175-a0d6df00-6ee3-11eb-9bf9-36f49bcde6a0.png)
Screenshot 4:
![image](https://user-images.githubusercontent.com/36479341/107894343-2fe3f700-6ee4-11eb-90ea-29dd05301d99.png)
ArchiveBox version
>docker -version
Docker version 20.10.2, build 2291f61
>docker-compose --version
docker-compose version 1.27.4, build 40524192
The text was updated successfully, but these errors were encountered: