[BUG]: Response could not be completed

[TheSirTigerr]

How are you running AnythingLLM?

Docker (remote machine)

What happened?

I have Anything-LLM on my server in a Docker and ollama i also have on this server.

When i try to import Youtube Transcript i get this error: "Response could not be completed"

How can i solve this? and when i use the agend it shows that it is finished but i dont get any response in the chat

Are there known steps to reproduce?

No response

[shatfield4]

Just to confirm, you are getting this error when trying to use the youtube transcript data connector, correct?

If so, this is most likely something wrong with the youtube link you are providing. If you share the link here with me I can take a look at it for you.

[TheSirTigerr]

This video for example: https://youtu.be/XwL_cRuXM2E?si=UdD6Atjk5HBRjMEg

Also when I want to use the agent in the chat it says that the agent session starter and then after a while thet it ended but I get no response. When I ask the same question again the same thing happens and when I refresh the site the output is there but only from the first question.

[shatfield4]

You are using a shortened version of the youtube video link. It needs to be in this format: https://www.youtube.com/watch?v=XwL_cRuXM2E&ab_channel=bycloud. You can get that link by just visiting the shortened link in your browser and it will expand to the full link.

Also, try using another LLM provider as your agent provider to see if it fixes the issue. It is highly likely the agent is getting stuck because of the quality of model you are using.

[TheSirTigerr]

That doesn't fix it I get the same error with the long URL.

[timothycarambat]

@shatfield4 The Youtube link collector works with both types of YT URLS

anything-llm/collector/utils/extensions/YoutubeTranscript/index.js

Lines 16 to 20 in 9a38b32

	const shortPatternMatch = new UrlPattern(
	"https\\://(www.)youtu.be/(:videoId)"
	).match(url);
	const fullPatternMatch = new UrlPattern(
	"https\\://(www.)youtube.com/watch?v=(:videoId)"

If @shatfield4 can scrape the URL and @TheSirTigerr cannot then something else is going on. Additionally, is the docker container running with --cap-add SYS_ADMIN? Most docker container hosting services (if applicable) do not allow this permission due to bot-scrapers

[TheSirTigerr]

I have a Ubuntu VM on digitalocean. How can I check if I have --cap-add SYS_ADMIN on?

[timothycarambat]

When you started the docker container in the ubuntu VM you would have to run some kind of docker command like docker run -p 3001:3001 mintplexlabs/anythingllm or something along those lines. The command needs to match what we have in our docs to get the full feature set due to some docker engine defaults.

https://github.com/Mintplex-Labs/anything-llm/blob/master/docker/HOW_TO_USE_DOCKER.md#recommend-way-to-run-dockerized-anythingllm

[brianorca]

I'm seeing a similar issue on Windows 10. (No docker containers.)

I can get transcripts for some channels, such as https://www.youtube.com/watch?v=O5GY7_aVBtk

But other channels fail, such as https://www.youtube.com/watch?v=V9KJ7nvhRWk
This results in a zero-byte file in the AppData\Roaming\anythingllm-desktop\storage\documents[channelname] folder.

I did confirm that Youtube shows a transcript for that video. This seems to be across all videos in a channel.

[TheSirTigerr]

So the transcript is fixed I used --cap-add SYS_ADMIN that fixed it. But web searching/agent still doesn't work in chat see picture.

[timothycarambat]

Do you know if the docker container is using a proxy or anything to reach your container? Some providers will do this and it makes using websockets (which is how agents work) unusable until worked around. I think that may be what is happening here?

You can also check to see if in the frontend network requests if the websocket connection is attempting to reach ws or wss when starting an agent chat.

[TheSirTigerr]

It sends the wss i use Nginx Proxy Manager

[timothycarambat]

When you have the Nginx proxy running - do you have an allowance for WSS to connect?

# Default server configuration
# Example config for regular setup + SSL + Websockets.
server {
	listen 80;
	server_name mysite.com;
	return 301 https://mysite.com$request_uri;
}

server {
	listen 443 ssl;
	ssl on;
	server_name mysite.useanything.com;
	ssl_certificate /etc/letsencrypt/live/mysite.com/fullchain.pem;
	ssl_certificate_key /etc/letsencrypt/live/mysite.com/privkey.pem;	

  # Enable websocket connections for agent protocol.
	location ~* ^/api/agent-invocation/(.*) {
		proxy_pass http://localhost:3001;
		proxy_http_version 1.1;
		proxy_set_header Upgrade $http_upgrade;
		proxy_set_header Connection "Upgrade";
	}

	location / {
		proxy_connect_timeout       605;
    proxy_send_timeout          605;
    proxy_read_timeout          605;
    send_timeout                605;
    keepalive_timeout           605;
    proxy_buffering off;
    proxy_cache off;
    proxy_pass         http://ip-of-server:3001$request_uri;
  }
}