Risk of crawler ending before all URLs crawled

[arnaudbreton]

Hi,

Thanks for posting your solution, it helped me a lot to figure out how-to solve the exercise with Go's channels.

I'm wondering if there is a risk of the crawler ending before all URLs have been crawled?
At https://github.com/absoludity/go-tour-exercises/blob/master/70-web-crawler.go#L68, you check if all URLs are crawled each time a crawl ends but I think it can happen that all URLs can be crawled but not all discovered.

What do you think ?

[absoludity]

Hi Arnaud. Hrm - from a quick look, I think you might be right... it's been a while since I've looked at this code, but it looks as though if we caused the Crawl() fn to sleep initially, so that on line:

https://github.com/absoludity/go-tour-exercises/blob/master/70-web-crawler.go#L33

all the go-routines adding new urls to be crawled would stall then the crawlComplete message would be processed first. I'll have to have a closer look in more detail. Let me know if you find anything first and thanks for the feedback!

[arnaudbreton]

Hey,

I came with my own solution, mostly inspired by yours, based on WaitGroup. It should solve this particular issue. You can find it here: https://github.com/arnaudbreton/gotour-exercises