Skip to content

Commit

Permalink
fixed: no search results limit in TelevisionTunes scrapers ends in a …
Browse files Browse the repository at this point in the history
…very long scraping duration

e.g. the movie "9" (or "#9" in german) ends with over 3600 search
results they has to be parsed (each theme with a "9" in the title will
be found, e.g. all themes with a year with "9" in title).

Now the limit is set to 20 results.
  • Loading branch information
DanCooper committed Aug 30, 2017
1 parent b29978b commit 7e6d3a8
Showing 1 changed file with 5 additions and 1 deletion.
Expand Up @@ -73,7 +73,7 @@ Namespace TelevisionTunes
Dim SearchURL As String

If Not String.IsNullOrEmpty(originaltitle) Then
SearchTitle = HttpUtility.UrlEncode(originaltitle)
SearchTitle = Uri.EscapeDataString(originaltitle)
SearchURL = String.Concat(BaseURL, SearchTitle)
Else
SearchURL = String.Empty
Expand All @@ -99,6 +99,10 @@ Namespace TelevisionTunes
Dim sResult As MatchCollection = Regex.Matches(Html, sPattern, RegexOptions.Singleline)

For ctr As Integer = 0 To sResult.Count - 1
If _themelist.Count = 20 Then
logger.Warn(String.Format("[TelevisionTunes] [GetThemes] Limit reached (20 themes of {0} has been added)", sResult.Count))
Exit For
End If
tWebURL = String.Concat("http://www.televisiontunes.com/", HttpUtility.HtmlDecode(sResult.Item(ctr).Groups("URL").Value).Trim)
tTitle = sResult.Item(ctr).Groups("TITLE").Value.Trim
tURL = GetDownloadURL(tWebURL)
Expand Down

0 comments on commit 7e6d3a8

Please sign in to comment.