Use whole html to find og tags

[eugene-kirzhanov]

Some web pages contain <meta property="og:*"> tags inside body section not in head.
For example, https://www.youtube.com/watch?v=G2icJffwJLY

For now, library loads only head from HTML in JsoupHtmlFetcher, so it finds nothing in such webpages.

class JsoupHtmlFetcher : HtmlFetcher {
    override fun fetchHead(url: URL): String? {
        val connection = Jsoup.connect(url.toString())
        connection.userAgent("Mozilla")
        connection.timeout(5000)
        return try {
            connection.get()
                    .head()
                    .html()
        } catch (e: HttpStatusException) {
            null
        }
    }
}

I know this is an issue on websites, but I think it will be good to cover this exceptional case by fetching whole html before finding og tags. Or add some boolean parameter to OgMapper.process() method to allow parser use whole html.