making links clickable

[JacobReynolds]

This will make it so that all of the applicable robots.txt links are clickable. This aids in quick navigation to any of these links. There is a little issue with whitespace, where some site cause an extra newline, but it does not affect usability at all. Github messed up some of the formatting (indentation, etc...), but not much we can do about that.

Thanks!
Jake Reynolds (https://jakereynolds.co)

[peternewnham]

I think this could be simplified using regular expressions, something like this could work:

return text
  .replace(/((Disallow|Allow|Sitemap):\s+)(\/[^\s\*]+)(\n|$)/ig, '$1<a href="' + host + '$3">$3</a>$4')
  .replace(/(\s|^)(((https?:\/\/)?[\w-]+(\.[\w-]+)+\.?(:\d+)?(\/\S*)?))/gi, '$1<a href="$2">$2</a>');

This will also match Sitemap directives and any other full urls

[JacobReynolds]

I agree, that's a really elegant solution, thanks for coming up with that. Quick testing shows that it works great.

[peternewnham]

There's a lot of repeating going on in these files to get the host and then convert html entities and add links. This could all be converted to a common util function and then used for both the robots and humans text display e.g.

RvH.common.Util.parseText(robots, 'robots.txt', function(text) {
  $('#robots').html(text);
});

...

// common/rvh/Util.js

parseText(data, file, callback) {
  if (data === false) {
    callback('<div class="alert alert-danger">' + chrome.i18n.getMessage("fileNotFound", [file]) + '</div>');
  }
  else {
    chrome.tabs.query({
      'active': true,
      'windowId': chrome.windows.WINDOW_ID_CURRENT
    }, function(tabs) {
      var tablink = tabs[0].url.match(/^[\w-]+:\/{2,}\[?([\w\.:-]+)\]?(?::[0-9]*)?/)[0];
      var parsedData = this.addLinks( // <-- rename the parseAsLink method
        this.htmlEntities(data)
      );
      callback('<pre>' + parsedData + '</pre>');
    });
  }
}

[peternewnham]

Hey thanks I like the idea! I also added another comment to reduce the amount of repetition when rendering the contents of each text file.

[JacobReynolds]

Didn't even occur to me to do it for humans.txt, forgot those had links as well.

[peternewnham]

Should be #humans

[JacobReynolds]

It wouldn't be monday if something didn't slip through the cracks, thanks for seeing that.

[JacobReynolds]

There were issues with links like
Disallow: subdomain.domain.com
because they wouldn't have a protocol attached, and would become
https://domain.com/subdomain.domain.com
This seems to fix that issue.