fc2 extractor does not correctly handle videos marked as adult

[ketsuban]

Non-adult videos on fc2, when accessed as adult videos, will automatically redirect to the non-adult page.

[thomas@chansey ~]♣ youtube-dl "http://video.fc2.com/en/a/content/20121103kUan1KHs" --verbose
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['http://video.fc2.com/en/a/content/20121103kUan1KHs', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2014.06.26
[debug] Python version 3.4.1 - Linux-3.15.2-1-ARCH-x86_64-with-arch
[debug] Proxy map: {}
[generic] 20121103kUan1KHs: Requesting header
[redirect] Following redirect to http://video.fc2.com/content/20121103kUan1KHs/

However, as the regular expression checks only for a region code when parsing the URL, it misinterprets the URL for an adult video which also has a region code.

[thomas@chansey ~]♣ youtube-dl "http://video.fc2.com/en/a/content/20130926eZpARwsF" --verbose 
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['http://video.fc2.com/en/a/content/20130926eZpARwsF', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2014.06.26
[debug] Python version 3.4.1 - Linux-3.15.2-1-ARCH-x86_64-with-arch
[debug] Proxy map: {}
[generic] 20130926eZpARwsF: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 20130926eZpARwsF: Downloading webpage
[generic] 20130926eZpARwsF: Extracting information
Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 9, in <module>
    load_entry_point('youtube-dl==2014.06.26', 'console_scripts', 'youtube-dl')()
  File "/usr/lib/python3.4/site-packages/youtube_dl/__init__.py", line 852, in main
    _real_main(argv)
  File "/usr/lib/python3.4/site-packages/youtube_dl/__init__.py", line 842, in _real_main
    retcode = ydl.download(all_urls)
  File "/usr/lib/python3.4/site-packages/youtube_dl/YoutubeDL.py", line 1050, in download
    self.extract_info(url)
  File "/usr/lib/python3.4/site-packages/youtube_dl/YoutubeDL.py", line 527, in extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/usr/lib/python3.4/site-packages/youtube_dl/YoutubeDL.py", line 564, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "/usr/lib/python3.4/site-packages/youtube_dl/YoutubeDL.py", line 830, in process_video_result
    self.process_info(new_info)
  File "/usr/lib/python3.4/site-packages/youtube_dl/YoutubeDL.py", line 1020, in process_info
    success = dl(filename, info_dict)
  File "/usr/lib/python3.4/site-packages/youtube_dl/YoutubeDL.py", line 996, in dl
    return fd.download(name, info)
  File "/usr/lib/python3.4/site-packages/youtube_dl/downloader/common.py", line 291, in download
    return self.real_download(filename, info_dict)
  File "/usr/lib/python3.4/site-packages/youtube_dl/downloader/http.py", line 58, in real_download
    data = self.ydl.urlopen(request)
  File "/usr/lib/python3.4/site-packages/youtube_dl/YoutubeDL.py", line 1225, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.4/urllib/request.py", line 455, in open
    response = self._open(req, data)
  File "/usr/lib/python3.4/urllib/request.py", line 473, in _open
    '_open', req)
  File "/usr/lib/python3.4/urllib/request.py", line 433, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.4/urllib/request.py", line 1215, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib/python3.4/urllib/request.py", line 1189, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "/usr/lib/python3.4/http/client.py", line 1090, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.4/http/client.py", line 1118, in _send_request
    self.putrequest(method, url, **skips)
  File "/usr/lib/python3.4/http/client.py", line 975, in putrequest
    self._output(request.encode('ascii'))
UnicodeEncodeError: 'ascii' codec can't encode character '\u3010' in position 41: ordinal not in range(128)

If I manually remove the URL indicator of an adult video, I get the correct incorrect behaviour as specified in #2912.

[thomas@chansey ~]♣ youtube-dl "http://video.fc2.com/en/content/20130926eZpARwsF" --verbose 
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['http://video.fc2.com/en/content/20130926eZpARwsF', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2014.06.26
[debug] Python version 3.4.1 - Linux-3.15.2-1-ARCH-x86_64-with-arch
[debug] Proxy map: {}
[fc2] 20130926eZpARwsF: Downloading webpage
[fc2] 20130926eZpARwsF: Downloading info page
ERROR: Error code: 403; please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type  youtube-dl -U  to update.
Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/youtube_dl/YoutubeDL.py", line 516, in extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python3.4/site-packages/youtube_dl/extractor/common.py", line 167, in extract
    return self._real_extract(url)
  File "/usr/lib/python3.4/site-packages/youtube_dl/extractor/fc2.py", line 50, in _real_extract
    raise ExtractorError('Error code: %s' % info['err_code'][0])
youtube_dl.utils.ExtractorError: Error code: 403; please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type  youtube-dl -U  to update.

[dstftw]

Works fine with latest version.

[h-collector]

@dstftw no, it still don't work, because of faulty git merge some time ago.
That is, links internally are still rewritten from /a/content/ to /a/a/content/
(check differences between current and forked code of extractor/fc2.py)

Yeah... maybe I should prepare next pull request with fixes, but I didn't have time and will :/