problem: parsing xn- domain #66

Closed
wesyoung opened this Issue Feb 10, 2017 · 0 comments

Projects

None yet

1 participant

@wesyoung
Member
csirtg-smrt -d -r /etc/cif/rules/default/alexa.yml
<< snip >>
2017-02-08 00:50:59,136 - DEBUG - csirtg_smrt.smrt[174] - adding: experienceproject.com/alexa.com/None/None
2017-02-08 00:50:59,136 - DEBUG - csirtg_smrt.smrt[174] - adding: skuola.net/alexa.com/None/None
2017-02-08 00:50:59,137 - DEBUG - csirtg_smrt.smrt[174] - adding: synology.me/alexa.com/None/None
Traceback (most recent call last):
  File "/usr/local/bin/csirtg-smrt", line 9, in <module>
    load_entry_point('csirtg-smrt==0.0.0a22', 'console_scripts', 'csirtg-smrt')()
  File "/usr/local/lib/python2.7/dist-packages/csirtg_smrt/smrt.py", line 359, in main
    'goback': goback
  File "/usr/local/lib/python2.7/dist-packages/csirtg_smrt/smrt.py", line 252, in _run_smrt
    for i in s.process(r, f, limit=args.limit, data=data, filters=filters):
  File "/usr/local/lib/python2.7/dist-packages/csirtg_smrt/smrt.py", line 212, in process
    for indicator_batch in feed_indicators_batches:
  File "/usr/local/lib/python2.7/dist-packages/csirtg_smrt/utils/__init__.py", line 92, in chunk
    for x in it:
  File "/usr/local/lib/python2.7/dist-packages/csirtg_smrt/smrt.py", line 208, in <genexpr>
    feed_indicators = (i for i in feed_indicators if not self.is_archived_with_log(i))
  File "/usr/local/lib/python2.7/dist-packages/csirtg_smrt/smrt.py", line 202, in <genexpr>
    feed_indicators = (self.clean_indicator(i, rule) for i in feed_indicators)
  File "/usr/local/lib/python2.7/dist-packages/csirtg_smrt/smrt.py", line 153, in clean_indicator
    i = normalize_itype(i)
  File "/usr/local/lib/python2.7/dist-packages/csirtg_indicator/utils/__init__.py", line 174, in normalize_itype
    i = _normalize_url(i)
  File "/usr/local/lib/python2.7/dist-packages/csirtg_indicator/utils/__init__.py", line 204, in _normalize_url
    if resolve_itype(i['indicator'], test_broken=True) == 'broken_url':
  File "/usr/local/lib/python2.7/dist-packages/csirtg_indicator/utils/__init__.py", line 154, in resolve_itype
    raise InvalidIndicator(error)
csirtg_indicator.exceptions.InvalidIndicator: unknown itype for "hdxturkceizle.xn--6frz82g"

@wesyoung wesyoung added the bug label Feb 10, 2017
@wesyoung wesyoung added a commit that closed this issue Feb 10, 2017
@wesyoung wesyoung fixes #66 #65 #64 ca8af43
@wesyoung wesyoung closed this in ca8af43 Feb 10, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment