Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unexpected errors in AwsInstance and AzureInstance #902

Closed
shreyamalviya opened this issue Dec 9, 2020 · 3 comments · Fixed by #924
Closed

Unexpected errors in AwsInstance and AzureInstance #902

shreyamalviya opened this issue Dec 9, 2020 · 3 comments · Fixed by #924
Labels
Bug An error, flaw, misbehavior or failure in the Monkey or Monkey Island. Complexity: Low Impact: High

Comments

@shreyamalviya
Copy link
Contributor

I'm getting the following errors during environment collection...

... in AwsInstance:

File "/home/shreya/Desktop/Projects/monkey/monkey/infection_monkey/system_info/collectors/environment_collector.py", line 1, in <module>
    from common.cloud.all_instances import get_all_cloud_instances
  File "/home/shreya/Desktop/Projects/monkey/monkey/common/cloud/all_instances.py", line 8, in <module>
    all_cloud_instances = [AwsInstance(), AzureInstance(), GcpInstance()]
  File "/home/shreya/Desktop/Projects/monkey/monkey/common/cloud/aws/aws_instance.py", line 49, in __init__
    AWS_LATEST_METADATA_URI_PREFIX + 'dynamic/instance-identity/document', timeout=2).read().decode())
  File "/home/shreya/Desktop/Projects/monkey/monkey/common/cloud/aws/aws_instance.py", line 81, in _extract_account_id
    return json.loads(instance_identity_document_response)[ACCOUNT_ID_KEY]
  File "/usr/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

which arises when the account ID is being extracted in _extract_account_id() because

urllib.request.urlopen(
    AWS_LATEST_METADATA_URI_PREFIX + 'dynamic/instance-identity/document', timeout=2).read().decode())

returns:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type" />
<meta content="no-cache" http-equiv="Pragma" />
<title>Waiting...</title>
<script type="text/javascript">
var pageName = '/';
top.location.replace(pageName);
</script>
</head>
<body> </body>
</html>

... and in AzureInstance:

File "/home/shreya/Desktop/Projects/monkey/monkey/infection_monkey/system_info/collectors/environment_collector.py", line 1, in <module>
    from common.cloud.all_instances import get_all_cloud_instances
  File "/home/shreya/Desktop/Projects/monkey/monkey/common/cloud/all_instances.py", line 8, in <module>
    all_cloud_instances = [AwsInstance(), AzureInstance(), GcpInstance()]
  File "/home/shreya/Desktop/Projects/monkey/monkey/common/cloud/azure/azure_instance.py", line 45, in __init__
    self.try_parse_response(response)
  File "/home/shreya/Desktop/Projects/monkey/monkey/common/cloud/azure/azure_instance.py", line 54, in try_parse_response
    response_data = response.json()
  File "/usr/local/lib/python3.7/dist-packages/requests/models.py", line 898, in json
    return complexjson.loads(self.text, **kwargs)
  File "/home/shreya/.local/lib/python3.7/site-packages/simplejson/__init__.py", line 525, in loads
    return _default_decoder.decode(s)
  File "/home/shreya/.local/lib/python3.7/site-packages/simplejson/decoder.py", line 370, in decode
    obj, end = self.raw_decode(s)
  File "/home/shreya/.local/lib/python3.7/site-packages/simplejson/decoder.py", line 400, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

which arises when the response is parsed here because

response = requests.get(AZURE_METADATA_SERVICE_URL,
                        headers={"Metadata": "true"},
                        timeout=SHORT_REQUEST_TIMEOUT)

returns:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type" />
<meta content="no-cache" http-equiv="Pragma" />
<title>Waiting...</title>
<script type="text/javascript">
var pageName = '/';
top.location.replace(pageName);
</script>
</head>
<body> </body>
</html>

I'm not sure why this suddenly came up but we should probably catch the exceptions.
@shreyamalviya shreyamalviya added Bug An error, flaw, misbehavior or failure in the Monkey or Monkey Island. Monkey labels Dec 9, 2020
@shreyamalviya shreyamalviya mentioned this issue Jan 17, 2021
Merged
@VakarisZ
Copy link
Contributor

@shreyamalviya maybe the AWS or Azure API changed? In that case we have bigger problems than decoding errors, we need to validate that these collectors still actually work

@VakarisZ
Copy link
Contributor

Can you elaborate more on how you're getting this? OS, on cloud or on prem?

@shreyamalviya
Copy link
Contributor Author

Probably not because I don't always get these errors. I've only ran into these 2-3 times, not sure why.
Linux, on prem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug An error, flaw, misbehavior or failure in the Monkey or Monkey Island. Complexity: Low Impact: High
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Catch exceptions in AwsInstance and AzureInstance shreyamalviya/monkey
2 participants
@VakarisZ @shreyamalviya