20190102T080502Z
20190102/eu-west-2/s3/aws4_request
497a3d160b4efa5af5ab0bc244ee0ba452127f8df6c0e595da881cdca921f021
2019-01-02 08:05:02 [botocore.auth] DEBUG: Signature:
df236b477c7e9574bf459cd6c3c39638d45ac231bbfae24afcf67d763ab09a1c
2019-01-02 08:05:02 [botocore.endpoint] DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=HEAD, url=https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com/mahindra.com/MM-Chairman-Presentation-at-AGM-2018.pdf, headers={'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAJNGI6DYOK7QJUQBQ/20190102/eu-west-2/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=df236b477c7e9574bf459cd6c3c39638d45ac231bbfae24afcf67d763ab09a1c', 'User-Agent': b'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Content-Length': '0', 'X-Amz-Date': b'20190102T080502Z', 'X-Amz-Content-SHA256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'}>
2019-01-02 08:05:02 [urllib3.util.retry] DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-01-02 08:05:02 [urllib3.connectionpool] DEBUG: https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com:443 "HEAD /mahindra.com/MM-Chairman-Presentation-at-AGM-2018.pdf HTTP/1.1" 404 0
2019-01-02 08:05:02 [botocore.parsers] DEBUG: Response headers: {'Content-Type': 'application/xml', 'Date': 'Wed, 02 Jan 2019 08:05:01 GMT', 'x-amz-id-2': 'BE8i9UQoL6jlzA20jCz6SxshuMArXa0t9LRlPEoPdtAaOwQy1M63HE9tgdGIXtpTzi/L5AbAp/A=', 'Server': 'AmazonS3', 'x-amz-request-id': 'D623412DB54480C2', 'Transfer-Encoding': 'chunked'}
2019-01-02 08:05:02 [botocore.parsers] DEBUG: Response body:
b''
2019-01-02 08:05:02 [botocore.hooks] DEBUG: Event needs-retry.s3.HeadObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f1fe54adef0>
2019-01-02 08:05:02 [botocore.retryhandler] DEBUG: No retry needed.
2019-01-02 08:05:02 [botocore.hooks] DEBUG: Event needs-retry.s3.HeadObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:02 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function sse_md5 at 0x7f1fe55d18c8>
2019-01-02 08:05:02 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function validate_bucket_name at 0x7f1fe55d1840>
2019-01-02 08:05:02 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:02 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function generate_idempotent_uuid at 0x7f1fe55d1488>
2019-01-02 08:05:02 [botocore.hooks] DEBUG: Event before-call.s3.HeadObject: calling handler <function add_expect_header at 0x7f1fe55d1d08>
2019-01-02 08:05:02 [botocore.hooks] DEBUG: Event before-call.s3.HeadObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:02 [botocore.utils] DEBUG: Updating URI from https://s3.amazonaws.com/scrapy-data-lseg/mahindra.com/MM-Consolidated-Results-for-30th-June-2018.pdf to https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/MM-Consolidated-Results-for-30th-June-2018.pdf
2019-01-02 08:05:02 [botocore.endpoint] DEBUG: Making request for OperationModel(name=HeadObject) with params: {'url_path': '/scrapy-data-lseg/mahindra.com/MM-Consolidated-Results-for-30th-June-2018.pdf', 'query_string': {}, 'url': 'https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/MM-Consolidated-Results-for-30th-June-2018.pdf', 'headers': {'User-Agent': 'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws'}, 'method': 'HEAD', 'body': b'', 'context': {'client_region': 'us-east-1', 'has_streaming_input': False, 'client_config': <botocore.config.Config object at 0x7f1fe54ad518>, 'auth_type': None, 'signing': {'region': 'eu-west-2', 'endpoint': 'https://s3.eu-west-2.amazonaws.com', 'bucket': 'scrapy-data-lseg'}}}
2019-01-02 08:05:02 [botocore.hooks] DEBUG: Event request-created.s3.HeadObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f1fe54ad4a8>>
2019-01-02 08:05:02 [botocore.hooks] DEBUG: Event choose-signer.s3.HeadObject: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x7f1fe7bdb278>>
2019-01-02 08:05:02 [botocore.hooks] DEBUG: Event choose-signer.s3.HeadObject: calling handler <function set_operation_specific_signer at 0x7f1fe55d1378>
2019-01-02 08:05:02 [botocore.hooks] DEBUG: Event before-sign.s3.HeadObject: calling handler <function fix_s3_host at 0x7f1fe571a730>
2019-01-02 08:05:02 [botocore.utils] DEBUG: Checking for DNS compatible bucket for: https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/MM-Consolidated-Results-for-30th-June-2018.pdf
2019-01-02 08:05:02 [botocore.utils] DEBUG: URI updated to: https://s3-path/mahindra.com/MM-Consolidated-Results-for-30th-June-2018.pdf
2019-01-02 08:05:02 [botocore.auth] DEBUG: Calculating signature using v4 auth.
2019-01-02 08:05:02 [botocore.auth] DEBUG: CanonicalRequest:
HEAD
/mahindra.com/MM-Consolidated-Results-for-30th-June-2018.pdf

host:scrapy-data-lseg.s3.eu-west-2.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20190102T080502Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2019-01-02 08:05:02 [botocore.auth] DEBUG: StringToSign:
AWS4-HMAC-SHA256
20190102T080502Z
20190102/eu-west-2/s3/aws4_request
1e9474abcd52dc5903c5e47c6e43e1f91e41fd0304f8623d496d9b8a7e9ea12c
2019-01-02 08:05:02 [botocore.auth] DEBUG: Signature:
a5b3c0b9e737eeada8cf2a1d30401eee48f3379d7bab48eab4a3f409c089d70a
2019-01-02 08:05:02 [botocore.endpoint] DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=HEAD, url=https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com/mahindra.com/MM-Consolidated-Results-for-30th-June-2018.pdf, headers={'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAJNGI6DYOK7QJUQBQ/20190102/eu-west-2/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=a5b3c0b9e737eeada8cf2a1d30401eee48f3379d7bab48eab4a3f409c089d70a', 'User-Agent': b'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Content-Length': '0', 'X-Amz-Date': b'20190102T080502Z', 'X-Amz-Content-SHA256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'}>
2019-01-02 08:05:02 [urllib3.util.retry] DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-01-02 08:05:02 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/resources/investor-reports/FY19/Events%20Update/FES-IR-Day-June-26-final-presentation.pdf> (referer: https://www.mahindra.com/investors/reports-and-presentations)
2019-01-02 08:05:02 [scrapy.extensions.logstats] INFO: Crawled 417 pages (at 104 pages/min), scraped 37 items (at 12 items/min)
2019-01-02 08:05:02 [urllib3.connectionpool] DEBUG: https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com:443 "HEAD /mahindra.com/MM-Consolidated-Results-for-30th-June-2018.pdf HTTP/1.1" 404 0
2019-01-02 08:05:02 [botocore.parsers] DEBUG: Response headers: {'Content-Type': 'application/xml', 'Date': 'Wed, 02 Jan 2019 08:05:01 GMT', 'x-amz-id-2': 'JDJZSbcN/CkE8Mac5OK0yjOSMjtG6NLGUb8rVC027S3I7c01mmHy6y0cePvLDrZgvcpH02qHDEA=', 'Server': 'AmazonS3', 'x-amz-request-id': '67C6015D1342BDA2', 'Transfer-Encoding': 'chunked'}
2019-01-02 08:05:02 [botocore.parsers] DEBUG: Response body:
b''
2019-01-02 08:05:02 [botocore.hooks] DEBUG: Event needs-retry.s3.HeadObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f1fe54adef0>
2019-01-02 08:05:02 [botocore.retryhandler] DEBUG: No retry needed.
2019-01-02 08:05:02 [botocore.hooks] DEBUG: Event needs-retry.s3.HeadObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:02 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/resources/investor-reports/FY19/Earnings%20Update/TRANSCRIPT-MM-Q1FY19-Earnings-Call-7th-Aug-2018-FINAL.pdf> (referer: https://www.mahindra.com/investors/reports-and-presentations)
2019-01-02 08:05:03 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function sse_md5 at 0x7f1fe55d18c8>
2019-01-02 08:05:03 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function validate_bucket_name at 0x7f1fe55d1840>
2019-01-02 08:05:03 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:03 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function generate_idempotent_uuid at 0x7f1fe55d1488>
2019-01-02 08:05:03 [botocore.hooks] DEBUG: Event before-call.s3.HeadObject: calling handler <function add_expect_header at 0x7f1fe55d1d08>
2019-01-02 08:05:03 [botocore.hooks] DEBUG: Event before-call.s3.HeadObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:03 [botocore.utils] DEBUG: Updating URI from https://s3.amazonaws.com/scrapy-data-lseg/mahindra.com/TRANSCRIPT-MM-Q1FY19-Earnings-Call-7th-Aug-2018-FINAL.pdf to https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/TRANSCRIPT-MM-Q1FY19-Earnings-Call-7th-Aug-2018-FINAL.pdf
2019-01-02 08:05:03 [botocore.endpoint] DEBUG: Making request for OperationModel(name=HeadObject) with params: {'url_path': '/scrapy-data-lseg/mahindra.com/TRANSCRIPT-MM-Q1FY19-Earnings-Call-7th-Aug-2018-FINAL.pdf', 'query_string': {}, 'url': 'https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/TRANSCRIPT-MM-Q1FY19-Earnings-Call-7th-Aug-2018-FINAL.pdf', 'headers': {'User-Agent': 'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws'}, 'method': 'HEAD', 'body': b'', 'context': {'client_region': 'us-east-1', 'has_streaming_input': False, 'client_config': <botocore.config.Config object at 0x7f1fe54ad518>, 'auth_type': None, 'signing': {'region': 'eu-west-2', 'endpoint': 'https://s3.eu-west-2.amazonaws.com', 'bucket': 'scrapy-data-lseg'}}}
2019-01-02 08:05:03 [botocore.hooks] DEBUG: Event request-created.s3.HeadObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f1fe54ad4a8>>
2019-01-02 08:05:03 [botocore.hooks] DEBUG: Event choose-signer.s3.HeadObject: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x7f1fe7bdb278>>
2019-01-02 08:05:03 [botocore.hooks] DEBUG: Event choose-signer.s3.HeadObject: calling handler <function set_operation_specific_signer at 0x7f1fe55d1378>
2019-01-02 08:05:03 [botocore.hooks] DEBUG: Event before-sign.s3.HeadObject: calling handler <function fix_s3_host at 0x7f1fe571a730>
2019-01-02 08:05:03 [botocore.utils] DEBUG: Checking for DNS compatible bucket for: https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/TRANSCRIPT-MM-Q1FY19-Earnings-Call-7th-Aug-2018-FINAL.pdf
2019-01-02 08:05:03 [botocore.utils] DEBUG: URI updated to: https://s3-path/mahindra.com/TRANSCRIPT-MM-Q1FY19-Earnings-Call-7th-Aug-2018-FINAL.pdf
2019-01-02 08:05:03 [botocore.auth] DEBUG: Calculating signature using v4 auth.
2019-01-02 08:05:03 [botocore.auth] DEBUG: CanonicalRequest:
HEAD
/mahindra.com/TRANSCRIPT-MM-Q1FY19-Earnings-Call-7th-Aug-2018-FINAL.pdf

host:scrapy-data-lseg.s3.eu-west-2.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20190102T080503Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2019-01-02 08:05:03 [botocore.auth] DEBUG: StringToSign:
AWS4-HMAC-SHA256
20190102T080503Z
20190102/eu-west-2/s3/aws4_request
249d440fcd667329a073b27bed78f0cd9ff3ff6c682e26894119302f6127859c
2019-01-02 08:05:03 [botocore.auth] DEBUG: Signature:
911122c8eae425c8e9fd25aaf34787b9300f89bacf2513a03fbbd4c15d53e8e8
2019-01-02 08:05:03 [botocore.endpoint] DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=HEAD, url=https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com/mahindra.com/TRANSCRIPT-MM-Q1FY19-Earnings-Call-7th-Aug-2018-FINAL.pdf, headers={'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAJNGI6DYOK7QJUQBQ/20190102/eu-west-2/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=911122c8eae425c8e9fd25aaf34787b9300f89bacf2513a03fbbd4c15d53e8e8', 'User-Agent': b'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Content-Length': '0', 'X-Amz-Date': b'20190102T080503Z', 'X-Amz-Content-SHA256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'}>
2019-01-02 08:05:03 [urllib3.util.retry] DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-01-02 08:05:03 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/resources/investor-reports/FY19/Earnings%20Update/MM-Q2-FY19-Results-pack.pdf> (referer: https://www.mahindra.com/investors/reports-and-presentations)
2019-01-02 08:05:03 [urllib3.connectionpool] DEBUG: https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com:443 "HEAD /mahindra.com/TRANSCRIPT-MM-Q1FY19-Earnings-Call-7th-Aug-2018-FINAL.pdf HTTP/1.1" 404 0
2019-01-02 08:05:03 [botocore.parsers] DEBUG: Response headers: {'Content-Type': 'application/xml', 'Date': 'Wed, 02 Jan 2019 08:05:03 GMT', 'x-amz-id-2': 'Ha6eXw2TXDLLbvsKSUltgyaTA7pqwpiUOW5LbIJAA0IT9pK+zn7Cdeajr5dnoL6ZKyxmgA42DGY=', 'Server': 'AmazonS3', 'x-amz-request-id': '7589D08D8B23B333', 'Transfer-Encoding': 'chunked'}
2019-01-02 08:05:03 [botocore.parsers] DEBUG: Response body:
b''
2019-01-02 08:05:03 [botocore.hooks] DEBUG: Event needs-retry.s3.HeadObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f1fe54adef0>
2019-01-02 08:05:03 [botocore.retryhandler] DEBUG: No retry needed.
2019-01-02 08:05:03 [botocore.hooks] DEBUG: Event needs-retry.s3.HeadObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:04 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function sse_md5 at 0x7f1fe55d18c8>
2019-01-02 08:05:04 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function validate_bucket_name at 0x7f1fe55d1840>
2019-01-02 08:05:04 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:04 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function generate_idempotent_uuid at 0x7f1fe55d1488>
2019-01-02 08:05:04 [botocore.hooks] DEBUG: Event before-call.s3.HeadObject: calling handler <function add_expect_header at 0x7f1fe55d1d08>
2019-01-02 08:05:04 [botocore.hooks] DEBUG: Event before-call.s3.HeadObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:04 [botocore.utils] DEBUG: Updating URI from https://s3.amazonaws.com/scrapy-data-lseg/mahindra.com/MM-Q2-FY19-Results-pack.pdf to https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/MM-Q2-FY19-Results-pack.pdf
2019-01-02 08:05:04 [botocore.endpoint] DEBUG: Making request for OperationModel(name=HeadObject) with params: {'url_path': '/scrapy-data-lseg/mahindra.com/MM-Q2-FY19-Results-pack.pdf', 'query_string': {}, 'url': 'https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/MM-Q2-FY19-Results-pack.pdf', 'headers': {'User-Agent': 'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws'}, 'method': 'HEAD', 'body': b'', 'context': {'client_region': 'us-east-1', 'has_streaming_input': False, 'client_config': <botocore.config.Config object at 0x7f1fe54ad518>, 'auth_type': None, 'signing': {'region': 'eu-west-2', 'endpoint': 'https://s3.eu-west-2.amazonaws.com', 'bucket': 'scrapy-data-lseg'}}}
2019-01-02 08:05:04 [botocore.hooks] DEBUG: Event request-created.s3.HeadObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f1fe54ad4a8>>
2019-01-02 08:05:04 [botocore.hooks] DEBUG: Event choose-signer.s3.HeadObject: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x7f1fe7bdb278>>
2019-01-02 08:05:04 [botocore.hooks] DEBUG: Event choose-signer.s3.HeadObject: calling handler <function set_operation_specific_signer at 0x7f1fe55d1378>
2019-01-02 08:05:04 [botocore.hooks] DEBUG: Event before-sign.s3.HeadObject: calling handler <function fix_s3_host at 0x7f1fe571a730>
2019-01-02 08:05:04 [botocore.utils] DEBUG: Checking for DNS compatible bucket for: https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/MM-Q2-FY19-Results-pack.pdf
2019-01-02 08:05:04 [botocore.utils] DEBUG: URI updated to: https://s3-path/mahindra.com/MM-Q2-FY19-Results-pack.pdf
2019-01-02 08:05:04 [botocore.auth] DEBUG: Calculating signature using v4 auth.
2019-01-02 08:05:04 [botocore.auth] DEBUG: CanonicalRequest:
HEAD
/mahindra.com/MM-Q2-FY19-Results-pack.pdf

host:scrapy-data-lseg.s3.eu-west-2.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20190102T080504Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2019-01-02 08:05:04 [botocore.auth] DEBUG: StringToSign:
AWS4-HMAC-SHA256
20190102T080504Z
20190102/eu-west-2/s3/aws4_request
a39dd17a9cd2481c034242e7b740e627511359f68deaf0d63568693c45ba8ebe
2019-01-02 08:05:04 [botocore.auth] DEBUG: Signature:
a4243299e583d99896f84a0104ecca1e1beec5f8729eeec01a51de2011b2f90b
2019-01-02 08:05:04 [botocore.endpoint] DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=HEAD, url=https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com/mahindra.com/MM-Q2-FY19-Results-pack.pdf, headers={'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAJNGI6DYOK7QJUQBQ/20190102/eu-west-2/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=a4243299e583d99896f84a0104ecca1e1beec5f8729eeec01a51de2011b2f90b', 'User-Agent': b'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Content-Length': '0', 'X-Amz-Date': b'20190102T080504Z', 'X-Amz-Content-SHA256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'}>
2019-01-02 08:05:04 [urllib3.util.retry] DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-01-02 08:05:04 [urllib3.connectionpool] DEBUG: https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com:443 "HEAD /mahindra.com/MM-Q2-FY19-Results-pack.pdf HTTP/1.1" 404 0
2019-01-02 08:05:04 [botocore.parsers] DEBUG: Response headers: {'Content-Type': 'application/xml', 'Date': 'Wed, 02 Jan 2019 08:05:03 GMT', 'x-amz-id-2': 'clZaY2/GNSXnp/JxQeFuMmCiD5MixMqhgivRqA7/JFp4ncSJNKPIH+iUnao5YLcF0moVJ453KzQ=', 'Server': 'AmazonS3', 'x-amz-request-id': '50891F131D4669AA', 'Transfer-Encoding': 'chunked'}
2019-01-02 08:05:04 [botocore.parsers] DEBUG: Response body:
b''
2019-01-02 08:05:04 [botocore.hooks] DEBUG: Event needs-retry.s3.HeadObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f1fe54adef0>
2019-01-02 08:05:04 [botocore.retryhandler] DEBUG: No retry needed.
2019-01-02 08:05:04 [botocore.hooks] DEBUG: Event needs-retry.s3.HeadObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:04 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/resources/investor-reports/FY19/Earnings%20Update/investor-presentation-post-Q2FY19-results.pdf> (referer: https://www.mahindra.com/investors/reports-and-presentations)
2019-01-02 08:05:05 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/resources/investor-reports/FY19/Earnings%20Update/AUDIO-MM-Q2FY19-Earnings-Call-14-Nov-2018.mp3> (referer: https://www.mahindra.com/investors/reports-and-presentations)
2019-01-02 08:05:05 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/resources/investor-reports/FY19/Earnings%20Update/TRANSCRIPT-M&M-Q2FY19-Earnings-Con-call-14th-Nov-2018-FINAL.pdf> (referer: https://www.mahindra.com/investors/reports-and-presentations)
2019-01-02 08:05:05 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function sse_md5 at 0x7f1fe55d18c8>
2019-01-02 08:05:05 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function validate_bucket_name at 0x7f1fe55d1840>
2019-01-02 08:05:05 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:05 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function generate_idempotent_uuid at 0x7f1fe55d1488>
2019-01-02 08:05:05 [botocore.hooks] DEBUG: Event before-call.s3.HeadObject: calling handler <function add_expect_header at 0x7f1fe55d1d08>
2019-01-02 08:05:05 [botocore.hooks] DEBUG: Event before-call.s3.HeadObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:05 [botocore.utils] DEBUG: Updating URI from https://s3.amazonaws.com/scrapy-data-lseg/mahindra.com/TRANSCRIPT-M%26M-Q2FY19-Earnings-Con-call-14th-Nov-2018-FINAL.pdf to https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/TRANSCRIPT-M%26M-Q2FY19-Earnings-Con-call-14th-Nov-2018-FINAL.pdf
2019-01-02 08:05:05 [botocore.endpoint] DEBUG: Making request for OperationModel(name=HeadObject) with params: {'url_path': '/scrapy-data-lseg/mahindra.com/TRANSCRIPT-M%26M-Q2FY19-Earnings-Con-call-14th-Nov-2018-FINAL.pdf', 'query_string': {}, 'url': 'https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/TRANSCRIPT-M%26M-Q2FY19-Earnings-Con-call-14th-Nov-2018-FINAL.pdf', 'headers': {'User-Agent': 'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws'}, 'method': 'HEAD', 'body': b'', 'context': {'client_region': 'us-east-1', 'has_streaming_input': False, 'client_config': <botocore.config.Config object at 0x7f1fe54ad518>, 'auth_type': None, 'signing': {'region': 'eu-west-2', 'endpoint': 'https://s3.eu-west-2.amazonaws.com', 'bucket': 'scrapy-data-lseg'}}}
2019-01-02 08:05:05 [botocore.hooks] DEBUG: Event request-created.s3.HeadObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f1fe54ad4a8>>
2019-01-02 08:05:05 [botocore.hooks] DEBUG: Event choose-signer.s3.HeadObject: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x7f1fe7bdb278>>
2019-01-02 08:05:05 [botocore.hooks] DEBUG: Event choose-signer.s3.HeadObject: calling handler <function set_operation_specific_signer at 0x7f1fe55d1378>
2019-01-02 08:05:05 [botocore.hooks] DEBUG: Event before-sign.s3.HeadObject: calling handler <function fix_s3_host at 0x7f1fe571a730>
2019-01-02 08:05:05 [botocore.utils] DEBUG: Checking for DNS compatible bucket for: https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/TRANSCRIPT-M%26M-Q2FY19-Earnings-Con-call-14th-Nov-2018-FINAL.pdf
2019-01-02 08:05:05 [botocore.utils] DEBUG: URI updated to: https://s3-path/mahindra.com/TRANSCRIPT-M%26M-Q2FY19-Earnings-Con-call-14th-Nov-2018-FINAL.pdf
2019-01-02 08:05:05 [botocore.auth] DEBUG: Calculating signature using v4 auth.
2019-01-02 08:05:05 [botocore.auth] DEBUG: CanonicalRequest:
HEAD
/mahindra.com/TRANSCRIPT-M%26M-Q2FY19-Earnings-Con-call-14th-Nov-2018-FINAL.pdf

host:scrapy-data-lseg.s3.eu-west-2.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20190102T080505Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2019-01-02 08:05:05 [botocore.auth] DEBUG: StringToSign:
AWS4-HMAC-SHA256
20190102T080505Z
20190102/eu-west-2/s3/aws4_request
8d162a49914a5769d0c53357c30f628b6d92865b3a8f54beb7f179deb7a0b469
2019-01-02 08:05:05 [botocore.auth] DEBUG: Signature:
8669358f1e96b85e2e9d0401a60c4bb45e015fe9cc8c42d58e6e5ce819b69fbe
2019-01-02 08:05:05 [botocore.endpoint] DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=HEAD, url=https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com/mahindra.com/TRANSCRIPT-M%26M-Q2FY19-Earnings-Con-call-14th-Nov-2018-FINAL.pdf, headers={'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAJNGI6DYOK7QJUQBQ/20190102/eu-west-2/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=8669358f1e96b85e2e9d0401a60c4bb45e015fe9cc8c42d58e6e5ce819b69fbe', 'User-Agent': b'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Content-Length': '0', 'X-Amz-Date': b'20190102T080505Z', 'X-Amz-Content-SHA256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'}>
2019-01-02 08:05:05 [urllib3.util.retry] DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-01-02 08:05:05 [urllib3.connectionpool] DEBUG: https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com:443 "HEAD /mahindra.com/TRANSCRIPT-M%26M-Q2FY19-Earnings-Con-call-14th-Nov-2018-FINAL.pdf HTTP/1.1" 404 0
2019-01-02 08:05:05 [botocore.parsers] DEBUG: Response headers: {'Content-Type': 'application/xml', 'Date': 'Wed, 02 Jan 2019 08:05:05 GMT', 'x-amz-id-2': 'fHmGLiuW/Nf5C23Hr/r1oztjMJpt1tbGN1zOfsifeWZy86ZHRlWnMAfeQacjizwrL6gnoKL7jpo=', 'Server': 'AmazonS3', 'x-amz-request-id': '99CC7C439591F078', 'Transfer-Encoding': 'chunked'}
2019-01-02 08:05:05 [botocore.parsers] DEBUG: Response body:
b''
2019-01-02 08:05:05 [botocore.hooks] DEBUG: Event needs-retry.s3.HeadObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f1fe54adef0>
2019-01-02 08:05:05 [botocore.retryhandler] DEBUG: No retry needed.
2019-01-02 08:05:05 [botocore.hooks] DEBUG: Event needs-retry.s3.HeadObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:05 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/resources/investor-reports/FY19/Announcements/REPLAY-Webcast-link-M-and-M-Annual-Analyst-Meet-May-29-2018.pdf> (referer: https://www.mahindra.com/investors/reports-and-presentations)
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function sse_md5 at 0x7f1fe55d18c8>
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function validate_bucket_name at 0x7f1fe55d1840>
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function generate_idempotent_uuid at 0x7f1fe55d1488>
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event before-call.s3.HeadObject: calling handler <function add_expect_header at 0x7f1fe55d1d08>
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event before-call.s3.HeadObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:06 [botocore.utils] DEBUG: Updating URI from https://s3.amazonaws.com/scrapy-data-lseg/mahindra.com/REPLAY-Webcast-link-M-and-M-Annual-Analyst-Meet-May-29-2018.pdf to https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/REPLAY-Webcast-link-M-and-M-Annual-Analyst-Meet-May-29-2018.pdf
2019-01-02 08:05:06 [botocore.endpoint] DEBUG: Making request for OperationModel(name=HeadObject) with params: {'url_path': '/scrapy-data-lseg/mahindra.com/REPLAY-Webcast-link-M-and-M-Annual-Analyst-Meet-May-29-2018.pdf', 'query_string': {}, 'url': 'https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/REPLAY-Webcast-link-M-and-M-Annual-Analyst-Meet-May-29-2018.pdf', 'headers': {'User-Agent': 'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws'}, 'method': 'HEAD', 'body': b'', 'context': {'client_region': 'us-east-1', 'has_streaming_input': False, 'client_config': <botocore.config.Config object at 0x7f1fe54ad518>, 'auth_type': None, 'signing': {'region': 'eu-west-2', 'endpoint': 'https://s3.eu-west-2.amazonaws.com', 'bucket': 'scrapy-data-lseg'}}}
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event request-created.s3.HeadObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f1fe54ad4a8>>
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event choose-signer.s3.HeadObject: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x7f1fe7bdb278>>
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event choose-signer.s3.HeadObject: calling handler <function set_operation_specific_signer at 0x7f1fe55d1378>
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event before-sign.s3.HeadObject: calling handler <function fix_s3_host at 0x7f1fe571a730>
2019-01-02 08:05:06 [botocore.utils] DEBUG: Checking for DNS compatible bucket for: https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/REPLAY-Webcast-link-M-and-M-Annual-Analyst-Meet-May-29-2018.pdf
2019-01-02 08:05:06 [botocore.utils] DEBUG: URI updated to: https://s3-path/mahindra.com/REPLAY-Webcast-link-M-and-M-Annual-Analyst-Meet-May-29-2018.pdf
2019-01-02 08:05:06 [botocore.auth] DEBUG: Calculating signature using v4 auth.
2019-01-02 08:05:06 [botocore.auth] DEBUG: CanonicalRequest:
HEAD
/mahindra.com/REPLAY-Webcast-link-M-and-M-Annual-Analyst-Meet-May-29-2018.pdf

host:scrapy-data-lseg.s3.eu-west-2.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20190102T080506Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2019-01-02 08:05:06 [botocore.auth] DEBUG: StringToSign:
AWS4-HMAC-SHA256
20190102T080506Z
20190102/eu-west-2/s3/aws4_request
e2218b05397500f22f9ff2fa3f389cb7fd1299f6cd3ee01f3ef8a30a150de66f
2019-01-02 08:05:06 [botocore.auth] DEBUG: Signature:
d2179fbaae4b1eef91ae826cc590e6476ce55505e8a6813c0ab45a7cfe6a41cf
2019-01-02 08:05:06 [botocore.endpoint] DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=HEAD, url=https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com/mahindra.com/REPLAY-Webcast-link-M-and-M-Annual-Analyst-Meet-May-29-2018.pdf, headers={'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAJNGI6DYOK7QJUQBQ/20190102/eu-west-2/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=d2179fbaae4b1eef91ae826cc590e6476ce55505e8a6813c0ab45a7cfe6a41cf', 'User-Agent': b'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Content-Length': '0', 'X-Amz-Date': b'20190102T080506Z', 'X-Amz-Content-SHA256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'}>
2019-01-02 08:05:06 [urllib3.util.retry] DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-01-02 08:05:06 [urllib3.connectionpool] DEBUG: https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com:443 "HEAD /mahindra.com/REPLAY-Webcast-link-M-and-M-Annual-Analyst-Meet-May-29-2018.pdf HTTP/1.1" 404 0
2019-01-02 08:05:06 [botocore.parsers] DEBUG: Response headers: {'Content-Type': 'application/xml', 'Date': 'Wed, 02 Jan 2019 08:05:05 GMT', 'x-amz-id-2': '0E2pjVUMNqZh2kXjRcjUDtLuEWaul1RcNarqUDgAPJM0EXXYPmjVfp+h3v5njX3Cw6qN4zqJakw=', 'Server': 'AmazonS3', 'x-amz-request-id': '23BA6243982962F9', 'Transfer-Encoding': 'chunked'}
2019-01-02 08:05:06 [botocore.parsers] DEBUG: Response body:
b''
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event needs-retry.s3.HeadObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f1fe54adef0>
2019-01-02 08:05:06 [botocore.retryhandler] DEBUG: No retry needed.
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event needs-retry.s3.HeadObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:06 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/resources/investor-reports/FY19/Announcements/1QFY19-Silent-Period-Notice.pdf> (referer: https://www.mahindra.com/investors/reports-and-presentations)
2019-01-02 08:05:06 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/resources/investor-reports/FY19/Announcements/MM-Q1FY19-Earnings-Concall-Invite-for-Tuesday-7th-Aug-2018.pdf> (referer: https://www.mahindra.com/investors/reports-and-presentations)
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function sse_md5 at 0x7f1fe55d18c8>
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function validate_bucket_name at 0x7f1fe55d1840>
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function generate_idempotent_uuid at 0x7f1fe55d1488>
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event before-call.s3.HeadObject: calling handler <function add_expect_header at 0x7f1fe55d1d08>
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event before-call.s3.HeadObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:06 [botocore.utils] DEBUG: Updating URI from https://s3.amazonaws.com/scrapy-data-lseg/mahindra.com/MM-Q1FY19-Earnings-Concall-Invite-for-Tuesday-7th-Aug-2018.pdf to https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/MM-Q1FY19-Earnings-Concall-Invite-for-Tuesday-7th-Aug-2018.pdf
2019-01-02 08:05:06 [botocore.endpoint] DEBUG: Making request for OperationModel(name=HeadObject) with params: {'url_path': '/scrapy-data-lseg/mahindra.com/MM-Q1FY19-Earnings-Concall-Invite-for-Tuesday-7th-Aug-2018.pdf', 'query_string': {}, 'url': 'https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/MM-Q1FY19-Earnings-Concall-Invite-for-Tuesday-7th-Aug-2018.pdf', 'headers': {'User-Agent': 'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws'}, 'method': 'HEAD', 'body': b'', 'context': {'client_region': 'us-east-1', 'has_streaming_input': False, 'client_config': <botocore.config.Config object at 0x7f1fe54ad518>, 'auth_type': None, 'signing': {'region': 'eu-west-2', 'endpoint': 'https://s3.eu-west-2.amazonaws.com', 'bucket': 'scrapy-data-lseg'}}}
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event request-created.s3.HeadObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f1fe54ad4a8>>
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event choose-signer.s3.HeadObject: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x7f1fe7bdb278>>
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event choose-signer.s3.HeadObject: calling handler <function set_operation_specific_signer at 0x7f1fe55d1378>
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event before-sign.s3.HeadObject: calling handler <function fix_s3_host at 0x7f1fe571a730>
2019-01-02 08:05:06 [botocore.utils] DEBUG: Checking for DNS compatible bucket for: https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/MM-Q1FY19-Earnings-Concall-Invite-for-Tuesday-7th-Aug-2018.pdf
2019-01-02 08:05:06 [botocore.utils] DEBUG: URI updated to: https://s3-path/mahindra.com/MM-Q1FY19-Earnings-Concall-Invite-for-Tuesday-7th-Aug-2018.pdf
2019-01-02 08:05:06 [botocore.auth] DEBUG: Calculating signature using v4 auth.
2019-01-02 08:05:06 [botocore.auth] DEBUG: CanonicalRequest:
HEAD
/mahindra.com/MM-Q1FY19-Earnings-Concall-Invite-for-Tuesday-7th-Aug-2018.pdf

host:scrapy-data-lseg.s3.eu-west-2.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20190102T080506Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2019-01-02 08:05:06 [botocore.auth] DEBUG: StringToSign:
AWS4-HMAC-SHA256
20190102T080506Z
20190102/eu-west-2/s3/aws4_request
49ec7c01e82a6fc48bf9f1ff56f008f62555bb36fadf9b2c150bd0417013c7ff
2019-01-02 08:05:06 [botocore.auth] DEBUG: Signature:
6a36479da35e60a8d904e34322ef3ecff978433a530f578b987b63508eb25067
2019-01-02 08:05:06 [botocore.endpoint] DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=HEAD, url=https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com/mahindra.com/MM-Q1FY19-Earnings-Concall-Invite-for-Tuesday-7th-Aug-2018.pdf, headers={'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAJNGI6DYOK7QJUQBQ/20190102/eu-west-2/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=6a36479da35e60a8d904e34322ef3ecff978433a530f578b987b63508eb25067', 'User-Agent': b'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Content-Length': '0', 'X-Amz-Date': b'20190102T080506Z', 'X-Amz-Content-SHA256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'}>
2019-01-02 08:05:06 [urllib3.util.retry] DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-01-02 08:05:06 [urllib3.connectionpool] DEBUG: https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com:443 "HEAD /mahindra.com/MM-Q1FY19-Earnings-Concall-Invite-for-Tuesday-7th-Aug-2018.pdf HTTP/1.1" 404 0
2019-01-02 08:05:06 [botocore.parsers] DEBUG: Response headers: {'Content-Type': 'application/xml', 'Date': 'Wed, 02 Jan 2019 08:05:06 GMT', 'x-amz-id-2': '/iLpu9gEFmnhBQcqbppWoFfD2YeXpdVZuD4+9I0CTqCDTsW+yCZykQMOO/9lD1VFKUk6YrzgPzQ=', 'Server': 'AmazonS3', 'x-amz-request-id': 'F25832AB3F41DD4F', 'Transfer-Encoding': 'chunked'}
2019-01-02 08:05:06 [botocore.parsers] DEBUG: Response body:
b''
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event needs-retry.s3.HeadObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f1fe54adef0>
2019-01-02 08:05:06 [botocore.retryhandler] DEBUG: No retry needed.
2019-01-02 08:05:06 [botocore.hooks] DEBUG: Event needs-retry.s3.HeadObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:07 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/resources/investor-reports/FY19/Announcements/Schedule-Presentation-of-Analyst-Institutional-Investor-Meetings-post-Q4FY18-results.zip> (referer: https://www.mahindra.com/investors/reports-and-presentations)
2019-01-02 08:05:07 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/resources/investor-reports/FY19/Announcements/schedule-presentation-of-analyst-institutional-investor-meetings-post-Q1FY19-results.zip> (referer: https://www.mahindra.com/investors/reports-and-presentations)
2019-01-02 08:05:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/annualreviewFY18/assets/Downloads/Analyst%20Day%20Presentations/Auto-Sector-Presentation-Post-FY18-Results.pdf> (referer: None)
2019-01-02 08:05:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://www.mahindra.com/annualreviewFY18/assets/Downloads/Analyst%20Day%20Presentations/Auto-Sector-Presentation-Post-FY18-Results.pdf> referred in <None>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_ascii_metadata at 0x7f1fe55d5510>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function sse_md5 at 0x7f1fe55d18c8>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function convert_body_to_file_like_object at 0x7f1fe55d5bf8>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_bucket_name at 0x7f1fe55d1840>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function generate_idempotent_uuid at 0x7f1fe55d1488>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function conditionally_calculate_md5 at 0x7f1fe55d17b8>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function add_expect_header at 0x7f1fe55d1d08>
2019-01-02 08:05:08 [botocore.handlers] DEBUG: Adding expect 100 continue header to request.
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:08 [botocore.utils] DEBUG: Updating URI from https://s3.amazonaws.com/scrapy-data-lseg/mahindra.com/Auto-Sector-Presentation-Post-FY18-Results.pdf to https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/Auto-Sector-Presentation-Post-FY18-Results.pdf
2019-01-02 08:05:08 [botocore.endpoint] DEBUG: Making request for OperationModel(name=PutObject) with params: {'url_path': '/scrapy-data-lseg/mahindra.com/Auto-Sector-Presentation-Post-FY18-Results.pdf', 'query_string': {}, 'url': 'https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/Auto-Sector-Presentation-Post-FY18-Results.pdf', 'headers': {'Cache-Control': 'max-age=172800', 'x-amz-acl': 'private', 'User-Agent': 'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Expect': '100-continue', 'Content-MD5': '9OR4dTO5Hp3s9BSuGGh5lQ=='}, 'method': 'PUT', 'body': <_io.BytesIO object at 0x7f1fe4c524c0>, 'context': {'client_region': 'us-east-1', 'has_streaming_input': True, 'client_config': <botocore.config.Config object at 0x7f1fe54ad518>, 'auth_type': None, 'signing': {'region': 'eu-west-2', 'endpoint': 'https://s3.eu-west-2.amazonaws.com', 'bucket': 'scrapy-data-lseg'}}}
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event request-created.s3.PutObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f1fe54ad4a8>>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x7f1fe7bdb278>>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <function set_operation_specific_signer at 0x7f1fe55d1378>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-sign.s3.PutObject: calling handler <function fix_s3_host at 0x7f1fe571a730>
2019-01-02 08:05:08 [botocore.utils] DEBUG: Checking for DNS compatible bucket for: https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/Auto-Sector-Presentation-Post-FY18-Results.pdf
2019-01-02 08:05:08 [botocore.utils] DEBUG: URI updated to: https://s3-path/mahindra.com/Auto-Sector-Presentation-Post-FY18-Results.pdf
2019-01-02 08:05:08 [botocore.auth] DEBUG: Calculating signature using v4 auth.
2019-01-02 08:05:08 [botocore.auth] DEBUG: CanonicalRequest:
PUT
/mahindra.com/Auto-Sector-Presentation-Post-FY18-Results.pdf

cache-control:max-age=172800
content-md5:9OR4dTO5Hp3s9BSuGGh5lQ==
host:scrapy-data-lseg.s3.eu-west-2.amazonaws.com
x-amz-acl:private
x-amz-content-sha256:UNSIGNED-PAYLOAD
x-amz-date:20190102T080508Z

cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date
UNSIGNED-PAYLOAD
2019-01-02 08:05:08 [botocore.auth] DEBUG: StringToSign:
AWS4-HMAC-SHA256
20190102T080508Z
20190102/eu-west-2/s3/aws4_request
3c1c4f0d46d2424353af84b4a84338fe3286802622cab117a6fae0bdef1c9f5f
2019-01-02 08:05:08 [botocore.auth] DEBUG: Signature:
2f1f4ae05fff267ce46167c718a3283c86a28cd428bdde1b674487a945677dd8
2019-01-02 08:05:08 [botocore.endpoint] DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=PUT, url=https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com/mahindra.com/Auto-Sector-Presentation-Post-FY18-Results.pdf, headers={'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAJNGI6DYOK7QJUQBQ/20190102/eu-west-2/s3/aws4_request, SignedHeaders=cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date, Signature=2f1f4ae05fff267ce46167c718a3283c86a28cd428bdde1b674487a945677dd8', 'Cache-Control': b'max-age=172800', 'X-Amz-Date': b'20190102T080508Z', 'Expect': b'100-continue', 'X-Amz-Content-SHA256': b'UNSIGNED-PAYLOAD', 'x-amz-acl': b'private', 'User-Agent': b'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Content-Length': '2733823', 'Content-MD5': b'9OR4dTO5Hp3s9BSuGGh5lQ=='}>
2019-01-02 08:05:08 [urllib3.util.retry] DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-01-02 08:05:08 [botocore.awsrequest] DEBUG: Waiting for 100 Continue response.
2019-01-02 08:05:08 [botocore.awsrequest] DEBUG: 100 Continue response seen, now sending request body.
2019-01-02 08:05:08 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): luoq61dim4.execute-api.eu-west-2.amazonaws.com:443
2019-01-02 08:05:08 [urllib3.connectionpool] DEBUG: https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com:443 "PUT /mahindra.com/Auto-Sector-Presentation-Post-FY18-Results.pdf HTTP/1.1" 200 0
2019-01-02 08:05:08 [botocore.parsers] DEBUG: Response headers: {'Date': 'Wed, 02 Jan 2019 08:05:09 GMT', 'x-amz-id-2': 'ia/1XNAbus1ceEuMEvj0z57waYrEFy5m9h4rDa2YVkHUcFp/ndLKQBLyaXptleJGuiZHpU6fOvk=', 'Server': 'AmazonS3', 'ETag': '"f4e4787533b91e9decf414ae18687995"', 'x-amz-request-id': '504041FC6551D6E5', 'Content-Length': '0'}
2019-01-02 08:05:08 [botocore.parsers] DEBUG: Response body:
b''
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f1fe54adef0>
2019-01-02 08:05:08 [botocore.retryhandler] DEBUG: No retry needed.
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:08 [urllib3.connectionpool] DEBUG: https://luoq61dim4.execute-api.eu-west-2.amazonaws.com:443 "POST /dev/ HTTP/1.1" 200 73
2019-01-02 08:05:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.mahindra.com/annualreviewFY18/assets/Downloads/Analyst%20Day%20Presentations/Auto-Sector-Presentation-Post-FY18-Results.pdf>
{'baseurl': 'http://www.mahindra.com',
 'companyname': 'Mahindra&Mahindra',
 'doc_type': 'application/pdf',
 'file_urls': 'https://www.mahindra.com/annualreviewFY18/assets/Downloads/Analyst%20Day%20Presentations/Auto-Sector-Presentation-Post-FY18-Results.pdf',
 'filename': 'Auto-Sector-Presentation-Post-FY18-Results.pdf',
 'lastmodifier': 'Mon, 30 Jul 2018 01:06:40 GMT',
 'name': 'mahindra.com/Auto-Sector-Presentation-Post-FY18-Results.pdf',
 's3_path': 's3://scrapy-data-lseg/mahindra.com/Auto-Sector-Presentation-Post-FY18-Results.pdf'}
2019-01-02 08:05:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/annualreviewFY18/assets/Downloads/Analyst%20Day%20Presentations/Investor-Presentation-post-FY18-Results.pdf> (referer: None)
2019-01-02 08:05:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://www.mahindra.com/annualreviewFY18/assets/Downloads/Analyst%20Day%20Presentations/Investor-Presentation-post-FY18-Results.pdf> referred in <None>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_ascii_metadata at 0x7f1fe55d5510>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function sse_md5 at 0x7f1fe55d18c8>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function convert_body_to_file_like_object at 0x7f1fe55d5bf8>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_bucket_name at 0x7f1fe55d1840>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function generate_idempotent_uuid at 0x7f1fe55d1488>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function conditionally_calculate_md5 at 0x7f1fe55d17b8>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function add_expect_header at 0x7f1fe55d1d08>
2019-01-02 08:05:08 [botocore.handlers] DEBUG: Adding expect 100 continue header to request.
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:08 [botocore.utils] DEBUG: Updating URI from https://s3.amazonaws.com/scrapy-data-lseg/mahindra.com/Investor-Presentation-post-FY18-Results.pdf to https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/Investor-Presentation-post-FY18-Results.pdf
2019-01-02 08:05:08 [botocore.endpoint] DEBUG: Making request for OperationModel(name=PutObject) with params: {'url_path': '/scrapy-data-lseg/mahindra.com/Investor-Presentation-post-FY18-Results.pdf', 'query_string': {}, 'url': 'https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/Investor-Presentation-post-FY18-Results.pdf', 'headers': {'Cache-Control': 'max-age=172800', 'x-amz-acl': 'private', 'User-Agent': 'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Expect': '100-continue', 'Content-MD5': 'Ny5iG9YThuFazMeOmyt/ZA=='}, 'method': 'PUT', 'body': <_io.BytesIO object at 0x7f1fe4c52728>, 'context': {'client_region': 'us-east-1', 'has_streaming_input': True, 'client_config': <botocore.config.Config object at 0x7f1fe54ad518>, 'auth_type': None, 'signing': {'region': 'eu-west-2', 'endpoint': 'https://s3.eu-west-2.amazonaws.com', 'bucket': 'scrapy-data-lseg'}}}
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event request-created.s3.PutObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f1fe54ad4a8>>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x7f1fe7bdb278>>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <function set_operation_specific_signer at 0x7f1fe55d1378>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-sign.s3.PutObject: calling handler <function fix_s3_host at 0x7f1fe571a730>
2019-01-02 08:05:08 [botocore.utils] DEBUG: Checking for DNS compatible bucket for: https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/Investor-Presentation-post-FY18-Results.pdf
2019-01-02 08:05:08 [botocore.utils] DEBUG: URI updated to: https://s3-path/mahindra.com/Investor-Presentation-post-FY18-Results.pdf
2019-01-02 08:05:08 [botocore.auth] DEBUG: Calculating signature using v4 auth.
2019-01-02 08:05:08 [botocore.auth] DEBUG: CanonicalRequest:
PUT
/mahindra.com/Investor-Presentation-post-FY18-Results.pdf

cache-control:max-age=172800
content-md5:Ny5iG9YThuFazMeOmyt/ZA==
host:scrapy-data-lseg.s3.eu-west-2.amazonaws.com
x-amz-acl:private
x-amz-content-sha256:UNSIGNED-PAYLOAD
x-amz-date:20190102T080508Z

cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date
UNSIGNED-PAYLOAD
2019-01-02 08:05:08 [botocore.auth] DEBUG: StringToSign:
AWS4-HMAC-SHA256
20190102T080508Z
20190102/eu-west-2/s3/aws4_request
d149bb5c2067074e1c2c42187d1aa6ab3c76c1a0cef711bc64bf12e91ee05b0e
2019-01-02 08:05:08 [botocore.auth] DEBUG: Signature:
87f53f6c97e38ac3f9b3873e6385a8b3b974df5891b0c3641e5b009ad1d9e192
2019-01-02 08:05:08 [botocore.endpoint] DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=PUT, url=https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com/mahindra.com/Investor-Presentation-post-FY18-Results.pdf, headers={'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAJNGI6DYOK7QJUQBQ/20190102/eu-west-2/s3/aws4_request, SignedHeaders=cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date, Signature=87f53f6c97e38ac3f9b3873e6385a8b3b974df5891b0c3641e5b009ad1d9e192', 'Cache-Control': b'max-age=172800', 'X-Amz-Date': b'20190102T080508Z', 'Expect': b'100-continue', 'X-Amz-Content-SHA256': b'UNSIGNED-PAYLOAD', 'x-amz-acl': b'private', 'User-Agent': b'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Content-Length': '7846632', 'Content-MD5': b'Ny5iG9YThuFazMeOmyt/ZA=='}>
2019-01-02 08:05:08 [urllib3.util.retry] DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-01-02 08:05:08 [botocore.awsrequest] DEBUG: Waiting for 100 Continue response.
2019-01-02 08:05:08 [botocore.awsrequest] DEBUG: 100 Continue response seen, now sending request body.
2019-01-02 08:05:08 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): luoq61dim4.execute-api.eu-west-2.amazonaws.com:443
2019-01-02 08:05:08 [urllib3.connectionpool] DEBUG: https://luoq61dim4.execute-api.eu-west-2.amazonaws.com:443 "POST /dev/ HTTP/1.1" 200 73
2019-01-02 08:05:08 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.mahindra.com/annualreviewFY18/assets/Downloads/Analyst%20Day%20Presentations/Investor-Presentation-post-FY18-Results.pdf>
{'baseurl': 'http://www.mahindra.com',
 'companyname': 'Mahindra&Mahindra',
 'doc_type': 'application/pdf',
 'file_urls': 'https://www.mahindra.com/annualreviewFY18/assets/Downloads/Analyst%20Day%20Presentations/Investor-Presentation-post-FY18-Results.pdf',
 'filename': 'Investor-Presentation-post-FY18-Results.pdf',
 'lastmodifier': 'Mon, 30 Jul 2018 01:06:40 GMT',
 'name': 'mahindra.com/Investor-Presentation-post-FY18-Results.pdf',
 's3_path': 's3://scrapy-data-lseg/mahindra.com/Investor-Presentation-post-FY18-Results.pdf'}
2019-01-02 08:05:08 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/resources/investor-reports/FY19/Announcements/M&MISIN30092018.pdf> (referer: None)
2019-01-02 08:05:08 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://www.mahindra.com/resources/investor-reports/FY19/Announcements/M&MISIN30092018.pdf> referred in <None>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_ascii_metadata at 0x7f1fe55d5510>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function sse_md5 at 0x7f1fe55d18c8>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function convert_body_to_file_like_object at 0x7f1fe55d5bf8>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_bucket_name at 0x7f1fe55d1840>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function generate_idempotent_uuid at 0x7f1fe55d1488>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function conditionally_calculate_md5 at 0x7f1fe55d17b8>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function add_expect_header at 0x7f1fe55d1d08>
2019-01-02 08:05:08 [botocore.handlers] DEBUG: Adding expect 100 continue header to request.
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:08 [botocore.utils] DEBUG: Updating URI from https://s3.amazonaws.com/scrapy-data-lseg/mahindra.com/M%26MISIN30092018.pdf to https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/M%26MISIN30092018.pdf
2019-01-02 08:05:08 [botocore.endpoint] DEBUG: Making request for OperationModel(name=PutObject) with params: {'url_path': '/scrapy-data-lseg/mahindra.com/M%26MISIN30092018.pdf', 'query_string': {}, 'url': 'https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/M%26MISIN30092018.pdf', 'headers': {'Cache-Control': 'max-age=172800', 'x-amz-acl': 'private', 'User-Agent': 'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Expect': '100-continue', 'Content-MD5': 'bQkSvjtOstGE6D/gYw9fhw=='}, 'method': 'PUT', 'body': <_io.BytesIO object at 0x7f1fe4c52e08>, 'context': {'client_region': 'us-east-1', 'has_streaming_input': True, 'client_config': <botocore.config.Config object at 0x7f1fe54ad518>, 'auth_type': None, 'signing': {'region': 'eu-west-2', 'endpoint': 'https://s3.eu-west-2.amazonaws.com', 'bucket': 'scrapy-data-lseg'}}}
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event request-created.s3.PutObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f1fe54ad4a8>>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x7f1fe7bdb278>>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <function set_operation_specific_signer at 0x7f1fe55d1378>
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event before-sign.s3.PutObject: calling handler <function fix_s3_host at 0x7f1fe571a730>
2019-01-02 08:05:08 [botocore.utils] DEBUG: Checking for DNS compatible bucket for: https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/M%26MISIN30092018.pdf
2019-01-02 08:05:08 [botocore.utils] DEBUG: URI updated to: https://s3-path/mahindra.com/M%26MISIN30092018.pdf
2019-01-02 08:05:08 [botocore.auth] DEBUG: Calculating signature using v4 auth.
2019-01-02 08:05:08 [botocore.auth] DEBUG: CanonicalRequest:
PUT
/mahindra.com/M%26MISIN30092018.pdf

cache-control:max-age=172800
content-md5:bQkSvjtOstGE6D/gYw9fhw==
host:scrapy-data-lseg.s3.eu-west-2.amazonaws.com
x-amz-acl:private
x-amz-content-sha256:UNSIGNED-PAYLOAD
x-amz-date:20190102T080508Z

cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date
UNSIGNED-PAYLOAD
2019-01-02 08:05:08 [botocore.auth] DEBUG: StringToSign:
AWS4-HMAC-SHA256
20190102T080508Z
20190102/eu-west-2/s3/aws4_request
f161471578f72d9bf695a5345b1d48d8e0ae56304b57c35cffc55185835af13b
2019-01-02 08:05:08 [botocore.auth] DEBUG: Signature:
678962f8cb900bb638783c59ab4c18e3afa5bd6663448196acd5ffa6c7b653ff
2019-01-02 08:05:08 [botocore.endpoint] DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=PUT, url=https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com/mahindra.com/M%26MISIN30092018.pdf, headers={'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAJNGI6DYOK7QJUQBQ/20190102/eu-west-2/s3/aws4_request, SignedHeaders=cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date, Signature=678962f8cb900bb638783c59ab4c18e3afa5bd6663448196acd5ffa6c7b653ff', 'Cache-Control': b'max-age=172800', 'X-Amz-Date': b'20190102T080508Z', 'Expect': b'100-continue', 'X-Amz-Content-SHA256': b'UNSIGNED-PAYLOAD', 'x-amz-acl': b'private', 'User-Agent': b'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Content-Length': '168328', 'Content-MD5': b'bQkSvjtOstGE6D/gYw9fhw=='}>
2019-01-02 08:05:08 [urllib3.util.retry] DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-01-02 08:05:08 [urllib3.connectionpool] DEBUG: Resetting dropped connection: scrapy-data-lseg.s3.eu-west-2.amazonaws.com
2019-01-02 08:05:08 [botocore.awsrequest] DEBUG: Waiting for 100 Continue response.
2019-01-02 08:05:08 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): luoq61dim4.execute-api.eu-west-2.amazonaws.com:443
2019-01-02 08:05:08 [urllib3.connectionpool] DEBUG: https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com:443 "PUT /mahindra.com/Investor-Presentation-post-FY18-Results.pdf HTTP/1.1" 200 0
2019-01-02 08:05:08 [botocore.parsers] DEBUG: Response headers: {'Date': 'Wed, 02 Jan 2019 08:05:09 GMT', 'x-amz-id-2': 'QuJS7AsP82RO0DrJcL5m8vdDdrK6uFYPWY8ghFHpn7aH92h54WjsYs1mOI3hDpwxXadQe7ri9oU=', 'Server': 'AmazonS3', 'ETag': '"372e621bd61386e15accc78e9b2b7f64"', 'x-amz-request-id': 'E7B73C52B300CB12', 'Content-Length': '0'}
2019-01-02 08:05:08 [botocore.parsers] DEBUG: Response body:
b''
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f1fe54adef0>
2019-01-02 08:05:08 [botocore.retryhandler] DEBUG: No retry needed.
2019-01-02 08:05:08 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:08 [botocore.awsrequest] DEBUG: 100 Continue response seen, now sending request body.
2019-01-02 08:05:09 [urllib3.connectionpool] DEBUG: https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com:443 "PUT /mahindra.com/M%26MISIN30092018.pdf HTTP/1.1" 200 0
2019-01-02 08:05:09 [botocore.parsers] DEBUG: Response headers: {'Date': 'Wed, 02 Jan 2019 08:05:09 GMT', 'x-amz-id-2': 'tTGhuuClYhtq5J1WahJTUn2tGYewh4C6lyHAFVJxCi9jGmiCB02zyU7mcyi6n6Et2TXoBnSG89w=', 'Server': 'AmazonS3', 'ETag': '"6d0912be3b4eb2d184e83fe0630f5f87"', 'x-amz-request-id': 'B0683A81817A9D84', 'Content-Length': '0'}
2019-01-02 08:05:09 [botocore.parsers] DEBUG: Response body:
b''
2019-01-02 08:05:09 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f1fe54adef0>
2019-01-02 08:05:09 [botocore.retryhandler] DEBUG: No retry needed.
2019-01-02 08:05:09 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:09 [urllib3.connectionpool] DEBUG: https://luoq61dim4.execute-api.eu-west-2.amazonaws.com:443 "POST /dev/ HTTP/1.1" 200 73
2019-01-02 08:05:09 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.mahindra.com/resources/investor-reports/FY19/Announcements/M&MISIN30092018.pdf>
{'baseurl': 'http://www.mahindra.com',
 'companyname': 'Mahindra&Mahindra',
 'doc_type': 'application/pdf',
 'file_urls': 'https://www.mahindra.com/resources/investor-reports/FY19/Announcements/M&MISIN30092018.pdf',
 'filename': 'M&MISIN30092018.pdf',
 'lastmodifier': 'Thu, 04 Oct 2018 05:51:42 GMT',
 'name': 'mahindra.com/M&MISIN30092018.pdf',
 's3_path': 's3://scrapy-data-lseg/mahindra.com/M&MISIN30092018.pdf'}
2019-01-02 08:05:09 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/resources/investor-reports/FY18/Annual%20Reports/Mahindra-and-Mahindra-Subsidiary-Annual-Report-2017-18.pdf> (referer: https://www.mahindra.com/investors/reports-and-presentations)
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function sse_md5 at 0x7f1fe55d18c8>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function validate_bucket_name at 0x7f1fe55d1840>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.HeadObject: calling handler <function generate_idempotent_uuid at 0x7f1fe55d1488>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-call.s3.HeadObject: calling handler <function add_expect_header at 0x7f1fe55d1d08>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-call.s3.HeadObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:27 [botocore.utils] DEBUG: Updating URI from https://s3.amazonaws.com/scrapy-data-lseg/mahindra.com/Mahindra-and-Mahindra-Subsidiary-Annual-Report-2017-18.pdf to https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/Mahindra-and-Mahindra-Subsidiary-Annual-Report-2017-18.pdf
2019-01-02 08:05:27 [botocore.endpoint] DEBUG: Making request for OperationModel(name=HeadObject) with params: {'url_path': '/scrapy-data-lseg/mahindra.com/Mahindra-and-Mahindra-Subsidiary-Annual-Report-2017-18.pdf', 'query_string': {}, 'url': 'https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/Mahindra-and-Mahindra-Subsidiary-Annual-Report-2017-18.pdf', 'headers': {'User-Agent': 'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws'}, 'method': 'HEAD', 'body': b'', 'context': {'client_region': 'us-east-1', 'has_streaming_input': False, 'client_config': <botocore.config.Config object at 0x7f1fe54ad518>, 'auth_type': None, 'signing': {'region': 'eu-west-2', 'endpoint': 'https://s3.eu-west-2.amazonaws.com', 'bucket': 'scrapy-data-lseg'}}}
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event request-created.s3.HeadObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f1fe54ad4a8>>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event choose-signer.s3.HeadObject: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x7f1fe7bdb278>>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event choose-signer.s3.HeadObject: calling handler <function set_operation_specific_signer at 0x7f1fe55d1378>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-sign.s3.HeadObject: calling handler <function fix_s3_host at 0x7f1fe571a730>
2019-01-02 08:05:27 [botocore.utils] DEBUG: Checking for DNS compatible bucket for: https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/Mahindra-and-Mahindra-Subsidiary-Annual-Report-2017-18.pdf
2019-01-02 08:05:27 [botocore.utils] DEBUG: URI updated to: https://s3-path/mahindra.com/Mahindra-and-Mahindra-Subsidiary-Annual-Report-2017-18.pdf
2019-01-02 08:05:27 [botocore.auth] DEBUG: Calculating signature using v4 auth.
2019-01-02 08:05:27 [botocore.auth] DEBUG: CanonicalRequest:
HEAD
/mahindra.com/Mahindra-and-Mahindra-Subsidiary-Annual-Report-2017-18.pdf

host:scrapy-data-lseg.s3.eu-west-2.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20190102T080527Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2019-01-02 08:05:27 [botocore.auth] DEBUG: StringToSign:
AWS4-HMAC-SHA256
20190102T080527Z
20190102/eu-west-2/s3/aws4_request
6171d4cdb31ce7c302bdb6305569f7019c22187b1f4f570107a9f69620170417
2019-01-02 08:05:27 [botocore.auth] DEBUG: Signature:
9e7148bcea6e2c2ed7ddb73631cd7bc5c6266d873de843eac3aaf29d2273078d
2019-01-02 08:05:27 [botocore.endpoint] DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=HEAD, url=https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com/mahindra.com/Mahindra-and-Mahindra-Subsidiary-Annual-Report-2017-18.pdf, headers={'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAJNGI6DYOK7QJUQBQ/20190102/eu-west-2/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=9e7148bcea6e2c2ed7ddb73631cd7bc5c6266d873de843eac3aaf29d2273078d', 'User-Agent': b'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Content-Length': '0', 'X-Amz-Date': b'20190102T080527Z', 'X-Amz-Content-SHA256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'}>
2019-01-02 08:05:27 [urllib3.util.retry] DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-01-02 08:05:27 [urllib3.connectionpool] DEBUG: Resetting dropped connection: scrapy-data-lseg.s3.eu-west-2.amazonaws.com
2019-01-02 08:05:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/annualreviewFY18/assets/Downloads/Analyst%20Day%20Presentations/Mahindra&MahindraAnnualReport2018.pdf> (referer: None)
2019-01-02 08:05:27 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://www.mahindra.com/annualreviewFY18/assets/Downloads/Analyst%20Day%20Presentations/Mahindra&MahindraAnnualReport2018.pdf> referred in <None>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_ascii_metadata at 0x7f1fe55d5510>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function sse_md5 at 0x7f1fe55d18c8>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function convert_body_to_file_like_object at 0x7f1fe55d5bf8>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_bucket_name at 0x7f1fe55d1840>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function generate_idempotent_uuid at 0x7f1fe55d1488>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function conditionally_calculate_md5 at 0x7f1fe55d17b8>
2019-01-02 08:05:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/resources/investor-reports/FY19/Announcements/pressreleaseleasing.pdf> (referer: None)
2019-01-02 08:05:27 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://www.mahindra.com/resources/investor-reports/FY19/Announcements/pressreleaseleasing.pdf> referred in <None>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_ascii_metadata at 0x7f1fe55d5510>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function sse_md5 at 0x7f1fe55d18c8>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function convert_body_to_file_like_object at 0x7f1fe55d5bf8>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_bucket_name at 0x7f1fe55d1840>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function generate_idempotent_uuid at 0x7f1fe55d1488>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function conditionally_calculate_md5 at 0x7f1fe55d17b8>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function add_expect_header at 0x7f1fe55d1d08>
2019-01-02 08:05:27 [botocore.handlers] DEBUG: Adding expect 100 continue header to request.
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:27 [botocore.utils] DEBUG: Updating URI from https://s3.amazonaws.com/scrapy-data-lseg/mahindra.com/pressreleaseleasing.pdf to https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/pressreleaseleasing.pdf
2019-01-02 08:05:27 [botocore.endpoint] DEBUG: Making request for OperationModel(name=PutObject) with params: {'url_path': '/scrapy-data-lseg/mahindra.com/pressreleaseleasing.pdf', 'query_string': {}, 'url': 'https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/pressreleaseleasing.pdf', 'headers': {'Cache-Control': 'max-age=172800', 'x-amz-acl': 'private', 'User-Agent': 'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Expect': '100-continue', 'Content-MD5': 'lKHTPmN0137k4AOdCoiBzQ=='}, 'method': 'PUT', 'body': <_io.BytesIO object at 0x7f1fe4c52e08>, 'context': {'client_region': 'us-east-1', 'has_streaming_input': True, 'client_config': <botocore.config.Config object at 0x7f1fe54ad518>, 'auth_type': None, 'signing': {'region': 'eu-west-2', 'endpoint': 'https://s3.eu-west-2.amazonaws.com', 'bucket': 'scrapy-data-lseg'}}}
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event request-created.s3.PutObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f1fe54ad4a8>>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x7f1fe7bdb278>>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <function set_operation_specific_signer at 0x7f1fe55d1378>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-sign.s3.PutObject: calling handler <function fix_s3_host at 0x7f1fe571a730>
2019-01-02 08:05:27 [botocore.utils] DEBUG: Checking for DNS compatible bucket for: https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/pressreleaseleasing.pdf
2019-01-02 08:05:27 [botocore.utils] DEBUG: URI updated to: https://s3-path/mahindra.com/pressreleaseleasing.pdf
2019-01-02 08:05:27 [botocore.auth] DEBUG: Calculating signature using v4 auth.
2019-01-02 08:05:27 [botocore.auth] DEBUG: CanonicalRequest:
PUT
/mahindra.com/pressreleaseleasing.pdf

cache-control:max-age=172800
content-md5:lKHTPmN0137k4AOdCoiBzQ==
host:scrapy-data-lseg.s3.eu-west-2.amazonaws.com
x-amz-acl:private
x-amz-content-sha256:UNSIGNED-PAYLOAD
x-amz-date:20190102T080527Z

cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date
UNSIGNED-PAYLOAD
2019-01-02 08:05:27 [botocore.auth] DEBUG: StringToSign:
AWS4-HMAC-SHA256
20190102T080527Z
20190102/eu-west-2/s3/aws4_request
0cdc66833cc37f7a4ced9e8c620cde35f968d001bd3cf4942cdbe34a67acfd7a
2019-01-02 08:05:27 [botocore.auth] DEBUG: Signature:
1b49326e7b90fb13064c9e57726d39bef8016a071eca5417c9de02e97f850d24
2019-01-02 08:05:27 [botocore.endpoint] DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=PUT, url=https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com/mahindra.com/pressreleaseleasing.pdf, headers={'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAJNGI6DYOK7QJUQBQ/20190102/eu-west-2/s3/aws4_request, SignedHeaders=cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date, Signature=1b49326e7b90fb13064c9e57726d39bef8016a071eca5417c9de02e97f850d24', 'Cache-Control': b'max-age=172800', 'X-Amz-Date': b'20190102T080527Z', 'Expect': b'100-continue', 'X-Amz-Content-SHA256': b'UNSIGNED-PAYLOAD', 'x-amz-acl': b'private', 'User-Agent': b'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Content-Length': '454682', 'Content-MD5': b'lKHTPmN0137k4AOdCoiBzQ=='}>
2019-01-02 08:05:27 [urllib3.util.retry] DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-01-02 08:05:27 [urllib3.connectionpool] DEBUG: Resetting dropped connection: scrapy-data-lseg.s3.eu-west-2.amazonaws.com
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function add_expect_header at 0x7f1fe55d1d08>
2019-01-02 08:05:27 [botocore.handlers] DEBUG: Adding expect 100 continue header to request.
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:27 [botocore.utils] DEBUG: Updating URI from https://s3.amazonaws.com/scrapy-data-lseg/mahindra.com/Mahindra%26MahindraAnnualReport2018.pdf to https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/Mahindra%26MahindraAnnualReport2018.pdf
2019-01-02 08:05:27 [botocore.endpoint] DEBUG: Making request for OperationModel(name=PutObject) with params: {'url_path': '/scrapy-data-lseg/mahindra.com/Mahindra%26MahindraAnnualReport2018.pdf', 'query_string': {}, 'url': 'https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/Mahindra%26MahindraAnnualReport2018.pdf', 'headers': {'Cache-Control': 'max-age=172800', 'x-amz-acl': 'private', 'User-Agent': 'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Expect': '100-continue', 'Content-MD5': '1YPUi8liIGBwoU3CAlWJNQ=='}, 'method': 'PUT', 'body': <_io.BytesIO object at 0x7f1fe4c52a98>, 'context': {'client_region': 'us-east-1', 'has_streaming_input': True, 'client_config': <botocore.config.Config object at 0x7f1fe54ad518>, 'auth_type': None, 'signing': {'region': 'eu-west-2', 'endpoint': 'https://s3.eu-west-2.amazonaws.com', 'bucket': 'scrapy-data-lseg'}}}
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event request-created.s3.PutObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f1fe54ad4a8>>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x7f1fe7bdb278>>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <function set_operation_specific_signer at 0x7f1fe55d1378>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-sign.s3.PutObject: calling handler <function fix_s3_host at 0x7f1fe571a730>
2019-01-02 08:05:27 [botocore.utils] DEBUG: Checking for DNS compatible bucket for: https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/Mahindra%26MahindraAnnualReport2018.pdf
2019-01-02 08:05:27 [botocore.utils] DEBUG: URI updated to: https://s3-path/mahindra.com/Mahindra%26MahindraAnnualReport2018.pdf
2019-01-02 08:05:27 [botocore.auth] DEBUG: Calculating signature using v4 auth.
2019-01-02 08:05:27 [botocore.auth] DEBUG: CanonicalRequest:
PUT
/mahindra.com/Mahindra%26MahindraAnnualReport2018.pdf

cache-control:max-age=172800
content-md5:1YPUi8liIGBwoU3CAlWJNQ==
host:scrapy-data-lseg.s3.eu-west-2.amazonaws.com
x-amz-acl:private
x-amz-content-sha256:UNSIGNED-PAYLOAD
x-amz-date:20190102T080527Z

cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date
UNSIGNED-PAYLOAD
2019-01-02 08:05:27 [botocore.auth] DEBUG: StringToSign:
AWS4-HMAC-SHA256
20190102T080527Z
20190102/eu-west-2/s3/aws4_request
8df04cffcbfe72cc74b6f242fc591303e3e15c9ae7930542ca0d51befb3814a4
2019-01-02 08:05:27 [botocore.auth] DEBUG: Signature:
0a24712e3db9db9566e2fcbcc43d1d7ffcc7b706dce565f30c0234501d374490
2019-01-02 08:05:27 [botocore.endpoint] DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=PUT, url=https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com/mahindra.com/Mahindra%26MahindraAnnualReport2018.pdf, headers={'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAJNGI6DYOK7QJUQBQ/20190102/eu-west-2/s3/aws4_request, SignedHeaders=cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date, Signature=0a24712e3db9db9566e2fcbcc43d1d7ffcc7b706dce565f30c0234501d374490', 'Cache-Control': b'max-age=172800', 'X-Amz-Date': b'20190102T080527Z', 'Expect': b'100-continue', 'X-Amz-Content-SHA256': b'UNSIGNED-PAYLOAD', 'x-amz-acl': b'private', 'User-Agent': b'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Content-Length': '11126031', 'Content-MD5': b'1YPUi8liIGBwoU3CAlWJNQ=='}>
2019-01-02 08:05:27 [urllib3.util.retry] DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-01-02 08:05:27 [urllib3.connectionpool] DEBUG: Resetting dropped connection: scrapy-data-lseg.s3.eu-west-2.amazonaws.com
2019-01-02 08:05:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/annualreviewFY18/assets/Downloads/Mahindra/Riseforgood/integ_report.pdf> (referer: None)
2019-01-02 08:05:27 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://www.mahindra.com/annualreviewFY18/assets/Downloads/Mahindra/Riseforgood/integ_report.pdf> referred in <None>
2019-01-02 08:05:27 [urllib3.connectionpool] DEBUG: https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com:443 "HEAD /mahindra.com/Mahindra-and-Mahindra-Subsidiary-Annual-Report-2017-18.pdf HTTP/1.1" 404 0
2019-01-02 08:05:27 [botocore.parsers] DEBUG: Response headers: {'Content-Type': 'application/xml', 'Date': 'Wed, 02 Jan 2019 08:05:27 GMT', 'x-amz-id-2': 'Njj/sU4fXSRmScJbANkBC5iOmvkQgzE6Ve4JHFcPO16tM55m71niYUX3MPTMA58TtBeT6c/8aUk=', 'Server': 'AmazonS3', 'x-amz-request-id': '6966765D0E5910F4', 'Transfer-Encoding': 'chunked'}
2019-01-02 08:05:27 [botocore.parsers] DEBUG: Response body:
b''
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event needs-retry.s3.HeadObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f1fe54adef0>
2019-01-02 08:05:27 [botocore.retryhandler] DEBUG: No retry needed.
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event needs-retry.s3.HeadObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_ascii_metadata at 0x7f1fe55d5510>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function sse_md5 at 0x7f1fe55d18c8>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function convert_body_to_file_like_object at 0x7f1fe55d5bf8>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_bucket_name at 0x7f1fe55d1840>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function generate_idempotent_uuid at 0x7f1fe55d1488>
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function conditionally_calculate_md5 at 0x7f1fe55d17b8>
2019-01-02 08:05:27 [botocore.awsrequest] DEBUG: Waiting for 100 Continue response.
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function add_expect_header at 0x7f1fe55d1d08>
2019-01-02 08:05:27 [botocore.handlers] DEBUG: Adding expect 100 continue header to request.
2019-01-02 08:05:27 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:27 [botocore.utils] DEBUG: Updating URI from https://s3.amazonaws.com/scrapy-data-lseg/mahindra.com/integ_report.pdf to https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/integ_report.pdf
2019-01-02 08:05:28 [botocore.endpoint] DEBUG: Making request for OperationModel(name=PutObject) with params: {'url_path': '/scrapy-data-lseg/mahindra.com/integ_report.pdf', 'query_string': {}, 'url': 'https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/integ_report.pdf', 'headers': {'Cache-Control': 'max-age=172800', 'x-amz-acl': 'private', 'User-Agent': 'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Expect': '100-continue', 'Content-MD5': 'UEOxUn8ISLOiUnJvoLhHIA=='}, 'method': 'PUT', 'body': <_io.BytesIO object at 0x7f1fe4c66ca8>, 'context': {'client_region': 'us-east-1', 'has_streaming_input': True, 'client_config': <botocore.config.Config object at 0x7f1fe54ad518>, 'auth_type': None, 'signing': {'region': 'eu-west-2', 'endpoint': 'https://s3.eu-west-2.amazonaws.com', 'bucket': 'scrapy-data-lseg'}}}
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event request-created.s3.PutObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f1fe54ad4a8>>
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x7f1fe7bdb278>>
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <function set_operation_specific_signer at 0x7f1fe55d1378>
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event before-sign.s3.PutObject: calling handler <function fix_s3_host at 0x7f1fe571a730>
2019-01-02 08:05:28 [botocore.utils] DEBUG: Checking for DNS compatible bucket for: https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/integ_report.pdf
2019-01-02 08:05:28 [botocore.utils] DEBUG: URI updated to: https://s3-path/mahindra.com/integ_report.pdf
2019-01-02 08:05:28 [botocore.auth] DEBUG: Calculating signature using v4 auth.
2019-01-02 08:05:28 [botocore.auth] DEBUG: CanonicalRequest:
PUT
/mahindra.com/integ_report.pdf

cache-control:max-age=172800
content-md5:UEOxUn8ISLOiUnJvoLhHIA==
host:scrapy-data-lseg.s3.eu-west-2.amazonaws.com
x-amz-acl:private
x-amz-content-sha256:UNSIGNED-PAYLOAD
x-amz-date:20190102T080528Z

cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date
UNSIGNED-PAYLOAD
2019-01-02 08:05:28 [botocore.auth] DEBUG: StringToSign:
AWS4-HMAC-SHA256
20190102T080528Z
20190102/eu-west-2/s3/aws4_request
76abc7b3dd583df8e7d0e031ce54f4ce1b8ec64ab2fc1d80deb62dc872e13e2b
2019-01-02 08:05:28 [botocore.auth] DEBUG: Signature:
6a6f836675bd4d74b66fff11c32a4334b24e47ba5b446a272d7cca47da909621
2019-01-02 08:05:28 [botocore.endpoint] DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=PUT, url=https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com/mahindra.com/integ_report.pdf, headers={'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAJNGI6DYOK7QJUQBQ/20190102/eu-west-2/s3/aws4_request, SignedHeaders=cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date, Signature=6a6f836675bd4d74b66fff11c32a4334b24e47ba5b446a272d7cca47da909621', 'Cache-Control': b'max-age=172800', 'X-Amz-Date': b'20190102T080528Z', 'Expect': b'100-continue', 'X-Amz-Content-SHA256': b'UNSIGNED-PAYLOAD', 'x-amz-acl': b'private', 'User-Agent': b'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Content-Length': '7081377', 'Content-MD5': b'UEOxUn8ISLOiUnJvoLhHIA=='}>
2019-01-02 08:05:28 [urllib3.util.retry] DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-01-02 08:05:28 [botocore.awsrequest] DEBUG: Waiting for 100 Continue response.
2019-01-02 08:05:28 [botocore.awsrequest] DEBUG: Waiting for 100 Continue response.
2019-01-02 08:05:28 [botocore.awsrequest] DEBUG: 100 Continue response seen, now sending request body.
2019-01-02 08:05:28 [botocore.awsrequest] DEBUG: 100 Continue response seen, now sending request body.
2019-01-02 08:05:28 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): luoq61dim4.execute-api.eu-west-2.amazonaws.com:443
2019-01-02 08:05:28 [botocore.awsrequest] DEBUG: 100 Continue response seen, now sending request body.
2019-01-02 08:05:28 [urllib3.connectionpool] DEBUG: https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com:443 "PUT /mahindra.com/pressreleaseleasing.pdf HTTP/1.1" 200 0
2019-01-02 08:05:28 [botocore.parsers] DEBUG: Response headers: {'Date': 'Wed, 02 Jan 2019 08:05:28 GMT', 'x-amz-id-2': 'sGmRzW92tjquEyXiGs63uje5S9VGBRcv+MyixL/puLEPL4Y79xLXkn1nIRu5cO5/UDooVX6GzE8=', 'Server': 'AmazonS3', 'ETag': '"94a1d33e6374d77ee4e0039d0a8881cd"', 'x-amz-request-id': '9A8059C625A41BDB', 'Content-Length': '0'}
2019-01-02 08:05:28 [botocore.parsers] DEBUG: Response body:
b''
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f1fe54adef0>
2019-01-02 08:05:28 [botocore.retryhandler] DEBUG: No retry needed.
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:28 [urllib3.connectionpool] DEBUG: https://luoq61dim4.execute-api.eu-west-2.amazonaws.com:443 "POST /dev/ HTTP/1.1" 200 73
2019-01-02 08:05:28 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.mahindra.com/annualreviewFY18/assets/Downloads/Analyst%20Day%20Presentations/Mahindra&MahindraAnnualReport2018.pdf>
{'baseurl': 'http://www.mahindra.com',
 'companyname': 'Mahindra&Mahindra',
 'doc_type': 'application/pdf',
 'file_urls': 'https://www.mahindra.com/annualreviewFY18/assets/Downloads/Analyst%20Day%20Presentations/Mahindra&MahindraAnnualReport2018.pdf',
 'filename': 'Mahindra&MahindraAnnualReport2018.pdf',
 'lastmodifier': 'Mon, 30 Jul 2018 01:06:40 GMT',
 'name': 'mahindra.com/Mahindra&MahindraAnnualReport2018.pdf',
 's3_path': 's3://scrapy-data-lseg/mahindra.com/Mahindra&MahindraAnnualReport2018.pdf'}
2019-01-02 08:05:28 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): luoq61dim4.execute-api.eu-west-2.amazonaws.com:443
2019-01-02 08:05:28 [urllib3.connectionpool] DEBUG: https://luoq61dim4.execute-api.eu-west-2.amazonaws.com:443 "POST /dev/ HTTP/1.1" 200 73
2019-01-02 08:05:28 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.mahindra.com/resources/investor-reports/FY19/Announcements/pressreleaseleasing.pdf>
{'baseurl': 'http://www.mahindra.com',
 'companyname': 'Mahindra&Mahindra',
 'doc_type': 'application/pdf',
 'file_urls': 'https://www.mahindra.com/resources/investor-reports/FY19/Announcements/pressreleaseleasing.pdf',
 'filename': 'pressreleaseleasing.pdf',
 'lastmodifier': 'Thu, 11 Oct 2018 04:37:34 GMT',
 'name': 'mahindra.com/pressreleaseleasing.pdf',
 's3_path': 's3://scrapy-data-lseg/mahindra.com/pressreleaseleasing.pdf'}
2019-01-02 08:05:28 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): luoq61dim4.execute-api.eu-west-2.amazonaws.com:443
2019-01-02 08:05:28 [urllib3.connectionpool] DEBUG: https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com:443 "PUT /mahindra.com/integ_report.pdf HTTP/1.1" 200 0
2019-01-02 08:05:28 [botocore.parsers] DEBUG: Response headers: {'Date': 'Wed, 02 Jan 2019 08:05:29 GMT', 'x-amz-id-2': '9bl9U0+KcyYBspQZeHYOi87SVQxEcF8DLzaokrsZP+DZ71a9KLmMf+6uqjfPqBz6HMDP2BwYMAw=', 'Server': 'AmazonS3', 'ETag': '"5043b1527f0848b3a252726fa0b84720"', 'x-amz-request-id': 'CC2A42A8B5FA118C', 'Content-Length': '0'}
2019-01-02 08:05:28 [botocore.parsers] DEBUG: Response body:
b''
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f1fe54adef0>
2019-01-02 08:05:28 [botocore.retryhandler] DEBUG: No retry needed.
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:28 [urllib3.connectionpool] DEBUG: https://luoq61dim4.execute-api.eu-west-2.amazonaws.com:443 "POST /dev/ HTTP/1.1" 200 73
2019-01-02 08:05:28 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.mahindra.com/annualreviewFY18/assets/Downloads/Mahindra/Riseforgood/integ_report.pdf>
{'baseurl': 'http://www.mahindra.com',
 'companyname': 'Mahindra&Mahindra',
 'doc_type': 'application/pdf',
 'file_urls': 'https://www.mahindra.com/annualreviewFY18/assets/Downloads/Mahindra/Riseforgood/integ_report.pdf',
 'filename': 'integ_report.pdf',
 'lastmodifier': 'Tue, 17 Jul 2018 06:21:30 GMT',
 'name': 'mahindra.com/integ_report.pdf',
 's3_path': 's3://scrapy-data-lseg/mahindra.com/integ_report.pdf'}
2019-01-02 08:05:28 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/resources/investor-reports/FY19/Announcements/CG-Report30092018.pdf> (referer: None)
2019-01-02 08:05:28 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://www.mahindra.com/resources/investor-reports/FY19/Announcements/CG-Report30092018.pdf> referred in <None>
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_ascii_metadata at 0x7f1fe55d5510>
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function sse_md5 at 0x7f1fe55d18c8>
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function convert_body_to_file_like_object at 0x7f1fe55d5bf8>
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_bucket_name at 0x7f1fe55d1840>
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function generate_idempotent_uuid at 0x7f1fe55d1488>
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function conditionally_calculate_md5 at 0x7f1fe55d17b8>
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function add_expect_header at 0x7f1fe55d1d08>
2019-01-02 08:05:28 [botocore.handlers] DEBUG: Adding expect 100 continue header to request.
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:28 [botocore.utils] DEBUG: Updating URI from https://s3.amazonaws.com/scrapy-data-lseg/mahindra.com/CG-Report30092018.pdf to https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/CG-Report30092018.pdf
2019-01-02 08:05:28 [botocore.endpoint] DEBUG: Making request for OperationModel(name=PutObject) with params: {'url_path': '/scrapy-data-lseg/mahindra.com/CG-Report30092018.pdf', 'query_string': {}, 'url': 'https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/CG-Report30092018.pdf', 'headers': {'Cache-Control': 'max-age=172800', 'x-amz-acl': 'private', 'User-Agent': 'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Expect': '100-continue', 'Content-MD5': '3Xk0OZxoA/f49FrVUae9sQ=='}, 'method': 'PUT', 'body': <_io.BytesIO object at 0x7f1fe4c52150>, 'context': {'client_region': 'us-east-1', 'has_streaming_input': True, 'client_config': <botocore.config.Config object at 0x7f1fe54ad518>, 'auth_type': None, 'signing': {'region': 'eu-west-2', 'endpoint': 'https://s3.eu-west-2.amazonaws.com', 'bucket': 'scrapy-data-lseg'}}}
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event request-created.s3.PutObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f1fe54ad4a8>>
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x7f1fe7bdb278>>
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <function set_operation_specific_signer at 0x7f1fe55d1378>
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event before-sign.s3.PutObject: calling handler <function fix_s3_host at 0x7f1fe571a730>
2019-01-02 08:05:28 [botocore.utils] DEBUG: Checking for DNS compatible bucket for: https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/CG-Report30092018.pdf
2019-01-02 08:05:28 [botocore.utils] DEBUG: URI updated to: https://s3-path/mahindra.com/CG-Report30092018.pdf
2019-01-02 08:05:28 [botocore.auth] DEBUG: Calculating signature using v4 auth.
2019-01-02 08:05:28 [botocore.auth] DEBUG: CanonicalRequest:
PUT
/mahindra.com/CG-Report30092018.pdf

cache-control:max-age=172800
content-md5:3Xk0OZxoA/f49FrVUae9sQ==
host:scrapy-data-lseg.s3.eu-west-2.amazonaws.com
x-amz-acl:private
x-amz-content-sha256:UNSIGNED-PAYLOAD
x-amz-date:20190102T080528Z

cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date
UNSIGNED-PAYLOAD
2019-01-02 08:05:28 [botocore.auth] DEBUG: StringToSign:
AWS4-HMAC-SHA256
20190102T080528Z
20190102/eu-west-2/s3/aws4_request
71252882bbd878882bb2f03dc8df49da5c98a23e5c2a3c1c1a8600d71686a466
2019-01-02 08:05:28 [botocore.auth] DEBUG: Signature:
c910c62998fdbe2d031d2cb35f50ca291d0f385a79fae3e6400e518c25bd6f14
2019-01-02 08:05:28 [botocore.endpoint] DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=PUT, url=https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com/mahindra.com/CG-Report30092018.pdf, headers={'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAJNGI6DYOK7QJUQBQ/20190102/eu-west-2/s3/aws4_request, SignedHeaders=cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date, Signature=c910c62998fdbe2d031d2cb35f50ca291d0f385a79fae3e6400e518c25bd6f14', 'Cache-Control': b'max-age=172800', 'X-Amz-Date': b'20190102T080528Z', 'Expect': b'100-continue', 'X-Amz-Content-SHA256': b'UNSIGNED-PAYLOAD', 'x-amz-acl': b'private', 'User-Agent': b'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Content-Length': '889254', 'Content-MD5': b'3Xk0OZxoA/f49FrVUae9sQ=='}>
2019-01-02 08:05:28 [urllib3.util.retry] DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-01-02 08:05:28 [botocore.awsrequest] DEBUG: Waiting for 100 Continue response.
2019-01-02 08:05:28 [botocore.awsrequest] DEBUG: 100 Continue response seen, now sending request body.
2019-01-02 08:05:28 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): luoq61dim4.execute-api.eu-west-2.amazonaws.com:443
2019-01-02 08:05:28 [urllib3.connectionpool] DEBUG: https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com:443 "PUT /mahindra.com/Mahindra%26MahindraAnnualReport2018.pdf HTTP/1.1" 200 0
2019-01-02 08:05:28 [botocore.parsers] DEBUG: Response headers: {'Date': 'Wed, 02 Jan 2019 08:05:29 GMT', 'x-amz-id-2': 'qqK1N1sXliPemvy+VYfIx3lqfFmY+Srig1fiibIAJZimVmZr65HiBqCJ+yJMV8ImXZPIjqMIRyU=', 'Server': 'AmazonS3', 'ETag': '"d583d48bc962206070a14dc202558935"', 'x-amz-request-id': 'C60D690CA6A0579E', 'Content-Length': '0'}
2019-01-02 08:05:28 [botocore.parsers] DEBUG: Response body:
b''
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f1fe54adef0>
2019-01-02 08:05:28 [botocore.retryhandler] DEBUG: No retry needed.
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:28 [urllib3.connectionpool] DEBUG: https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com:443 "PUT /mahindra.com/CG-Report30092018.pdf HTTP/1.1" 200 0
2019-01-02 08:05:28 [botocore.parsers] DEBUG: Response headers: {'Date': 'Wed, 02 Jan 2019 08:05:29 GMT', 'x-amz-id-2': '1dw3u4IMibvaOQO33DHAGtnoWNKlbxK1z/sazlLi7QwrfZLw7/TV87DeLqjJrns9M4ru0cVLrg4=', 'Server': 'AmazonS3', 'ETag': '"dd7934399c6803f7f8f45ad551a7bdb1"', 'x-amz-request-id': '01984F1494850BFA', 'Content-Length': '0'}
2019-01-02 08:05:28 [botocore.parsers] DEBUG: Response body:
b''
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f1fe54adef0>
2019-01-02 08:05:28 [botocore.retryhandler] DEBUG: No retry needed.
2019-01-02 08:05:28 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:29 [urllib3.connectionpool] DEBUG: https://luoq61dim4.execute-api.eu-west-2.amazonaws.com:443 "POST /dev/ HTTP/1.1" 200 73
2019-01-02 08:05:29 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.mahindra.com/resources/investor-reports/FY19/Announcements/CG-Report30092018.pdf>
{'baseurl': 'http://www.mahindra.com',
 'companyname': 'Mahindra&Mahindra',
 'doc_type': 'application/pdf',
 'file_urls': 'https://www.mahindra.com/resources/investor-reports/FY19/Announcements/CG-Report30092018.pdf',
 'filename': 'CG-Report30092018.pdf',
 'lastmodifier': 'Fri, 12 Oct 2018 16:34:19 GMT',
 'name': 'mahindra.com/CG-Report30092018.pdf',
 's3_path': 's3://scrapy-data-lseg/mahindra.com/CG-Report30092018.pdf'}
2019-01-02 08:05:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/resources/investor-reports/FY19/Announcements/press-release-treo.pdf> (referer: None)
2019-01-02 08:05:29 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://www.mahindra.com/resources/investor-reports/FY19/Announcements/press-release-treo.pdf> referred in <None>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_ascii_metadata at 0x7f1fe55d5510>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function sse_md5 at 0x7f1fe55d18c8>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function convert_body_to_file_like_object at 0x7f1fe55d5bf8>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_bucket_name at 0x7f1fe55d1840>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function generate_idempotent_uuid at 0x7f1fe55d1488>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function conditionally_calculate_md5 at 0x7f1fe55d17b8>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function add_expect_header at 0x7f1fe55d1d08>
2019-01-02 08:05:29 [botocore.handlers] DEBUG: Adding expect 100 continue header to request.
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:29 [botocore.utils] DEBUG: Updating URI from https://s3.amazonaws.com/scrapy-data-lseg/mahindra.com/press-release-treo.pdf to https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/press-release-treo.pdf
2019-01-02 08:05:29 [botocore.endpoint] DEBUG: Making request for OperationModel(name=PutObject) with params: {'url_path': '/scrapy-data-lseg/mahindra.com/press-release-treo.pdf', 'query_string': {}, 'url': 'https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/press-release-treo.pdf', 'headers': {'Cache-Control': 'max-age=172800', 'x-amz-acl': 'private', 'User-Agent': 'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Expect': '100-continue', 'Content-MD5': '9THQtxnYck7lWlVdosxmAw=='}, 'method': 'PUT', 'body': <_io.BytesIO object at 0x7f1fe4c66c50>, 'context': {'client_region': 'us-east-1', 'has_streaming_input': True, 'client_config': <botocore.config.Config object at 0x7f1fe54ad518>, 'auth_type': None, 'signing': {'region': 'eu-west-2', 'endpoint': 'https://s3.eu-west-2.amazonaws.com', 'bucket': 'scrapy-data-lseg'}}}
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event request-created.s3.PutObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f1fe54ad4a8>>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x7f1fe7bdb278>>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <function set_operation_specific_signer at 0x7f1fe55d1378>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-sign.s3.PutObject: calling handler <function fix_s3_host at 0x7f1fe571a730>
2019-01-02 08:05:29 [botocore.utils] DEBUG: Checking for DNS compatible bucket for: https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/press-release-treo.pdf
2019-01-02 08:05:29 [botocore.utils] DEBUG: URI updated to: https://s3-path/mahindra.com/press-release-treo.pdf
2019-01-02 08:05:29 [botocore.auth] DEBUG: Calculating signature using v4 auth.
2019-01-02 08:05:29 [botocore.auth] DEBUG: CanonicalRequest:
PUT
/mahindra.com/press-release-treo.pdf

cache-control:max-age=172800
content-md5:9THQtxnYck7lWlVdosxmAw==
host:scrapy-data-lseg.s3.eu-west-2.amazonaws.com
x-amz-acl:private
x-amz-content-sha256:UNSIGNED-PAYLOAD
x-amz-date:20190102T080529Z

cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date
UNSIGNED-PAYLOAD
2019-01-02 08:05:29 [botocore.auth] DEBUG: StringToSign:
AWS4-HMAC-SHA256
20190102T080529Z
20190102/eu-west-2/s3/aws4_request
aa13fd7c32f806d84889863a71f83406c96c0c0486a3e276111cac63da1555a0
2019-01-02 08:05:29 [botocore.auth] DEBUG: Signature:
286803261593a2eafeb41180b19be884578d41a2dfa12d6c86e30c02806bab87
2019-01-02 08:05:29 [botocore.endpoint] DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=PUT, url=https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com/mahindra.com/press-release-treo.pdf, headers={'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAJNGI6DYOK7QJUQBQ/20190102/eu-west-2/s3/aws4_request, SignedHeaders=cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date, Signature=286803261593a2eafeb41180b19be884578d41a2dfa12d6c86e30c02806bab87', 'Cache-Control': b'max-age=172800', 'X-Amz-Date': b'20190102T080529Z', 'Expect': b'100-continue', 'X-Amz-Content-SHA256': b'UNSIGNED-PAYLOAD', 'x-amz-acl': b'private', 'User-Agent': b'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Content-Length': '568688', 'Content-MD5': b'9THQtxnYck7lWlVdosxmAw=='}>
2019-01-02 08:05:29 [urllib3.util.retry] DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-01-02 08:05:29 [botocore.awsrequest] DEBUG: Waiting for 100 Continue response.
2019-01-02 08:05:29 [botocore.awsrequest] DEBUG: 100 Continue response seen, now sending request body.
2019-01-02 08:05:29 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): luoq61dim4.execute-api.eu-west-2.amazonaws.com:443
2019-01-02 08:05:29 [urllib3.connectionpool] DEBUG: https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com:443 "PUT /mahindra.com/press-release-treo.pdf HTTP/1.1" 200 0
2019-01-02 08:05:29 [botocore.parsers] DEBUG: Response headers: {'Date': 'Wed, 02 Jan 2019 08:05:30 GMT', 'x-amz-id-2': 'hKmR/J+UxTTfEj6D2q071d6ZoFtIFxcnEbZH7VX2UigzGz3OAM0aL6866xuT1hSjz7hFjFOECJ8=', 'Server': 'AmazonS3', 'ETag': '"f531d0b719d8724ee55a555da2cc6603"', 'x-amz-request-id': '0265063066D58B8C', 'Content-Length': '0'}
2019-01-02 08:05:29 [botocore.parsers] DEBUG: Response body:
b''
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f1fe54adef0>
2019-01-02 08:05:29 [botocore.retryhandler] DEBUG: No retry needed.
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:29 [urllib3.connectionpool] DEBUG: https://luoq61dim4.execute-api.eu-west-2.amazonaws.com:443 "POST /dev/ HTTP/1.1" 200 73
2019-01-02 08:05:29 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.mahindra.com/resources/investor-reports/FY19/Announcements/press-release-treo.pdf>
{'baseurl': 'http://www.mahindra.com',
 'companyname': 'Mahindra&Mahindra',
 'doc_type': 'application/pdf',
 'file_urls': 'https://www.mahindra.com/resources/investor-reports/FY19/Announcements/press-release-treo.pdf',
 'filename': 'press-release-treo.pdf',
 'lastmodifier': 'Thu, 15 Nov 2018 06:43:05 GMT',
 'name': 'mahindra.com/press-release-treo.pdf',
 's3_path': 's3://scrapy-data-lseg/mahindra.com/press-release-treo.pdf'}
2019-01-02 08:05:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/resources/investor-reports/FY19/Announcements/press-release-jawa.pdf> (referer: None)
2019-01-02 08:05:29 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://www.mahindra.com/resources/investor-reports/FY19/Announcements/press-release-jawa.pdf> referred in <None>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_ascii_metadata at 0x7f1fe55d5510>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function sse_md5 at 0x7f1fe55d18c8>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function convert_body_to_file_like_object at 0x7f1fe55d5bf8>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_bucket_name at 0x7f1fe55d1840>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function generate_idempotent_uuid at 0x7f1fe55d1488>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function conditionally_calculate_md5 at 0x7f1fe55d17b8>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function add_expect_header at 0x7f1fe55d1d08>
2019-01-02 08:05:29 [botocore.handlers] DEBUG: Adding expect 100 continue header to request.
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:29 [botocore.utils] DEBUG: Updating URI from https://s3.amazonaws.com/scrapy-data-lseg/mahindra.com/press-release-jawa.pdf to https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/press-release-jawa.pdf
2019-01-02 08:05:29 [botocore.endpoint] DEBUG: Making request for OperationModel(name=PutObject) with params: {'url_path': '/scrapy-data-lseg/mahindra.com/press-release-jawa.pdf', 'query_string': {}, 'url': 'https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/press-release-jawa.pdf', 'headers': {'Cache-Control': 'max-age=172800', 'x-amz-acl': 'private', 'User-Agent': 'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Expect': '100-continue', 'Content-MD5': 'hmtVd5RgjTZdbZoAxprtIQ=='}, 'method': 'PUT', 'body': <_io.BytesIO object at 0x7f1fe4c669e8>, 'context': {'client_region': 'us-east-1', 'has_streaming_input': True, 'client_config': <botocore.config.Config object at 0x7f1fe54ad518>, 'auth_type': None, 'signing': {'region': 'eu-west-2', 'endpoint': 'https://s3.eu-west-2.amazonaws.com', 'bucket': 'scrapy-data-lseg'}}}
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event request-created.s3.PutObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f1fe54ad4a8>>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x7f1fe7bdb278>>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <function set_operation_specific_signer at 0x7f1fe55d1378>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-sign.s3.PutObject: calling handler <function fix_s3_host at 0x7f1fe571a730>
2019-01-02 08:05:29 [botocore.utils] DEBUG: Checking for DNS compatible bucket for: https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/press-release-jawa.pdf
2019-01-02 08:05:29 [botocore.utils] DEBUG: URI updated to: https://s3-path/mahindra.com/press-release-jawa.pdf
2019-01-02 08:05:29 [botocore.auth] DEBUG: Calculating signature using v4 auth.
2019-01-02 08:05:29 [botocore.auth] DEBUG: CanonicalRequest:
PUT
/mahindra.com/press-release-jawa.pdf

cache-control:max-age=172800
content-md5:hmtVd5RgjTZdbZoAxprtIQ==
host:scrapy-data-lseg.s3.eu-west-2.amazonaws.com
x-amz-acl:private
x-amz-content-sha256:UNSIGNED-PAYLOAD
x-amz-date:20190102T080529Z

cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date
UNSIGNED-PAYLOAD
2019-01-02 08:05:29 [botocore.auth] DEBUG: StringToSign:
AWS4-HMAC-SHA256
20190102T080529Z
20190102/eu-west-2/s3/aws4_request
640d898cc8f741cbad086d479e94b1aa6227ce06dc1dee83679cf3df90e2c1f3
2019-01-02 08:05:29 [botocore.auth] DEBUG: Signature:
ecf1392bd44811275546cc0f65a0110010d7156fc7371dd37aff2d0f44949905
2019-01-02 08:05:29 [botocore.endpoint] DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=PUT, url=https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com/mahindra.com/press-release-jawa.pdf, headers={'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAJNGI6DYOK7QJUQBQ/20190102/eu-west-2/s3/aws4_request, SignedHeaders=cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date, Signature=ecf1392bd44811275546cc0f65a0110010d7156fc7371dd37aff2d0f44949905', 'Cache-Control': b'max-age=172800', 'X-Amz-Date': b'20190102T080529Z', 'Expect': b'100-continue', 'X-Amz-Content-SHA256': b'UNSIGNED-PAYLOAD', 'x-amz-acl': b'private', 'User-Agent': b'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Content-Length': '479832', 'Content-MD5': b'hmtVd5RgjTZdbZoAxprtIQ=='}>
2019-01-02 08:05:29 [urllib3.util.retry] DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-01-02 08:05:29 [botocore.awsrequest] DEBUG: Waiting for 100 Continue response.
2019-01-02 08:05:29 [botocore.awsrequest] DEBUG: 100 Continue response seen, now sending request body.
2019-01-02 08:05:29 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): luoq61dim4.execute-api.eu-west-2.amazonaws.com:443
2019-01-02 08:05:29 [urllib3.connectionpool] DEBUG: https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com:443 "PUT /mahindra.com/press-release-jawa.pdf HTTP/1.1" 200 0
2019-01-02 08:05:29 [botocore.parsers] DEBUG: Response headers: {'Date': 'Wed, 02 Jan 2019 08:05:30 GMT', 'x-amz-id-2': 'L/eWOCmV9eip6+++Xg3utSeT3ga3+wpPxcjYHGpL8CJFzA5HSJRVBRkBS5pDmFfUVKQNOrhDmEI=', 'Server': 'AmazonS3', 'ETag': '"866b557794608d365d6d9a00c69aed21"', 'x-amz-request-id': 'FC66C6A51C045905', 'Content-Length': '0'}
2019-01-02 08:05:29 [botocore.parsers] DEBUG: Response body:
b''
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f1fe54adef0>
2019-01-02 08:05:29 [botocore.retryhandler] DEBUG: No retry needed.
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:29 [urllib3.connectionpool] DEBUG: https://luoq61dim4.execute-api.eu-west-2.amazonaws.com:443 "POST /dev/ HTTP/1.1" 200 73
2019-01-02 08:05:29 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.mahindra.com/resources/investor-reports/FY19/Announcements/press-release-jawa.pdf>
{'baseurl': 'http://www.mahindra.com',
 'companyname': 'Mahindra&Mahindra',
 'doc_type': 'application/pdf',
 'file_urls': 'https://www.mahindra.com/resources/investor-reports/FY19/Announcements/press-release-jawa.pdf',
 'filename': 'press-release-jawa.pdf',
 'lastmodifier': 'Thu, 15 Nov 2018 06:44:11 GMT',
 'name': 'mahindra.com/press-release-jawa.pdf',
 's3_path': 's3://scrapy-data-lseg/mahindra.com/press-release-jawa.pdf'}
2019-01-02 08:05:29 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/resources/investor-reports/FY19/Announcements/mahindra-xuv-300.pdf> (referer: None)
2019-01-02 08:05:29 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://www.mahindra.com/resources/investor-reports/FY19/Announcements/mahindra-xuv-300.pdf> referred in <None>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_ascii_metadata at 0x7f1fe55d5510>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function sse_md5 at 0x7f1fe55d18c8>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function convert_body_to_file_like_object at 0x7f1fe55d5bf8>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_bucket_name at 0x7f1fe55d1840>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function generate_idempotent_uuid at 0x7f1fe55d1488>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function conditionally_calculate_md5 at 0x7f1fe55d17b8>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function add_expect_header at 0x7f1fe55d1d08>
2019-01-02 08:05:29 [botocore.handlers] DEBUG: Adding expect 100 continue header to request.
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:29 [botocore.utils] DEBUG: Updating URI from https://s3.amazonaws.com/scrapy-data-lseg/mahindra.com/mahindra-xuv-300.pdf to https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/mahindra-xuv-300.pdf
2019-01-02 08:05:29 [botocore.endpoint] DEBUG: Making request for OperationModel(name=PutObject) with params: {'url_path': '/scrapy-data-lseg/mahindra.com/mahindra-xuv-300.pdf', 'query_string': {}, 'url': 'https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/mahindra-xuv-300.pdf', 'headers': {'Cache-Control': 'max-age=172800', 'x-amz-acl': 'private', 'User-Agent': 'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Expect': '100-continue', 'Content-MD5': 'R6W0TBvQD7ZQ+IK8XSoAEw=='}, 'method': 'PUT', 'body': <_io.BytesIO object at 0x7f1fe4c7a468>, 'context': {'client_region': 'us-east-1', 'has_streaming_input': True, 'client_config': <botocore.config.Config object at 0x7f1fe54ad518>, 'auth_type': None, 'signing': {'region': 'eu-west-2', 'endpoint': 'https://s3.eu-west-2.amazonaws.com', 'bucket': 'scrapy-data-lseg'}}}
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event request-created.s3.PutObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f1fe54ad4a8>>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x7f1fe7bdb278>>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <function set_operation_specific_signer at 0x7f1fe55d1378>
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event before-sign.s3.PutObject: calling handler <function fix_s3_host at 0x7f1fe571a730>
2019-01-02 08:05:29 [botocore.utils] DEBUG: Checking for DNS compatible bucket for: https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/mahindra-xuv-300.pdf
2019-01-02 08:05:29 [botocore.utils] DEBUG: URI updated to: https://s3-path/mahindra.com/mahindra-xuv-300.pdf
2019-01-02 08:05:29 [botocore.auth] DEBUG: Calculating signature using v4 auth.
2019-01-02 08:05:29 [botocore.auth] DEBUG: CanonicalRequest:
PUT
/mahindra.com/mahindra-xuv-300.pdf

cache-control:max-age=172800
content-md5:R6W0TBvQD7ZQ+IK8XSoAEw==
host:scrapy-data-lseg.s3.eu-west-2.amazonaws.com
x-amz-acl:private
x-amz-content-sha256:UNSIGNED-PAYLOAD
x-amz-date:20190102T080529Z

cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date
UNSIGNED-PAYLOAD
2019-01-02 08:05:29 [botocore.auth] DEBUG: StringToSign:
AWS4-HMAC-SHA256
20190102T080529Z
20190102/eu-west-2/s3/aws4_request
78f8124eaad2af36c4331d8acd41bbb2552522779d2e5766f19b08a7554b86da
2019-01-02 08:05:29 [botocore.auth] DEBUG: Signature:
b0631b5f5c4a9b3b4bc62083c70af42e87885a4728021f9fb117d26571fc6580
2019-01-02 08:05:29 [botocore.endpoint] DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=PUT, url=https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com/mahindra.com/mahindra-xuv-300.pdf, headers={'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAJNGI6DYOK7QJUQBQ/20190102/eu-west-2/s3/aws4_request, SignedHeaders=cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date, Signature=b0631b5f5c4a9b3b4bc62083c70af42e87885a4728021f9fb117d26571fc6580', 'Cache-Control': b'max-age=172800', 'X-Amz-Date': b'20190102T080529Z', 'Expect': b'100-continue', 'X-Amz-Content-SHA256': b'UNSIGNED-PAYLOAD', 'x-amz-acl': b'private', 'User-Agent': b'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Content-Length': '656517', 'Content-MD5': b'R6W0TBvQD7ZQ+IK8XSoAEw=='}>
2019-01-02 08:05:29 [urllib3.util.retry] DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-01-02 08:05:29 [botocore.awsrequest] DEBUG: Waiting for 100 Continue response.
2019-01-02 08:05:29 [botocore.awsrequest] DEBUG: 100 Continue response seen, now sending request body.
2019-01-02 08:05:29 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): luoq61dim4.execute-api.eu-west-2.amazonaws.com:443
2019-01-02 08:05:29 [urllib3.connectionpool] DEBUG: https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com:443 "PUT /mahindra.com/mahindra-xuv-300.pdf HTTP/1.1" 200 0
2019-01-02 08:05:29 [botocore.parsers] DEBUG: Response headers: {'Date': 'Wed, 02 Jan 2019 08:05:30 GMT', 'x-amz-id-2': 'w0ZxR2RWrhIgP0bYMt45IoUQr1stqNrZuvn3v7QrNC+2V+GXOYjwmAFUSB/ZNbXMJv/UQ0lNIXw=', 'Server': 'AmazonS3', 'ETag': '"47a5b44c1bd00fb650f882bc5d2a0013"', 'x-amz-request-id': '7B3822CAAB280C5A', 'Content-Length': '0'}
2019-01-02 08:05:29 [botocore.parsers] DEBUG: Response body:
b''
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f1fe54adef0>
2019-01-02 08:05:29 [botocore.retryhandler] DEBUG: No retry needed.
2019-01-02 08:05:29 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:30 [urllib3.connectionpool] DEBUG: https://luoq61dim4.execute-api.eu-west-2.amazonaws.com:443 "POST /dev/ HTTP/1.1" 200 73
2019-01-02 08:05:30 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.mahindra.com/resources/investor-reports/FY19/Announcements/mahindra-xuv-300.pdf>
{'baseurl': 'http://www.mahindra.com',
 'companyname': 'Mahindra&Mahindra',
 'doc_type': 'application/pdf',
 'file_urls': 'https://www.mahindra.com/resources/investor-reports/FY19/Announcements/mahindra-xuv-300.pdf',
 'filename': 'mahindra-xuv-300.pdf',
 'lastmodifier': 'Wed, 19 Dec 2018 08:40:40 GMT',
 'name': 'mahindra.com/mahindra-xuv-300.pdf',
 's3_path': 's3://scrapy-data-lseg/mahindra.com/mahindra-xuv-300.pdf'}
2019-01-02 08:05:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.mahindra.com/resources/investor-reports/FY19/Earnings%20Update/MM-Q1FY19-Results-Pack.pdf> (referer: None)
2019-01-02 08:05:30 [scrapy.pipelines.files] DEBUG: File (downloaded): Downloaded file from <GET https://www.mahindra.com/resources/investor-reports/FY19/Earnings%20Update/MM-Q1FY19-Results-Pack.pdf> referred in <None>
2019-01-02 08:05:30 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_ascii_metadata at 0x7f1fe55d5510>
2019-01-02 08:05:30 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function sse_md5 at 0x7f1fe55d18c8>
2019-01-02 08:05:30 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function convert_body_to_file_like_object at 0x7f1fe55d5bf8>
2019-01-02 08:05:30 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function validate_bucket_name at 0x7f1fe55d1840>
2019-01-02 08:05:30 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:30 [botocore.hooks] DEBUG: Event before-parameter-build.s3.PutObject: calling handler <function generate_idempotent_uuid at 0x7f1fe55d1488>
2019-01-02 08:05:30 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function conditionally_calculate_md5 at 0x7f1fe55d17b8>
2019-01-02 08:05:30 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <function add_expect_header at 0x7f1fe55d1d08>
2019-01-02 08:05:30 [botocore.handlers] DEBUG: Adding expect 100 continue header to request.
2019-01-02 08:05:30 [botocore.hooks] DEBUG: Event before-call.s3.PutObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>
2019-01-02 08:05:30 [botocore.utils] DEBUG: Updating URI from https://s3.amazonaws.com/scrapy-data-lseg/mahindra.com/MM-Q1FY19-Results-Pack.pdf to https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/MM-Q1FY19-Results-Pack.pdf
2019-01-02 08:05:30 [botocore.endpoint] DEBUG: Making request for OperationModel(name=PutObject) with params: {'url_path': '/scrapy-data-lseg/mahindra.com/MM-Q1FY19-Results-Pack.pdf', 'query_string': {}, 'url': 'https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/MM-Q1FY19-Results-Pack.pdf', 'headers': {'Cache-Control': 'max-age=172800', 'x-amz-acl': 'private', 'User-Agent': 'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Expect': '100-continue', 'Content-MD5': 'wQU2My+M1ONQP4xZDcx15g=='}, 'method': 'PUT', 'body': <_io.BytesIO object at 0x7f1fe4cb6c50>, 'context': {'client_region': 'us-east-1', 'has_streaming_input': True, 'client_config': <botocore.config.Config object at 0x7f1fe54ad518>, 'auth_type': None, 'signing': {'region': 'eu-west-2', 'endpoint': 'https://s3.eu-west-2.amazonaws.com', 'bucket': 'scrapy-data-lseg'}}}
2019-01-02 08:05:30 [botocore.hooks] DEBUG: Event request-created.s3.PutObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f1fe54ad4a8>>
2019-01-02 08:05:30 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x7f1fe7bdb278>>
2019-01-02 08:05:30 [botocore.hooks] DEBUG: Event choose-signer.s3.PutObject: calling handler <function set_operation_specific_signer at 0x7f1fe55d1378>
2019-01-02 08:05:30 [botocore.hooks] DEBUG: Event before-sign.s3.PutObject: calling handler <function fix_s3_host at 0x7f1fe571a730>
2019-01-02 08:05:30 [botocore.utils] DEBUG: Checking for DNS compatible bucket for: https://s3.eu-west-2.amazonaws.com/scrapy-data-lseg/mahindra.com/MM-Q1FY19-Results-Pack.pdf
2019-01-02 08:05:30 [botocore.utils] DEBUG: URI updated to: https://s3-path/mahindra.com/MM-Q1FY19-Results-Pack.pdf
2019-01-02 08:05:30 [botocore.auth] DEBUG: Calculating signature using v4 auth.
2019-01-02 08:05:30 [botocore.auth] DEBUG: CanonicalRequest:
PUT
/mahindra.com/MM-Q1FY19-Results-Pack.pdf

cache-control:max-age=172800
content-md5:wQU2My+M1ONQP4xZDcx15g==
host:scrapy-data-lseg.s3.eu-west-2.amazonaws.com
x-amz-acl:private
x-amz-content-sha256:UNSIGNED-PAYLOAD
x-amz-date:20190102T080530Z

cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date
UNSIGNED-PAYLOAD
2019-01-02 08:05:30 [botocore.auth] DEBUG: StringToSign:
AWS4-HMAC-SHA256
20190102T080530Z
20190102/eu-west-2/s3/aws4_request
d64dbd4fdc394d3a9152d81bba7885f91c3e4cc813abdb8702d36f76df2e534e
2019-01-02 08:05:30 [botocore.auth] DEBUG: Signature:
80395d4512d73eaa1096e8d6dab1e41bfa262ef3ef0db61e7e1373639702c0a8
2019-01-02 08:05:30 [botocore.endpoint] DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=PUT, url=https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com/mahindra.com/MM-Q1FY19-Results-Pack.pdf, headers={'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAJNGI6DYOK7QJUQBQ/20190102/eu-west-2/s3/aws4_request, SignedHeaders=cache-control;content-md5;host;x-amz-acl;x-amz-content-sha256;x-amz-date, Signature=80395d4512d73eaa1096e8d6dab1e41bfa262ef3ef0db61e7e1373639702c0a8', 'Cache-Control': b'max-age=172800', 'X-Amz-Date': b'20190102T080530Z', 'Expect': b'100-continue', 'X-Amz-Content-SHA256': b'UNSIGNED-PAYLOAD', 'x-amz-acl': b'private', 'User-Agent': b'Botocore/1.11.8 Python/3.5.2 Linux/4.15.0-1021-aws', 'Content-Length': '791889', 'Content-MD5': b'wQU2My+M1ONQP4xZDcx15g=='}>
2019-01-02 08:05:30 [urllib3.util.retry] DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-01-02 08:05:30 [botocore.awsrequest] DEBUG: Waiting for 100 Continue response.
2019-01-02 08:05:30 [botocore.awsrequest] DEBUG: 100 Continue response seen, now sending request body.
2019-01-02 08:05:30 [urllib3.connectionpool] DEBUG: https://scrapy-data-lseg.s3.eu-west-2.amazonaws.com:443 "PUT /mahindra.com/MM-Q1FY19-Results-Pack.pdf HTTP/1.1" 200 0
2019-01-02 08:05:30 [botocore.parsers] DEBUG: Response headers: {'Date': 'Wed, 02 Jan 2019 08:05:31 GMT', 'x-amz-id-2': 'uoec5kvJRvZhwK4gMoHHY1LScIAk1UaZpIvumB1AlqblNAchl+5zPiD7iuA5codgrWshJePz6tI=', 'Server': 'AmazonS3', 'ETag': '"c10536332f8cd4e3503f8c590dcc75e6"', 'x-amz-request-id': 'DBD427D55BBC7ED1', 'Content-Length': '0'}
2019-01-02 08:05:30 [botocore.parsers] DEBUG: Response body:
b''
2019-01-02 08:05:30 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f1fe54adef0>
2019-01-02 08:05:30 [botocore.retryhandler] DEBUG: No retry needed.
2019-01-02 08:05:30 [botocore.hooks] DEBUG: Event needs-retry.s3.PutObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f1fe54adf28>>