The following changes have been done:

1) Removed HTTPClient dependency by moving to Requests.jl 2) Removed dependency on LibExpat by moving to LightXML.jl 3) Implementation of SQS APIs 4) Documentation update
JuliaCloud · Jan 14, 2016 · 9f5bc6d · 9f5bc6d
1 parent 0b8603c
commit 9f5bc6d
Show file tree

Hide file tree

Showing 17 changed files with 4,335 additions and 2,169 deletions.
diff --git a/README.md b/README.md
@@ -1,14 +1,20 @@
 ## AWS - Julia interface to Amazon Web Services
 
-This package is a WIP for providing a native Julia interface to the Amazon Web Services API
+This package provides a native Julia interface to the Amazon Web Services API
 
-Initially, the EC2 and S3 API will be supported.
+The following services are supported as of now:
+
+- EC2
+- S3
+- SQS
 
 
 ### Current status
-- Most of the APIs are yet untested. Any testing will be helpful. The REST API does not match exactly in certain cases
+- The basic APIs of EC2, S3 and SQS are tested
+- The advanced APIs (e.g. VPC related APIs, etc) of EC2 are yet untested. Any testing will be helpful. 
+- The REST API does not match exactly in certain cases
   with the WSDL. For the EC2 API, the bulk of the code is generated from the WSDL while it has been translated by hand for the
-  S3 API. In certain cases breakage may occur due to wrong request syntax.
+  S3 API.
 
   Please file issues on GitHub with the output from running the request in debug mode, i.e., with env.dbg = true.
 
@@ -22,6 +28,7 @@ type AWSEnv
     aws_seckey::ASCIIString     # AWS Secret key for signing requests
     aws_token::ASCIIString      # AWS Security Token for temporary credentials
     region::AbstractString      # region name
+    ep_scheme::ASCIIString      # URL scheme: http or https
     ep_host::AbstractString     # region endpoint (host)
     ep_path::AbstractString     # region endpoint (path)
     sig_ver::Int                # AWS signature version (2 or 4)
@@ -34,7 +41,7 @@ end
 Constructors:
 
 ```
-AWSEnv(; id=AWS_ID, key=AWS_SECKEY, token=AWS_TOKEN, ec2_creds=false, region=US_EAST_1, ep="", sig_ver=4, timeout=0.0, dr=false, dbg=false)
+AWSEnv(; id=AWS_ID, key=AWS_SECKEY, token=AWS_TOKEN, ec2_creds=false, scheme="https", region=AWS_REGION, ep="", sig_ver=4, timeout=0.0, dr=false, dbg=false)
 ```
 
 - The ```AWS_ID``` and ```AWS_SECKEY``` are initialized from env if available. Else a file ~/.awssecret is read (if available) for the same.
@@ -132,7 +139,7 @@ type S3Response
                             # it is parsed and set here.
                             # Else it will contain an IOBuffer object
 
-    pd::Union(ETree, Nothing)
+    pd::Union(LightXML.XMLElement, Nothing)
 end
 ```
 
@@ -147,7 +154,7 @@ EC2 has two sets of APIs
 
 ### EC2 Simple API
 
-- Currently the following are available:
+- Currently the following APIs are available:
 
 - ec2_terminate
 - ec2_launch
@@ -213,7 +220,7 @@ type EC2Response
     http_code::Int
     headers
     body::Union(String, Nothing)
-    pd::Union(ETree, Nothing)
+    pd::Union(LightXML.XMLElement, Nothing)
     obj::Any
 end
 ```
@@ -233,22 +240,319 @@ For succcessful requests, EC2Response.obj will contain an object of the appropri
 For example, for RunInstances, the EC2Response.obj will be of type RunInstancesResponseType
 
 
+### SQS API
+
+This package uses the REST interface of SQS
+
+- The type names, function names, etc follow the names specified in http://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/Welcome.html as well as in http://queue.amazonaws.com/doc/2012-11-05/QueueService.wsdl
+
+- Sample code
+
+```
+using AWS, AWS.SQS
+
+include("config.jl")
+
+
+env=AWSEnv(; id=id, key=key, dbg=dbg, region=region)
+## env=AWSEnv(; id=id, key=key, dbg=dbg)
+
+queueName="MyTest1"
+
+println("List all queues !!!")
+queues = SQS.ListQueues(env)
+
+println("Create queue")
+attributes = AttributeType[]
+push!(attributes, AttributeType(name="DelaySeconds",value="300"))
+push!(attributes, AttributeType(name="VisibilityTimeout",value="120"))
+resp = CreateQueue(env; queueName=queueName, attributeSet=attributes)
+if resp.http_code < 299 
+    println("Test for Create Queue Passed")
+else
+    println("Test for Create Queue Failed")
+end
+    
+qurl = resp.obj.queueUrl
+@show qurl
+
+println("Get queue attributes")
+resp = GetQueueAttributes(env; queueUrl=qurl, attributeNameSet=["All"])
+if resp.http_code < 299
+    println("Test for Get Queue Attributes Passed")
+else
+    println("Test for Get Queue Attributes Failed")
+end
+
+println("Send Message")
+msgAttributes = MessageAttributeType[]
+push!(msgAttributes, MessageAttributeType(name="some-attribute",
+  value=MessageAttributeValueType(dataType="Number.integer-ish", stringValue="0")))
+push!(msgAttributes, MessageAttributeType(name="other-attribute",
+  value=MessageAttributeValueType(dataType="String.yyy", stringValue="My yyy string")))
+push!(msgAttributes, MessageAttributeType(name="bin-attribute",
+  value=MessageAttributeValueType(dataType="Binary.jpg", binaryValue=[0x0,0x1,0x2,0x3,0x4,0x5,0x6])))
+resp = SendMessage(env; queueUrl=qurl, delaySeconds=0, messageBody="test", messageAttributeSet=msgAttributes)
+if resp.http_code < 299
+    println("Test for Send Message Passed")
+else
+    println("Test for Send Message Failed")
+end
+
+
+println("Receive Message")
+resp=ReceiveMessage(env; queueUrl=qurl, attributeNameSet=["All"], messageAttributeNameSet=["All"])
+if resp.http_code < 299
+    println("Test for Receive Message Passed")
+else
+    println("Test for Receive Message Failed")
+end
+
+msg = resp.obj.messageSet[1]
+msg_body = msg.body
+  # msg_body == "test"
+
+println("Delete Message")
+resp = DeleteMessage(env; queueUrl=qurl, receiptHandle=msg.receiptHandle)
+if resp.http_code < 299
+    println("Test for Delete Message Passed")
+else
+    println("Test for Delete Message Failed")
+end
+
+
+println("Send Message Batch")
+qurl=GetQueueUrl(env; queueName=queueName).obj.queueUrl
+## Test for SendMessageBatch
+id1 = "SendMessageBatchMessage1"
+delaySeconds1 = 10
+msgAttributes1 = MessageAttributeType[]
+push!(msgAttributes1, MessageAttributeType(name="some-attribute",
+  value=MessageAttributeValueType(dataType="Number.integer-ish", stringValue="111")))
+push!(msgAttributes1, MessageAttributeType(name="other-attribute",
+  value=MessageAttributeValueType(dataType="String.yyy", stringValue="My 111 string")))
+push!(msgAttributes1, MessageAttributeType(name="bin-attribute",
+  value=MessageAttributeValueType(dataType="Binary.jpg", binaryValue=[0x0,0x1,0x2,0x3,0x4,0x5,0x6])))
+messageBody1 = "Message Body 1"
+sendMessageBatchRequestEntryType1 = SendMessageBatchRequestEntryType(; delaySeconds=delaySeconds1, id=id1, messageAttributeSet=msgAttributes1, messageBody=messageBody1)
+
+id2 = "SendMessageBatchMessage2"
+delaySeconds2 = 10
+msgAttributes2 = MessageAttributeType[]
+push!(msgAttributes2, MessageAttributeType(name="some-attribute",
+  value=MessageAttributeValueType(dataType="Number.integer-ish", stringValue="222")))
+push!(msgAttributes2, MessageAttributeType(name="other-attribute",
+  value=MessageAttributeValueType(dataType="String.yyy", stringValue="My 222 string")))
+push!(msgAttributes2, MessageAttributeType(name="bin-attribute",
+  value=MessageAttributeValueType(dataType="Binary.jpg", binaryValue=[0x0,0x1,0x2,0x3,0x4,0x5,0x6])))
+messageBody2 = "Message Body 2"
+sendMessageBatchRequestEntryType2 = SendMessageBatchRequestEntryType(; delaySeconds=delaySeconds2, id=id2, messageAttributeSet=msgAttributes2, messageBody=messageBody2)
+
+
+id3 = "SendMessageBatchMessage3"
+delaySeconds3 = 10
+messageBody3 = "Message Body 3"
+sendMessageBatchRequestEntryType3 = SendMessageBatchRequestEntryType(; delaySeconds=delaySeconds3, id=id3, messageBody=messageBody3)
+
+id4 = "SendMessageBatchMessage4"
+delaySeconds4 = 10
+messageBody4 = "Message Body 4"
+sendMessageBatchRequestEntryType4 = SendMessageBatchRequestEntryType(; delaySeconds=delaySeconds4, id=id4, messageBody=messageBody4)
+
+sendMessageBatchRequestEntrySet = Vector{SendMessageBatchRequestEntryType}()
+push!(sendMessageBatchRequestEntrySet, sendMessageBatchRequestEntryType1)
+push!(sendMessageBatchRequestEntrySet, sendMessageBatchRequestEntryType2)
+push!(sendMessageBatchRequestEntrySet, sendMessageBatchRequestEntryType3)
+push!(sendMessageBatchRequestEntrySet, sendMessageBatchRequestEntryType4)
+
+sendMessageBatchType = SendMessageBatchType(; sendMessageBatchRequestEntrySet=sendMessageBatchRequestEntrySet, queueUrl=qurl)
+resp = SendMessageBatch(env, sendMessageBatchType)
+if resp.http_code < 299
+    println("Test for Send Message Batch Passed")
+else
+    println("Test for Send Message Batch Failed")
+end
+
+println("Sleeping for 20 secs !!!")
+sleep(20)
+
+println("Delete Message Batch")
+## Test case for DeleteMessageBatch
+qurl=GetQueueUrl(env; queueName=queueName).obj.queueUrl
+## resp1=ReceiveMessage(env; queueUrl=qurl, attributeNameSet=["All"], messageAttributeNameSet=["All"])
+resp1=ReceiveMessage(env; queueUrl=qurl)
+msg1 = resp1.obj.messageSet[1]
+## resp2=ReceiveMessage(env; queueUrl=qurl, attributeNameSet=["All"], messageAttributeNameSet=["All"])
+resp2=ReceiveMessage(env; queueUrl=qurl)
+msg2 = resp2.obj.messageSet[1]
+
+deleteMessageBatchRequestEntryType1 = DeleteMessageBatchRequestEntryType(; id=id1, receiptHandle=msg1.receiptHandle)
+deleteMessageBatchRequestEntryType2 = DeleteMessageBatchRequestEntryType(; id=id2, receiptHandle=msg2.receiptHandle)
+deleteMessageBatchRequestEntrySet = Vector{DeleteMessageBatchRequestEntryType}()
+push!(deleteMessageBatchRequestEntrySet, deleteMessageBatchRequestEntryType1)
+push!(deleteMessageBatchRequestEntrySet, deleteMessageBatchRequestEntryType2)
+
+deleteMessageBatchType = DeleteMessageBatchType(; deleteMessageBatchRequestEntrySet=deleteMessageBatchRequestEntrySet, queueUrl=qurl)
+resp = DeleteMessageBatch(env, deleteMessageBatchType)
+if resp.http_code < 299
+    println("Test for Delete Message Batch Passed")
+else
+    println("Test for Delete Message Batch Failed")
+end
+
+
+
+println("Change Message Visibility")
+## Test case for ChangeMessageVisibility
+qurl=GetQueueUrl(env; queueName=queueName).obj.queueUrl
+resp=ReceiveMessage(env; queueUrl=qurl)
+msg = resp.obj.messageSet[1]
+
+changeMessageVisibilityType = ChangeMessageVisibilityType(; queueUrl=qurl, receiptHandle=msg.receiptHandle, visibilityTimeout=30)
+resp = ChangeMessageVisibility(env, changeMessageVisibilityType)
+if resp.http_code < 299
+    println("Test for Change Message Visibility Passed")
+else
+    println("Test for Change Message Visibility Failed")
+end
+
+
+
+println("Change Message Visibility Batch")
+## Test case for ChangeMessageVisibilityBatch
+qurl=GetQueueUrl(env; queueName=queueName).obj.queueUrl
+resp1=ReceiveMessage(env; queueUrl=qurl)
+msg1 = resp1.obj.messageSet[1]
+id1=resp1.obj.messageSet[1].messageId
+
+resp2=ReceiveMessage(env; queueUrl=qurl)
+msg2 = resp2.obj.messageSet[1]
+id2=resp2.obj.messageSet[1].messageId
+
+changeMessageVisibilityBatchRequestEntryType1 = ChangeMessageVisibilityBatchRequestEntryType(; id=id1, receiptHandle=msg1.receiptHandle, visibilityTimeout=60)
+changeMessageVisibilityBatchRequestEntryType2 = ChangeMessageVisibilityBatchRequestEntryType(; id=id2, receiptHandle=msg2.receiptHandle, visibilityTimeout=90)
+changeMessageVisibilityBatchRequestEntrySet = Vector{ChangeMessageVisibilityBatchRequestEntryType}()
+push!(changeMessageVisibilityBatchRequestEntrySet, changeMessageVisibilityBatchRequestEntryType1)
+push!(changeMessageVisibilityBatchRequestEntrySet, changeMessageVisibilityBatchRequestEntryType2)
+
+changeMessageVisibilityBatchType = ChangeMessageVisibilityBatchType(; queueUrl=qurl, changeMessageVisibilityBatchRequestEntrySet=changeMessageVisibilityBatchRequestEntrySet)
+resp = ChangeMessageVisibilityBatch(env, changeMessageVisibilityBatchType)
+if resp.http_code < 299
+    println("Test for Change Message Visibility Batch Passed")
+else
+    println("Test for Change Message Visibility Batch Failed")
+end
+
+
+
+println("List Dead Letter Source Queues")
+## Test case for ListDeadLetterSourceQueues
+qurl=GetQueueUrl(env; queueName=queueName).obj.queueUrl
+resp = ListDeadLetterSourceQueues(env; queueUrl=qurl)
+if resp.http_code < 299
+    println("Test for List Dead Letter Source Queues Passed")
+else
+    println("Test for List Dead Letter Source Queues Failed")
+end
+
+
+println("Purge Queue")
+## Test case for PurgeQueue
+qurl=GetQueueUrl(env; queueName=queueName).obj.queueUrl
+resp = PurgeQueue(env; queueUrl=qurl)
+if resp.http_code < 299
+    println("Test for Purge Queue Passed")
+else
+    println("Test for Purge Queue Failed")
+end
+
+
+println("Add  Permission")
+## Test case for AddPermission
+qurl=GetQueueUrl(env; queueName=queueName).obj.queueUrl
+
+aWSAccountIdSet = Vector{ASCIIString}()
+actionNameSet = Vector{ASCIIString}()
+
+push!(aWSAccountIdSet, awsAccountID)
+push!(actionNameSet, "SendMessage")
+
+push!(aWSAccountIdSet, awsAccountID)
+push!(actionNameSet, "ReceiveMessage")
+
+resp = AddPermission(env; queueUrl=qurl, label="My Permission 1", aWSAccountIdSet=aWSAccountIdSet, actionNameSet=actionNameSet)
+if resp.http_code < 299
+    println("Test for Add Permission Passed")
+else
+    println("Test for Add Permission Failed")
+    println("Check if the AWS Account ID exists !!")
+end
+
+
+println("Remove  Permission")
+## Test case for RemovePermission
+qurl=GetQueueUrl(env; queueName=queueName).obj.queueUrl
+resp = RemovePermission(env; queueUrl=qurl, label="My Permission 1")
+if resp.http_code < 299
+    println("Test for Remove Permission Passed")
+else
+    println("Test for Remove Permission Failed")
+end
+
+
+println("Testing DeleteQueue")
+qurl=GetQueueUrl(env; queueName=queueName).obj.queueUrl
+DeleteQueue(env; queueUrl=qurl)
+if resp.http_code < 299
+    println("Test for Delete Queue Passed")
+else
+    println("Test for Delete Queue Failed")
+end
+
+
+
+```
+
+The response object is as below:
+
+```
+type SQSResponse
+    http_code::Int
+    headers
+    body::Union{AbstractString, Void}
+    pd::Union{LightXML.XMLElement, Void}
+    obj::Any
+end
+
+```
+
+The error object is as below:
+
+```
+type SQSError
+    typ::AbstractString
+    code::AbstractString
+    msg::AbstractString
+    detail::AbstractString
+    request_id::Union{AbstractString, Void}
+end
+```
 
 
 ### Julia Dependencies
 
 Calendar
 
-LibExpat
+Requests
 
-libCURL
+LightXML
 
 
 ### Binary dependencies
 
-libcurl must be installed
+libz must be installed
 
-libexpat must be installed
+libxml2 must be installed
 
 ### NOTE