This repository has been archived by the owner. It is now read-only.
Use AWS Lambda to run scalable server-side code from the web browser (NOTE: project no longer maintained)
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
docs/examples
lib errors from lambda-job should be displayed, even if debugging is off Dec 30, 2014
.gitignore added bower files Dec 28, 2014
.gitmodules
.npmignore added runcasperjs example Dec 30, 2014
LICENSE added license and git repo Dec 28, 2014
README.md project no longer maintained Jan 4, 2017
bower.json bower config added Dec 28, 2014
package.json package update Dec 28, 2014

README.md

LambdaJob

Version npmVersion bower

THIS PROJECT IS NO LONGER MAINTAINED -- Amazon has integrated this functionality straight into Lambda!

LambdaJob provides real-time invoking and completion detection of AWS Lambda functions from JavaScript on the clientside's web browser. This makes it easy to scale things like: scraping websites, thumbnail generation of images/videos, running custom binaries, etc.

Features

  • Lambdas triggered by S3 Object creation
  • Real-time results via SQS long-polling
  • No back-end needed, just an AWS account and public IAM permissions
  • Low-latency and performance focus throughout
  • Maximum job throttling (to stay under the 25 Lambda limit)
  • Shared SQS queue per client
  • Helper for calling commandline applications on Lambda

Usage

From the web browser:

var lambdaJob = new LambdaJobClient();
lambdaJob.invoke("bash", {cmd: "/sbin/ifconfig"}, function (err, output) {
  console.log(output);
});

The Lambda job:

var LambdaJob = require('lambda-job');
var lambdaJob = new LambdaJob.LambdaJobWorker(AWS, jobReceived);
exports.handler = lambdaJob.lambdaHandler;

function jobReceived(params, errDataCallback) {
  console.log("Will run: bash -c " + params.cmd + "...");
  lambdaJob.execHelper(params.cmd, function(err, consoleOutput) {
    errDataCallback(err, consoleOutput);
  }
}

Outputs in the web browser:

vsb_20    Link encap:Ethernet  HWaddr 26:3A:68:14:69:21  
          inet addr:192.168.20.21  Bcast:0.0.0.0  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:54 errors:0 dropped:0 overruns:0 frame:0
          TX packets:52 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:23982 (23.4 KiB)  TX bytes:10948 (10.6 KiB)
[...]

Options

  • debugLog: Turn debug logging on/off. Useful to turn on to see how LambdaJob works. (default: false)
  • maxActiveJobs: Throttle how many jobs can run at once. Note AWS has a hard limit of 25 active jobs at any one point. (default: 25)
  • jobTimeout: The maximum amount of time (in ms) that a job can run for. (default: 30000)
  • s3BucketPrefix: Since we create S3 objects for every desired job, the bucket they're inside of has this prefix. Note that S3 buckets are in a global namespace with other people. (default: lambda-job-)
  • sqsQueuePrefix: Like the s3BucketPrefix except for SQS queues. (default: lambda-job-)

Prerequisites

  1. Create an IAM user with the following permissions:

     {
       "Version": "2012-10-17",
       "Statement": [
         {
           "Sid": "Stmt1418615018000",
           "Effect": "Allow",
           "Action": [
             "s3:PutObject",
             "s3:PutObjectAcl",
             "s3:GetObject",
             "s3:DeleteObject",
             "sqs:CreateQueue",
             "sqs:DeleteQueue",
             "sqs:ReceiveMessage",
             "sqs:SendMessage",
             "sqs:DeleteMessage"
           ],
           "Resource": [
             "arn:aws:s3:::lambda-job-*",
             "arn:aws:sqs:us-west-2:YOURACCOUNTNUM:lambda-job-*"
           ]
         }
       ]
     }
    
  2. Create an S3 bucket named lambda-job-LAMBDANAME

  3. Add CORS permissions on the S3 bucket. Here's a CORS config sample:

     <?xml version="1.0" encoding="UTF-8"?>
     <CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
       <CORSRule>
         <AllowedOrigin>*</AllowedOrigin>
         <AllowedMethod>GET</AllowedMethod>
         <AllowedMethod>POST</AllowedMethod>
         <AllowedMethod>PUT</AllowedMethod>
         <AllowedMethod>DELETE</AllowedMethod>
         <AllowedMethod>HEAD</AllowedMethod>
         <MaxAgeSeconds>3000</MaxAgeSeconds>
         <AllowedHeader>*</AllowedHeader>
       </CORSRule>
     </CORSConfiguration>
    
  4. Create your Lambda and set this S3 bucket as your S3 source. It's recommended to use the full 1024MB of RAM since it seems things are faster this way.

  5. In your Lambda, upload your files to AWS, making sure you included the npm modules you'll need in the archive.

  6. Browse to index.html either where you've hosted it, or locally.

Examples

Before running any of these examples make sure you do the permissions related things above and also change the CHANGEME text in the index.html and index.js files to be your AWS keys.

  • bash shows how you can run custom binaries on Lambda. It also provides convenient access to explore the Lambda VM and what's available.
  • nomocors shows how you could use LambdaJob to proxy around CORS restrictions. Great for client-side automation.
  • runcasperjs shows how you can do web scriping from Lambda.