Webpack support #603

Closed
anthonator opened this Issue May 13, 2015 · 93 comments

Projects

None yet
@anthonator

Has anybody gotten this library to work with webpack? The webpack team recently fixed a ternary require issue which fixed a show stopper.

I was able to fix an issue with loading a JSON file with the json-loader. Now I'm getting:

WARNING in ./~/aws-sdk/lib/util.js
Critical dependencies:
40:30-45 the request of a dependency is an expression
43:11-53 the request of a dependency is an expression
 @ ./~/aws-sdk/lib/util.js 40:30-45 43:11-53

WARNING in ./~/aws-sdk/lib/api_loader.js
Critical dependencies:
13:15-59 the request of a dependency is an expression
104:12-46 the request of a dependency is an expression
108:21-58 the request of a dependency is an expression
114:18-52 the request of a dependency is an expression

Which is causing the individual services to not be included in the packaged bundle.

So doing new AWS.S3() returns a is not a function error.

It looks like you might be able to fix this with the ContextReplacementPlugin but I'm not sure how to configure it properly for this scenario.

Does anybody have any examples or anything?

@lsegal lsegal added the Question label May 14, 2015
@lsegal
Contributor
lsegal commented May 14, 2015

I think in this case a site like StackOverflow might be a better venue to ask this kind of question. The SDK unfortunately does not currently support Webpack, since we currently have our tooling working with browserify. It seems like this specific issue might be a lot more complicated to solve based on the error, but someone who knows more about the webpack tool may be of more assistance-- it might not be as complicated as it seems. If you do resolve this we'd be happy to hear how you were able to do it!

@AdityaManohar
Member

I'm marking this as closed since this isn't an SDK issue. Feel free to open another issue if you have other questions.

@jblack10101

I'm having this same issue. Was a stack overflow ever created?

@geminiyellow

@AdityaManohar @lsegal

+1 i want webpack support too.

WARNING in ./~/aws-sdk/lib/util.js
Critical dependencies:
40:30-45 the request of a dependency is an expression
43:11-53 the request of a dependency is an expression
 @ ./~/aws-sdk/lib/util.js 40:30-45 43:11-53

WARNING in ./~/aws-sdk/lib ^\.\/.*$
Module not found: Error: Cannot resolve directory '.' in /Users/me/Documents/Sources/my-project/client/node_modules/aws-sdk/lib
 @ ./~/aws-sdk/lib ^\.\/.*$

WARNING in ./~/aws-sdk/lib/api_loader.js
Critical dependencies:
13:15-59 the request of a dependency is an expression
104:12-46 the request of a dependency is an expression
108:21-58 the request of a dependency is an expression
114:18-52 the request of a dependency is an expression
 @ ./~/aws-sdk/lib/api_loader.js 13:15-59 104:12-46 108:21-58 114:18-52

WARNING in ./~/aws-sdk/lib/region_config.json
Module parse failed: /Users/me/Documents/Sources/my-project/client/node_modules/aws-sdk/lib/region_config.json Line 2: Unexpected token :
You may need an appropriate loader to handle this file type.
| {
|   "rules": {
|     "*/*": {
|       "endpoint": "{service}.{region}.amazonaws.com"
 @ ./~/aws-sdk/lib ^\.\/.*$

ERROR in ./~/aws-sdk/lib/api_loader.js
Module not found: Error: Cannot resolve module 'fs' in /Users/me/Documents/Sources/my-project/client/node_modules/aws-sdk/lib
 @ ./~/aws-sdk/lib/api_loader.js 1:9-22

ERROR in ./~/aws-sdk/lib/services.js
Module not found: Error: Cannot resolve module 'fs' in /Users/me/Documents/Sources/my-project/client/node_modules/aws-sdk/lib
 @ ./~/aws-sdk/lib/services.js 1:9-22

it feel so suck.

@ryanyogan

This issue isn't specific to this repo, having said that Webpack does not care what you throw at it, you may use Browserify transforms in Webpack.

This particular problem looks to be specific to JSON so try adding this to your Webpack config:

npm i --save-dev json-loader

module: {
  loaders: [
    { test: /\.json$/, loader: 'json' }
  ]
}

If that does not fix everything have a look at google "webpack transform packegify"

@rstormsf

Nope, it doesn't fix it. +1 for webpack support

@prescottprue

+1 for webpack support. This stack overflow question says it has been solved, but I am still seeing errors about Node related stuff.

The main errors include the following: Module not found: Error: Cannot resolve module 'fs' in .... If it is in a package that gets browserified before being imported, then I was able to get it to work (still haven't gotten it to work with a loader).

@rstormsf

The same here :-(

@dbernstein

+1 for webpack support. I'm seeing the same problem.

@geminiyellow

@prescottprue @rstormsf @dbernstein maybe what you need is no parse it.

webpack.conf.js

module: {
  noParse: [/aws-sdk.js/]
}
@dbernstein

Thanks @geminiyellow for your suggestion: I tried that - no luck. when I run npm install I still get the warnings below. When I try running my code none of the AWS api's are available ( javascript console error = "AWS.SNS is not a function"). It would appear that the messages below are related. If you have any other ideas I would appreciate it. I've been cycling on this for a few hours.

WARNING in .//aws-sdk/lib/util.js
Critical dependencies:
40:30-45 the request of a dependency is an expression
43:11-53 the request of a dependency is an expression
@ ./
/aws-sdk/lib/util.js 40:30-45 43:11-53

WARNING in .//aws-sdk/lib/api_loader.js
Critical dependencies:
13:15-59 the request of a dependency is an expression
104:12-46 the request of a dependency is an expression
108:21-58 the request of a dependency is an expression
114:18-52 the request of a dependency is an expression
@ ./
/aws-sdk/lib/api_loader.js 13:15-59 104:12-46 108:21-58 114:18-52

@prescottprue

I was able to get something working using a solution similiar to @ geminiyellow:

module:{
 noParse: [/aws-sdk/]
}

Notice that the regex is for the whole aws-sdk module folder, not just the main aws-sdk.js file (as the error was coming from parsing of other files).

@wesleyyee

+1

@prescottprue

Posted another solution

Basically:
Using the noParse method should work if you are creating a node package but not for umd format. To do umd I had to use loaders to Browserify and handle json files.

Install the loaders:
npm install json-loader --save-dev
npm install transform-loader brfs --save-dev

Webpack Config:

module: {
  loaders: [
    { test: /aws-sdk/, loaders: ["transform?brfs"]},
    { test: /\.json$/, loaders: ['json']},
  ]
}
@jkudish
jkudish commented Dec 27, 2015

This solution worked for me, but it would be great to see better webpack support in the future.

@dukedougal

Please provide webpack support.

@amowu
amowu commented Jan 4, 2016

+1

@jontewks

+1

@mdramos
mdramos commented Feb 8, 2016

+1 for webpack support.

@bendenoz

+1

@dukedougal

Is this likely to be reviewed? Webpack is fundamental now to the JavaScript community.

@chrisradek chrisradek added Feature Request and removed Question labels Feb 19, 2016
@chrisradek
Member

Webpack support is something we're looking into, but don't currently have an estimate on when work will be completed to support it. I'm reopening this as a feature request for now.

@chrisradek chrisradek reopened this Feb 19, 2016
@slessans

+1

@bendenoz

this provides the best work-around so far : http://andrewhfarmer.com/aws-sdk-with-webpack/

basically

require('aws-sdk/dist/aws-sdk');
var AWS = window.AWS;

instead of require('aws-sdk')

@joscha
Contributor
joscha commented Mar 3, 2016

You can also use a loader to make that a bit cleaner and have no global exposed:

var prePackagedAwsPath = path.join('aws-sdk', 'dist', 'aws-sdk');

module.exports = {
  resolve: {
      alias: {
          'aws-sdk': prePackagedAwsPath
      }
  },
  module: {
      loaders: [
          {
              test: require.resolve(prePackagedAwsPath),
              loader: 'imports?window=>null'
          }
      ]
  },

then you can just use

const AWS = require('aws-sdk');
@rotemtam rotemtam referenced this issue in apex/apex Mar 9, 2016
Open

Babel runtime #217

@EloB
EloB commented Mar 31, 2016

+1

@mhahn
mhahn commented Apr 8, 2016

This is so painful...

@dukedougal

It's really hard to understand why one of the most popular build tools is not supported. Surely this justifies priority action.

@chrisradek
Member
chrisradek commented Apr 20, 2016 edited

#603 (comment) had a very good suggestion, though it looks like it doesn't include any of the service clients.

By supplying the transform that the SDK uses with browserify (https://github.com/aws/aws-sdk-js/blob/master/dist-tools/transform.js) instead, it looks like the SDK will work properly with webpack. I had to hack around to actually have our transform used instead of brfs, but it shouldn't be too difficult if we can release our transform as a separate npm module.

This would also mean that you could define process.env.AWS_SERVICES to contain the list of services you want included with the SDK, which may result in a smaller SDK footprint.

So, to include an SDK that has just S3, my webpack.config.js file for my application would look something like this:

process.env.AWS_SERVICES = 's3'; // optional
module.exports = {
    entry: './src/index.js',
    output: {
        path: 'builds',
        filename: 'bundle.js'
    },
    // Relevant config starts here
    node: {
        fs: 'empty'
    },
    module: {
        loaders: [
            {
                test: /aws-sdk/,
                loaders: [
                    'transform?aws'
                ] 
            },
            {
                test: /\.json$/, loaders: ['json']
            }
        ]
    },
};

Because we use browserify for our browser builder, and the way the SDK is currently architected, it will take a lot of work to update the SDK so that it feels more 'natural' to use with webpack. That said, how would everyone feel about including configuration like the above, at the very least as an interim solution.

@chrisradek
Member

Just to follow up, the above configuration can actually be used today with a modification to what gets passed to the transform loader.

First, you need to make sure you npm install the json-loader and transform loader:
npm install json-loader --save-dev
npm install transform-loader --save-dev

Then assuming you've also npm installed the aws-sdk, the config would look like this:

process.env.AWS_SERVICES = 's3'; // optional
module.exports = {
    entry: './src/index.js',
    output: {
        path: 'builds',
        filename: 'bundle.js'
    },
    // Relevant config starts here
    node: {
        fs: 'empty'
    },
    module: {
        loaders: [
            {
                test: /aws-sdk/,
                loaders: [
                    'transform?aws-sdk/dist-tools/transform'
                ] 
            },
            {
                test: /\.json$/, loaders: ['json']
            }
        ]
    },
};

There will be a few warnings generated but they should be safely ignored. Both warnings are irrelevant when using the SDK in the browser with the above transform.

Again, I know it isn't the ideal way for the SDK to work with webpack, but I hope this provides an interim solution until we can make it better.

@bergben
bergben commented May 24, 2016

+1

@mikedizon

+1

@alfonsodev
alfonsodev commented May 28, 2016 edited

@chrisradek what would be the solution for running it in node.js ?

@Iuriy-Budnikov

+1

@pruett
pruett commented Jun 24, 2016

Not sure if this is useful, but I'm using webpack's exports-loader and can access the AWS variable locally within the js file where it was imported. Note, the AWS object is still attached to the window object. I tried using the imports loader to mitigate that, but wasn't successful. No further webpack config needed.

npm i -D exports-loader

index.js

import AWS from 'exports?AWS!aws-sdk/dist/aws-sdk'
console.log(AWS) // => Object {...}
@basarat
basarat commented Jul 27, 2016

I tried a few ways, here is my webpack setup with reasons:

  • Want to redirect aws-sdk to aws-sdk/dist/aws-sdk
 resolve: {
    alias: {
      'aws-sdk': 'aws-sdk/dist/aws-sdk'
    }
  },
  • Next aws-sdk doesn't do a nice export just take its local AWS variable and make it the export:
// npm install exports-loader --save-dev

{
  test: /aws-sdk.js/,
  loader: 'exports?AWS'
}
  • Finally webpack will complain that don't use prebuilt files so tell webpack not to parse this sdk file:
noParse: [
  /aws-sdk.js/
],

Done 🌹 Now you can import * as AWS from "aws-sdk" both in the Browser and NodeJS

@prescottprue

Did the same thing as @basarat separately and it works.

@deviantony

Is it possible to only import S3 service using @basarat solution?

@basarat
basarat commented Aug 8, 2016

@deviantony nope 🌹 You'd need to make a custom build ❤️

@preetsethi

I am attempting to use the @basarat solution but I get the following:
aws-sdk.js?085a:9867 Uncaught TypeError: Cannot read property 'crypto' of undefined
Anyone seen this or know how to resolve?

@konsumer

I don't think this is webpack-specific. I am having similar issues trying to pull out just what I need in browserify, and I suspect AMD (requirejs) people will too. The interface exposed externally is a giant monkey-patched AWS object with lots of internal dependencies that aren't being tracked. I think the first step is to figure out the right require order/placement for every file and make a top-level require that pulls in all the parts. That could be fed to browserify in this project to generate UMD, but also give developers like me, who just want a few things, a hint as to what needs to be pulled in, to just use some subset

I think this is the start of that, but because of the dynamic stuff in api_loader it's throwing off everyone who needs a static dependency tree (direct import in ES6, webpack, browserify without brfs, etc.)

If that dynamic stuff is really needed (I'm not sure it is) I'd recommend a separate build-step that generates static require-lists, instead, and keep the generated files in this repo.

@aj-ptw aj-ptw referenced this issue in EmergingTechnologyAdvisors/node-serialport Aug 17, 2016
Closed

Tracking serialport with webpack #901

@jagi
jagi commented Aug 17, 2016

I don't think that it was resolved as I'm still getting this error.
I'm trying to use the create-react-app cli with aws-sdk. Just trying to import AWS from "aws-sdk" but I'm getting the following error

Error in ./~/aws-sdk/lib/api_loader.js
Module not found: Error: Cannot resolve module 'fs' in /Users/jagi/workspace/react/client/node_modules/aws-sdk/lib
resolve module fs in /Users/jagi/workspace/react/client/node_modules/aws-sdk/lib
  looking for modules in /Users/jagi/workspace/react/client/node_modules/aws-sdk/node_modules
    /Users/jagi/workspace/react/client/node_modules/aws-sdk/node_modules/fs doesn't exist (module as directory)
    resolve 'file' fs in /Users/jagi/workspace/react/client/node_modules/aws-sdk/node_modules
      resolve file
        /Users/jagi/workspace/react/client/node_modules/aws-sdk/node_modules/fs doesn't exist
        /Users/jagi/workspace/react/client/node_modules/aws-sdk/node_modules/fs.js doesn't exist
        /Users/jagi/workspace/react/client/node_modules/aws-sdk/node_modules/fs.json doesn't exist
  looking for modules in /Users/jagi/workspace/react/client/node_modules
    /Users/jagi/workspace/react/client/node_modules/fs doesn't exist (module as directory)
    resolve 'file' fs in /Users/jagi/workspace/react/client/node_modules
      resolve file
        /Users/jagi/workspace/react/client/node_modules/fs doesn't exist
        /Users/jagi/workspace/react/client/node_modules/fs.js doesn't exist
        /Users/jagi/workspace/react/client/node_modules/fs.json doesn't exist
[/Users/jagi/workspace/react/client/node_modules/aws-sdk/node_modules/fs]
[/Users/jagi/workspace/react/client/node_modules/aws-sdk/node_modules/fs]
[/Users/jagi/workspace/react/client/node_modules/aws-sdk/node_modules/fs.js]
[/Users/jagi/workspace/react/client/node_modules/aws-sdk/node_modules/fs.json]
[/Users/jagi/workspace/react/client/node_modules/fs]
[/Users/jagi/workspace/react/client/node_modules/fs]
[/Users/jagi/workspace/react/client/node_modules/fs.js]
[/Users/jagi/workspace/react/client/node_modules/fs.json]
 @ ./~/aws-sdk/lib/api_loader.js 1:9-22

Error in ./~/aws-sdk/lib/services.js
Module not found: Error: Cannot resolve module 'fs' in /Users/jagi/workspace/react/client/node_modules/aws-sdk/lib
resolve module fs in /Users/jagi/workspace/react/client/node_modules/aws-sdk/lib
  looking for modules in /Users/jagi/workspace/react/client/node_modules/aws-sdk/node_modules
    /Users/jagi/workspace/react/client/node_modules/aws-sdk/node_modules/fs doesn't exist (module as directory)
    resolve 'file' fs in /Users/jagi/workspace/react/client/node_modules/aws-sdk/node_modules
      resolve file
        /Users/jagi/workspace/react/client/node_modules/aws-sdk/node_modules/fs doesn't exist
        /Users/jagi/workspace/react/client/node_modules/aws-sdk/node_modules/fs.js doesn't exist
        /Users/jagi/workspace/react/client/node_modules/aws-sdk/node_modules/fs.json doesn't exist
  looking for modules in /Users/jagi/workspace/react/client/node_modules
    /Users/jagi/workspace/react/client/node_modules/fs doesn't exist (module as directory)
    resolve 'file' fs in /Users/jagi/workspace/react/client/node_modules
      resolve file
        /Users/jagi/workspace/react/client/node_modules/fs doesn't exist
        /Users/jagi/workspace/react/client/node_modules/fs.js doesn't exist
        /Users/jagi/workspace/react/client/node_modules/fs.json doesn't exist
[/Users/jagi/workspace/react/client/node_modules/aws-sdk/node_modules/fs]
[/Users/jagi/workspace/react/client/node_modules/aws-sdk/node_modules/fs]
[/Users/jagi/workspace/react/client/node_modules/aws-sdk/node_modules/fs.js]
[/Users/jagi/workspace/react/client/node_modules/aws-sdk/node_modules/fs.json]
[/Users/jagi/workspace/react/client/node_modules/fs]
[/Users/jagi/workspace/react/client/node_modules/fs]
[/Users/jagi/workspace/react/client/node_modules/fs.js]
[/Users/jagi/workspace/react/client/node_modules/fs.json]
 @ ./~/aws-sdk/lib/services.js 1:9-22
@dukedougal

It's challenging to understand why Amazon doesn't fix Webpack support.

@prescottprue

@jagi and @dukedougal did you try the solution posted by @basarat? I have used this same method to implement aws-sdk in tons of projects.

It makes more sense to setup a more specific webpack config rather than modify how this library works just for "supporting webpack". This is a nodejs library, so it makes sense that it uses node specific packages like fs, you just need to make sure that your build config (webpack) knows about this and makes the appropriate adjustments.

@devhyunjae

+1

@dukedougal
dukedougal commented Aug 18, 2016 edited

@prescottprue it should work out of the box.

AWS should work seamlessly with the major JavaScript build systems. Users should not need to initially try and fail to make it work then somehow know to dig down in github and to know that one specific comment by one guy is the way to make it work. Amazon should fix it and they haven't more than a year after the initial report on this thread.

This thread is labelled as "feature request" but its really a bug. No milestone, no-one assigned.

@simonbuchan

I've had success with the default entrypoint and ignoring require() calls with module.exprContext*, which doesn't include any services, so I use aws-sdk/dist-tools/service-collector to generate a services definition module:

// build-aws-services.js
const collector = require('aws-sdk/dist-tools/service-collector');

module.exports = `var AWS = require('aws-sdk');
${collector(process.env.AWS_SERVICES)
  .replace(/\brequire\('\.\//g, 'require(\'aws-sdk/lib/')}
`;

Then just add the output to the entry dynamically with val-loader:

// See `node -p 'require("aws-sdk/lib/api_loader").services'`
const services = 'cognitoidentityserviceprovider,cognitoidentity,firehose';
process.env.AWS_SERVICES = services;

module.exports = {
  entry: {
    app: [
      'regenerator-runtime/runtime',
      'val!./build-aws-services', // Enhance AWS with services from AWS_SERVICES
      './src',
    ],
  },
  module: {
    // Ignore require(expr) in aws-sdk
    exprContextRegExp: /$^/,
    exprContextCritical: false,
// ...
@jagi
jagi commented Aug 19, 2016

@prescottprue I can use solution proposed by @basarat because I'm using create-react-app tool which does not allow modifying webpack config. Actually it does but it would require "ejecting" project and that will make it harder later to update to newest versions of react.

@ajmurmann

@jagi I also am trying to use this with create-react-app and would prefer not to eject. Where you able to solve this?

@simonbuchan

See aws/amazon-cognito-identity-js#117 where I have to add a bunch of extra stuff in the docs for webpack usage that looks a bit weird to have in another aws project :)

@alfonsodev

@prescottprue The title of the repo is "AWS SDK for JavaScript in the browser and Node.js http://aws.amazon.com/javascript" Description of the repo "The official AWS SDK for JavaScript, available for browsers and mobile devices, or Node.js backends" so I don't agree this is just a nodejs library.

Aws JS team should listen the needs of users and give support to Webpack out of the box, or write simple instructions in the main Readme. The same problem is happening in other libraries like aws-iot-device-sdk
I'm don't want to be negative, I think this could be solved if the AWS Javascript team meets and decide a general approach to take in all the JS packages to give true universal support when possible for all Javascript engines, (nodejs, browser, react-native). And adds the information to the Readme.

@jagi
jagi commented Aug 22, 2016

@ajmurmann for now I've just setup endpoints and I'm making http requests. It's temporary solution util aws-sdk support for webpack will be fixed.

@chrisradek chrisradek self-assigned this Aug 29, 2016
@chrisradek
Member

Hello everyone,
Given all the feedback, we're actively looking at how to make the experience of using the AWS SDK with webpack better.

First, PR #1116 adds a sample webpack configuration to the SDK's README to help users get started. Thank you @basarat for providing the configuration you use!

In the short-term, we're looking into how we can make the SDK work 'out of the box' with webpack without making any breaking changes. The goal here is to make the browser SDK work, but not necessarily support custom builds (additional webpack configuration would be necessary for that.)

To fully be able to support webpack and other bundlers, we will have to make some breaking changes. We're actively looking into what is possible with a major version bump, but don't have a timeline for that yet.

@VladShcherbin

It would be great to have a possibility to import only needed parts of the library in the project.

Currently, you can import the whole library. So, after adding s3 functionality to your 300kb project file, it becomes 1.3mb.

@chrisradek
Member

I've posted pr #1117 as a WIP for webpack support out of the box. I've done some testing, including with the create-react-app project to verify the SDK is being bundled and works correctly.

It still needs further testing but given the amount of demand, I wanted to post something as soon as possible to give those who want a chance to start playing with it.

@simonbuchan simonbuchan referenced this issue in aws/amazon-cognito-identity-js Aug 30, 2016
Closed

ES6 and npm usage #133

@chrisradek
Member

@VladShcherbin
The latest changes in the PR allow for pulling in individual services.

In your code, instead of writing something like:

var AWS = require('aws-sdk');
var s3 = new AWS.S3();

You could write something like:

var S3 = require('aws-sdk/browser/s3');
var s3 = new S3();

Then tools like webpack and browserify will only pull in S3 and the SDK core.

If you're not using a bundler, you could continue creating custom builds of the SDK using the browser builder.

This isn't live yet, but the code is available in the PR.

@VladShcherbin

@chrisradek sounds awesome, will give it a try this week. Thanks! 👍

@dukedougal
dukedougal commented Sep 1, 2016 edited

Any guesses as to approx when this might become available in the released version?

thanks

@chrisradek
Member

@dukedougal
You can npm install specific git branches.
So for this PR, you could run:
npm install git://github.com/chrisradek/aws-sdk-js.git#webpack
This is still a work in progress, so the code isn't completely finalized yet.

You'll still need the generic json-loader in your webpack config, but that should be it. I checked, and the create-react-app that people reported using includes this by default.

For example, this is the webpack config I was testing with:

var path = require('path');


module.exports = {
    entry: [
        path.join(process.cwd(), 'app', 'index.js')
    ],

    output: {
        path: path.join('dist'),
        filename: 'aws-bundle.js'
    },
    module: {
        loaders: [
            {
                test: /\.json$/, 
                loaders: ['json-loader']
            },
        ]
    }
}
@daumann
daumann commented Sep 6, 2016 edited

@chrisradek
When following the solutions in this thread, the script fails (with webpack) with the error message:
Error: Missing credentials in config(…) "CredentialsError: Missing credentials in config

But I have both the shared credentials file in ~/aws/credentials as well as environment variables set up.

Only when I hardcode the credentials it works... (why ?)

@jagi
jagi commented Sep 6, 2016 edited

I'm trying to use solution by @chrisradek but I can't make it work on the server. Everything works properly on the client. On the server I'm using aws-sdk API in Lambda function. I have a project using webpack that creates bundle which I upload to Lambda function. And when using aws-sdk I get the following error:

{
    "errorMessage": "Cannot find module \"./region_config.json\"",
    "errorType": "Error",
    "stackTrace": [
        "Object.<anonymous> (/var/task/build/handler.js:23392:182)",
        "__webpack_require__ (/var/task/build/handler.js:21:30)",
        "Object.<anonymous> (/var/task/build/handler.js:22843:21)",
        "__webpack_require__ (/var/task/build/handler.js:21:30)",
        "Object.<anonymous> (/var/task/build/handler.js:17663:2)",
        "__webpack_require__ (/var/task/build/handler.js:21:30)",
        "Object.<anonymous> (/var/task/build/handler.js:16601:12)",
        "Object.module.exports.module.exports (/var/task/build/handler.js:16624:31)",
        "__webpack_require__ (/var/task/build/handler.js:21:30)"
    ]
}

The handler.js file is a bundle file. Here is a fragment of the handler.js:16624:31 file in which is error (the last line)

    /* WEBPACK VAR INJECTION */(function(process) {var util = __webpack_require__(165);

    // browser specific modules
    util.crypto.lib = __webpack_require__(286);
    util.Buffer = __webpack_require__(287).Buffer;

    // browser stubs
    util.nodeRequire = function nodeRequire(mod) {};

    var AWS = __webpack_require__(166);

    // Load browser API loader
    AWS.apiLoader = function(svc, version) {
      return AWS.apiLoader.services[svc][version];
    };

    /**
     * @api private
     */
    AWS.apiLoader.services = {};

    // Load the DOMParser XML parser
    AWS.XML.Parser = __webpack_require__(296);

    // Load the XHR HttpClient
    __webpack_require__(297);

    if (typeof process === 'undefined') {
      process = {
        browser: true
      };
    }
    /* WEBPACK VAR INJECTION */}.call(exports, __webpack_require__(164)))
@chrisradek
Member

@daumann
The solutions provided in this thread only allow the browser version of the SDK to be used with webpack. In the browser, the SDK won't be able to access the credentials on your hard drive, or via environment variables.

There is still more work to be done to allow the SDK to support webpack for node bundles.

@chrisradek
Member

@jagi
When creating your webpack bundle, are you requiring the SDK completely, or are you pulling in specific services? Are you configuring webpack to build a node.js bundle?

@jagi
jagi commented Sep 6, 2016

@chrisradek I'm requiring entire library. What do you mean by "to build a node.js bundle". It's building project into one file/bundle and my output webpack is:

  output: {
    path: paths.serverAppBuild,
    filename: 'handler.js',
    library: 'lib',
    libraryTarget: 'commonjs2'
  },

And I've tested it works properly if I'm not using aws-sdk.

@chrisradek
Member

@jagi
In webpack, you can specify a target By default the target is web, which causes webpack to look at the browser field in the package.json file. In the SDK's case, this essentially means you'll be getting the browser version of the SDK.

For Lambda, you won't want the browser version of the SDK. The PR only covers supporting the browser version of the SDK with webpack right now, so if you change the target to node, you'll likely face some issues still. I'll work on updating the PR to also support webpack/browserify when setting the target to node.

@jagi
jagi commented Sep 6, 2016 edited

@chrisradek oh I see... I didn't know about the target option. I've changed it and now getting another error Cannot find module 'domain'. but it's probably a problem you're talking about.

Sorry for probably stupid question, but why it's not working with webpack? At the beginning I though that the problem is using import statement but even when I use require in webpack it's still not working. So is it like webpack take the aws-sdk library and just exports it but library is just not prepared to be used with webpack? And actually why it's not webpack compatible? There are many other libraries that when being created, haven't been designed to work with webpack but they work.

Do you know how much time will it take you to make this change?

@chrisradek
Member

@jagi
The SDK doesn't already work in webpack/browserify for node.js projects largely due to the way API models are consumed. The models are loaded dynamically, when many of these tools require static imports (or custom configuration) to work properly. In the PR, for the browser use-case the APIs are now imported statically. We'll need to do something similar to get this working for node.js as well.

@jagi
jagi commented Sep 6, 2016

@chrisradek oh I see. Thanks for explanation. Do you know when can it be ready? Is it a big feature to implement?

@chrisradek
Member

@jagi
Since we've gotten a couple issues now regarding how to use browserify/webpack with the SDK for node.js projects, I'll try to include that with the existing PR. I can't say with certainty when it will be ready, but I plan to work on that this week. Hopefully we can borrow a lot of what we did to support the browser version to also support the node.js version.

@dukedougal

I need the server version for node.js, not browser.

@deviantony

Got the following issues when trying to use @chrisradek aws-sdk branch:

ERROR in ./~/aws-sdk/browser/s3.js
Module not found: Error: Cannot resolve 'file' or 'directory' ../apis/s3-2006-03-01.min in /.../website/web/node_modules/aws-sdk/browser
 @ ./~/aws-sdk/browser/s3.js 8:45-81

ERROR in ./~/aws-sdk/browser/s3.js
Module not found: Error: Cannot resolve 'file' or 'directory' ../apis/s3-2006-03-01.paginators in /.../website/web/node_modules/aws-sdk/browser
 @ ./~/aws-sdk/browser/s3.js 9:56-99

ERROR in ./~/aws-sdk/browser/s3.js
Module not found: Error: Cannot resolve 'file' or 'directory' ../apis/s3-2006-03-01.waiters2 in /.../website/web/node_modules/aws-sdk/browser
 @ ./~/aws-sdk/browser/s3.js 10:53-94

I'm using the following code in my app:

import S3 from 'aws-sdk/browser/s3';

const s3 = new S3({...});
...
@chrisradek
Member

@deviantony
I'll be putting up a new PR shortly that should address some problems. In your webpack config, are you specifying a json loader like in the above comment:
#603 (comment)

@deviantony

@chrisradek yeap, I'm using a json loader. Thanks for your work !

@chrisradek
Member

@deviantony Do you mind sharing your config? With my latest changes I've been testing with a pretty bare bones config, for web-based and node-based projects. If you can share what yours looks like I can expand my test cases.

@deviantony

@chrisradek here's the webpack config I've used:

var path = require('path');
var webpack = require('webpack');
var autoprefixer = require('autoprefixer');

module.exports = {
  entry: [
    'webpack-dev-server/client?http://0.0.0.0:8080',
    'webpack/hot/only-dev-server',
    './src/index.jsx',
  ],

  output: {
    path: path.join(__dirname, 'public'),
    filename: 'bundle.js',
    publicPath: '/',
  },

  plugins: [
    new webpack.NoErrorsPlugin(),
  ],

  resolve: {
    extensions: ['', '.js', '.jsx'],
  },

  module: {
    loaders: [
      // Javascript & React JSX files
      {
        test: /\.jsx?$/,
        loaders: ['react-hot', 'babel?presets[]=react,presets[]=es2015'],
        include: path.join(__dirname, 'src'),
        exclude: /node_modules/,
      },
      {
        test: /\.scss$/,
        loader: 'style-loader!css-loader!sass-loader!postcss-loader',
      },
      // Auth0-Lock Build
      {
        test: /node_modules[\\\/]auth0-lock[\\\/].*\.js$/,
        loaders: ['transform-loader/cacheable?brfs',
        'transform-loader/cacheable?packageify',
      ],
    }, {
      test: /node_modules[\\\/]auth0-lock[\\\/].*\.ejs$/,
      loader: 'transform-loader/cacheable?ejsify',
    }, {
      test: /\.json$/,
      loader: 'json',
    },
    // Bootstrap CSS
    {
      test: /\.css$/,
      loader: 'style-loader!css-loader',
    }, {
      test: /\.(woff|woff2)(\?v=\d+\.\d+\.\d+)?$/,
      loader: 'url?limit=10000&mimetype=application/font-woff',
    }, {
      test: /\.ttf(\?v=\d+\.\d+\.\d+)?$/,
      loader: 'url?limit=10000&mimetype=application/octet-stream',
    }, {
      test: /\.eot(\?v=\d+\.\d+\.\d+)?$/,
      loader: 'file',
    }, {
      test: /\.svg(\?v=\d+\.\d+\.\d+)?$/,
      loader: 'url?limit=10000&mimetype=image/svg+xml',
    },
    {
      test: /\.less$/,
      loader: 'style!css!less',
    },
    {
      test: /\.(jpe?g|png|gif|svg)$/,
      loader: 'file?hash=sha512&digest=hex&name=[hash].[ext]!image-webpack?bypassOnDebug&optimizationLevel=7&interlaced=false',
    },
  ],
  postLoaders: [
    { test: /\.jsx?$/, loader: 'transform?envify' },
  ],
},
postcss: function postCss() {
  return [autoprefixer];
},
devServer: {
  historyApiFallback: true,
  contentBase: 'public/',
},
};
@simonbuchan
simonbuchan commented Sep 8, 2016 edited

@deviantony @chrisradek Looks like #1117 needs .json in (edit) resolve.extensions right now, which it probably shouldn't. Workaround is to add it for now, but requires of .json files should include the extension.

@chrisradek
Member

@simonbuchan Could that be because @deviantony's loader is json? In my config above, mine is json-loader (I'm using json-loader@0.5.4)

@chrisradek
Member

@simonbuchan
Weird, so for whatever reason, using json-loader works in my test, but just json breaks. If I add the .json extension to all my imports then both options work. Either way, it doesn't hurt to add the extension, that was just unexpected behavior.

@simonbuchan

@chrisradek Weird? Unless you've changed your resolveLoaders.moduleTemplates, that should only happen if you only have a json-loader-loader module!

@chrisradek
Member

I've created #1123 that contains all the changes needed to support webpack. It also allows webpack and browserify to be used to generate node bundles as well. Like the other PR, it also allows for importing individual services.

I incorporated some of the feedback from the previous PR and resolved some merge conflicts. There shouldn't be anymore major changes, and I think we've settled on the method for requiring services individually.

@deviantony

@chrisradek I just retried with npm install git://github.com/chrisradek/aws-sdk-js.git#webpack, is it updated yet? As I still have the same issue.

@chrisradek
Member

@deviantony
Apologies, I created a different branch when resolving some merge conflicts and adding support for node bundles.
npm install git://github.com/chrisradek/aws-sdk-js.git#full-webpack should work.

@deviantony

@chrisradek

This works perfectly:

import S3 from 'aws-sdk/clients/s3';

const s3 = new S3({...});

Amazing job 👏

@chrisradek chrisradek added a commit that closed this issue Sep 9, 2016
@chrisradek chrisradek Adds webpack support for browser and nodejs projects.
Enables browserify to include SDK in nodejs projects.

Fixes #603
Fixes #696
de21c81
@chrisradek chrisradek closed this in de21c81 Sep 9, 2016
@chrisradek chrisradek added a commit that referenced this issue Sep 9, 2016
@chrisradek chrisradek Tag release v2.6.0
References:
  #1114, #1123, #603, #696
8fd2060
@chrisradek
Member

The PR has been merged and released as part of 2.6.0. We'll work on updating our documentation on how to use webpack/browserify.

@danbrianwhite
danbrianwhite commented Sep 9, 2016 edited

Heads up to other devs

If you start getting webpack build errors from ./~/aws-sdk/lib/util.js and have something like

{
                test: /aws-sdk/,
                loaders: ['transform?brfs']
 }

in your webpack config, remove it and the build will work again.

@rameshsubramanian

@chrisradek I keep getting TypeError: Cannot read property 'crypto' of undefined error. Anyone else facing the issue?

Thanks
Ramesh

@chrisradek
Member

@rameshsubramanian
Which version of the SDK are you using?

If you're having trouble getting the SDK to work with webpack, take a look at these blog posts that walk through a simple example and see if they help:
Using webpack and the AWS SDK for JavaScript to Create and Bundle an Application - Part 1
Using webpack and the AWS SDK for JavaScript to Create and Bundle an Application - Part 2

@manubhat90 manubhat90 referenced this issue in erikras/react-redux-universal-hot-example Nov 3, 2016
Open

Integrating the aws-sdk #1312

@nickbreaton

@rameshsubramanian I was still getting this when I was using babel-loader. If you are also using babel, and easy fix is to ensure you have exclude: /node_modules/ in your loader configuration.

{ test: /\.js$/,   loader: 'babel-loader', exclude: /node_modules/ }
@kejsiStruga

Hello, I am using react and I am trying to connect to DynamoDB via aws-sdk. How could this be accomplished ? I have tried the above methods, still cannot set up credentials for dynamo. This works on Javascript but not on React !

import AWS from 'aws-sdk/dist/aws-sdk';
AWS.config.loadFromPath('./config.json');
@chrisradek
Member

@kejsiStruga
The browser doesn't have access to the file system, so AWS.config.loadFromPath doesn't work when your project is run in a browser. There are a couple of options for getting credentials loaded in the browser SDK, with Cognito being the preferred way in most cases. Here's some documentation that goes into setting up credentials for the browser SDK:
http://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/setting-credentials-browser.html

Also, since version 2.6.0 of the SDK, it is no longer necessary to import the SDK from aws-sdk/dist/aws-sdk. This was a temporary solution that had cons such as webpack being unable to do any 'tree-shaking' to optimize the bundles it generates, and having to use complicated webpack config.
Here are the docs that explain how to use the SDK with webpack (you can simply import AWS from 'aws-sdk' now):
http://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/webpack.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment