Requiring Node Built-ins with Webpack

Webpack is infamous for being complicated, but it actually does quite a lot for you out of the box. Bundling a Node/CommonJS module for the browser can be as easy as

webpack index.js bundle.js

But when you look at what Webpack has produced, you may find that it has gone too far: its automatic shimming of Node built-ins can add hundreds of kilobytes of unused code to your bundle, and encourage developers to use 3rd-party re-implementations of Node built-ins rather than perfectly good browser APIs.

The Problem

At a high level, what webpack index.js bundle.js does for you is convert index.js’ require calls into a format that’ll work in the browser. If you have modules as such:

// salutation.js
module.exports = 'Hello';

// index.js
var Salutation = require('./salutation');
module.exports = Salutation + ' world!';

webpack index.js bundle.js will produce a file containing (in part):

function(module, exports, __webpack_require__) {
  var Salutation = __webpack_require__(1);
  module.exports = Salutation + " world!";
}

You can see how it has defined module for index.js, and replaced require('./salutation') with a call to a __webpack_require__ function.

This is very simple and entirely unproblematic so long as your requires are for your own, pure-TC39 modules. But what if you require a Node module like os?

Well, Webpack will automatically handle that too. I can’t find this documented anywhere, but if you require a Node built-in or use a Node global, Webpack will download a browser shim for it and bundle that with your code. This lets you do something like

var os = require('os');

// Webpack's shim for `os.platform` returns 'browser'.
if (os.platform() === 'browser') {
  console.log('in the browser');
} else {
  console.log('in Node');
}

However, Webpack’s test for whether you need the shim is painfully simple: if you use the Node built-in anywhere in your file, Webpack will add its shim to your bundle—regardless of whether that code will be evaluated.

This bit me when developing a cross-platform library that made use of cryptographic functionality. Since I was only targeting modern browsers, I intended to use Node’s crypto module in Node, but SubtleCrypto in the browser. I tried to accomplish this with the following code:

var os = require('os');
var IS_BROWSER = os.platform() === 'browser';

var crypto;
if (IS_BROWSER) {
  crypto = window.crypto;
} else {
  crypto = require('crypto');
}

Only to find that Webpack had still replaced the require('crypto') call … with 100kB of unused code.

Ouch.

Further compounding the problem was Webpack’s documentation. I eventually found two ways of suppressing this behavior.

Solutions

First, you could mark crypto as an external: in your webpack.config.js file (no more simple command-line usage, alas), do:

module.exports = {
  externals: {
    'crypto': 'crypto'
  }
};

This means that Webpack will attempt to import crypto from the environment at runtime, rather than bundling its definition: require('crypto') will end up executing code that looks like this:

function(module, exports) {
  module.exports = crypto;  // i.e. `window.crypto`
}

If crypto and SubtleCrypto had identical APIs, this could have actually let me use require('crypto') in both Node and the browser. Unfortunately, they don’t (most notably, crypto’s APIs are synchronous whereas SubtleCrypto uses promises), so I had to use if (IS_BROWSER) conditionals throughout the module anyway. And given that, the external definition above was still a bit of unused code.

The second way of suppressing the shim is to make Webpack aware that the !IS_BROWSER branch is dead code. In webpack.config.js, do:

module.exports = {
  plugins: [
    new webpack.DefinePlugin({
      IS_BROWSER: true
    })
  ]
};

Now, since IS_BROWSER is a compile-time constant, Webpack will only crawl the true branches:

// This condition is `(typeof IS_BROWSER === 'undefined')` in Node, pre-compilation.
if (false) IS_BROWSER = false;

var crypto;
if (true) {
  crypto = window.crypto;
} else {
  crypto = require('crypto');
}

And the false branches can even be removed during minification!

Since this approach was truest to my intention, and the most efficient (no need for the os shim, no “external” crypto definition, and with Node-specific code stripped during minification), this is the approach I ended up taking.

Conclusion

I think automatic shimming is a reasonable default for Webpack. It should be obvious to developers that something has to happen to make requiring a Node built-in work in the browser, and indeed I knew that Webpack would shim os—I just didn’t think about it shimming crypto as well.

However, I think Webpack should be a lot more transparent about what it does. At a minimum, I think it should document when it shims, what it shims, and how shimming can be disabled without developers having to examine 3-year-old comment threads and examples that talk only about 3rd-party browser libraries.

I also think that Webpack should suggest that developers use browser APIs where available rather than relying on the shims. SubtleCrypto is almost equivalent to crypto, the former's use of promises aside—developers can save 100kB of JS, and worry less about correctness, by using the browser's API vs. a 3rd-party implementation.

If you agree with any/all of the above, please upvote/chime in on https://github.com/webpack/webpack/issues/2871.

And if you’d like to work in open-source software and use cutting edge JS tech, however painful at times 😉 , email careers@mixmax.com and let’s grab coffee!