Requiring Node Built-ins with Webpack

SHARE ON

Webpack is infamousfor being complicated, but it actually does quite a lot for you out of the box. Bundling a
Node/CommonJS module for the browser can be as easy as

webpack index.js bundle.js

But when you look at what Webpack has produced, you may find that it has gone too far: its automatic
shimming of Node built-ins can add hundreds of kilobytes of unused code to your bundle, and encourage
developers to use 3rd-party re-implementations of Node built-ins rather than perfectly good browser APIs.

The Problem

At a high level, what webpack index.js bundle.js does for you is convert index.js’ require calls
into a format that’ll work in the browser. If you have modules as such:

// salutation.js
module.exports = 'Hello';

// index.js
var Salutation = require('./salutation');
module.exports = Salutation + ' world!';

webpack index.js bundle.js will produce a file containing (in part):

function(module, exports, __webpack_require__) {
  var Salutation = __webpack_require__(1);
  module.exports = Salutation + " world!";
}

You can see how it has defined module for index.js, and replaced require('./salutation') with a
call to a __webpack_require__ function.

This is very simple and entirely unproblematic so long as your requires are for your own,
pure-TC39 modules. But what if you require a Node module like os?

Well, Webpack will automatically handle that too. I can’t find this documented
anywhere,
but if you require a Node built-in or use a Node global, Webpack will download a browser
shim for it and bundle that with your code. This lets
you do something like

var os = require('os');

// Webpack's shim for `os.platform` returns 'browser'.
if (os.platform() === 'browser') {
  console.log('in the browser');
} else {
  console.log('in Node');
}

However, Webpack’s test for whether you need the shim is painfully simple: if you use the Node
built-in anywhere in your file, Webpack will add its shim to your bundle—regardless of whether that
code will be evaluated.

This bit me when developing a cross-platform library that
made use of cryptographic functionality. Since I was only targeting modern browsers, I intended to
use Node’s crypto module in Node, but
SubtleCrypto in the browser. I
tried to accomplish this with the following code:

var os = require('os');
var IS_BROWSER = os.platform() === 'browser';

var crypto;
if (IS_BROWSER) {
  crypto = window.crypto;
} else {
  crypto = require('crypto');
}

Only to find that Webpack had still replaced the require('crypto') call… with
100kB of unused code.

Ouch.

Further compounding the problem was Webpack’s documentation. I eventually found two ways of
suppressing this behavior.

Solutions

First, you could mark crypto as an external:
in your webpack.config.js file (no more simple
command-line usage, alas), do:

module.exports = {
  externals: {
    'crypto': 'crypto'
  }
};

This means that Webpack will attempt to import crypto from the environment at runtime, rather than
bundling its definition: require('crypto') will end up executing code that looks like this:

function(module, exports) {
  module.exports = crypto;  // i.e. `window.crypto`
}

If crypto and SubtleCrypto had identical APIs, this could have actually let me use
require('crypto') in both Node and the browser. Unfortunately, they don’t (most notably, crypto’s
APIs are synchronous whereas SubtleCrypto uses promises), so I had to use if (IS_BROWSER)
conditionals throughout the module anyway. And given that, the external definition above was still a
bit of unused code.

The second way of suppressing the shim is to make Webpack aware that the !IS_BROWSER branch is
dead code. In webpack.config.js,
do:

module.exports = {
  plugins: [
    new webpack.DefinePlugin({
      IS_BROWSER: true
    })
  ]
};

Now, since IS_BROWSER is a compile-time constant,
Webpack will only crawl the true branches:

// This condition is `(typeof IS_BROWSER === 'undefined')` in Node, pre-compilation.
if (false) IS_BROWSER = false;

var crypto;
if (true) {
  crypto = window.crypto;
} else {
  crypto = require('crypto');
}

And the false branches can even be removed during minification!

Since this approach was truest to my intention, and the most efficient (no need for the os shim,
no “external” crypto definition, and with Node-specific code stripped during minification), this
is the approach I ended up taking.

Conclusion

I think automatic shimming is a reasonable default for Webpack. It should be obvious to developers
that something has to happen to make requiring a Node built-in work in the browser, and indeed I knew
that Webpack would shim os—I just didn’t think about it shimming crypto as well.

However, I think Webpack should be a lot more transparent about what it does. At a minimum, I think
it should document when it shims, what it shims, and how shimming can be disabled without developers
having to examine 3-year-old comment threads
and examples that talk only about 3rd-party browser libraries.

I also think that Webpack should suggest that developers use browser APIs where available rather than
relying on the shims. SubtleCrypto is almost equivalent to crypto, the former’s use of promises
aside—developers can save 100kB of JS, and worry less about correctness, by using the browser’s API
vs. a 3rd-party implementation.

If you agree with any/all of the above, please upvote/chime in on
https://github.com/webpack/webpack/issues/2871.

And if you’d like to work in open-source software and use cutting edge JS tech, however painful at
times
? , email careers@mixmax.com and let’s grab coffee!

SHARE ON

Written By

Jeff Wear

Jeff Wear

From Your Friends At