Adding JavaScript modules to the web platform
One thing we’ve been meaning to do more of is tell our blog readers more about new features we’ve been working on across WHATWG standards. We have quite a backlog of exciting things that have happened, and I’ve been nominated to start off by telling you the story of <script type="module">.
JavaScript modules have a long history. They were originally slated to be finalized in early 2015 (as part of the “ES2015” revision of the JavaScript specification), but as the deadline drew closer, it became clear that although the syntax was ready, the semantics of how modules load each other were still up in the air. This is a hard problem anyway, as it involves extensive integration between the JavaScript engine and its “host environment”—which could be either a web browser, or something else, like Node.js.
The compromise that was reached was to have the JavaScript specification specify the syntax of modules, but without any way to actually run them. The host environment, via a hook called HostResolveImportedModule, would be responsible for resolving module specifiers (the "x" in import x from "x") into module instances, by executing the modules and fetching their dependencies. And so a year went by with JavaScript modules not being truly implementable in web browsers, as while their syntax was specified, their semantics were not yet.
In the epic whatwg/html#433 pull request, we worked on specifying these missing semantics. This involved a lot of deep changes to the script execution pipeline, to better integrate with the modern JavaScript spec. The WHATWG community had to discuss subtle issues like how cross-origin module scripts were fetched, or how/whether the async, defer, and charset attributes applied. The end result can be seen in a number of places in the HTML Standard, most notably in the definition of the script element and the scripting processing model sections. At the request of the Edge team, we also added support for worker modules, which you can see in the section on creating workers. (This soon made it over to the service workers spec as well!) To wrap things up, we included some examples: a couple for <script type="module">, and one for module workers.
Of course, specifying a feature is not the end; it also needs to be implemented! Right now there is active implementation work happening in all four major rendering engines, which (for the open source engines) you can follow in these bugs:
- Blink: issue 594639
- Gecko: bug 1240072
- WebKit: bug 148897
And there's more work to do on the spec side, too! There's ongoing discussion of how to add more advanced dynamic module-loading APIs, from something simple like a promise-returning self.importModule, all the way up to the experimental ideas being prototyped in the whatwg/loader repository.
We hope you find the addition of JavaScript modules to the HTML Standard as exciting as we do. And we'll be back to tell you more about other recent important changes to the world of WHATWG standards soon!
Hi,
that’s great achievement. Why type=”module” is used when MIME type is otherwise used for content of type attribute? This seems rather inconsistent.
Jirka
Awesome news! 🙂
I feel like I “get” the benefits of JS modules at the language level (declarative syntax, strict by default, no globals etc.).
I also feel I understand the role that current build-time bundling (webpack et. al) plays, from a performance perspective (consolidate separate JS scripts into a single HTTP fetch, tree shaking etc.).
What I don’t (yet) fully comprehend is: what is the end-game for script type=”module” with regards to web performance?
If I understand correctly, when a browser encounters a module script, it will recursively resolve all dependencies for the module tree; triggering (potentially) multiple HTTP fetches for the dependent resources.
In HTTP/1.x; best practice was to reduce the number of HTTP fetches via concatenating/bundling.
In HTTP/2.x, concatenation is somewhat of an anti-pattern (though there has been some research suggesting that, for compression reasons, *some* amount of packaging is still beneficial in HTTP2. See: http://engineering.khanacademy.org/posts/js-packaging-http2.htm).
Is the default position that script type=”module” will rely on HTTP/2.x efficiencies to make up for the absence of build-time packaging? (Perhaps with some smarts on the server to enable HTTP2 pushing of dependent resources before the browser actually asks for them).
Or is there still a role for build-time packaging with script type=”module”?
I appreciate that this is all still very new and currently being worked out, but something I’m yet to see are guidelines for developers on what to expect.
Something along the lines of “here are some best practices when using script type=”module”, that will ensure performance is maintained”.
I have nothing against the syntax but there’s no point in having a syntax if there’s no “real” implementation for it, aside from extremely inefficient naive ones. You only need a module system if you’re building big application. Nobody is actually complaining about using simple scripts right now for simple things (ie. things with only 4-10 functions). And this is just essentially bottom of the barrel as far as options would go for a big application.
This entire thing just makes the web slower since to move to this from bundles (especially on http1) involves a lot of needless blocking along with who knows how many load/wait/load/wait/load/wait instead of what should really just be load/done, as it is for things like webpack with use of require.ensure.
Why doesn’t the spec just include at least something simple like:
loadpath ui/* from /bundles/ui.js
loadpath lib/* from /bundles/lib.js
import Window from lib/ui/Window
import SigninPage from ui/SigninPage
That’s still primitive and inefficient compared to something like webpack which can produce the smallest unit for all use cases (and doesn’t require declarations to do it), but at least it’s not dead slow due to having no way of loading deep dependency trees with out an entire round trip for all unique files it has to bring out.
Looking at npm most modules have extremely deep dependency trees, so deep that windows users could be cosidered as having gotten an usable version of npm with the advent of npm3 that tries to flatten it out rather then go deep.
The examples in the spec use games as a use case. I’ve seen games built with RequireJS, which is very close to this model, and they run atrociously. Because what happens in practice, especially when trying to do pure MVC on the fronted (which I also recommend against due to the costs of the boilerplating), is constantly get unique models, views, controllers and their unique smaller dependencies, over and over. It’s slow enough to be measured in seconds. In large part because any “lag” is simply blown up by the dependency crawling. Some of that can be considered bad coding perhaps but there’s also something to be said of the loading strategy just not being usable.
Jirka:
The type attribute has never actually been related to MIME types. It was an attribute with two states:
We’ve simply added a third state, where the value “module” triggers “module script mode”.
This is also the way to get backward-compatibility, by preventing old browsers from interpreting module scripts as classic scripts.
This was discussed previously in the pull request at https://github.com/whatwg/html/pull/443#issuecomment-167639239.