Please leave your sense of logic at the door, thanks!

First set of Review Drafts published

(If you’re interested in the IPR status of WHATWG standards this post is for you; otherwise, feel free to skip.)

One aspect of last year's working mode changes is the periodic publication of a Review Draft for each WHATWG Living Standard, as per the IPR Policy. The WHATWG published an initial five of these just now:

To get notified of future Review Draft publications you need to watch the Living Standard’s corresponding GitHub repository. You’ll find these repositories linked from the top of the standard.

Then you can filter the emails generated by watching this repository by looking for the subject-line prefix “Review Draft Publication:” which the WHATWG will consistently use as per the Maintainer Guidelines.

Watching the repository is the only way to get notified going forward. This blog post serves as an initial announcement and as a way to make everyone familiar with the process going forward.

Thanks for reading through three non-technical posts in a row. It will get technical again soon!

Posted in WHATWG | No Comments »

Changing the copyright license for Living Standards

The recent changes to the WHATWG are designed to help the web-standards development community work together across the broadest possible spectrum of developers, implementers, and end-users. As part of this change, we decided to move our standards from being licensed under the CC0 public domain dedication to the CC BY license.

This decision was not made lightly. We valued the alignment of CC0 with the original public domain dedication of the WWW project; the ease with which it allows remixing and refactoring of standards; and the way in which it allows copying of snippets or examples into software that implements or uses the standards.

While CC0 is beneficial in some ways, it also has some disadvantages. The lack of an attribution requirement may seem convenient, but it can also cause confusion in the standards community. For example, there are derivative specifications ("forks") that are not prominently identified as copies of WHATWG standards. Such unattributed forks create confusion both about the origin of the standard and about any applicable intellectual property rights (IPR) commitments. Use of the CC BY license makes it easier for implementers to trace what they are implementing back to the original source and its accompanying IPR commitments.

Additionally, CC0 is not approved by the Open Source Initiative. Although their reasoning is not very applicable in the context of the WHATWG, which has its own system for patent grants, this lack of approval from a prominent organization involved in defining "open source" can discourage participation.

Moving to CC BY addresses these disadvantages and also allows for a reduction in license proliferation. We were able to replace the custom license of the HTML Standard, which was a source of some confusion, with CC BY. CC BY still allows modification and redistribution, but adds on the relatively modest burden of attribution. We believe that, on balance, this change is the best path forward for the community.

Posted in WHATWG | Comments Off on Changing the copyright license for Living Standards

Further working mode changes

The WHATWG has been going great since it began in 2004, but without participation from all the browser engine implementers, was only partially meeting its goals. Over the last year, engineers and attorneys from the organizations behind Blink, Edge, Gecko, and WebKit have worked together to find a way forward that works for all the stakeholders.

What’s happening?

The organizations behind the four major integrated browser engines — Apple, Google, Microsoft, and Mozilla — have developed an Intellectual Property Rights (IPR) Policy and governance structure for the WHATWG. This enables more people to collaborate on Living Standards.

The WHATWG day-to-day activities can mostly proceed as-is. Legalities are kept to a minimum and do not disrupt WHATWG’s culture of pragmatic collaboration. Very briefly:

Why is this happening?

The WHATWG has operated successfully since 2004 with no formal governance structure, guided by a strong culture of pragmatism and collaboration. Although this has worked well for driving the web forward, we realized that we could get broader participation by being clear about what rights and responsibilities members of the community have. Concretely, this involves creating an IPR Policy and governance structure.

To stay true to our culture, we’ve created the most effective legal and governance policies we think can serve that purpose. The new Steering Group is empowered to implement the policies, address problems that arise, and to modify them to minimize problems in the future.

Overview of new structure and operations

The WHATWG remains a community. Contributions are invited from all, with no membership or fees. Communication is conducted in public, with proposals judged by their technical merit, the community, and implementers.

A noticeable change is a request to make a legally binding promise to offer a royalty-free license on any relevant patents, via our new Contributor and Workstream Participant Agreement. Functionally, this process is similar to signing a CLA when contributing to an open-source project. Once you’ve done that, we hope participants will notice the changes to WHATWG mainly as an exception handling mechanism, and not part of the day-to-day workflow.

To give employers and individuals a fair chance to review the agreement, the IPR Policy, and their patents, there's an initial grace period. Until January 11th, 2018, you can continue to participate and contribute even if you haven't yet signed the agreement. After the grace period is over, you'll have to sign the agreement to continue to participate. Of course, you're welcome (and encouraged!) to sign early.

There are a number of documents describing all this in detail:

The Steering Group may modify these documents by strong consensus or supermajority vote if necessary.


What do I need to do to participate in the WHATWG now?

See our new participation page! (Effectively, you or your employer will need to sign the Contributor and Workstream Participant Agreement.)

What effect will this have on the WHATWG’s standing as a standards organization?

It closes a gap, as standards organizations are expected to have an IPR Policy and governance structure. And with more implementers on board, it should also reduce the confusion around which version of a standard to use.

Who controls the WHATWG?

The community working there. Living Standards are informed by input from contributors, driven by workstream participants, articulated by editors, and coordinated by the Steering Group. If necessary, controversies are resolved by the Steering Group with members appointed from the organizations that develop browser engines. Substantive technical objections are considered and resolved by the Steering Group, consistent with the principles of the WHATWG.

The WHATWG continues to develop standards as a public community, with input from all. Anyone can contribute to standards here. Even with the Steering Group in place, editors still adhere to the Working Mode, making a judgment call as to whether a change or addition will have multi-implementer support. The main practical difference is that now there is a formal appeals path, in case a Workstream Participant disagrees with the editor’s judgment around multi-implementer support. Additionally, the founding members have helped the community put in place a legal framework to promote royalty-free licensing for Living Standards.

Does the WHATWG operate by consensus?

The WHATWG strives for rough, informal consensus among contributors when drafting Living Standards. After considering input from all parties, the editor of a Living Standard makes the judgment as to whether a feature has enough support behind it to include. Those disagreeing with the editor's judgment can, under what we hope are exceptional circumstances, appeal to the Steering Group, which does have a formal consensus policy.

What are Review Drafts and how do they relate to Living Standards?

The Living Standard is the living, changing standard with the most recent feedback incorporated. This is the document that developers and implementers should use, and that other standards organizations should normatively reference. Review Drafts are used by attorneys to determine if any patent exclusions are necessary.

Features in Living Standards that are controversial will be clearly labeled with a warning. They will be omitted from the next Review Draft if the controversy cannot be resolved in time.

I believe there are problems with a Living Standard, but the Editor does not agree. What can I do?

Present evidence that the standard does not accurately describe how browser engines interoperate today or how they will work in the near future, and convince other workstream participants to support your change proposal. If there is sustained disagreement in the workstream, appeal to the Steering Group to resolve the dispute.

I was not able to engage with the WHATWG productively in the past. Why should I try again?

The WHATWG now has a clearly documented Working Mode that makes the process more transparent and less surprising. Additionally, everyone is expected to abide by the Code of Conduct.

Can other organizations become part of the Steering Group?

If additional integrated browser engines beyond Blink, Edge, Gecko, and WebKit get significant mindshare and market share, the Steering Group is empowered to invite the organizations behind them to sign the Steering Group Agreement and participate there. This fits with the primary role of the Steering Group in resolving disagreements about whether something will be widely implemented in leading browser engines; apart from that, organizations in the community all participate in the same way.

Posted in WHATWG | Comments Off on Further working mode changes

Progressing Streams

Back in 2014 we announced the Streams Standard. It's about time for an update on where we are and what's coming up.

Streaming the response to fetch() via the response.body attribute was standardized last year and is now implemented in several major browsers. Recently streaming uploads have been added to the Fetch Standard. fetch(url, {method: 'POST', body: readable}) will start an upload. The expected properties of a streaming function apply:

Service Workers are another area where Streams are indispensable. They are used internally to allow the page to start processing bytes as soon as any are available, and are being used by developers as a powerful tool to synthesize responses. A Response object can be constructed with a ReadableStream body and then passed to fetchEvent.respondWith() like any other response.

The real power of Streams is unlocked when sources, sinks and transforms from disparate authors are combined in novel ways. The most exciting action is not in platform built-ins but in streams created in the wider developer ecosystem. With this in mind, every aspect of the standard has been fine-tuned for productivity. Take the following example:

let appendChildWritableStream = new WritableStream({
  write(domNode) {

Notice what isn't there:

With a recent browser you can see this in action in a live demo (video). At time of writing, this demo and the others in this post work in Chrome stable version 59 and Safari stable version 10.1.

The WritableStream API is now stable. We've added a getWriter() method which is the analogue of ReadableStream's getReader(). It adds locking semantics so that multiple writers cannot interfere with each other. Recent work has focused on predictability, for example by preventing underlying sink methods from running concurrently, and robustness, like dealing with badly-behaving strategy size functions that call into other methods reentrantly.

The strength of the algorithmic style of specification is that even unintended behavior will be the same between implementations. On the other hand, when specifying the pipeTo() method of ReadableStream, providing latitude for browsers to optimize was a high priority. As well as bypassing JavaScript when copying data between built-in streams, user agents may need to change the timing or ordering of calls to underlying methods to get the best performance for their architecture. For this reason, we specified pipeTo() in a requirements style. This presents its own challenges, for example how to specify the "least work" that an implementation can do and still be compliant.

Streams also challenge our fundamental assumptions about how the web platform works. You may not want to have to modify the DOM directly if you already have a template engine producing HTML. Shouldn't you be able to pipe a stream of HTML to an element?

We don't yet know how this capability would fit in the web platform, but Jake Archibald has created a custom element providing a compelling vision of what we could do with it. The demo demonstrates inserting a stream of HTML directly from the server.

Depending on your environment, you may have seen some significant jank in that demo. The problem is that the server supplies data faster than the browser can layout and render it. This is where backpressure comes in. Any data sink can apply backpressure just by returning a promise from its write() method. In many cases this happens as a natural consequence of the implementation. In this case, we want to delay until the browser has had a chance to render the HTML. A slight modification to the custom element and the page becomes much smoother: demo (side-by-side video).

It's clear that we should prioritize interactivity when adding content to an existing page. Maybe browsers need a special low-jank path for streaming HTML. But what about initial page load? You've probably seen pages that didn't respond to input because they were still performing some expensive layout below the fold. Should we prioritize interactivity there, too? We're still working through all the implications.

Ensuring low friction for all participants has really helped drive the progress of the standard. From bug fixes to the BYOB readable byte stream design from implementers, to large scale contributions from external contributors, the benefits of the community process are clear.

Transform streams are the final key piece needed to make the stream ecosystem complete. We have a working, tested reference implementation that we are using as the basis for active design discussions. Full standardization and implementer adoption is expected to follow in the next few months.

In past two years streams have gone from being a promising idea to having multiple independent implementations and wide adoption. Implementation work is accelerating, and there is already a critical mass of shipping functionality.

We're looking to widen developer involvement with Streams. Check out the examples, contribute some web platform tests, or help improve the documentation.

Posted in What's Next | Comments Off on Progressing Streams

The Developer’s Edition of HTML makes a comeback

Back in 2011, Ben Schwarz took on the ambitious project of curating an edition of the HTML Standard specifically for web developers. It omitted details aimed specifically at browser vendors, and had several additional features to make the experience more pleasant to read.

Ben did an amazing job maintaining this for many years, but some time ago it fell behind the changes to the HTML Standard. Since the move to make HTML more community-driven, we've been hoping to find a way to synchronize the developer's edition with the mainstream specification. That day has finally arrived!

We've deployed an initial version of the new developer's edition at a new URL, It's rough around the edges, missing several of the features of the old version. And it needs some curation to omit implementer-specific sections; many have crept in during the downtime. We're tracking these and other issues in the issue tracker. But now, the developer's edition is integrated into our build process and editing workflow, and will forever remain synchronized with the HTML Standard itself.

Hereby we issue a call to the community to help us with the revitalized developer's edition. Two of the biggest areas of potential improvement are helping us properly mark up the source according to the guidelines for what goes in the developer's edition, and contributing to the design of the developer's edition in order to make it more beautiful and usable.

Finally, I want to thank Michael™ Smith for getting this process started, via a series of pull requests to our build tools which did most of the foundational work. And of course Ben Schwarz, without whom none of this would have happened in the first place.

Posted in Tutorials, What's Next | Comments Off on The Developer’s Edition of HTML makes a comeback