Author Archive
Tuesday, January 26th, 2010
Welcome back to “What’s Next in HTML,” where I’ll try to summarize the major activity in the ongoing standards process in the WHAT Working Group. With HTML5 in Last Call, the WHATWG has moved to an unversioned development model for HTML. While browser vendors are busy implementing HTML5, let’s talk about what’s next.
The big news in HTML this week is r1643. ... Well, technically that revision is over 20 months old, but there have been a flurry of updates that affect the underlying feature. What feature, you might ask? Sandboxing untrusted content.
The sandbox
attribute, when specified [on an <iframe>
element], enables a set of extra restrictions on any content hosted by the iframe. ... When the attribute is set, the content [hosted by the iframe] is treated as being from a unique origin, forms and scripts are disabled, links are prevented from targeting other browsing contexts, and plugins are disabled.
This could be useful for all kinds of scenarios. The HTML5 spec lists some examples of blog comments, but I think that’s mostly a red herring. Think about what’s hosted in iframes today: third-party advertising and third-party widgets. In each case, a web author wants to embed something on their page that they have little or no control over. In practice, that usually works fine. Advertising iframes don’t do anything (except display ads). Most widgets are well-behaved, and most widget frameworks (like Google Gadgets) enforce terms of service that forbid widgets from “taking over” the parent page in which they are embedded. Still, that’s a social/legal solution, not a technical one. Sandboxing is a complementary technical solution, where the parent page can actually tell the browser “Hey, I don’t fully trust this thing, but I’m embedding it anyway. Can you reduce its privileges?”
What privileges? Well, by default, “sandboxed” iframes can not
- access the DOM of the parent page (technically speaking, because the iframe is relegated to a different “origin” than the parent page)
- execute scripts
- embed their own forms, or manipulate forms via script
- read or write cookies, local storage, or local SQL databases
There are ways for the parent page to add back each of these privileges, if the third-party content needs it.
[The sandbox
attribute’s] value must be an unordered set of unique space-separated tokens. The allowed values are allow-same-origin
, allow-forms
, and allow-scripts
. The allow-same-origin
keyword allows the content to be treated as being from the same origin instead of forcing it into a unique origin, and the allow-forms
and allow-scripts
keywords re-enable forms and scripts respectively (though scripts are still prevented from creating popups).
So it’s a security feature. You could restrict an advertising iframe to have no privileges whatsoever, but you could give a widget iframe privileges to execute its own scripts or embed its own forms.
Yes. Well, no more than they are now. In fact, very few browsers support the sandbox
attribute today, so we’re not just talking about users of older browsers — we’re talking about pretty much everyone. But that’s OK. The sandbox
attribute is designed to be an incremental security feature. It’s an additional layer of security, not the only layer. Browsers have supported iframes for a long time, and thousands of web authors are using them despite the very real risks of embedding untrusted content. Advertising networks can and have been hacked; malicious widgets can and have been published; bad actors can and do try to do bad things to as many people as possible until they’re caught and taken down. You need to keep doing all the things you’re doing now to prevent iframe-based attacks. Then add sandbox
, too.
I can’t do any filtering or sanitizing. Can I rely solely on browser-based sandboxing?
Someday, you might — might! — be able to throw out all your sanitizing code and rely solely on the sandbox
attribute. Of course, you can’t do that today, because users of older browsers would still be vulnerable. So we need a “clean break” solution — a way to serve untrusted content to supporting browsers while absolutely, positively, 100% ensuring that older browsers never render the untrusted content under any circumstances. Enter the text/html-sandboxed
MIME type.
All HTML pages are served with the text/html
MIME type. It’s part of the HTTP headers, normally invisible to end users, but nevertheless sent by web servers every time a client requests a page. Every resource type (images, scripts, CSS files) has its own MIME type. Untrusted content could have its own MIME type. And this is where text/html-sandboxed
comes in. If my web server serves up an HTML page with a MIME type of text/html
, your browser will render it. If my web server serves up the same HTML page with a MIME type of text/html-sandboxed
, you browser will download it (or offer to download it). Your browser doesn’t recognize that MIME type, so it falls back to the default action, which is to download it and save it as a file on your local disk. We can use this behavior to our advantage.
As browsers start supporting the sandbox
attribute, they can also start supporting the text/html-sandboxed
MIME type. What does it mean to “support” this new MIME type? If a user navigates directly to a page served with the new MIME type, don’t do anything special. Just download it, which is what happens already. BUT... if the user navigates to a page that includes an <iframe>
element, AND the iframe has a sandbox
attribute, AND the src
of the iframe points to an HTML page that is served with the text/html-sandboxed
MIME type, THEN render the iframe as normal (but still subject to the restrictions listed in the sandbox
attribute).
Older browsers will download (or offer to download) the untrusted content. From a security perspective, that’s a good thing — at least, it means the content won’t be rendered as HTML. From a usability perspective, that’s terrible. Who wants to go to a page and suddenly have the browser offering to download a bunch of useless files? That means that you won’t really be able to use this technique until all users have upgraded to a browser that supports both the sandbox
attribute and the text/html-sandboxed
MIME type. That will be... a while. But it might happen someday!
There have been a number of proposals for a <sandbox>
element, which you could wrap around untrusted content. All such proposals suffer fatal flaws, stemming from how today’s browsers parse HTML markup. You, the author who wants to “wrap” untrusted content, would need to ensure that the content did not “break out” of the sandbox. For instance, it could include an </sandbox>
element. (Hey, it’s untrusted! That’s why we’re here in the first place.) There are a surprising number of variations of markup that are recognized as end tags (having to do with inserting whitespace characters in strange places), and you would be responsible for sanitizing all of these variations. Furthermore, you would need to ensure that the untrusted content did not include a script that called document.write()
, which could be used for writing out a matching </sandbox>
end tag programmatically. Think about the number of ways that script could be obfuscated, and pretty soon you’re asking individual web authors to solve the halting problem just to wrap some untrusted content.
If a wrapper element is the wrong solution, what’s the right one? This is where the “flurry of updates” has been happening. The current solution is r4619: the srcdoc
attribute (with minor updates in r4623, r4624, and r4626). The best way to explain it is by example:
<iframe sandbox srcdoc="<p>Markup in an attribute, woohoo!</p>"></iframe>
Yeah, that’s pretty janky. But it has the following nice qualities:
- The “sandbox” is an attribute value, not children of a wrapper element. That means the only thing you need to escape is quotation marks.
- Legacy browsers just ignore it and render nothing at all.
It also has the following not-so-nice qualities:
- The “sandbox” is an attribute value. Markup in an attribute? Srsly? Puke.
- Legacy browsers render nothing at all.
- When you’re assembling this markup on the server side, there’s no way to know in advance whether the browser will render it or not. Except User-Agent sniffing... ick.
There is one exception to that last rule. There are a few comment systems that are entirely client-side. That is, the comments are not part of the page markup that comes down from the web server; they are programmatically added after the page is rendered. Such comment systems could use JavaScript-based feature detection to check whether the browser supported the srcdoc
attribute, and write out the appropriate markup either way. I wrote the book on HTML5 feature detection. (No really! A whole fscking book!) Detecting srcdoc
support would use detection technique #2:
if ("srcdoc" in document.createElement("iframe")) { ... }
But this would only help in the case where you were adding untrusted content to the page at runtime, on the client side. Server-side cases will have to wait until everybody upgrades.
So when can I use all this stuff?
Hahahahahaha. You must be new here.
No really, when?
There are several pieces here, each with their own compatibility story.
- The
sandbox
attribute, for reducing privileges of untrusted content. Chromium and Google Chrome support the sandbox
attribute (I tested the dev channel version 4.0.302.3); Safari, Firefox, Internet Explorer, and Opera ignore it. So you can start using the sandbox
attribute today — just be sure to test in Chromium or Google Chrome to ensure you’ve set the sandbox privileges properly. It won’t have any effect in other browsers, but that’s OK. Remember, the sandbox
attribute isn’t designed to be your only line of defense; it’s a complement to your existing defenses. Keep doing whatever you’re doing now (sanitizing input, auditing code, enforcing legal terms with your partners, etc), then add sandbox
for extra protection.
- The
text/html-sandboxed
MIME type, for ensuring that users can’t navigate to untrusted content. There are two parts to this. First, browsers must not render pages served with a text/html-sandboxed
MIME type, if you navigate to the page directly. This part works in all browsers, today; they all download (or offer to download) the page markup instead of rendering it. Second, browsers that support the sandbox
attribute need to render iframes served with the text/html-sandboxed
MIME type (subject to the privilege restrictions listed in the sandbox
attribute). No browser supports this yet, not even Google Chrome. (It renders the parent page but downloads the iframe content instead of rendering it within the frame.) So you can’t use this technique yet, until Google updates Chrome to support it. (In theory, other browser vendors will implement support for this at the same time they implement support for the sandbox
attribute, but I suppose we’ll just have to wait and see.)
- The
srcdoc
attribute, for including untrusted content inline. Since the fallback behavior in legacy browsers for this feature is “render nothing at all” (by design), this attribute won’t be useful until pretty much all of your visitors upgrade to browsers that support the attribute. At the moment, no current browser supports the srcdoc
attribute, so it’ll be a while. If I had to guess, I’d say January 29, 2022, at 4:37pm. Plus or minus 10 years.
And now you know “What’s Next in HTML.”
Posted in What's Next | 24 Comments »
Wednesday, January 13th, 2010
Welcome to "What's Next in HTML," where I'll try to summarize the major activity in the ongoing standards process in the WHAT Working Group. Wait... what happened to This Week in HTML5? Hell, what happened to HTML5? Well, nothing. It took over five years to create, but it's in Last Call now. By all measures, it has already been wildly successful. Browser vendors are implementing it, books are being written, we have a kick-ass validator, web developers are slowly catching on, and there's still plenty of time to send us your feedback. But in the meantime, the WHAT Working Group has begun work on new, experimental features for the next version of HTML.
The next version of HTML doesn't have a name yet. In fact, it may never have a name, because the working group is switching to an unversioned development model. Various parts of the specification will be at varying degrees of stability, as noted in each section. But if all goes according to plan, there will never be One Big Cutoff that is frozen in time and dubbed "HTML6." HTML is an unbroken line stretching back almost two decades, and version numbers are a vestige of an older development model for standards that never really matched reality very well anyway. HTML5 is so last week. Let's talk about what's next.
The big news in HTML is r4439, which adds the device
element. What's a <device>
? I'm glad you asked.
The device
element represents a device selector, to allow the user to give the page access to a device, for example a video camera.
The type
attribute allows the author to specify which kind of device the page would like access to.
So it's for video conferencing, something you can currently only do with Adobe Flash or other proprietary plugins that sit on top of your browser. In fact, most of the pieces for browser-based video chat are already in place. The idea is that a device
element would go hand in hand with a video
element and a web socket. The device
records a video stream (using the also-newly-defined Stream API) and sends the stream of video along a web socket to the other party (perhaps via an intermediate server) which renders the stream in a video
element. And like the video
element, the device
element would be native to your browser, so browser vendors would not have to wait for third parties to add specific support for their platform.
Does all that work yet? Hell no. We don't even have a standard video codec yet! Google Chrome is the only browser that has shipped an implementation of web sockets (although it's part of WebKit, so presumably Apple could ship it in a future version of Safari if they choose). And the entire device
API is still in its infancy. Nobody has even started implementing a prototype of that piece yet, and the whole idea might be scrapped by my next episode. But that's life on the bleeding edge.
And now you know "What's Next in HTML."
Posted in What's Next | 43 Comments »
Tuesday, October 20th, 2009
Welcome back to "This Week in HTML 5," where I'll try to summarize the major activity in the ongoing standards process in the WHATWG and W3C HTML Working Group.
This week, there were some more refinements to microdata. r4139 changes the names of the DOM properties that reflect microdata markup. r4140 renames the content
property to itemValue
Since no browser has actually implemented this API yet, these changes shouldn't make any difference. Standards are like sex; one mistake, and you're stuck supporting it forever! r4141 and r4147 fix up some microdata examples, in particular this example from Gavin Carothers about marking up O'Reilly's book catalog. Hooray for real-world examples!
There were also some noteworthy changes to the <video>
and <audio>
API. r4131 says that setting the src
attribute on one of those elements should call its load()
method. r4132 removes the load
event for multimedia elements, and r4133 removes the "in progress" events (loadstart
, loadend
, and progress
) that used to be fired while the video/audio file was downloading.
Other noteworthy changes this week:
- r4097 defines fallback content for the obsolete
<applet>
element.
- r4098 "dramatically simplifies
<script defer>
and <script async>
handling." [Background: bug 7792]
- r4106 makes the step argument to the
<input>
element's stepUp()
and stepDown()
methods optional.
- r4111 removes
<link rel=feed>
. As I documented earlier this year, rel=feed
was a reasonable idea that never took off. Only one browser ever implemented it, and in a survey of 3 billion pages I could only find a single page that used it.
- r4126 lists suggested default encodings for different locales. [Background: RE: HTML5 Issue 11 (encoding detection): I18N WG response...]
- r4138 drops support for non-UTF-8 encodings in Web Workers. [Background: [whatwg] Please always use utf-8 for Web Workers]
- r4099 marks the creation of Web Applications 1.0, a super-spec that contains HTML5, pre-defined microdata vocabularies, Web Workers, Web Storage, Web Database, Server-sent Events, and Web Sockets. This marks the first time that some of those specs have been published by the WHATWG, rather than the W3C, and therefore the first time that said specs have been published under a Free-Software-compatible license. (The W3C is still deciding whether to use such a license.)
Around the web:
Tune in next week for another exciting edition of "This Week in HTML5."
Posted in Weekly Review | 5 Comments »
Friday, October 9th, 2009
The big news this week is microdata. Google sponsored a usability study on microdata syntax, which resulted in significant changes to the spec [r4066]. Also related: r4067 fixes a microdata example, r4068 updates the algorithm for extracting RDF triples from microdata, r4069 does some spec cleanup, and r4070 splits out the predefined microdata syntaxes into their own specs:
There was also work on events this week. r4032 defines what events are involved in copy and paste, closing bug 7668. r4037 defines when the reset
event fires, closing bug 7699. r4039 defines when the abort
event fires, closing bug 7700.
This week brings another milestone, one which went mostly unremarked in mailing lists, blogs, and IRC chatter. As with any large project, Ian Hickson has maintained an informal wishlist of things he would like to clarify, define, or otherwise include in HTML5. The list has grown and shrunk over the years. The list was stored in HTML comments, so it has never been visible unless you viewed the source of the HTML5 specification itself. And as with any large project, there comes a time when you realize you're not going to get to everything on your wishlist.
This week, the wishlist shrunk a lot, as Ian finally "punted" on several issues. Some of them may be tackled in HTML6. (Of course, if someone feels strongly enough, they can certainly argue that an issue still needs to be tackled in HTML5.) r4023 shows the deletions from the wishlist, including: "ability for a web app to save a file to the local disk," proposals for new attributes on the <title>
element, partial form validation, multi-column select widgets, auto-formatting of number fields (like many spreadsheet programs do), relative dates, input controls for repeating dates (like anniversaries or other repeating events), and input controls for currency.
Other noteworthy changes this week:
- r4011 syncs with the latest Origin spec, closing bug 7599.
- r4031 allows user agents to explicitly disable
<canvas>
support.
- r4042 limits
PUT
and DELETE
actions on web forms to the same origin as the page. This is similar to the restriction on XMLHttpRequest
.
- r4057 defines
<applet>
.
- r4076 disallows the backtick (
`
) character in unquoted attribute values, because Internet Explorer will treat it as an attribute value delimiter.
- r4082 adds the
document.head
property, which makes me very happy.
- r4083 states that an
<audio>
element without controls should always be hidden. (You can still make a visible <audio>
element; just give it a controls
attribute.)
- r4086 tries to clarify the ever-elusive
WindowProxy
object.
- r4091 registers the various HTTP headers that are used in the new features of HTML5, including
Ping-From
and Ping-To
.
- r4092 and r4094 add a non-normative index of HTML elements and attributes. Think of it as an "HTML5 cheat sheet." Various third parties have attempted such a list, but none have been able to keep up with the maintenance required as HTML5 evolved.
Around the web:
Tune in next week for another exciting edition of "This Week in HTML5."
Posted in Weekly Review | 2 Comments »
Tuesday, September 29th, 2009
I recently found myself testing how browsers sniff for RSS 1.0 feeds that are served with an incorrect MIME type. (Yes, my life is full of delicious irony.) I thought I'd share my findings so far.
Firefox
Firefox's feed sniffing algorithm is located in nsFeedSniffer.cpp
. As you can see, starting at line 353, it takes the first 512 bytes of the page, looks for a root tag called rss
(for RSS 2.0), atom
(for Atom 0.3 and 1.0), or rdf:RDF
(for RSS 1.0). The RSS 1.0 marker is really a generic RDF marker, so it then does some additional checks for the two required namespaces of an RSS 1.0 feed, http://www.w3.org/1999/02/22-rdf-syntax-ns#
and http://purl.org/rss/1.0/
. This check is quite simple; it literally just checks for the presence of both strings, not caring whether they are the value of an xmlns
attribute (or indeed any attribute at all).
Firefox has an additional feature which tripped up my testing until I understood it. IE and Safari both have a mode where they essentially say "I detected this page as a feed and tried to parse it, but I failed, so now I'm giving up, and here's an error message describing why I gave up." Firefox does not have a mode like this. As far as I can tell, if it decides that a resource is a feed but then fails to parse the resource as a feed, it reparses the resource with feed handling disabled. So an non-well-formed feed served as application/rss+xml
will actually trigger a "Do you want to download this file" dialog, because Firefox tried to parse it as a feed, failed, then reparsed it as some-random-media-type-that-I-don't-handle. A non-well-formed feed served as text/html
will actually render as HTML, but only after Firefox silently tries (and fails) to parse it as a feed.
There's nothing wrong with this approach; in fact, it seems much more end-user-friendly than throwing up an incomprehensible error message. I just mention it because it tripped me up while testing.
Internet Explorer
Internet Explorer's feed sniffing algorithm is documented by the Windows RSS team. About RSS 1.0, it states:
IE7 detects a RSS 1.0 feed using the content types application/xml
or text/xml
. ... The document is checked for the strings <rdf:RDF
, http://www.w3.org/1999/02/22-rdf-syntax-ns#
and http://purl.org/rss/1.0/
. IE7 detects that it is a feed if all three strings are found within the first 512 bytes of the document. ... IE7 also supports other generic Content-Type
s by checking the document for specific Atom and RSS strings.
Now that I understand IE's algorithm, I have to concede that this documentation is 100% accurate. However, it doesn't tell the full story. Here's what actually happens. If the Content-Type
is
application/xml
text/xml
application/octet-stream
text/plain
text/html
- the empty string, or
- missing altogether
...then IE will trigger its feed sniffing. Once IE triggers its feed sniffing, it will never change its mind (unlike Firefox). If feed parsing fails, IE will throw up an error message complaining of feed coding errors or an unsupported feed format. The presence or absence of a charset
parameter in the Content-Type
header made absolutely no difference in any of the cases I tested.
And how exactly does IE detect an RSS 1.0 feed, once it decides to sniff? The documentation on MSDN is literally true: "The document is checked for the strings <rdf:RDF
, http://www.w3.org/1999/02/22-rdf-syntax-ns#
and http://purl.org/rss/1.0/
. IE7 detects that it is a feed if all three strings are found within the first 512 bytes of the document." Combined with our knowledge of which Content-Type
s IE considers "generic," we can conclude that the following page, served as text/html
, will be treated as a feed in IE:
<!-- <rdf:RDF -->
<!-- http://www.w3.org/1999/02/22-rdf-syntax-ns# -->
<!-- http://purl.org/rss/1.0/ -->
<script>alert('Hi!');</script>
[live demonstration]
Why Bother?
I am working with Adam Barth and Ian Hickson to update draft-abarth-mime-sniff-01 (the content sniffing algorithm referenced by HTML5) to sniff RSS 1.0 feeds served as text/html
. It is unlikely that we will adopt IE's algorithm, since it seems unnecessarily pathological. I am proposing the following change, which would bring the content sniffing specification in line with Firefox's sniffing algorithm:
In the "Feed or HTML" section, insert the following steps between step 10 and step 11:
10a. Initialize /RDF flag/ to 0.
10b. Initialize /RSS flag/ to 0.
10c. If the bytes with positions pos to pos+23 in s are exactly equal to 0x68, 0x74, 0x74, 0x70, 0x3A, 0x2F, 0x2F, 0x70, 0x75, 0x72, 0x6C, 0x2E, 0x6F, 0x72, 0x67, 0x2F, 0x72, 0x73, 0x73, 0x2F, 0x31, 0x2E, 0x30, 0x2F respectively (ASCII for "http://purl.org/rss/1.0/"), then:
- Increase pos by 23.
- Set /RSS flag/ to 1.
10d. If the bytes with positions pos to pos+42 in s are exactly equal to 0x68, 0x74, 0x74, 0x70, 0x3A, 0x2F, 0x2F, 0x77, 0x77, 0x77, 0x2E, 0x77, 0x33, 0x2E, 0x6F, 0x72, 0x67, 0x2F, 0x31, 0x39, 0x39, 0x39, 0x2F, 0x30, 0x32, 0x2F, 0x32, 0x32, 0x2D, 0x72, 0x64, 0x66, 0x2D, 0x73, 0x79, 0x6E, 0x74, 0x61, 0x78, 0x2D, 0x6E, 0x73, 0x23 respectively (ASCII for "http://www.w3.org/1999/02/22-rdf-syntax-ns#"), then:
- Increase pos by 42.
- Set /RDF flag/ to 1.
10e. Increase pos by 1.
10f. If /RDF flag/ is 1, and /RSS flag/ is 1, then the /sniffed type/ of the resource is "application/rss+xml". Abort these steps.
10g. If pos points beyond the end of the byte stream s, then continue to step 11 of this algorithm.
10h. Jump back to step 10c of this algorithm.
Further Reading
You can see the results of my research to date and test the feeds for yourself. Because my research results are plain text with embedded HTML tags, I have added 512 bytes of leading whitespace to the page to foil browsers' plain-text-or-HTML content sniffing. Mmmm -- delicious, delicious irony.
Update: Belorussian translation is available.
Posted in Browsers | 4 Comments »