I have released a new version of the Validator.nu HTML Parser (an implementation of the HTML5 parsing algorithm in Java). The new release supports SVG and MathML subtrees, is faster than the old version, fixes bugs, is more portable and supports applications that want to do document.write().
The parser comes with a sample app that makes it possible to use XSLT programs written for XHTML5+SVG+MathML with text/html.
Warning! The internal APIs have changed. Please refer to the Upgrade Guide below.
Change Log
- Made the SAX, DOM and XOM parser entry point constructors default to altering the infoset instead of throwing when the input needs coercing to be an XML 1.0 4th ed. plus Namespaces infoset.
- Isolated Java IO dependent code from the parser core. The parser core now compiles on Google Web Toolkit.
- Refactored the tokenizer to use a
switch branch per state instead of method per state.
- Made various performance tweaks to the tokenizer.
- Implemented support for MathML and SVG foreign content. (Note that the SVG part is based on spec text that has been commented out from the spec at the request of the SVG WG.)
- Made the parser suspendable after any input character.
- Made it possible for custom
TreeBuilder subclasses to request parser suspension. (Applications wishing to implement document.write() should provide their own TreeBuilder subclass and a document.write()-aware replacement of the Driver class. Look in the gwt-src/ directory for sample code.)
- Made changes to the parser core to make it more suitable for mechanical translation into other object-oriented programming languages that have C-like control structures but not necessarily a garbage collector (with focus on targeting C++). This work is not complete.
- Made the HTML serializer do the right thing when input represents a conforming XHTML+SVG+MathML tree. (Results may be bad for non-conforming input trees.)
- Developed sample programs for converting between HTML5 and XHTML5 when the input is known to be conforming.
- Provided an XML serializer so that the sample code no longer depends on the Xalan serializer.
- Improved API documentation.
- Fixed bugs in the tokenizer, tree builder and the input stream character encoding decoder.
- Made coercion to an XML infoset work according to the HTML5 spec.
- Added ID uniqueness checking.
- Various other fixes.
Upgrade Guide from 1.0.7 to 1.1.0
In all cases, you need to check that your application does not break when it receives SVG or MathML subtrees.
- If you use the parser through the SAX, DOM or XOM API and do not pass an explicit
XmlViolationPolicy to the constructor of HtmlParser, HtmlDocumentBuilder or HtmlBuilder:
If you really wanted the old default behavior, you should now pass XmlViolationPolicy.FATAL to the constructor.
If you did not really want to have fatal errors by default, you do not need to do anything, since ALTER_INFOSET is now the default.
- If you use the parser through the SAX, DOM or XOM API and do pass an explicit
XmlViolationPolicy to the constructor of HtmlParser, HtmlDocumentBuilder or HtmlBuilder:
You do not need to change your code to upgrade.
- If you have your own subclass of
TreeBuilder:
The abstract methods on TreeBuilder now have additional arguments for passing the namespace URI. You should upgrade your subclass to deal with the namespace URIs. (The URI is always an interned string, so you can use == to compare.)
The entry point for passing in a SAX InputSource has moved from the Tokenizer class to the Driver class (in the io package), so you should change your references from Tokenizer to Driver.
- If you have your own implementation of
TokenHandler:
Please refer to the JavaDocs of TokenHandler. Also note the new separation of Tokenizer and Driver mentioned above.
Tags: gwt, java, mathml, parser, Parsing, svg, validator.nu
Posted in Syntax | Comments Off on Validator.nu HTML Parser 1.1.0
Welcome back to "This Week in HTML 5," where I'll try to summarize the major activity in the ongoing standards process in the WHATWG and W3C HTML Working Group.
The biggest news this week is the birth of the event loop.
To coordinate events, user interaction, scripts, rendering, networking, and so forth, user agents must use event loops as described in this section.
... An event loop has one or more task queues. A task queue is an ordered list of tasks, which can be:
- Events
Asynchronously dispatching an Event object at a particular EventTarget object is a task.
- Parsing
The HTML parser tokenising a single byte, and then processing any resulting tokens, is a task.
- Callbacks
Calling a callback asynchronously is a task.
- Using a resource
When an algorithm fetches a resource, if the fetching occurs asynchronously then the processing of the resource once some or all of the resource is available is a task.
- Reacting to DOM manipulation
Some elements have tasks that trigger in response to DOM manipulation, e.g. when that element is inserted into the document.
The purpose of defining an event loop is to unify the definition of things that happen asychronously. (I want to avoid saying "events" since that term is already overloaded.) For example, if an image defines an onload callback function, exactly when does it get called? Questions like this are now answered in terms of adding tasks to a queue and processing them in an event loop.
- Revision 2074 defines event loops and task queues (as quoted above).
- Revision 2076, 2079, 2080, 2081, 2082, and 2083 define the behavior of media elements (like
<audio> and <video>) in terms of the event loop.
- Revision 2084 defines the behavior of
template and ref attributes, local database storage, and remote events in terms of the event loop.
- Revision 2085 defines the behavior of web sockets,
postMessage, message ports, and setTimeout in terms of the event loop.
- Revision 2097 defines the behavior of an image's
load event in terms of the event loop.
The other major news this week is the addition of the hashchange event, which occurs when the user clicks an in-page link that goes somewhere else on the same page, or when a script programmatically sets the location.hash property. This is primarily useful for AJAX applications that wish to maintain a history of user actions while remaining on the same page. As a concrete example, executing a search of your messages in GMail takes you to a list of search results, but does not change the base URL, just the hash; clicking the Back button takes you back to the previous view within GMail (such as your inbox), again without changing the base URL (just the hash). GMail employs some nasty hacks to make this work in all browsers; the hashchange event is designed to make those hacks slightly less nasty. Microsoft Internet Explorer 8 pioneered the hashchange event, and its definition in HTML 5 is designed to match Internet Explorer's behavior.
Other interesting changes this week:
- In last week's episode, I mentioned revision 2063, which allows HTML documents to contain both
xml:lang and lang attributes as long as they are identical. Revision 2091 relaxes this restriction slightly to allow the xml:lang and lang attributes to differ by case (i.e. one could be uppercase and the other could be lowercase, and that is no longer an error). Discussion: xml:lang="" and lang=""
- Revision 2092 defines the parsing algorithm for empty table rows.
- Revision 2094 clarifies the meaning of whitespace by deferring to the Unicode definitions.
- Revision 2096 forbids content sniffing for SVG images. In order to use an SVG image in an
<img src=""> attribute, the web server must ensure that the SVG image is served with a Content-Type: image/svg+xml HTTP header.
Tune in next week for another exciting episode of "This Week in HTML 5."
Tags: thisweekinhtml5
Posted in Weekly Review | 4 Comments »
Welcome back to "This Week in HTML 5," where I'll try to summarize the major activity in the ongoing standards process in the WHATWG and W3C HTML Working Group.
The biggest news this week is revision 2020, which standardizes the navigator object:
The navigator attribute of the Window interface must return an instance of the Navigator interface, which represents the identity and state of the user agent (the client), and allows Web pages to register themselves as potential protocol and content handlers.
Currently, HTML 5 defines four properties and two methods:
appNameappVersionplatformuserAgentregisterProtocolHandlerregisterContentHandler
This is only a subset of navigator properties and methods that browsers already support. See Navigator Object on Google Doctype for complete browser compatibility information.
Next up: Content-Language. No, not the HTTP header, not even the <html lang> attribute, but the <meta> tag! As reported by Henri Sivonen,
It seems that some authoring tools and authors use <meta http-equiv='content-language' content='languagetag'> instead of <html lang='languagetag'>.
This led to revision 2057, which defines the <meta> http-equiv="Content-Language"> directive and its relationship with lang, xml:lang, and the Content-Language HTTP header.
In the continuing saga of the alt attribute, the new syntax for alternate text of auto-generated images (which I covered in last week's episode) has generated some followup discussion. Philip Taylor is concerned that it will increase complexity for authoring tools; others feel the complexity is worth the cost. James Graham suggested a no-text-equivalent attribute; similar proposals have been discussed before and rejected.
Switching to the new Web Workers specification (which I also covered last week), Aaron Boodman (one of the developers of Google Gears) posted his initial feedback. This kicked off a long discussion and led to the creation of the Worker object.
Other interesting changes this week:
Administrivia: "This Week in HTML 5" now has its own feed.
Tune in next week for another exciting episode of "This Week in HTML 5."
Tags: thisweekinhtml5
Posted in Weekly Review | 3 Comments »
Earlier, I blogged about running the Validator.nu HTML Parser inside Hixie’s Live DOM Viewer using the magic of the hosted mode of the Google Web Toolkit. Back then, a compiler bug in GTW 1.5 RC1 prevented the parser from running as JavaScript in the Web mode. Google has now released GWT 1.5 RC2, which contains a fix for the bug.
So without further ado, here’s Live DOM Viewer with an HTML5 parser running as JavaScript in your browser.
Try pasting in the SVG lion or some MathML in Firefox 3 and Opera 9.5.
Known problems:
- SVG
use does not work in Firefox. Update: Fixed in Minefield nightlies.
- SVG does not render is Safari.
- IE does not support
createElementNS and, thus, does not work at all.
A big thanks for the GWT team for making this work!
Tags: parsing svg mathml javascript gwt validator.nu
Posted in DOM, Syntax | Comments Off on HTML5 Live DOM Viewer—Now in Your Browser
Welcome to a new semi-regular column, "This Week in HTML 5," where I'll try to summarize the major activity in the ongoing standards process in the WHATWG and W3C HTML Working Group.
The biggest news is the birth of the Web Workers draft specification. Quoting the spec, "This specification defines an API that allows Web application authors to spawn background workers running scripts in parallel to their main page. This allows for thread-like operation with message-passing as the coordination mechanism." This is the standardization of the API that Google Gears pioneered last year. See also: initial Workers thread, announcement of new spec, response to Workers feedback.
Also notable this week: even more additions to the Requirements for providing text to act as an alternative for images. 4 new cases were added:
- A link containing nothing but an image
- A group of images that form a single larger image
- An image not intended for the user (such as a "web bug" tracking image)
- Text that has been rendered to a graphic for typographical effect
Additionally, the spec now tries to define what authors should do if they know they have an image but don't know what it is. Quoting again from the spec:
If the src attribute is set and the alt attribute is set to a string whose first character is a U+007B LEFT CURLY BRACKET character ({) and whose last character is a U+007D RIGHT CURLY BRACKET character (}), the image is a key part of the content, and there is no textual equivalent of the image available. The string consisting of all the characters between the first and the last character of the value of the alt attribute gives the kind of image (e.g. photo, diagram, user-uploaded image). If that value is the empty string (i.e. the attribute is just "{}"), then even the kind of image being shown is not known.
- If the image is available, the element represents the image specified by the src attribute.
- If the image is not available or if the user agent is not configured to display the image, then the user agent should display some sort of indicator that the image is not being rendered, and, if possible, provide to the user the information regarding the kind of image that is (as derived from the alt attribute).
See also: revision 1972, revision 1976, revision 1978, revision 1979, Images and alternate text.
Other interesting changes this week:
- revision 1951: define
window.top
- revision 1956: "User agents must not run executable code embedded in the image resource."
- revision 1958: more notes on what is a valid image (a surprisingly difficult question)
- revision 1965: allow
<a> elements to straddle paragraphs
- revision 1998: define what happens when you set
onclick='' on a document outside a Window
- revision 1999: define
javascript: in Window-less environments
- revision 2001: define 'directionality' in terms of the
dir='' attribute for cases where the 'direction' property has no computed value
- revision 2002: define processing for the second argument to
getDataURL() for image/jpeg
- revision 2004: specify how to handle transparent images in the
toDataURL() method
- revision 2008: make patterns required in the
<canvas> API
- revision 2016: when
<script type=''> is given, it must match the type of the script, even if the script is Javascript
- revision 2019: remove
autosubmit='' from the <menu> element
Tune in next week for another exciting episode of "This Week in HTML 5."
Tags: thisweekinhtml5
Posted in Processing Model, Weekly Review, WHATWG | 21 Comments »
