Paul CHEYROU-LAGREZE
INRIA Rhône-Alpes
Zirst - 655 avenue de l'Europe
38334 Montbonnot -Saint Ismier Cedex FRANCE
Phone : +33 (0)4 76 61 53 58
Fax : +33 (0)4 76 61 52 52
Paul.Cheyrou-Lagreze@inrialpes.fr
Webpage : http://www.w3.org/Amaya
Keywords: Edition; SVG; XML; MathML; XHTML; CSS; Universal Canvas; Compound Documents
Abstract
Amaya is a Web client
that acts both as a browser and as an authoring tool. It has been designed by
W3C with the primary purpose of
demonstrating new Web technologies in a WYSIWYG (What You See Is What You
Get) environment. Documents mixing XHTML text, MathML equations, and SVG
graphics can be browsed and edited with Amaya. All these XML elements can be
styled in a uniform way with CSS (Cascading Style Sheets) style sheets. When
edited, those compound documents can be either saved in the local file system
or published on remote servers through the HTTP PUT
method.
Amaya is also able to browse and edit any XML document and to format it
through CSS style sheets. This paper presents the overall architecture of
Amaya putting the emphasis on the advantage it provides for the
implementation of SVG. Specific developments made to handle SVG efficiently
are detailed. This includes the use of OpenGL and FreeType,as well as user interaction
issues.
Modern editing tools provide a way to handle graphics as well as texts and mathematics directly in the canvas, allowing users to author compound documents. But those tools do not aim at interoperability, accessibility and easy distribution. XML [1] languages are a good solution to address these issues. SVG [2] is the graphic part of the XML effort. SVG must be integrated in editing tools as a Web language in order to give its full power to users. The plug-in [3] approach can then be avoided, solving many integration issues. This is important as different XML languages can be combined in a same document.
The architecture of Amaya [4], a full featured Web client, was designed so that its low-level functions handle all XML languages the same way. This leads to several advantages on the implementation side, as a number of functions and structures can be shared. But SVG also implies specific techniques as it is the XML language that require the most powerful canvas. OpenGL [5] and FreeType [6] libraries was chosen for Amaya as it fits in those strong requirements. It also provides a way to handle SVG as well as other XML languages.
SVG was developed by W3C [7] to allow rich vector graphics to be represented and shared on the Web. XML was chosen as a basis for this graphics language. XML is the common infrastructure for document formats on the Web. It is used not only by SVG but, to name only a few other languages, by XHTML [8] and MathML [9].
Figure 1shows a screenshot of a document that mixes different XML languages. When a document contains a chunk of XHTML text, some MathML equations, and SVG graphics, it is not just a canvas with three things stuck onto it. It's a fabric woven from three kinds of threads. Textual elements belonging to each thread are subject to the same search and styling mechanisms. Every element is part of the document object model (DOM [10]) and, as such, available to controlling scripts. XML allows different data types to blend and interact. What makes XML interesting is that it promises to fundamentally unify the representation and manipulation of structured data:
The benefit of XML is the synergy of different technologies, all contributing to create a product that is more than the sum of its parts. That is what editing SVG with other XML languages is about, and what Amaya tries to do.
The above remarks about the benefits SVG draw from the underlying Web technologies are not taken into account by all SVG tools. Most of them process drawings in isolation, in particular authoring tools and players. There is already a broad range of editors for producing SVG files, either natively or through format conversion, but most of them just allow the author to manipulate graphics, without any connection with their environment on the Web. A number of SVG displaying engines are also available, but it seems that most of them are used as plug-ins associated with browsers that have no knowledge of SVG. This greatly reduces many of the advantages high-lighted above.
With plug-ins, SVG is processed like raster images: graphics must be transmitted and updated as separate documents. The plug-in approach also makes integration of SVG with the rest of the document more complex and sometimes impossible. This is important as different XML languages can be combined in a single document. For example, due to current plug-in API limitations, CSS rules associated with the main document can not be transmitted to the embedded SVG objects. Plug-ins break a lot of expected behaviors in editing/viewing drawings:
Moreover, the use of plug-ins forces documents to be split into several files. Therefore compound documents are difficult to handle and make maintaining Web sites more complex.
Amaya tries to address these issues by putting the emphasis on integration. Integration is considered at three different levels : integrating the tool with the Web, integrating complementary functionalities in the same tool, and integrating various types of data in compound documents.
Amaya is a Web client that has direct access to the Web. From the user perspective, there is no difference between a document stored on the local file system and a resource accessed through a Web server. At the user interface level, the same commands are used in the same way in both cases. The Web is seen as a uniform information space and there is no boundary between local and remote resources.
Amaya is a full-featured Web client, in the sense that it provides total access to Web resources. You can not only browse the Web, but you can also modify the resources you access and you can save them back, either locally or on remote servers, provided you have the required access rights. You can also create new Web pages and publish them instantly on remote servers. This approach gets back to the pre-Mosaic [13] times, when Web clients where not only browsers, but included an editing functionality, thus allowing users to play a more active role on the Web.
Amaya can processes XML documents which include not only text (in XHTML), but also graphics (in SVG) and mathematics (in MathML) in any number of combinations. For instance, an XHTML file may contain many MathML expressions and SVG graphics. SVG objects may include mathematics or formatted (XHTML) text which in turn contains mathematics and graphics, and so on, without any limitation.
Amaya handles several XML vocabularies (XHTML, SVG, MathML, RDF [14], etc.). The first step in handling all those languages is to parse the XML syntax. For this task, Amaya uses the Expat [15] parser.
Figure 2 : Document cycle through Amaya Engine
The XML parser builds a DOM tree corresponding to each XML document.
Documents may contain pieces written in different XML languages, provided
that namespaces are used for each language transition. DOM trees therefore
contain subtrees representing different kinds of structures. Amaya can handle
at the same time several documents belonging to several namespaces. For SVG,
the multi-namespace feature allows typically a single XHTML document to
include several SVG drawings and MathML expressions (including MathML within
SVG, through the foreignObject
element). The multi-document
feature allows users to work simultaneously on several SVG drawings and on a
related XHTML document (Figure 2).
Wherever XML structures contain text, content is represented internally in
Unicode [16]. Amaya
can support different document encodings. However, at parsing time, every
text string is converted into UTF-16. At saving time, documents are
transcoded back to their original encoding by default, but the user may also
change encoding at that time. A single internal encoding is a great help for
editing. Users are free to copy (or move) text from one document to another,
even if documents were encoded differently and if they belong to different
namespaces. This also allows Amaya to process all pieces of text with the
same algorithms, wherever they are located in the XML structures. Users can
then search for text, replace strings, or check spelling everywhere,
including in SVG elements included in XHTML pages. This also allows the text
formatter to hyphenate words according to the current language (indicated by
the xml:lang
attribute). It will then be very easy to implement
text formatting within SVG, as soon as this feature is added to the SVG
language.
Unicode contributes to a large extent to the internationalization of the Web. It also provides a lot of internationalization features to Amaya. By sharing Unicode with XHTML, SVG benefits from the developments that were made to handle multiple scripts and various writing directions.
To allow users to clearly understand the structure of XML documents, Amaya
can display different views of each document. Each view is rendered
in a different window. There is a Formatted view that typically
shows the document in the same way as most Web browsers do. There is also a
Structure view which displays an indented list of elements with
their attributes, making the tree structure very clear. There is a
Source view, which just displays the XML code. Depending on the
document type, additional views may be provided, such as a Table of
Contents view for XHTML pages (it renders only Hn
elements), or a Link view gathering all hypertext links originating
in the document.
The Formatted view is of special interest. Whereas the graphical aspect of other views is predefined, formatting in this view is controlled by CSS style sheets. Formatting is based on the CSS box model and CSS properties apply to these boxes. The same CSS parser, formatting, and style engine are used for all documents, and all parts of documents, whatever their type. This is made possible by the uniform representation of all structures within Amaya. This is the way that SVG elements are styled. In fact, some specific SVG properties have been added by extending the CSS parser and styling engine, but most of the properties are shared with the XHTML and MathML elements.
The immediate advantage of this approach is for style inheritance. As SVG elements are part of the same DOM tree as their surrounding document, they can inherit any CSS property from their environment, without any limitation. This unified approach allows SVG to benefit from all the nice CSS features.
Hypertext links are of paramount importance in the Web, and Amaya provides
a lot of features for handling them efficiently. For instance, a simple
point-and-click mechanism allows the user to create or change hyperlinks very
easily, without manipulating URIs [17]. Links are also
automatically transformed (the href
attribute is updated) when
material is moved from one document to another, with a different base. Many
other services are provided to manipulate links. SVG takes advantage of these
services. All the links it uses (elements such as a
and
use
, for instance, or links to style sheets) are handled by the
same services, with the same interface.
Amaya accesses the Web through the Libwww [18] library
which implements the HTTP [19] 1.1 protocol. Amaya uses
that protocol to fetch all the resources it needs (Web pages, images, style
sheets). It uses the PUT
method from HTTP to publish pages
remotely. There is a single implementation of HTTP, which is used for
transferring all files whatever their format. This was proven especially
interesting recently, when the WebDAV [20] extension was added to Libwww. At once, it was possible
to use WebDAV to share not only XHTML pages, but also SVG drawings.
Amaya is a WYSIWYG editor. As described earlier, Amaya displays different views of each document. Views not only provide a way to see a document, but they are also used to interact on the document. Any view can be edited at any time, and changes made in one view are reflected simultaneously in all the other views. Users can thus work on one view and see how the other views are modified at the same time. Obviously, all editing commands are the same in all the views, which provides a uniform user interface.
The user interface provides users a way to handle multiple XML languages at once. In order to achieve this integration, Amaya dynamically adapts its interface according to the languages used in documents. Users can thus edit XHTML, MathML and SVG structures, without need of a complete knowing of underlying technologies.
Experience has shown that many editing operations are performed directly on the Formatted view, probably because it offers a WYSIWYG editing mode. The editing commands provided to style text elements are also available for SVG. Selections (mouse and key events) and commands like copy/paste, spell-checking, search/replace, etc., work in the same way on the XHTML and the SVG text.
Amaya provides a command to create hypertext links. The same generic command is able to generate an XHTML anchor around a XHTML piece of text or a SVG anchor around any SVG element.
CSS is supported in Amaya to allow web authors to define the style of documents independently of their contents. Even if CSS is often used only for styling text, most of its properties apply to SVG equally well, and having a single mechanism to change the color, the font, or the size of characters, wherever they are, makes the user interface more consistent. So painting some text in red is done exactly in the same way as changing the color of a red circle.
The architecture described above clearly shows how important it is to actually share the internal structures and the code to process different types of data in Web tools. When introducing SVG in Amaya, a large part of the implementation was already provided by the overall architecture. It was the same when MathML was introduced. Obviously some specific work was needed for each language, to implement of their own semantics, but all the features they share with other XML languages were already implemented. This not only saves a lot of time to the developers, but it also reduces the hardware resources required to run the tool. Plug-ins typically duplicate a lot of code that is already implemented in their host browser. For SVG they have their own XML parser, their own DOM tree engine, their own implementation of CSS, their own protocol stack. With the architecture chosen for Amaya, all those things are implemented only once.
The specific developments that were made for SVG are detailed in the next sections, but it is important to note how much time and effort can be saved by sharing the implementation of the common features. Annotea [21], an annotation application that was developed on top of Amaya, may reinforce this observation. Annotea allows a user to create and read shared annotations associated with any Web page. Annotations are themselves Web pages that are stored on specific servers and related to the documents they annotate by XLink [22]. This allows users to annotate documents without modifying them, and to attach annotations to any part of an XML document.
As soon as SVG was implemented in Amaya, it was possible to associate annotations to any SVG file, and even to any element within an SVG drawing, included or not within another document. It was also possible to put SVG drawings within annotations.
To completely implement SVG in Amaya, some specific work was required, mostly due to the high graphic quality implied by SVG. Special attention was paid to the display and more specifically to the canvas. Extensions were also needed to display SVG text and to interactively manipulate graphics.
While the basic features provided by the X Window System (Linux) and the Windows API were sufficient to render XHTML and MathML, a more sophisticated graphic library was needed for SVG. SVG rendering requires a graphic toolkit able to handle complex paths, transparency, gradients, animation, etc. We looked for a such toolkit available on Windows and Unix platforms. As Amaya is an open source software, the toolkit license also need to be compatible with the Amaya license. We estimate that OpenGL is the library that provides the best answer to these issues.
OpenGL is a procedural 3D graphics API introduced in 1992. OpenGL has become the industry's most widely used and supported 2D and 3D graphics application programming interface (API), bringing thousands of applications to a wide variety of computer platforms. OpenGL fosters innovation and speeds application development by incorporating a broad set of rendering, texture mapping, special effects, and other powerful visualization functions. Numerous books have been published about OpenGL, and a great deal of sample code is readily available, making information about OpenGL inexpensive and easy to obtain. Efficient OpenGL routines typically result in applications with fewer lines of code than those that make up programs generated using other graphics libraries or packages. In addition, OpenGL drivers encapsulate information about the underlying hardware, freeing the application developer from having to design for specific hardware features. All OpenGL applications produce consistent visual display results on any OpenGL API compliant hardware, regardless of operating system or windowing system.
OpenGL defines a state machine that controls the rendering pipeline. State machine attributes are modified through procedure calls. Vertices are always sent the same way to the graphic pipeline. Depending on which states are enabled, rendering is different.
This following code shows how to enable a state and then draw a line using
the glvertex
function:
... /* Draw a line defined two points */ /* Enabling state */ glbegin (GL_LINE); /* Define points*/ glvertex (x1, y1); glvertex (x2, y2); /* Disabling state */ glend (); ...
The following example enables different states, and as a results it draws
a rectangle with a blue to red gradient. Notice the use of the same set of
function glenable
, glend
, glvertex
. Figure 3 shows the result of the execution of this
code.
... /* Draw a rectangle by providing a point a width and a height */ /* Enabling state */ glBegin (GL_QUADS); red = 255; blue = 0; green = 0; opacity = 255; /* define the color state for the left part of rectangle */ glColor4ub (red, green, blue, opacity); /* Define points */ glVertex2f (x, y ); glVertex2f (x, y + height); /* define the color state for the right part of rectangle */ red = 0; blue = 255; green = 0; opacity = 255; glColor4ub (red, green, blue, opacity); glVertex2f (x + width, y); glVertex2f (x + width, y + height); /* Disabling state */ glEnd (); ...
OpenGL drivers encapsulate information about the underlying hardware, freeing the application developer from having to design for specific hardware features. All OpenGL applications produce consistent visual display results on any OpenGL API-compliant hardware, regardless of operating system or windowing system.
Finally, OpenGL already provides a large part of the capabilities needed by an universal canvas [23]. This greatly simplifies the implementation of SVG as well as other XML language implementation (Table 1) .
Table 1: OpenGL features that ease SVG and other XML language implementation
Needs | OpenGL facilities |
---|---|
Pictures | They are mapped on rectangle as textures with bi-cubic interpolation, giving high quality rendering even when zoomed. They are also cached in the video card memory. |
Paths | They are computed using the tessellation facility of the GLU library, that compose even non convex polygons with a minimal set of triangles and all SVG fill-rule. |
Transformation | Rotation, scaling, skewing are applied directly on elements or group of elements |
Selection | OpenGL renders a scene in a "selection" state and can give informations on objects that will be drawn |
Opacity | Alpha blending provides a means to create transparent objects. Using alpha information, an object can be defined as anything from totally transparent to totally opaque. |
Anti-aliasing | This technique averages the color of the pixels adjacent to the line. Rendering is smoother and suffers less from screen pixellisation, and thus are more device-dependent. |
Animation | Double buffering is used to provide smooth animation of objects. Each successive scene of an object in motion can be constructed in an "hidden" buffer and then displayed. This allows only complete images to ever be displayed on the screen. |
Speed | A list of OpenGL commands may be preprocessed. When rendering these lists, unmodified objects render up to fifty times faster. You can also apply after effects like transformation on those display lists.. |
Text that is part of an SVG document (or fragment) is specified as standard XML character data. For that reason Unicode support is required for SVG. However, OpenGL is targeted at applications manipulating graphics in 2D and 3D; it does not provide API for text rendering
To address this limitation, we have merged a TrueType font renderer with the OpenGL toolkit. The FreeType 2 library is a free rasterizer for all types of fonts, including TrueType fonts, that works on any machine with an ANSI C compiler and requires no particular operating system feature. FreeType 2 also provides font information like the character bounding box, the kerning and the Unicode character mapping for a font file. Finally, other projects had reported positive results when integrating of FreeType 2 with OpenGL.
The FreeType 2 library can deliver characters in vector and bitmap format. In our SVG implementation, we had to use both type of formats for different reasons:
Because the previous implementations were not adapted to our needs, we had to redo the integration ourselves. Specifically, our implementation addresses speed and memory consumption problems when using the bitmap character rendering for text:
OpenGL and FreeType 2 allows Amaya to render graphics, text and, animation using the same architecture in a transparent way. This transparency must also apply to the editing interface.
As explained above, SVG takes advantage of the generic editing commands provided by Amaya that allow users to add CSS properties, to select and edit text, and to create hypertext links. It also takes advantage of the multi-view feature and the possibility to perform editing commands in any of these views. For example, if the user wants to reorder a set of SVG shapes, that can be more easily performed on the structure view than on the formatted view. As all Amaya views are synchronized, the user can see what specific shape is manipulated when editing in the structure view.
However, the generic editing interface is not sufficient to create complex
SVG images. A user interface with direct graphic manipulation is necessary.
The development of this specific graphic interface is not achieved yet in
Amaya. Today, a SVG palette (Fig. 4) permits to insert
lines, rectangles, ellipses, polylines, polygons and paths. Selecting a shape
in that palette generates the necessary structure that inserts the new SVG
element at the current selected position. If the selection is within a XHTML
document, Amaya will create the graphic SVG element but also the necessary
parent SVG
element with its namespace.
The selection was also adapted to graphics elements. When a graphic is selected, its control points are hilighted. The user uses these control points to move and resize that SVG element.
In addition to the basic SVG editing functions, Amaya provides component
libraries. These libraries are used to store predefined drawings and symbol
that are often reused. A library is implemented as a set of SVG
def
elements (components like arrows, blocks, trees, ...) and
can be used to create more complex SVG images. Users use them either by
copying (with the copy/paste commands of Amaya) or by referencing (with SVG
use
elements). A benefit of using def
and
use
is that any update of library components will be
automatically transmitted to all SVG images that "use" them without any other
change.
A library manager allows a user to create and maintain his own set of reusable graphic components. it provides commands to add, remove, classify, select, and seek components. Libraries are simply XHTML documents and then can be edited and browsed as any other XHTML document. That allows Amaya to work with local SVG libraries but also with remote shared libraries. The graphic library manager functionality is intended to be extended to other XML elements that Amaya supports.
As SVG support grows, its integration in the Web as an XML language becomes critical. Therefore tools must integrate SVG inline, not as a separate language. Integration must be considered at three different levels: integrating the tool with the Web, integrating complementary functionalities in the same tool, and integrating various types of data in compound documents. Amaya addresses those issues by providing authors with means to handle documents containing SVG and other XML languages transparently, through a large set of common functions, and with the ability to save these changes over the Web.
In order to fully support SVG, the Amaya canvas had to be redesigned. We chose OpenGL for its compliance to SVG requirements, its performance and its well defined API. Graphics, text and animation can be displayed at once using a large common set of routines. Using a WYSIWYG interface, Amaya tends to hide document formats and technologies to provide a universal canvas. Users have a single, convenient way to accomplish their task, without requiring more knowledge.
Now that XML lies everywhere, it must sneak into user hands to give them its full power, without confusing them with the low-level details. Amaya contributes to this goal by providing users with the means to not only browse, but also to create, to edit and to publish text, tables, graphics and mathematics in a seamless way.
[1] Extensible Markup Language (XML), http://www.w3c.org/XML/
[2] SVG Recommendation, http://www.w3c.org/TR/SVG/.
[3] Plugins: http://www.adobe.com/svg/main.html (adobe), http://www.kde.org/svg/ (ksvg), http://www.dessci.com/webmath/mathplayer/features.stm (MathPlayer).
[4] Amaya, http://www.w3.org/Amaya/
[5] OpenGL, http://www.opengl.org/
[6] FreeType 2, http://www.freetype.org/
[7] The World Wide Web Consortium, http://www.w3c.org/
[8] XHTML 1.1 Recommendation, http://www.w3c.org/TR/xhtml11/
[9] MathML 2 Recommendation, http://www.w3c.org/TR/MathML2/
[10] Document Object Model (DOM), http://www.w3c.org/TR/REC-DOM-Level-1/
[11] Cascading Style Sheets (CSS), http://www.w3.org/TR/REC-CSS2/.
[12] XSLT Recommendation, http://www.w3c.org/TR/xslt
[13] Mosaic, http://archive.ncsa.uiuc.edu/SDG/Software/Mosaic/NCSAMosaicHome.html
[14] Resource Description Framework (RDF), http://www.w3.org/RDF/
[15] Expat, http://www.jclark.com/xml/expat.html
[16] Unicode, http://www.unicode.org/
[17] Uniform Resource Identifiers (URIs, aka URLs), http://www.w3c.org/Addressing/
[18] Libwww, http://www.w3.org/Library/
[19] HTTP 1.1 protocol, http://www.w3.org/Protocols/rfc2616/rfc2616.html
[20] WebDAV, http://www.ietf.org/rfc/rfc2518.txt
[21] Annotea, http://www.w3c.org/2001/Annotea
[22] XLink Recommendation, http://www.w3c.org/TR/2001/REC-xlink-20010627/
[23] Opengl implementation notes, http://www.w3.org/Amaya/User/gldev.html