Editing SVG with other XML languages

Paul CHEYROU-LAGREZE
INRIA Rhône-Alpes
Zirst - 655 avenue de l'Europe
38334 Montbonnot -Saint Ismier Cedex FRANCE
Phone : +33 (0)4 76 61 53 58
Fax : +33 (0)4 76 61 52 52
Paul.Cheyrou-Lagreze@inrialpes.fr
Webpage : http://www.w3.org/Amaya

Keywords: Edition; SVG; XML; MathML; XHTML; CSS; Universal Canvas; Compound Documents

Abstract

Amaya is a Web client that acts both as a browser and as an authoring tool. It has been designed by W3C with the primary purpose of demonstrating new Web technologies in a WYSIWYG (What You See Is What You Get) environment. Documents mixing XHTML text, MathML equations, and SVG graphics can be browsed and edited with Amaya. All these XML elements can be styled in a uniform way with CSS (Cascading Style Sheets) style sheets. When edited, those compound documents can be either saved in the local file system or published on remote servers through the HTTP PUT method. Amaya is also able to browse and edit any XML document and to format it through CSS style sheets. This paper presents the overall architecture of Amaya putting the emphasis on the advantage it provides for the implementation of SVG. Specific developments made to handle SVG efficiently are detailed. This includes the use of OpenGL and FreeType,as well as user interaction issues.

Introduction

Modern editing tools provide a way to handle graphics as well as texts and mathematics directly in the canvas, allowing users to author compound documents. But those tools do not aim at interoperability, accessibility and easy distribution. XML [1] languages are a good solution to address these issues. SVG [2] is the graphic part of the XML effort. SVG must be integrated in editing tools as a Web language in order to give its full power to users. The plug-in [3] approach can then be avoided, solving many integration issues. This is important as different XML languages can be combined in a same document.

The architecture of Amaya [4], a full featured Web client, was designed so that its low-level functions handle all XML languages the same way. This leads to several advantages on the implementation side, as a number of functions and structures can be shared. But SVG also implies specific techniques as it is the XML language that require the most powerful canvas. OpenGL [5] and FreeType [6] libraries was chosen for Amaya as it fits in those strong requirements. It also provides a way to handle SVG as well as other XML languages.

SVG as a Web Language

SVG was developed by W3C [7] to allow rich vector graphics to be represented and shared on the Web. XML was chosen as a basis for this graphics language. XML is the common infrastructure for document formats on the Web. It is used not only by SVG but, to name only a few other languages, by XHTML [8] and MathML [9].

Figure 1 : Screenshot of compound document mixing graphics with text and mathematics rendered by Amaya:

SVG+MATHML+XHTML sample

Figure 1shows a screenshot of a document that mixes different XML languages. When a document contains a chunk of XHTML text, some MathML equations, and SVG graphics, it is not just a canvas with three things stuck onto it. It's a fabric woven from three kinds of threads. Textual elements belonging to each thread are subject to the same search and styling mechanisms. Every element is part of the document object model (DOM [10]) and, as such, available to controlling scripts. XML allows different data types to blend and interact. What makes XML interesting is that it promises to fundamentally unify the representation and manipulation of structured data:

The benefit of XML is the synergy of different technologies, all contributing to create a product that is more than the sum of its parts. That is what editing SVG with other XML languages is about, and what Amaya tries to do.

Integration

The above remarks about the benefits SVG draw from the underlying Web technologies are not taken into account by all SVG tools. Most of them process drawings in isolation, in particular authoring tools and players. There is already a broad range of editors for producing SVG files, either natively or through format conversion, but most of them just allow the author to manipulate graphics, without any connection with their environment on the Web. A number of SVG displaying engines are also available, but it seems that most of them are used as plug-ins associated with browsers that have no knowledge of SVG. This greatly reduces many of the advantages high-lighted above.

With plug-ins, SVG is processed like raster images: graphics must be transmitted and updated as separate documents. The plug-in approach also makes integration of SVG with the rest of the document more complex and sometimes impossible. This is important as different XML languages can be combined in a single document. For example, due to current plug-in API limitations, CSS rules associated with the main document can not be transmitted to the embedded SVG objects. Plug-ins break a lot of expected behaviors in editing/viewing drawings:

Moreover, the use of plug-ins forces documents to be split into several files. Therefore compound documents are difficult to handle and make maintaining Web sites more complex.

Amaya tries to address these issues by putting the emphasis on integration. Integration is considered at three different levels : integrating the tool with the Web, integrating complementary functionalities in the same tool, and integrating various types of data in compound documents.

Amaya is a Web client that has direct access to the Web. From the user perspective, there is no difference between a document stored on the local file system and a resource accessed through a Web server. At the user interface level, the same commands are used in the same way in both cases. The Web is seen as a uniform information space and there is no boundary between local and remote resources.

Amaya is a full-featured Web client, in the sense that it provides total access to Web resources. You can not only browse the Web, but you can also modify the resources you access and you can save them back, either locally or on remote servers, provided you have the required access rights. You can also create new Web pages and publish them instantly on remote servers. This approach gets back to the pre-Mosaic [13] times, when Web clients where not only browsers, but included an editing functionality, thus allowing users to play a more active role on the Web.

Amaya can processes XML documents which include not only text (in XHTML), but also graphics (in SVG) and mathematics (in MathML) in any number of combinations. For instance, an XHTML file may contain many MathML expressions and SVG graphics. SVG objects may include mathematics or formatted (XHTML) text which in turn contains mathematics and graphics, and so on, without any limitation.

Amaya and XML structures

Architecture

Amaya handles several XML vocabularies (XHTML, SVG, MathML, RDF [14], etc.). The first step in handling all those languages is to parse the XML syntax. For this task, Amaya uses the Expat [15] parser.

Figure 2 : Document cycle through Amaya Engine

Document cycle through Amaya engine

The XML parser builds a DOM tree corresponding to each XML document. Documents may contain pieces written in different XML languages, provided that namespaces are used for each language transition. DOM trees therefore contain subtrees representing different kinds of structures. Amaya can handle at the same time several documents belonging to several namespaces. For SVG, the multi-namespace feature allows typically a single XHTML document to include several SVG drawings and MathML expressions (including MathML within SVG, through the foreignObject element). The multi-document feature allows users to work simultaneously on several SVG drawings and on a related XHTML document (Figure 2).

Wherever XML structures contain text, content is represented internally in Unicode [16]. Amaya can support different document encodings. However, at parsing time, every text string is converted into UTF-16. At saving time, documents are transcoded back to their original encoding by default, but the user may also change encoding at that time. A single internal encoding is a great help for editing. Users are free to copy (or move) text from one document to another, even if documents were encoded differently and if they belong to different namespaces. This also allows Amaya to process all pieces of text with the same algorithms, wherever they are located in the XML structures. Users can then search for text, replace strings, or check spelling everywhere, including in SVG elements included in XHTML pages. This also allows the text formatter to hyphenate words according to the current language (indicated by the xml:lang attribute). It will then be very easy to implement text formatting within SVG, as soon as this feature is added to the SVG language.

Unicode contributes to a large extent to the internationalization of the Web. It also provides a lot of internationalization features to Amaya. By sharing Unicode with XHTML, SVG benefits from the developments that were made to handle multiple scripts and various writing directions.

To allow users to clearly understand the structure of XML documents, Amaya can display different views of each document. Each view is rendered in a different window. There is a Formatted view that typically shows the document in the same way as most Web browsers do. There is also a Structure view which displays an indented list of elements with their attributes, making the tree structure very clear. There is a Source view, which just displays the XML code. Depending on the document type, additional views may be provided, such as a Table of Contents view for XHTML pages (it renders only Hn elements), or a Link view gathering all hypertext links originating in the document.

The Formatted view is of special interest. Whereas the graphical aspect of other views is predefined, formatting in this view is controlled by CSS style sheets. Formatting is based on the CSS box model and CSS properties apply to these boxes. The same CSS parser, formatting, and style engine are used for all documents, and all parts of documents, whatever their type. This is made possible by the uniform representation of all structures within Amaya. This is the way that SVG elements are styled. In fact, some specific SVG properties have been added by extending the CSS parser and styling engine, but most of the properties are shared with the XHTML and MathML elements.

The immediate advantage of this approach is for style inheritance. As SVG elements are part of the same DOM tree as their surrounding document, they can inherit any CSS property from their environment, without any limitation. This unified approach allows SVG to benefit from all the nice CSS features.

Hypertext links are of paramount importance in the Web, and Amaya provides a lot of features for handling them efficiently. For instance, a simple point-and-click mechanism allows the user to create or change hyperlinks very easily, without manipulating URIs [17]. Links are also automatically transformed (the href attribute is updated) when material is moved from one document to another, with a different base. Many other services are provided to manipulate links. SVG takes advantage of these services. All the links it uses (elements such as a and use, for instance, or links to style sheets) are handled by the same services, with the same interface.

Amaya accesses the Web through the Libwww [18] library which implements the HTTP [19] 1.1 protocol. Amaya uses that protocol to fetch all the resources it needs (Web pages, images, style sheets). It uses the PUT method from HTTP to publish pages remotely. There is a single implementation of HTTP, which is used for transferring all files whatever their format. This was proven especially interesting recently, when the WebDAV [20] extension was added to Libwww. At once, it was possible to use WebDAV to share not only XHTML pages, but also SVG drawings.

Editing

Amaya is a WYSIWYG editor. As described earlier, Amaya displays different views of each document. Views not only provide a way to see a document, but they are also used to interact on the document. Any view can be edited at any time, and changes made in one view are reflected simultaneously in all the other views. Users can thus work on one view and see how the other views are modified at the same time. Obviously, all editing commands are the same in all the views, which provides a uniform user interface.

The user interface provides users a way to handle multiple XML languages at once. In order to achieve this integration, Amaya dynamically adapts its interface according to the languages used in documents. Users can thus edit XHTML, MathML and SVG structures, without need of a complete knowing of underlying technologies.

Experience has shown that many editing operations are performed directly on the Formatted view, probably because it offers a WYSIWYG editing mode. The editing commands provided to style text elements are also available for SVG. Selections (mouse and key events) and commands like copy/paste, spell-checking, search/replace, etc., work in the same way on the XHTML and the SVG text.

Amaya provides a command to create hypertext links. The same generic command is able to generate an XHTML anchor around a XHTML piece of text or a SVG anchor around any SVG element.

CSS is supported in Amaya to allow web authors to define the style of documents independently of their contents. Even if CSS is often used only for styling text, most of its properties apply to SVG equally well, and having a single mechanism to change the color, the font, or the size of characters, wherever they are, makes the user interface more consistent. So painting some text in red is done exactly in the same way as changing the color of a red circle.

Sharing code and structures

The architecture described above clearly shows how important it is to actually share the internal structures and the code to process different types of data in Web tools. When introducing SVG in Amaya, a large part of the implementation was already provided by the overall architecture. It was the same when MathML was introduced. Obviously some specific work was needed for each language, to implement of their own semantics, but all the features they share with other XML languages were already implemented. This not only saves a lot of time to the developers, but it also reduces the hardware resources required to run the tool. Plug-ins typically duplicate a lot of code that is already implemented in their host browser. For SVG they have their own XML parser, their own DOM tree engine, their own implementation of CSS, their own protocol stack. With the architecture chosen for Amaya, all those things are implemented only once.

The specific developments that were made for SVG are detailed in the next sections, but it is important to note how much time and effort can be saved by sharing the implementation of the common features. Annotea [21], an annotation application that was developed on top of Amaya, may reinforce this observation. Annotea allows a user to create and read shared annotations associated with any Web page. Annotations are themselves Web pages that are stored on specific servers and related to the documents they annotate by XLink [22]. This allows users to annotate documents without modifying them, and to attach annotations to any part of an XML document.

As soon as SVG was implemented in Amaya, it was possible to associate annotations to any SVG file, and even to any element within an SVG drawing, included or not within another document. It was also possible to put SVG drawings within annotations.

Addressing the specific SVG Requirements

To completely implement SVG in Amaya, some specific work was required, mostly due to the high graphic quality implied by SVG. Special attention was paid to the display and more specifically to the canvas. Extensions were also needed to display SVG text and to interactively manipulate graphics.

OpenGL features and benefits

While the basic features provided by the X Window System (Linux) and the Windows API were sufficient to render XHTML and MathML, a more sophisticated graphic library was needed for SVG. SVG rendering requires a graphic toolkit able to handle complex paths, transparency, gradients, animation, etc. We looked for a such toolkit available on Windows and Unix platforms. As Amaya is an open source software, the toolkit license also need to be compatible with the Amaya license. We estimate that OpenGL is the library that provides the best answer to these issues.

OpenGL is a procedural 3D graphics API introduced in 1992. OpenGL has become the industry's most widely used and supported 2D and 3D graphics application programming interface (API), bringing thousands of applications to a wide variety of computer platforms. OpenGL fosters innovation and speeds application development by incorporating a broad set of rendering, texture mapping, special effects, and other powerful visualization functions. Numerous books have been published about OpenGL, and a great deal of sample code is readily available, making information about OpenGL inexpensive and easy to obtain. Efficient OpenGL routines typically result in applications with fewer lines of code than those that make up programs generated using other graphics libraries or packages. In addition, OpenGL drivers encapsulate information about the underlying hardware, freeing the application developer from having to design for specific hardware features. All OpenGL applications produce consistent visual display results on any OpenGL API compliant hardware, regardless of operating system or windowing system.

OpenGL defines a state machine that controls the rendering pipeline. State machine attributes are modified through procedure calls. Vertices are always sent the same way to the graphic pipeline. Depending on which states are enabled, rendering is different.

This following code shows how to enable a state and then draw a line using the glvertex function:

...
/* Draw a line defined two points */
/* Enabling state */
glbegin (GL_LINE); 
/* Define points*/
glvertex (x1, y1); 
glvertex (x2, y2); 
/* Disabling state */
glend ();
...

The following example enables different states, and as a results it draws a rectangle with a blue to red gradient. Notice the use of the same set of function glenable, glend, glvertex. Figure 3 shows the result of the execution of this code.

...
/* Draw a rectangle by providing a point 
a width and a height */
/* Enabling state */
glBegin (GL_QUADS);
red = 255; blue = 0; green = 0; opacity = 255;
/* define the color state for the left part of rectangle */
glColor4ub (red, green, blue, opacity);
/* Define points */
glVertex2f (x, y );
glVertex2f (x, y + height);
/* define the color state for the right part of rectangle */
red = 0; blue = 255; green = 0; opacity = 255;
glColor4ub (red, green, blue, opacity);
glVertex2f (x + width, y);
glVertex2f (x + width, y + height);
/* Disabling state */
glEnd ();
...

Figure 3: Result of OpenGL rendering the example 2 code.

Rendering of the rectangle with gradient with OpenGL

OpenGL drivers encapsulate information about the underlying hardware, freeing the application developer from having to design for specific hardware features. All OpenGL applications produce consistent visual display results on any OpenGL API-compliant hardware, regardless of operating system or windowing system.

Finally, OpenGL already provides a large part of the capabilities needed by an universal canvas [23]. This greatly simplifies the implementation of SVG as well as other XML language implementation (Table 1) .

Table 1: OpenGL features that ease SVG and other XML language implementation

Needs OpenGL facilities
Pictures They are mapped on rectangle as textures with bi-cubic interpolation, giving high quality rendering even when zoomed. They are also cached in the video card memory.
Paths They are computed using the tessellation facility of the GLU library, that compose even non convex polygons with a minimal set of triangles and all SVG fill-rule.
Transformation Rotation, scaling, skewing are applied directly on elements or group of elements
Selection OpenGL renders a scene in a "selection" state and can give informations on objects that will be drawn
Opacity Alpha blending provides a means to create transparent objects. Using alpha information, an object can be defined as anything from totally transparent to totally opaque.
Anti-aliasing This technique averages the color of the pixels adjacent to the line. Rendering is smoother and suffers less from screen pixellisation, and thus are more device-dependent.
Animation Double buffering is used to provide smooth animation of objects. Each successive scene of an object in motion can be constructed in an "hidden" buffer and then displayed. This allows only complete images to ever be displayed on the screen.
Speed A list of OpenGL commands may be preprocessed. When rendering these lists, unmodified objects render up to fifty times faster. You can also apply after effects like transformation on those display lists..

Handling Text

Text that is part of an SVG document (or fragment) is specified as standard XML character data. For that reason Unicode support is required for SVG. However, OpenGL is targeted at applications manipulating graphics in 2D and 3D; it does not provide API for text rendering

To address this limitation, we have merged a TrueType font renderer with the OpenGL toolkit. The FreeType 2 library is a free rasterizer for all types of fonts, including TrueType fonts, that works on any machine with an ANSI C compiler and requires no particular operating system feature. FreeType 2 also provides font information like the character bounding box, the kerning and the Unicode character mapping for a font file. Finally, other projects had reported positive results when integrating of FreeType 2 with OpenGL.

The FreeType 2 library can deliver characters in vector and bitmap format. In our SVG implementation, we had to use both type of formats for different reasons:

Vector character format:
Drawing text in SVG requires high level text capabilities in order to render very large fonts or transformed characters (rotated text, text on path). Vector fonts are used for transformed characters. They need more computation as they are described by paths, but give much smoother results. Like all other OpenGL computations, they can be encapsulated in display lists, and thus are rendered very fast.
Bitmap character format:
Amaya also renders text with less needs. XHTML or MathML text is rendered with bitmaps, which is a lot faster. FreeType 2 delivers an anti-aliased bitmap of each character. An opacity value is associated to each pixel of that bitmap, so anti-aliasing and transparency can work together.

Because the previous implementations were not adapted to our needs, we had to redo the integration ourselves. Specifically, our implementation addresses speed and memory consumption problems when using the bitmap character rendering for text:

OpenGL and FreeType 2 allows Amaya to render graphics, text and, animation using the same architecture in a transparent way. This transparency must also apply to the editing interface.

SVG Editing

As explained above, SVG takes advantage of the generic editing commands provided by Amaya that allow users to add CSS properties, to select and edit text, and to create hypertext links. It also takes advantage of the multi-view feature and the possibility to perform editing commands in any of these views. For example, if the user wants to reorder a set of SVG shapes, that can be more easily performed on the structure view than on the formatted view. As all Amaya views are synchronized, the user can see what specific shape is manipulated when editing in the structure view.

However, the generic editing interface is not sufficient to create complex SVG images. A user interface with direct graphic manipulation is necessary. The development of this specific graphic interface is not achieved yet in Amaya. Today, a SVG palette (Fig. 4) permits to insert lines, rectangles, ellipses, polylines, polygons and paths. Selecting a shape in that palette generates the necessary structure that inserts the new SVG element at the current selected position. If the selection is within a XHTML document, Amaya will create the graphic SVG element but also the necessary parent SVG element with its namespace.

Figure 4: Amaya SVG palette

Graphics toolbar

The selection was also adapted to graphics elements. When a graphic is selected, its control points are hilighted. The user uses these control points to move and resize that SVG element.

In addition to the basic SVG editing functions, Amaya provides component libraries. These libraries are used to store predefined drawings and symbol that are often reused. A library is implemented as a set of SVG def elements (components like arrows, blocks, trees, ...) and can be used to create more complex SVG images. Users use them either by copying (with the copy/paste commands of Amaya) or by referencing (with SVG use elements). A benefit of using def and use is that any update of library components will be automatically transmitted to all SVG images that "use" them without any other change.

A library manager allows a user to create and maintain his own set of reusable graphic components. it provides commands to add, remove, classify, select, and seek components. Libraries are simply XHTML documents and then can be edited and browsed as any other XHTML document. That allows Amaya to work with local SVG libraries but also with remote shared libraries. The graphic library manager functionality is intended to be extended to other XML elements that Amaya supports.

Conclusion

As SVG support grows, its integration in the Web as an XML language becomes critical. Therefore tools must integrate SVG inline, not as a separate language. Integration must be considered at three different levels: integrating the tool with the Web, integrating complementary functionalities in the same tool, and integrating various types of data in compound documents. Amaya addresses those issues by providing authors with means to handle documents containing SVG and other XML languages transparently, through a large set of common functions, and with the ability to save these changes over the Web.

In order to fully support SVG, the Amaya canvas had to be redesigned. We chose OpenGL for its compliance to SVG requirements, its performance and its well defined API. Graphics, text and animation can be displayed at once using a large common set of routines. Using a WYSIWYG interface, Amaya tends to hide document formats and technologies to provide a universal canvas. Users have a single, convenient way to accomplish their task, without requiring more knowledge.

Now that XML lies everywhere, it must sneak into user hands to give them its full power, without confusing them with the low-level details. Amaya contributes to this goal by providing users with the means to not only browse, but also to create, to edit and to publish text, tables, graphics and mathematics in a seamless way.

References

[1] Extensible Markup Language (XML), http://www.w3c.org/XML/

[2] SVG Recommendation, http://www.w3c.org/TR/SVG/.

[3] Plugins: http://www.adobe.com/svg/main.html (adobe), http://www.kde.org/svg/ (ksvg), http://www.dessci.com/webmath/mathplayer/features.stm (MathPlayer).

[4] Amaya, http://www.w3.org/Amaya/

[5] OpenGL, http://www.opengl.org/

[6] FreeType 2, http://www.freetype.org/

[7] The World Wide Web Consortium, http://www.w3c.org/

[8] XHTML 1.1 Recommendation, http://www.w3c.org/TR/xhtml11/

[9] MathML 2 Recommendation, http://www.w3c.org/TR/MathML2/

[10] Document Object Model (DOM), http://www.w3c.org/TR/REC-DOM-Level-1/

[11] Cascading Style Sheets (CSS), http://www.w3.org/TR/REC-CSS2/.

[12] XSLT Recommendation, http://www.w3c.org/TR/xslt

[13] Mosaic, http://archive.ncsa.uiuc.edu/SDG/Software/Mosaic/NCSAMosaicHome.html

[14] Resource Description Framework (RDF), http://www.w3.org/RDF/

[15] Expat, http://www.jclark.com/xml/expat.html

[16] Unicode, http://www.unicode.org/

[17] Uniform Resource Identifiers (URIs, aka URLs), http://www.w3c.org/Addressing/

[18] Libwww, http://www.w3.org/Library/

[19] HTTP 1.1 protocol, http://www.w3.org/Protocols/rfc2616/rfc2616.html

[20] WebDAV, http://www.ietf.org/rfc/rfc2518.txt

[21] Annotea, http://www.w3c.org/2001/Annotea

[22] XLink Recommendation, http://www.w3c.org/TR/2001/REC-xlink-20010627/

[23] Opengl implementation notes, http://www.w3.org/Amaya/User/gldev.html


Valid XHTML 1.0!