Document-Driven Visualization

Alright, here's the flow on document driven visualization. Documents are the lifeblood of the Internet. Right now everything we experience on the web is tied to documents. If it's Flash it's a document...instructions that tell the Flash engine what and how to render. HTML is a document. Any markup is a document. Even if it's not XML...if it's data, it's a document. Don't argue it, just accept it. Be okay with it because documents are just written communication. What are we doing when we use computers to collaborate if not communicating? So, it stands to reason that if visualization is going to thrive on the Internet it had better come to terms a document model...The Document Object Model, to be specific.

So I realized that the DOM makes for an ideal match to visualization, as I was taught it. I learned visualization programming using SGI's IRIS Performer. Performer was a...no, THE rich, powerful API driving almost every immersive virtual environment on the planet in the late, late 90s (maybe it still is). A central concept in Performer is the Scene Graph. A Scene Graph is made of Nodes. Nodes have Parents and Children. Graphs of Nodes can be Traversed. When Nodes are visited (during Traversal) they are evaluated and sometimes modified. The way we created and manipulated an immersive virtual environment was to add, remove, modify, and traverse the Nodes of a Scene Graph. So, it could be said that the complexity of writing any visualization application can be reduced to the repetitive execution of these operations over these Objects in performance of some domain-specific task. Well, it wasn't sooo easy, but the concept was really starting to gel.

Then I learned about a couple of really handy patterns from the Object-Oriented Paradigm that were in the Java core language. "Handy" doesn't do them justice, in fact. To me, Reflection and Serialization are the most important aspects of OOP. In a nutshell, Serialization is the ability for Objects to write themselves (or be written) out to text (XML) and Reflection is the ability for an Object's Attributes to be Get and Set by name.

Scene Graph has Nodes. DOM has Nodes. (Scene Graph is a Graph. DOM is a Tree. Turns out this is not a constraint that matters.) The fundamental operations on a Scene Graph of Nodes are the fundamental operations on a DOM Tree. Nodes are Objects. Even operations are Objects. Objects can be Reflected upon and Serialized to XML. Visualization Objects can be Reflected upon and Serialized to XML. Visualization is a Document. I don't know if that is a straight line but it seems pretty damn solid to me.

This is powerful. Visualization (and interaction) can be published. Visualization (and interaction) can be transformed. Visualization (and interaction) is interoperable. Visualization (and interaction) can be mashed up. Visualization gets the Internet. (Visualization gets natural language?)

Here are some other important factors:

A. We don't serialize everything to XML. Very rarely do we need to spell out all the vertices and normals of a geometry. 3D models do not need to be XML. To me expressing geometry as XML (some is okay) is like expressing the 0-255 value of every pixel in a bitmap. Do what HTML does with images:

"img src="url/to/image.jpg""

"Model url="url/to/model.lwo""

B. Driving visualization with APIs is powerful and there are plenty of occasions for it. But point-to-point API integration does not scale. You want graphs and maps and charts. Prefuse does graphs. Earth does maps. ChartFX does charts. Have fun coding. Instead, programs should use XML as a standard input/output format. Then at least integrators can use ubiquitous technologies to move and transform data from program to program. Visualization programs can even be written by other software programs. It's just rules producing markup fed to a render context. All of this reminds me of the Unix operating system. But with apps and APIs we still aren't getting to the heart of the matter. Apps and APIs are laden with cognitive artifacts for both end users and developers. Whether we are talking about the presentation and interaction layer or the business layer, apps and APIs all have their own ways of doing things. And APIs really only open up apps to developers (i.e., specifically those who take the trouble to learn the API). XML opens apps to developers (i.e., most any developer who understands how to work with XML) and other apps. We're not so much trying to bridge apps with document-driven visualization. Instead, we're trying to bridge visualization across two extremes. At one extreme is the render context...the canvas on which the computer paints. At the other extreme is the human mind; the mental models we construct, the metaphors we understand. Perhaps it is more accurate to say that we are trying to enable visualization through communication medium connecting two points. There is a language that we will express to describe what we want to see and how it should be rendered and behave, regardless of any tool set or application.

C. Document-driven visualization does not make visualization slow, inefficient, or a resource hog. SAXParsers are lightning fast. XML is highly compressable. Either the render engine is fast or it isn't, but that's not XML's fault. We're only using XML to pass instructions to the engine. If anything, the Reflection and OO in general adds overhead. But getting hung up on clocks and RAM as matters of efficiency is to lose site of all the inefficiency created by so much heterogeneiety. The benefits avoid the costs of having low interoperability and far outweigh the costs of clock cycles and memory.

Ok, that's enough for now.

Update 09.04.08: I have since realized that document-driven can "well-formed Unicode text," and not necessarily XML. JSON is a good example of an alternative. Of course there are trade-offs, too. I considered that here