Models and Simulations as Data

A colleague, Anthony Watkins, and I recently conducted a feasbility assessment of ETL-V applied to a DoD tactical simulation called JCATS:  Joint Conflict and Tactical Simulation.  JCATS was created by Lawrence Livermore National Labs in 1997 and has been supported in some fashion or another ever since.  JCATS is not unlike other military sims I've encountered over the years; OTB, TDF.  In fact, JCATS is generally not unlike all other models and sims with which I have worked in the past 11 years - aerospace apps, power/energy sims, custom apps.

I AM NOT A MODELING AND SIMULATION EXPERT.  

I took a class in graduate school, did pretty well.  But I studied Computer Science and Applications.  That's my slant.

When I reflect on my scant 11 years of experience designing, developing, implementing, and integrating business processes and software - a fair amount of it in the M&S domain - I come to one conclusion.

Interoperability stinks.

A bit of personal history on why that I think that - three words:  separation of concerns.  As in, we haven't maintained it.  I observed the problem in my first job at ODU Research Foundation, Center for Coastal Physical Oceanography:  oceanographers were writing code...bad code.  Eventually the computer scientists got involved, but all we did was write occasionally elegant code wrapped tightly around a domain application.

Then, in September of 2001, I found Object Serialization, Reflection, and XML all at the same time (XML, casually, in '99).  My colleagues and I were using these to create 3D visualization, behavior, and human-computer interaction for the Web.  With the introduction of XML-RPC we were communicating with "back end" data services to drive client-side 3D.  The 3D applications were entirely dynamic without compilation,  the same way all other content was being delivered over HTTP to a standard browser.  It was cooler than dirt.  

Along came Web Services, circa 2003-4 for us. Finally, everything was starting to click.

That brings me back to models and simulations as data and where I am in 2009.  

Simulations aren't applications, they're data sources.

I've written previously about how we use enterprise architecture models as data sources for analytics.  It's basically the same thing for sim integration.  There is one additional component.  Simulations often stream data in real time.  The transactional model of the Web and relational databases is not always sufficient.  Regardless, the methodology is not dependent on transfer rate or protocol.  The methodology is based on separation of concerns.  The difference between yesterday and today is that we realized how to take what were doing above the level of a single application.

Most of the applications with which I am experienced are heavy...very heavy.  Whether we are talking about footprint, documentation, training, number of participants, facilities, or duration most simulation apps require a lot of it.  Technology insertion and training are costly.  Learning curves are steep.  Exposure and knowledge is limited to very small groups.

It doesn't have to be that way, either.

[Update:  12/10/08 - Well, maybe it does.  I appreciate that M&S is hard and hard takes time.  The point I really want to make is that we can and should reuse the data generated by M&S to increase their ROI and overall value of their insights as information assets.  We do this by decoupling their data so it can be more easily integrated with other things - things people who aren't modelers or enterprise architects care about.]

If I could wave a magic wand, I'd strip every simulation down to it's algorithms and databases.  To be sure, there are some sweet front-ends out there, but they aren't maintaining separation of concerns.

JCATS, for example, is a powerful tactical simulation.  There are good smarts in there.  But JCATS has a limited graphical user interface (2D) and is strictly designed for tactical operational scenarios on maps.  While the designers of JCATS may have thought about 3D and some statistical diagnostics, they certainly did not, nor could they have anticipated or accomodated all the many ways we can use the output of JCATS simulations.

The good news is that JCATS saves huge DAT files, i.e., data files, i.e., JCATS data is portable.  JCATS produces ordinary delimited text files (comma-separated) and puts similar data on the wire in real time (either TCP/IP or UDP, I think).

From here it's easy:

All of these interactive views are transformations of JCATS data using ETL-V

Pentagon Internet Utility Model

Pentagon gets it right:

Now that you have a space in your cloud to put all your videos, you can come up with much simpler plans for multi-level security in that cloud using technology means that have been thoroughly vetted for decades.  Better results with less technology burden.  This can solve nearly any of your needs, not only the just-in-time requirements driven by the popularity of personal video production and sharing.  Certainly every unclassified activity is covered and that alone makes the world better by several orders of magnitude.  But, replicate this model on NIPR, SIPR, and JWICS and every possible thing you could want to do with digital video will be covered by (roughly) the same technology solution.  People who have to manage security layers will still have to manage security layers, but that concern is more easily separated from the entanglements of technology now.  This is a good thing all around in ways I hope and think you realized before I ever commented here.  

This is why you're hot.

More thoughts on Federal Agency Internet Utility Model.

P.S. "OneSource" is a bad term.  When it comes to shared content, there is never one source.  That's the point.  You are the infrastructure, just need to figure out how to make that catchy for PR and marketing.  One Destination, maybe.  ...needs work in any case.

A Portable Language for Real Time, Interactive, Rich Media

Following is a draft section from an in-progress whitepaper on ETL-V:

The key to enabling visualization as a core capability is robust description of functionality that is machine portable and parsable through a universal processor that enables the serialization and deserialization of any object, attribute, and behavior in a standard way. Such a description captures the language of visualization in a way that can be understood by any program or person.

Our machine portable language is not intended to express low-level artifacts like vertex points and surface normal vectors, although it can. Most often we follow the model of HTML, which uses URLs to reference images, movies, and other digital multimedia. A 3D Model is not unlike an image:

<img src="images/car.jpg" />

<Model url="objects/car.lwo"/>

Moreover, our language expresses high level artifacts that describe things a person without prior subject-matter expertise might easily recognize. Camera, Model, and Light are but a few examples of rudimentary artifacts in our language. Yet, to establish a robust platform for visualization we must be able to describe artifacts of greater complexity. We must describe the attributes of, for example, ColumnChart, AreaChart, FlowChart, TreeMap, and RadialTree.

Some objects have implied behaviors such as a Sequencer that iterates through a list of items or a Directive that traverses a graph.

Our language must contain more than nouns; it must also contain verbs. Therefore we have command objects like Append, Remove, Interpolate, Morph, Notify, and many others (see also Command schema, below).

Our language must be capable of describing attributes having multiple dimensions, for example a 3-dimensional position. We classify attributes as belonging to one of four basic categories: primitive, complex, compound, and collection. These are defined as follows:

1. Primitive Attribute – an attribute that is expressed using a single value; ex., integer, floating point number, character; we also consider string to be primitive
2. Complex Attribute - an attribute that is expressed using multiple primitives; ex., 3D vector of floats
3. Compound Attribute – an attribute that is itself an Object containing other attributes; Animation has Keyframe attribute, Keyframe is an Object with its own attributes
4. Collection Attribute - an attribute that represents a data structure, such as a list, vector, or hash map. A Collection attribute can hold any other kind of attribute.

Given this language, we can express any visualization technique, behavior, or interaction style as a human and machine readable document capable of being interpreted at runtime. To demonstrate our concept for this machine portable language, we have created Bridgeworks Markup Language (BML). BML is defined using Extensible Schema Definition language (XSD). Documentation and source schemata are available online in the following locations:

Documentation of BML:

http://dev.bridgeborn.com/Bridgeworks/Schemata/doc/Bridgeworks.html

Schemata: (relative to http://dev.bridgeborn.com/Bridgeworks)
/Schemata/Bridgeworks/Bridgeworks.xsd
/Schemata/Common/Command/Command.xsd
/Schemata/Common/Directive/Directive.xsd
/Schemata/Common/Evaluator/Evaluator.xsd
/Schemata/Common/Node/Node.xsd



[Update 12.02.08: realized the schemata listed above do not contain all of the elements documented...not sure what happened...see BwSchools for full list. Schemata are best viewed in XMLSpy.]

We use Bridgeworks to provide our customers with the largest and most flexible set of visualization capabilities available to them on the market. We deliver these capabilities through a single lightweight runtime that scales between personal devices and large facilities. The runtime engine operates through simple input/output mechanism for parsing text-base, well-formed data, making it easy to integrate without a lot of API programming.

Platform Functional Requirements:

1. The capability must provide graph layouts (ex., radial tree, cone tree, force-directed graph)
2. The capability must provide chart layouts (ex., bar charts, line charts)
3. The capability must provide geospatial layouts (ex., maps, globe, terrains)
4. The capability must provide high-fidelity 3D modeling and animation
5. The capability must provide video and audio
6. The capability must provide hybridized views (ex., Extrude Map: geospatial + chart)
7. The capability must provide split views (ex., side-by-side comparison)
8. The capability must provide layered views
9. The capability must provide a temporal dimension
10. The capability must provide a mechanism for user-defined attributes
11. The capability must provide a mechanism for user-defined events, and triggers
    

Platform Technical Requirements:

1. The capability must provide multi-dimensional graphics (2D, 3D, 4D+)
2. The capability must provide an engine that can change views and behaviors at runtime using only a text based, interpreted language that does not require compilation in order to add/modify/delete content in real time
   
3. The capability must provide serialization and de-serialization of data and objects into and out of rich media views using:
   1. XML
   2. JSON
   3. CSV
4. The capability must provide serialization and de-serialization of standard object notations (SONs, ex., JSON) into and out of views
5. The capability must provide Object/Attribute Reflection
   
6. The capability must provide storage of domain data as Objects’ attributes loaded in memory
   
7. The capability must provide an extensible API
   
8. The capability must provide ECMA script language binding
   
9. The capability must provide COM language binding
   
10. The capability must provide Java language binding
    
11. The capability must be able to render from multiple 2D and 3D file formats, including but not limited to:
    
1. ESRI ArcView Shape
   
2. LightWave Object
   
3. Keyhole Markup Language (KML)
4. CAD Models (AutoDesk, SolidWorks)
   
5. JPEG
   
6. Bitmap
7. Extensible to other formats as needed


12. The capability must be able to render from data located in local and remote URLs
    
13. The capability must embeddable in the following software applications:
    
1. Internet Explorer versions 6 & 7
   
2. Microsoft Office 2003 Products: Word, Excel, PowerPoint
   
3. Mozilla Firefox
   
4. Win32 Graphical User Interfaces (GUIs)
   
5. .Net/C# GUIs
   
6. Java GUIs
   
14. The capability must provide an XML schema and a document object model (DOM)
    

Visualization as a Service

Visualization as a service is an extension of our core capability concept of visualization as a platform. Because we can express views, behaviors, and interaction models without deviating from our standards for reasonable portability we can publish and share views, behaviors, and interaction models just like any other content. This means that web services and other data servers can produce human and machine readable documents that describe visualization, behavior, and interaction. Furthermore, these documents can be parsed to create or modify any visualization, behavior, or interaction technique at runtime without intermediate compilation. Depending on transfer frequency, volume of data, and security constraints, data describing visualizations, behavior, and interaction can also be cached for use in “offline-mode” with so-called “rich Internet applications” (RIAs).


[Update 03.14.2009 - Just watched Tim Berners-Lee's TED Talk on Linked Open Data and recently watched Vint Cerf's Feb 09 lecture to FAA. Want to add here that a document-based approach is the best way to ensure that visualization follows the important lessons taught by these great teachers. Binary objects can be be linked using HTTP and metadata, but binary objects are not open. Binary objects also always require a specialized software reader to be interpreted. Documents, on the other hand, are what the Web was designed to link and text will always be readable by readily available software on the web. Even if one rendering engine disappears it is always possible for another to come along; ex., if rendering an old visualization document as graphics ever becomes important hundreds of years in the future. I realize I am contradicting this principle by encouraging the use of existing binary formats for images and 3D models, but this is a compromise I am willing to make at this point. Check out BML Hello World. KML is important and wonderful, but it is just scratching the surface.]

Tufte on document-based platforms

This just posted at the end of a new thread on Ask E.T. about Google Chrome:

(Emphasis added by me)

"How's the browser?

Thanks to the many contributors who pointed out Scott's kind link in the example of amazon search.

David Pogue's report in The New York Times suggests this browser leads to an integrated platform with non-proprietary formats (very good) but is still application-based rather than document-based (not very good)."

-- Edward Tufte, September 3, 2008

This is something I think constantly about - document-based platforms can serve end-to-end needs from data extraction, transformation, portability, and visualization. See also, the flow on Document Driven Visualization

Wonderfully Simple Praise of the Hyperlink?

(Prologue: This title comes from posts written by Ben Fry and Jeremy Keith. I hope they will not mind. I find myself increasingly attracted to what they have to say on technology matters I care about.)

Here is near perfect example of a complexity problem, that I have yet to define, when comparing the integration of two APIs: Facebook and del.icio.us. One API seems to indicate what happens when you let developers define use cases and one shows what happens when you actually stop to think about how information is used by consumers. (I can say that because I write code.)

So in this case, I want to get a list of my friends. It's a dead simple request. I tell the platform I want my friends. I get back a well-formed (hopefully) document containing my data about my friends, right?

Exhibit A, The Facebook API:

facebook.friends.get

The name suggests a clean, intuitive interface...until we get into how we use the interface.

facebook.friends.get has nine parameters. Two of the parameters are separate authentication values.

The parameter call_id is documented as follows:

"The request's sequence number. Each successive call for any session must use a sequence number greater than the last. We suggest using the current time in milliseconds, such as PHP's microtime(true) function."

Which is to say that the caller is responsible for scheduling calls with the service?! Why do we have to agree on a "sequence number" in order to do business?

How about the "variable" that isn't:

string v This must be set to 1.0 to use this version of the API.

Which is to say, "Please pass me this parameter even though I already know the value."

It's a "placeholder," right? Because in "the future" we might want to leave room for specifying which version we want...per interface.

It's goes on and on. Then you tell facebook.friends.get what format you want, then whether or not you want the results wrapped in a function call so you can get around cross-domain data security, yada, yada... And don't forget to hash the URL first! I'm only talking about one endpoint here.

Ugh.

Am I wrong to be disappointed? Am I the only one who is confused? Who thinks this is unnecessary? This is the standard API-centric view of interoperability but I'm not buying it.

Now, I have little doubt there is a good reason for every parameter and for this kind of programming. I've heard every reason. I argued many of them myself...until I learned better. To gnash teeth over the legitimacy of each thing is to argue details regardless of their relevancy to the matter at hand. The matter at hand is that I am requesting my friends from the platform and I am expecting back a well-formed document as a result. That's all. From here the platform should do everything it possibly can limit the amount of additional information it requires of me. I'll grant that platform expects me to authenticate my request.

The problem with facebook.friends.get is that it is trying to serve multiple use cases through a single request, as if that is more efficient or more simple. Most of the use cases serve developers (or so they may think). Most use cases are more easily supported than the basic task I am trying to perform. Remember? It's "get my friends. " I find myself restating this important detail like a mantra in order for it to have a greater voice. Why is the service burdening the consumer with all of this minutia for a simple data call that should return a document?

Exhibit B: The del.icio.us interface for the same data:

http://del.icio.us/feeds/json/network/prestidigital

It's one simple URL. It returns well-formed data as JSON. Oh, the simplicity! Oh, the possibilities!

Detractors would no doubt point to the fact that this URL is neither authenticated nor encrypted. I would counter: that was a choice made to serve customers in this case as delicious certainly supports authenticated, encrypted interfaces that are just as easily used as /feeds/json/network. Beyond that, everything else is a decoration.

And, OK, so developers do need more options and features than can be provided through a simple URL interface. But those should be added as one or more layers on top of a layer that just provides well-formed data. I'll accept that if someone can show me: where's the layer that provides the data? It's obfuscated by the business layer. This is no doubt an artifact of the traditional three-tiered architecture (a model of which I am a proponent by the way).

On the Internet, the way in which data is located is via the URI and the way it is passed is HTTP. It's rudimentary: every container of data has a uniquely addressable identifier. Web service endpoints are URLs. We need to get developers and Internet platform providers back to this fundamental principle.

Exhibit C: Yahoo! Internet Location Platform:

http://where.yahooapis.com/v1/place/2507854

Click the URL, get back an XML document. Now THAT'S data portability; that's interoperability.


Update 11.17.09: BUMMER, this link is broken. It looks like an auth issue.

Best Practices of Well-Formed Data?

Update 2008/05/06: It occurs to me now that this works for me due to my dependence on Western (American English) style characters: http://benfry.com/writing/archives/82... My concern is readability by humans and for me that means American English. Unicode is a super set, so I have modified my post accordingly.

I have come to realize that there are exactly three (3) formats for well-formed data which suit all of my interoperability needs:

1. XML
2. ASCII Unicode standard object notation
1. (i.e., script object notation, i.e., SON, i.e., Object.property as text, ex. JSON)
3. ASCII Unicode standard delimited notation
   
1. (ex., comma-separated values)

Each of these has upsides and downsides.

Use XML when you want to be the most versatile with your data. Besides being ubiquitous, XML serializes and deserializes quickly and easily, represents OO clearly, and can be easily, automatically validated. XML presents itself well for both people and software. Especially use XML when you want to do advanced searches on data or transform the data from one domain-specific language (DSL) to another. DOM and XPath queries do lose their utility as data sets get large, but these situations can be avoided and parsing remains fast even on files many megabytes in size. Batch processing gigs of data is not a job for XML. (Use a relational database for that.)

Use SON when all you want to do is serialize/deserialize data to/from business objects or standard data structures (ex., arrays, maps). JavaScript Object Notation, or JSON, is super convenient for Javascript apps and JSON "liblets" (very small libraries, usually only 1 file) are easily found and maintained with your own code. SON also represents OO clearly, and given a pretty-print function, presents itself well to both people and software. But, there is currently no way to validate a SON. Not everything can handle a SON either. JSON doesn't work with languages that aren't JavaScript*, although there's nothing about JSON that is proprietary to Javascript so it's just a matter of time. JSON is just formatted text. ASCII Unicode-based SONs may one day be the XML killer but that time is a long way off.

Use CSVs when you want to be super fast on the throughput and can make assumptions that the data is simple to interpret. CSVs work well when moving very large amounts of data over a wire and/or into and out of relational databases. CSV data does not represent OO in any way, it does not present itself well to people, and there is no standard way to automatically validate it. CSV can be super efficient, but it is error-prone, can be difficult to debug, and is not nearly as versatile as the other two formats.

A good data service should support all three of these whenever and wherever it is feasible. And here is the obvious point that I do not want to escape from me - these 3 formats are all any data service ever need provide. Any service that supports each of these will be "universally" (practically) interoperable. If all data services supported exactly these 3 then all the Web would be an oyster.

*Update: over at JSON.org you can find ports for every language under the sun. Actually, what I meant is that support for JSON isn't native in the main platforms. Hopefully it will be built into all of these languages in the future.

Social Network Portability or Portable Social Networks?

I find it wickedly ironic that all the worthwhile activity around social network portability and data portability is trumped by ordinary human behavior:

Note that what Julian asserts there is valid, in my opinion: a whole lot of thought and knowledge, both good and bad, was poured into SNP in the "early days." If the people have moved from one group to another then perhaps it is safe to assume the knowledge went with them. But if this group died and the other emerged...then what?

And if that weren't confusion enough:


So here we are trying to keep up with the conversation - which can be extremely difficult in active groups - and the conversation keeps moving...figuratively and literally.

"Wait. It's not just social network portability. It's ALL data portability. Let's all move up to data portability."

"But 'all' data portability is too big. There are too many facets. Let's all move down into the various facets (presumably according to our interests and perspectives)."

Down that path madness lies if one isn't careful. Managing the order of things when it comes to the Internet can drive one crazy enough as is.

I do admire them for trying. Progress is being made. Data portability is intensely important for connected computing and process interoperability.