Interactive Realities Laboratory

Much work in distributed environments within simulation has been performed and standards have been defined and used. However, these efforts have only taken the field so far and generally is a solution without the problem being fully understood. In the recent years the Institute for Simulation and Training at the University of Central Florida has been performing research in the area of “shared multi-modal environments” for interactive, collaborative worlds.

The goal of VESS is to provide an application base that is useful and functional using today's hardware and graphics and audio libraries, extensible to support future hardware and software libraries, and easily portable to multiple platforms, graphics and audio systems, and application programming interfaces (API's).

Background

IST has previously developed the concept of the “Shared Environment” that would contain all data regarding the dynamic environment in a fully distributed fashion. Clients connect into the “Shared Environment” and query the data each needs, making updates as necessary. While the “Shared Environment” is thought of as a single conceptual component, its initial implementation was as a number of distributed components known as “Services.” To avoid bottlenecks, a “Service” would run on each host in the distributed environment and was responsible for propagating all updates and dealing with any data coherency issues (although the latter was ignored in this initial work).

These “Services” provided a good start to research in distributed, dynamic environments and provided much functionality that was needed. However, there were some drawbacks. The “Services” were all based on the military simulation standard, Distributed Interactive Simulation (DIS). While this worked at the time, as time went on we found that DIS was restrictive for researching new capabilities in distributed virtual environments. In addition to DIS being restrictive, architecturally the “Services” were not as scalable as we hoped. Within a “Service” it worked well as each host contained the “Service” it needed and it avoided the bottlenecks of other architectures (e.g. central servers). However, as the number of databases increased a new “Service” had to be created to contain it (for example, the addition of atmospheric effects required the creation of an “Atmosphere Service”).

To address the issue of multiple “Services” (including the maintenance headache caused by it) we prototyped a Java-based solution based around a single “Communication Service.” The conceptual architecture was similar to that of the original “Services” with distributed instances of the “Communication Service” and potentially multiple clients. In this case, however, client communication was achieved through Remote Method Invocation (RMI). The use of RMI had the advantage of using either a central server and fully distributed architecture.

Conceptual Design

In developing a new architecture we wanted to address many of the issues previously identified. In addition, we wanted to provide a testbed for research throughout the field of distributed interactive (multi-modal) environments. This included a flexible number of databases, a flexible architecture design (to provide for the extremes of a central server and fully distributed system, and everything in between), and flexible network protocols. The latter comes from the need to still support DIS in our applied (military) research yet with a desire to pursue advanced research in interactive environments (with techniques never thought of when DIS was conceived).

To fulfill these needs, we built the General-purpose, Environmental, Modular and Interactive Network Interface (GEMINI). Conceptually, it appears similar to the “Shared Environment” but there are a number of differences including communication and database support.

Flexible Communication

Our first step in developing GEMINI was to generalize all forms of interprocess communication. This included TCP/IP, UDP/IP, shared memory, message queues, and even serial ports. This allows the connection between the instances of the GEMINI servers as well as the connections between clients and GEMINI servers to be of any form. Therefore, any architecture can be formed for research investigations. For example, if multiple GEMINI servers are used (one per host) with a shared memory connection from clients, then a fully distributed system is in use (much like the “Services” discussed earlier). However, if a single GEMINI server is used and clients connect using TCP/IP, then a central server architecture is being used. Using this flexibility, you can even have a hierarchical arrangement within the overall distributed system.

Flexible Protocols

The second step for GEMINI was to design a system for flexible protocols. As mentioned earlier, there is still a need to interoperate with military simulations using primarily DIS (or the High Level Architecture), but we wish to explore advanced multimodal interactions, which will require new protocols. In addition, we could also interface to game engines with our simulations. In order to make a system that could handle both needs, we incorporated a “translator” concept. As data comes into GEMINI from other external components (whether other GEMINI servers or other simulators) it is translated from the protocol on that communication link into an internal “GEMINI” protocol. Similarly, as data leaves GEMINI to be transmitted to other external components, it is converted from the internal protocol to the protocol defined for that link. Flexibility is achieved by having the “translator” dynamically loaded at run-time. Support for a new protocol is added by simply implementing the “translator” for that protocol into and out of the internal “GEMINI” protocol (see figure below). As an aside we also included the translator feature on the client-communication side. While rarely used, it would allow a client whose protocol cannot be changed (such as a commercial game engine) be able to connect into the overall distributed environment.

An XML-based approach for the internal protocol was used as it provides great flexibility including re-ordering, or even elimination, of fields. However, the client interface is designed to hide knowledge of the internal protocol from the clients themselves. In order to translate from protocol to protocol a mapping definition is required that establishes how fields are converted back-and-forth.