This project aim at providing new tools for fast development of application-control User Interfaces (UIs) on Virtual Environments (VEs). Our approach allows developers to build sophisticated UIs containing both simple widgets (such as windows, buttons, menus and sliders) and advanced widgets (such as hierarchical views and web browsers) with minimum effort. Existing GUI toolkits for VEs are still too simple; they allow only a limited number of GUI components and often lack visual authoring tools. In contrast, existing GUI toolkits for 2D desktop environments are mature, include powerful authoring tools and provide a wide range of widgets.

We proposed a new approach to improve GUI prototyping for virtual environments. This approach allows developers to re-use existing 2D GUIs and embed them into virtual environments. It is based on monitoring and capturing the 2D displayed contents (windows) and embed them into the virtual environment as 3D windows. Once embedded, the user is able to interact with them using standard 3D interaction techniques (selection and manipulation), and in contrast to VNC and other frame-based approaches, the application has knowledge of the GUI structure. Our approach minimizes the number of lines of source code that need to be modified to migrate the GUI of an existing application to a VE. As migration, we refer not only to the graphical representation of the widgets, but also to their behavior.

This approach allows 3D UI developers to take advantage of the increasing number of components, layout managers and graphical design tools provided by 2D UI toolkits. The architecture of the system is described in the following paper.


Library (C++) capable of generating 2D graphical user interfaces embedded in 3D virtual environments. Provides a lightweight framework for embedding 2D GUI in 3D virtual environments generated by virtual reality applications. The library uses the existing well-known GUI toolkit Qt in order to generate the 2D GUI and creates 3D proxies of the GUI embedded inside the virtual world. The main features are that the generation of the GUI is transparent to the application, it can be integrated which any application using OpenGL as graphical renderer and the user can interact with the embedded GUI using standard 3D selection and manipulation techniques.