John Light
Intel Architecture Labs
2111 NE 25th Ave. JF2-64
Hillsboro, OR 97124 USA
jjlight@ibeam.intel.com
Introduction
Intel Architecture Labs’ (IAL) job is to ensure that new uses for personal computers continue to appear and evolve. To that end we research areas that appear important for future uses of computers, but which currently have insufficient support. That is, we try to anticipate future technological bottlenecks.
The Information Architecture group is charged with discovering new ways to use information on the desktop. This field is wide open because almost all Information Retrieval (IR) work is focused on servers, relegating the typical desktop to collecting typed input and displaying text output. Recent advances in processors and disk storage have raised personal computers to performance levels equivalent to most servers, inviting new applications and approaches.
Our work currently falls into two areas: document exploration and personal information assistance. Both areas have both an Information Management component and an Information Visualization component.
In keeping with the purpose of this conference, after summarizing the work being done, I will share some convictions and predictions about Information Visualization that have come out of our work.
Document Exploration
On the document exploration side, we face the dual problems that the current search experience is not satisfying and that the user’s desktop computer is grossly underutilized. We have prototyped a system that delivers metadata to the desktop and allows visual search using the graphics and processing power of the user’s personal computer. The scope of our work includes the definition of the metadata format, tools for expert definition of metadata content, tools for user filtering of the metadata, and tools for creating robust visualizations of selected portions of the metadata.
Personal Information
The personal information assistance side of our work has concentrated on managing small bits of information that are interconnected. The idea is that people need help remembering the important parts of the vast information that overwhelms them each day. The problem is that current filing and retrieval methods are inadequate. The approach is to allow the user to save information without explicit regard to how it relates to other information. When the information is needed, multiple paths are provided to the user, increasing the likelihood that one will be successful.
Visualization is used to allow the user to review, edit, and reorganize the information model. This is generally a background activity by the user in anticipation of later need for the information. We use 2D and 3D representations of the user’s information as a semantic net.
By allowing the user to visualize and manipulate models of personal information, the information store is made both more understandable and more useful.
Convictions
In our work on Information Visualization we have acquired some convictions about the future of visualization which are guiding our work. Our perspective, which comes from working in what is otherwise a multimedia laboratory, may provide a counterpoint to more traditional views. Our primary convictions follow.
Screen real estate will be the most serious shortage in computing for the next ten years. This is already the case, but the shortage will become more acute and obvious in short order. CPU performance has increased according to Moore’s Law. Storage sizes, both primary and secondary, have kept up with the CPU. Graphics performance has lagged the CPU but has reached adequate levels, with no end of improvement in site. The only critical resource that isn’t growing (much) is the size of display screens that every one of us depends on. Even as we talk about larger screens, more people work in smaller cubicles and more people depend on portable computers.
Graphical user interfaces (GUIs) will continue to evolve. Current commercial GUIs, however successful, are not effective in exploiting scarce screen real estate. Not only do they waste screen real estate with their excessive decoration and inflexible layouts, they impose a user interface design discipline which limits the creativity of designers. (For example, they provide little support for integration of 3D.)
3D user interfaces will predominate. Ware has shown that use of the third dimension can multiply effective screen size. As typical 3D graphics performance grows, our need for better screen utilization will overwhelm the problems of effective 3D user-interface design. The new 3D user interfaces will use 2D presentation where it is appropriate, but it won’t look like today’s commercial GUIs.
3D really means 4D. Since true 3D display devices will probably not be commonly available in the near future, 3D will continue to mean for some time three dimensions projected onto the surface of a flat screen. In most cases static 3D images are hard to interpret. What will make 3D successful is the effective use of animation. It takes a 4D presentation (three spatial dimensions plus time) to effectively present 3D on a flat screen.
Unstructured information represents the biggest challenge to visualization. Information that has a natural physical representation such as a map or real world object is not the problem. Years of work on geography and object modeling have inspired commercial packages to handle visualization representations of information that is tightly coupled to already recognizable forms. When information is readily quantifiable, computer graphing software is available to make all sorts of 2D and 3D graphs. Software is even reaching the marketplace for handling general hierarchical information. The unmet challenge is for visual representations of information that doesn’t have a natural structure. The challenged is to provide multiple visualizations that don’t prejudice or limit the user’s ability to make sense of the data.
Best doesn’t always mean most expensive. We all know this in real life, but we have to carry it into our visualization work as well. Especially as we start using 3D visualization of information, we must recognize that some of the shortcuts we take for reasons of development time, processing time, graphics performance, etc. will also make better visualizations. We have found techniques that we originally thought were compromises that turned out better than the uncompromised solution. (An example is the false shadows shown in figure 1.)
Best doesn’t always mean most real. While our ability to render realistic 2D and 3D images on computer screens grows continuously, we mustn’t be seduced by those images. Sometimes virtual unreality can be more effective at delivering information than virtual reality. By recognizing that we are inventing a new medium, we will find ways to use it that don’t have equivalent mechanisms in the real world. If we choose wisely, users will accept them without complaint (or even notice) and be empowered by them. (Examples include false shadows and the constrained distortions introduced by Carpendale’s work.)
Computer game designers are doing the most innovative visualization work today. Unfettered by preconceptions and bursting with innovation, game designers are pushing the limits of the visual presentation. The games are engaging and fun without requiring training.
Corollaries and Predictions
We’re not so sure about the corollaries and predictions as we are about the convictions.
The current user interface paradigm wastes screen real estate. Much screen real estate is dedicated to textual display of information or pseudo-graphical representation (e.g., Windows™ Explorer). While useful, neither fully utilizes the scarce pixels they consume. One of the biggest wastes is the vast expanse of gray background material between the controls of current GUIs. Seldom used controls, rectangular layouts, etc., waste screen real estate because each user interface designer is concerned only with optimizing his/her own application, not the user’s multi-application problems.
Future GUIs will put the user in charge of screen real estate usage. Current GUIs allow users to move and resize windows. Some current applications allow users to scroll inside the windows. Beyond that, with few exceptions, the application designer is in charge of what the user sees. The layout within each application window is so important that product success can depend on it. In the future, applications success will depend on how flexible and configurable window real estate is. How will that flexibility be accomplished:
Future GUIs will incorporate visualization technology. What we see now as visualization tools (e.g., lenses) will become GUI tools, unifying visual presentation on the desktop. Visualizations will not just be "things we do in windows". They will transcend window boundaries and involve multimedia, multiple windows and multiple applications.
Effective visualizations must respect the "left brain/right brain" hypothesis. Whether the hypothesis truly represents brain physiology is beside the point. I believe significant evidence shows that we have two major input channels. One is verbal, symbolic and limited in its ability to handle complexity. The other is visual, concrete, and capable of handling great complexity. Too much visualization today tries to combine these input channels inappropriately, with limited success. Tufte demonstrates his understanding of this in his admonitions to avoid textual clutter (including numbers) in complex pictures. I believe that in the future we will explicitly identify phases of our visualizations as appealing to one input channel or the other, whatever we end up calling them. I think of the hypothesis as a useful metaphor for processes not fully understood.
The most powerful visualizations appeal to the "right brain". This lesson seems to be well known in "print" visualizations, like those exemplified in Tufte’s books. Visual clutter in general is minimized, and text and numbers (requiring right-brain involvement) are largely subjugated to captions, allowing users to shift attention between "left brain" and "right brain" channels. This lesson is not followed as well in computer visualizations. Too often, prominent text appears in juxtaposition with complex graphic information, inhibiting full use of either visual channel. Since both channels are needed for information transfer, our challenge is to make our visual images dynamic enough to appeal at different times, meeting the user’s needs, to each input mode.
Visual distortion will become a major visualization tool. Currently, lenses are used sparingly to provide emphasis in some applications. As we learn to generalize the effects and value of visual distortion beyond simple lenses, its use will become more widespread. For instance, sophisticated distortion can become the basis of new GUI tools that unify visual applications and the computer desktop. A poster presented at this workshop by Carpendale seems to be the first comprehensive attempt to parameterize distortion functionality, a critical step in exploiting this exciting technology.
Advanced visualizations five years from now will seem to today’s eyes like computer games. That is, the user interfaces of applications will employ the same multimedia, dynamics and immersion seen in the best computer games. If you want to see tomorrow’s applications, play Diablo™, You Don’t Know Jack™, and Super Mario Brothers 64™ today. Not only do these games make effective use of scarce screen real estate, they engage the user in a way that can improve productivity and satisfaction.
Game-like user interfaces will require new users. Young people who have grown up with computer games will have no problems adapting. Open minded older users will quickly adapt. Some people will not be able to cope (this includes user interface designers as well as users.) Part of our challenge is to make sure the new world is compatible with as many current users as possible and provide a path for them to get from here to there.
Conclusion
Our work at Intel, targeted almost exclusively at textual information such as document and knowledge repositories, has led us in a somewhat different direction than most other visualization work. We have no standard landscape, geographical or otherwise, on which to base our visualizations. We have tried to invent new landscapes, with varying degrees of success.
The primary difference between our work and others dealing with document and knowledge repositories is our willingness to deal with unstructured spaces. When I asked one prominent researcher why his lab didn’t do the same, his response was "That’s hard." Hard or not, we feel that is where the real opportunities and challenges are.
Meeting those challenges requires a fundamental rethinking of visualization than we have seen recently. That is why I have taken this opportunity to present our most controversial opinions, even though they are currently unproven and not even completely thought out.
To summarize the major points of the disparate opinions:
Acknowledgements
Windows is a registered trademark of Microsoft, Inc. Diablo is a trademark of Davidson & Associates, Inc. You Don’t Know Jack is a trademark of Jellyvision, Inc. Super Mario Brothers 64 is a trademark of Nintendo, Inc.
References
[1] M. S. T. Carpendale, D. J. Cowperthwaite, and F. D. Fracchia,. Distortion viewing techniques for 3-dimensional data. In Proceedings: IEEE Symposium on Information Visualization ’96, pages 46-53.
[2] G. Franck and C. Ware, Viewing a graph in a virtual reality display is three times as good as a 2D diagram. In IEEE Conference on Visual Languages, Conference Proceedings, 1994, pages 182-183.