19th International CODATA Conference
Category: Interoperability

The Space Physics Interactive Data Resource (SPIDR) development in preparation for the electronic Geophysical Year eGY-2007

Dr. Mikhail N. Zhizhin (jjn@wdcb.ru), RAS/CGDS, Head of Telematics Lab, Institute of Physics of Earth and Geophysical Center, Russia
Eric A. Kihn (Eric.A.Kihn@noaa.gov), NOAA/NGDC, USA


The Space Physics Interactive Data Resource (SPIDR) is a distributed network of synchronous databases and application servers designed to allow a space weather modeling and prediction customer, or application, to intelligently access and manage historical space physics data for integration with virtual environment models and real-time space weather forecasts. Each SPIDR server resides at a parallel computer cluster and provides fuzzy logic based searching on a set of databases of space weather parameters. The system is designed to allow a user to specify desired spatial, temporal, and parameter conditions in fuzzy linguistic and/or numeric terms and to receive a ranked list of events best matching the desired conditions in the historical archive. Once discovered, the client can request dynamical temporal and spatial visualization using a set of communicating Java applets, browse the archive of Sun and Earth satellite images, and request delivery of the data formatted for inclusion in model runs. Each SPIDR server has a database management interface, which allows data updates performed either by a local user or by another SPIDR server from the Net. The servers communicate to each other for scheduled mirroring of the data and software. Algorithmic and programming patterns developed and utilized in SPIDR are general enough to be used in other Internet-based scientific data mining and visualization systems.

International bodies involved in the data exchange within SPIDR are World Data Centers for Solar-Terrestrial Physics. The project has two key development centers: one at National Geophysics Data Center (NGDC) in Boulder, CO, USA; another in Center of Geophysical Data Studies Russian Acad. Sci. (CGDS) in Moscow, Russia. The SPIDR network nodes are installed in USA, Russia, Australia, Japan, South Africa, and China.

In 2003 NGDC performed a SPIDR usability study using an independent web-development company. Results and recommendations of the study are incorporated in the new version of SPIDR system. In 2004-2006 we plan to extend the new version of SPIDR with the three main goals: 1) to implement real-time data loading and synchronization between nodes, 2) to provide automated and manual data quality verification tools, and 3) to improve usability of the web interface. All the tasks require international cooperation between SPIDR software developers, data managers, and system reviewers and testers. The resulting portable and extendable open source software product will be used by CODATA task forces and World Data Center system as a data storage, verification and dissemination tool in the framework of electronic Geophysical Year in 2007.