GAIAN Database (hereafter referred to as GaianDB) is a small (four-megabyte) dynamic distributed federated database (DDFD) that was written in Java™. GaianDB is an extension to Apache Derby that federates heterogenous back-end data sources (RDBMS databases, files, text indexes, and so on) using a logical table abstraction layer. However, its principal feature is its ability to autonomically discover and federate other GaianDBs, federating nodes such that a whole network of nodes can be formed. Its connectivity model is "biologically inspired" in that it strives to minimize network diameter and maximize connections to the most fit nodes. GaianDB advocates a flexible "store locally, query anywhere" (SLQA) paradigm.

GaianDB also provides a wide variety of other features, such as dynamic re-configuration; an "explain" network-route query option; in-memory tables and indexing; MQTT (MQ Telemetry Transport) integration; and contextual metadata. GaianDB has already been used in complex, distributed, semantic join queries for text analytics applications and has drawn the attention of significant customers in the military.

GaianDB combines multiple technologies to reach data wherever it resides, whether on sensors, file systems, text indexes or RDBMS. It also includes some concepts taken from federation techniques, mobile ad-hoc networks, grid computing, and autonomic computing technologies in order to manage the overall database with a minimum of human intervention.

GaianDB's self-organizing implementation makes it easy to manage and to cope with failures or network topology changes, and it is easy to deploy GaianDB on thousands of devices and computers with no central management. An automatic query propagation mechanism allows an application to get to the specific data anywhere in the network just as if it were a local database.

The fundamental research enabling the development of GaianDB was part of the USA-UK international technology alliance (ITA) research project. The research and subsequent development were performed by the Emerging Technology Services team in SWG Strategy and Technology.

How does it work?

After you unzip the GaianDB image on disk or on a USB drive, GaianDB is immediately ready to use and can propagate queries to other GaianDBs in the vicinity. As soon as you add new data sources locally by associating them with a logical table definition, GaianDB will be able to respond to SQL queries targeting that logical table and send the result sets back to the callers, wherever they are. A SQL API is available for managing logical tables, data sources, or specific, hard-wired connections to out-of-domain networks. GaianDB automatically recognizes other GaianDBs in the same network domain, so no manual connection is required in that case.

Keep up to date and join the conversation about GaianDB at the GaiandDB blog on developerWorks.

Patrick Dantressangle is a senior technical staff member working in information management for IBM Emerging Technology Services. He has worked on RDBMs since 1990 and at IBM since 1996. Mr. Dantressangle is now working on the next version of DB2®, very large database projects, and the International Technology Alliance (ITA) project. The ideas underpinning GaianDB were initially developed as part of this research project.

Graham Bent, Ph.D., is a senior technical staff member at IBM Emerging Technology Services. He works on a wide variety of customer engagements, specializing in data- and text-mining applications. Dr. Bent is currently a technical lead on the International Technology Alliance (ITA) project. The ideas underpinning GaianDB were initially developed as part of this research project.

David Vyvyan is an IT specialist in IBM Software Group. He has worked at IBM Emerging Technology Services for the past four years on various data/grid and WebSphere® projects, using a wide variety of languages and leading edge technologies. Before that, he worked for three years in Voice Systems as a C/Java developer on WebSphere Voice Response for AIX®.