Data integration is a key issue for exploiting the availability of large, heterogeneous, distributed and highly dynamic data volumes on Grids. In the Grid, a centralized structure for coordinating all the nodes may not be efficient because it becomes a bottleneck when a large number of nodes are involved and, most of all, it does not benefit from the dynamic and distributed nature of Grid resources. In this chapter, we present a decentralized service-based data integration architecture for Grid databases that we refer to as Grid Data Integration System (GDIS). GDIS main concern is reconciliation of schema heterogeneity among data sources. GDIS adopts the decentralized XMAP integration approach that can effectively exploit the available Grid resources and their dynamic allocation. The GDIS architecture offers a set of Grid services that allow users to submit queries over a single database and receive the results from multiple databases that are semantically correlated with the former one. The underlying model of such architecture is discussed and its implementation based on the OGSA Globus architecture is described. Moreover, we show how it fits the XMAP formalism/algorithm.