Entity Matching for Intelligent Information Integration

Abstract

Due to the rapid development of information technologies, especially the network technologies, business activities have never been as integrated as they are now. Business decision making often requires gathering information from different sources. This dissertation focuses on the problem of entity matching, associating corresponding information elements within or across information systems. It is devoted to providing complete and accurate information for business decision making. Three challenges have been identified that may affect entity matching performance: feature selection for entity representative, matching techniques, and searching strategy. This dissertation first provides a theoretical foundation for entity matching by connecting entity matching to the similarity and categorization theories developed in the field of cognitive science. The theories provide guidance for tackling the three challenges identified. First, based on the feature contrast similarity model, we propose a case-study-based methodology that identifies key features that uniquely identify an entity. Second, we propose a record comparison technique and a multi-layer naïve Bayes model that correspond respectively to the deterministic and the probability response selection models defined in the categorization theory. Experiments show that both techniques are effective in linking deceptive criminal identities. However, the probabilistic matching technique is preferable because it uses a semi-supervised learning method, which requires less human intervention during training. Third, based on the prototype access assumption proposed in the categorization theory, we apply an adaptive detection algorithm to entity matching so that efficiency can be greatly improved by the reduced search space. Experiments show that this technique significantly improves matching efficiency without significant accuracy loss. Based on the above findings we developed the Arizona IDMatcher, an identity matching system based on the multi-layer naïve Bayes model and the adaptive detection method. We compare the proposed system against the IBM Identity Resolution tool, a leading commercial product developed using heuristic decision rules. Experiments do not suggest a clear winner, but provide the pros and cons of each system. The Arizona IDMatcher is able to capture more true matches than IBM Identity Resolution (i.e., high recall). On the other hand, the matches identified by IBM Identity Resolution are mostly true matches (i.e., high precision).