Research Repository

On Memory and I/O Efficient Duplication Detection for Multiple Self-clean Data Sources

Zhang, J, Shu, Y and Wang, Hua ORCID: 0000-0002-8465-0996 (2010) On Memory and I/O Efficient Duplication Detection for Multiple Self-clean Data Sources. In: Database Systems for Advanced Applications, 01 April 2010-04 April 2010, Tsukuba, Japan.

Full text for this resource is not available from the Research Repository.


In this paper, we propose efficient algorithms for duplicate detection from multiple data sources that are themselves duplicate-free. When developing these algorithms, we take the full consideration of various possible cases given the workload of data sources to be cleaned and the available memory. These algorithms are memory and I/O efficient, being able to reduce the number of pair-wise record comparison and minimize the total page access cost involved in the cleaning process. Experimental evaluation demonstrates that the algorithms we propose are efficient and are able to achieve better performance than SNM and random access methods.

Item Type: Conference or Workshop Item (Paper)
ISBN: 9783642145889
Additional Information:

Series title: Lecture Notes in Computer Science, vol. 6193

Uncontrolled Keywords: algorithms; duplicate detection; data sources; data management
Subjects: Current > FOR Classification > 0806 Information Systems
Historical > Faculty/School/Research Centre/Department > Centre for Applied Informatics
Depositing User: Symplectic Elements
Date Deposited: 29 Jun 2015 00:18
Last Modified: 21 Jun 2018 04:23
ePrint Statistics: View download statistics for this item
Citations in Scopus: 0 - View on Scopus

Repository staff only

View Item View Item

Search Google Scholar