A Database Preprocessing approach for Association Rule Mining

Existing Apriori-like association rule mining (ARM) algorithms suffer from high database scanning overhead especially for high dimensional database. We propose a database preprocessing technique to optimize Apriori-like ARM algorithms addressing the problem. The classic ARM algorithm Apriori and its variations such as DHP, Partition require multiple database scans for finding frequent k-itemsets, which is a time consuming procedure. Our method optimizes Apriori families by utilizing a memory-resident filter to stop unnecessary items in the database transactions from being scanned
during the database iterations. The proposed method can also be applied to reduce data collection cost of ARM in distributed environments. The performance of the method is validated based on both synthetic datasets and real life datasets. Moreover, the simulation results of a mobile agent-based filtering design show that this filtering technique can reduce significant amount of communication cost against various network delays.