Optimization For Dynamic Inverted Index Maintenance

Abstract

For free-text search over rapidly evolving corpora, dynamic update of inverted indices is a basic requirement. B-trees are an effective tool in implementing such indices. The Zipfian distribution of postings suggests space and time optimizations unique to this task. In particular, we present two novel optimizations, <i>merge update</i>, which performs better than straight forward block update, and <i>pulsing</i> which significantly reduces space requirements without sacrificing performance.