Frequent value locality and its applications to energy efficient memory design

Abstract

The need in low power processor design is growing due to the reliability problem for high frequency, high temperature processor chips and the expanding market for battery powered mobile devices. The memory hierarchy is a known source of significant power consumption. This dissertation develops low power techniques for two parts in the memory hierarchy, namely the data cache and the off-chip data bus. The proposed techniques are based on new observations of the memory residing frequent values. The study on memory values shows that a small set of frequent values occupy a substantial fraction of memory spaces allocated to an executing program. Those values remain fairly stable over a program run. Moreover, the frequent values are distributed in the memory quite uniformly and periodically. Techniques in identifying the set of frequent values through software method and hardware methods are developed. Those techniques are adopted in the low power applications for the data cache and data bus. A conventional data cache is redesigned into frequent value cache (FVC) so that power consumption is reduced for every access of frequent values. However, this comes with a cost of extra cycles for nonfrequent value accesses. To overcome the loss in speed, a load marking technique is developed so that for a substantial number of nonfrequent value accesses there is no degradation in speed. Experimental results of the FVC design show an energy reduction of 28.8% in L1 data cache is achieved. On the off-chip data bus, an FV encoding technique is developed exploring frequent values. The encoding scheme reduces the total bus switching by using "one-hot" codes for frequent values. Variations of the FV encoding technique are also designed to achieve maximum switching reduction across different configurations and different benchmarks. The FV encoding technique can reduce the total number of bus switching counts 1.5 to 4 times more than that is achieved by other data bus encoding schemes. In addition to the frequent value based cache design, a cache access limiting mechanism is developed to achieve low power from a different angle. A subset of cache accesses is removed by reusing their results in history. The reuse hardware is fine tuned to keep the overhead minimum while achieving low power in the data cache. The reuse hardware for the data cache can achieve 11% net cache energy saving.