Question

Which of the caches/TLBs in a processor are in the critical path of instruction execution? How does this impact on the design of these structures?

Answer 1

Modern desktop PC and server CPUs have three independent caches:

An Instruction cache which speeds the executable instruction fetch,

Data cache to speed up data fetch and store, and

A Translation lookaside buffer (TLB) used to speed up virtual-to-physical address translation for both executable instructions and data.

Among these three data cache and TLB are present in the  critical path of instruction execution. As data gets fetched, decoded and executed.

Critical-path instructions, and their dependents, tend to get stalled in the instruction queue, and often become the oldest (bottom) instruction in the queue at some point. The columns labeled “IQ latency” and “Oldest in IQ” in Figure 1 show for each instruction, the average number of cycles spent in the instruction queue, and at the bottom of the instruction queue, respectively. These numbers correlate well with the critical path.

caches modeled are more modest, to compensate for the relatively small memory footprint of most of our benchmarks. The fetch unit can fetch up to 16 instructions per cycle from up to three basic blocks per cycle. This simulates the behavior of an effective trace cache.

As this effects the performance of the processor RAM has to increase inorder to provide more space to get the instructions to be executed. As processors increase their ability to exploit ILP in the instruction stream, application performance becomes more tied to the execution of the critical dependence path. Optimizations that accelerate critical path execution will have an increasingly large advantage