General performance optimization approach

Up to now, we listed the following classic optimization techniques:

• Optimal algorithms
• Optimal data structures
• Caching
• Precomputed tables
• Preallocation and custom allocators
• Buffering and batching
• Read-ahead
• Copy avoidance
• Finding a better library

With our current stand of knowledge, we can formulate the following general-performance optimization procedure:

1. Write your code, avoiding unnecessary pessimizations where it doesn't cost much, as in the following examples:
   • Pass parameters by reference.
   • Use reasonably good, widely known algorithms and data structures.
   • Avoid copying data and unnecessary allocations.

This alone should give you a pretty decent baseline performance.

1. Measure the performance, find the tight spots, and use some of the standard techniques listed. Then, measure again and iterate. This step must be done if the performance of our program isn't satisfactory despite our sound programming practices. Unfortunately, we can't know or anticipate everything that will happen in the complex interplay of hardware and software—there can always be surprises waiting for us.
2. If you still can't achieve good performance, then your hardware is probably too slow. Even with performance optimization techniques, we still can't do magic, sorry!

The preceding advice looks quite reasonable, and you might ask: Are we done? That wasn't that scary! Unfortunately, it's not the whole story. Enter the leaky abstraction of modern processor architectures.