Tuesday, May 21, 2013

Some benchmarking principles

Here's a nice slide deck on general benchmarking principles that I came across recently: Tokutek benchmarking principles

For hard-core perf types, there's not much new here, but I thought it was worth passing along in case some found it interesting.

Among my favorite observationss from the slide deck:

  • benchmark frequently, to catch performance regressions soon after they occur
  • keep a benchmark history, for analysis of data over time
  • use graphs and plots to help with data interpretation
  • share benchmarking data widely ("developers love data")

At various companies, at various times, I've worked with benchmarking teams, and it's nice to see the slow-but-steady "systematization of knowledge" going on here, since some of these are hard lessons and it's worth your time not to re-learn them yourself.

No comments:

Post a Comment