12.18 — Timing your code

When writing your code, sometimes you’ll run across cases where you’re not sure whether one method or another will be more performant. So how do you tell?

One easy way is to time your code to see how long it takes to run. C++11 comes with some functionality in the chrono library to do just that. However, using the chrono library is a bit arcane. The good news is that we can easily encapsulate all the timing functionality we need into a class that we can then use in our own programs.

Here’s the class:

That’s it! To use it, we instantiate a Timer object at the top of our main function (or wherever we want to start timing), and then call the elapsed() member function whenever we want to know how long the program took to run to that point.

Now, let’s use this in an actual example where we sort an array of 10000 elements. First, let’s use the selection sort algorithm we developed in a previous chapter:

On the author’s machine, three runs produced timings of 0.0507, 0.0506, and 0.0498. So we can say around 0.05 seconds.

Now, let’s do the same test using std::sort from the standard library.

On the author’s machine, this produced results of: 0.000693, 0.000692, and 0.000699. So basically right around 0.0007.

In other words, in this case, std::sort is 100 times faster than the selection sort we wrote ourselves!

A few caveats about timing

Timing is straightforward, but your results can be significantly impacted by a number of things, and it’s important to be aware of what those things are.

First, make sure you’re using a release build target, not a debug build target. Debug build targets typically turn optimization off, and that optimization can have a significant impact on the results. For example, using a debug build target, running the above std::sort example on the author’s machine took 0.0235 seconds -- 33 times as long!

Second, your timing results will be influenced by other things your system may be doing in the background. For best results, make sure your system isn’t doing anything CPU or memory intensive (e.g. playing a game) or hard drive intensive (e.g. searching for a file or running an antivirus scan).

Then measure at least 3 times. If the results are all similar, take the average. If one or two results are different, run the program a few more times until you get a better sense of which ones are outliers. Note that seemingly innocent things, like web browsers, can temporarily spike your CPU to 100% utilization when the site you have sitting in the background rotates in a new ad banner and has to parse a bunch of javascript. Running multiple times helps identify whether your initial run may have been impacted by such an event.

Third, when doing comparisons between two sets of code, be wary of what may change between runs that could impact timing. Your system may have kicked off an antivirus scan in the background, or maybe you’re streaming music now when you weren’t previously. Randomization can also impact timing. If we’d sorted an array filled with random numbers, the results could have been impacted by the randomization. Randomization can still be used, but ensure you use a fixed seed (e.g. don’t use the system clock) so the randomization is identical each run. Also, make sure you’re not timing waiting for user input, as how long the user takes to input something should not be part of your timing considerations.

Finally, note that results are only valid for your machine’s architecture, OS, compiler, and system specs. You may get different results on other systems that have different strengths and weaknesses.

12.x -- Chapter 12 comprehensive quiz
12.17 -- Nested types in classes

95 comments to 12.18 — Timing your code

  • Waldo Lemmer

    Here's the Timer class split into Timer.cpp and Timer.h:



  • Rishi

    I got a small doubt here. When I run the sort algorithm given in this lesson, I get a run time of about 0.77 seconds. That made sense as I'm running it on my android phone. But then I ran the library sort algorithm and got 0.0006(same result as Alex's). Does anyone know what's happening here? Why is the custom sort algorithm 10 times slower then Alex's machine?

    P.S: I have ran the code like ten times and without much background or foreground activity. So consider that this results are unbaised. Thank you already ^-^

    • 1th1

      "Finally, note that results are only valid for your machine’s architecture, OS, compiler, and system specs. You may get different results on other systems that have different strengths and weaknesses."

      Could be you're own device. For me, my custom sort algorithm took 7 seconds so yeah... (laptop not for win :()

      • Rishi

        Yes I read that, but isn't it weird? The standard sort algorithm takes the same time as Alex in my system, but the custom sort is alone slow ¯\_(ツ)_/¯

  • Ludovic

    Hi, maybe the use of `std::chrono::steady_clock` instead of `std::chrono::high_resolution_clock` is more appropriate, as the later may be an alias for `std::chrono::steady_clock` or `std::chrono::system_clock` and may lead to inconsistencies between implementations because we don't know if it is steady or not as stated on cppreference

    Great work with this site by the way

  • Hi Rishi,

    I would be happy to answer your question if you ask nicely.

    • Rishi

      Oh sorry and thank you. I got the answer

    • Kanaetochi Okiyi

      People don't see your response if you don't respond directly. Whether or not you include their username in it. Unless there were some hidden function invisible to us.

      For example, Nascar_driver and Alex are likely notified of any first level comments  in addition to direct responses.

      And asking people to "ask nicely" before offering them information is an unnecessary way to create an uncomfortable environment for learning.

  • Rishi

    Explain this line. What's that ".count()" there?

Leave a Comment

Put all code inside code tags: [code]your code here[/code]