[libcxx] Add support for benchmark tests using Google Benchmark.

Summary:
This patch does the following:

1. Checks in a copy of the Google Benchmark library into the libc++ repo under `utils/google-benchmark`.
2. Teaches libc++ how to build Google Benchmark against both (A) in-tree libc++ and (B) the platforms native STL.
3. Allows performance benchmarks to be built as part of the libc++ build.

Building the benchmarks (and Google Benchmark) is off by default. It must be enabled using the CMake option `-DLIBCXX_INCLUDE_BENCHMARKS=ON`. When this option is enabled the tests under `libcxx/benchmarks`  can be built using the `libcxx-benchmarks` target.

On Linux platforms where libstdc++ is the default STL the CMake option `-DLIBCXX_BUILD_BENCHMARKS_NATIVE_STDLIB=ON` can be used to build each benchmark test against libstdc++ as well. This is useful for comparing performance between standard libraries.

Support for benchmarks is currently very minimal. They must be manually run by the user and there is no mechanism for detecting performance regressions.

Known Issues:

* `-DLIBCXX_INCLUDE_BENCHMARKS=ON` is only supported for Clang, and not GCC, since the `-stdlib=libc++` option is needed to build Google Benchmark.








Reviewers: danalbert, dberlin, chandlerc, mclow.lists, jroelofs

Subscribers: chandlerc, dberlin, tberghammer, danalbert, srhines, hfinkel

Differential Revision: https://reviews.llvm.org/D22240

git-svn-id: https://llvm.org/svn/llvm-project/libcxx/trunk@276049 91177308-0d34-0410-b5e6-96231b3b80d8
This commit is contained in:
Eric Fiselier
2016-07-19 23:07:03 +00:00
parent 7310ec83f3
commit d9b9ef75a8
73 changed files with 9310 additions and 31 deletions

View File

@@ -227,7 +227,7 @@ ABI Library Specific Options
libc++abi is the C++ ABI library used.
libc++ Feature options
libc++ Feature Options
----------------------
.. option:: LIBCXX_ENABLE_EXCEPTIONS:BOOL
@@ -242,9 +242,25 @@ libc++ Feature options
Build libc++ with run time type information.
.. option:: LIBCXX_INCLUDE_BENCHMARKS:BOOL
libc++ Feature options
----------------------
**Default**: ``OFF``
Build the libc++ benchmark tests and the Google Benchmark library needed
to support them.
.. option:: LIBCXX_BUILD_BENCHMARK_NATIVE_STDLIB:BOOL
**Default**:: ``OFF``
Build the libc++ benchmark tests and Google Benchmark library against the
native standard library on the platform. On linux this can be used to compare
libc++ to libstdc++ by building the benchmark tests against both standard
libraries.
libc++ ABI Feature Options
--------------------------
The following options allow building libc++ for a different ABI version.

View File

@@ -198,3 +198,62 @@ Environment Variables
If ``LIBCXX_COLOR_DIAGNOSTICS`` is defined then the test suite will attempt
to use color diagnostic outputs from the compiler.
Also see :option:`color_diagnostics`.
Benchmarks
==========
Libc++ contains benchmark tests separately from the test of the test suite.
The benchmarks are written using the `Google Benchmark`_ library, a copy of which
is stored in the libc++ repository.
For more information about using the Google Benchmark library see the
`official documentation <https://github.com/google/benchmark>`_.
.. _`Google Benchmark`: https://github.com/google/benchmark
Building Benchmarks
-------------------
The benchmark tests are not enabled by default. To build the benchmarks
libc++ must be configured using the CMake option ``-DLIBCXX_INCLUDE_BENCHMARKS=ON``.
Then the benchmarks can be built using the ``libcxx-benchmarks`` target.
An example build would look like:
.. code-block:: bash
$ cd build
$ cmake [options] -DLIBCXX_INCLUDE_BENCHMARKS=ON <path to libcxx sources>
$ make libcxx-benchmarks
This will build all of the benchmarks under ``<libcxx-src>/benchmarks`` to be
built against the just-built libc++. The compiled tests are output into
``build/benchmarks``.
The benchmarks can also be built against the platforms native standard library
using the ``-DLIBCXX_BUILD_BENCHMARKS_NATIVE_STDLIB=ON`` CMake option. This
is useful for comparing the performance of libc++ to other standard libraries.
The compiled benchmarks are named ``<test>.libcxx.out`` if they test libc++ and
``<test>.native.out`` otherwise.
Also See:
* :ref:`Building Libc++ <build instructions>`
* :ref:`CMake Options`
Running Benchmarks
------------------
The benchmarks must be run manually by the user. Currently there is no way
to run them as part of the build.
For example:
.. code-block:: bash
$ cd build/benchmarks
$ make libcxx-benchmarks
$ ./algorithms.libcxx.out # Runs all the benchmarks
$ ./algorithms.libcxx.out --benchmark_filter=BM_Sort.* # Only runs the sort benchmarks
For more information about running benchmarks see `Google Benchmark`_.