Multivariate performance testing

Run performance tests with multiple candidates data collections. Outputs a figure and a pandas.DataFrame of long-format raw data. For an example, see the Select IN vs BETWEEN notebook.

A performance summary figure.

perf.run_multivariate_test(test_data: Union[Any, Dict[str, Any]], time_per_candidate: float = 6.0, plot: bool = True, **figure_kwargs: Any) → DataFrame

Run performance tests for multiple candidate methods on collections of test data.

This method combines rics.utility.perf.MultiCaseTimer.run(), to_dataframe() and, if plotting is enabled, plot_run(). For full functionally these methods should be used directly.

Parameters

• candidate_method – Candidate methods to evaluate.

• test_data – Test data to evaluate.

• time_per_candidate – Desired runtime for each repetition per candidate label.

• plot – If True, plot a figure using plot_run().

• **figure_kwargs – Keyword arguments for the barplot. Ignored if plot=False.

Returns

A long-format DataFrame of results.

Raises

ModuleNotFoundError – If Seaborn isn’t installed and plot=True.

Warning

By default, this function reports averages of all runs (repetitions), as opposed to the built-in timeit module which reports only the best result (in non-verbose mode).