Aggregated Results

We run our benchmark on 2 different configurations: The Short tab below contains results of the smaller problems (less than a million variables), run with a 1 hour timeout on a less powerful machine. The Long tab contains larger problems (more than a million variables), run with a 24 hour timeout on a more powerful machine. Select the desired tab to view a summary of the results on that configuration, along with the technical specifications of the machine used.

Short

Long

Configuration

Machine:c4-standard-2
vCPUs:2
Memory:7 GB
Timeout:1h

No benchmark problems match the selected filters.

Note: As with all benchmarks, our results provide only an indication of which solvers might be good for your problems. We recommend using our scripts to benchmark on your own problems before picking a solver. See also the section on caveats below.

Caveats

Here are some key points to keep in mind when interpreting these results:

We run the benchmark on commercial cloud virtual machines (VMs) for efficiency and cost reasons. The shared nature of cloud resources means there is some error in our runtime measurements, which we estimate as a coefficient of variation of no larger than 6%. More details on this here.
All solvers are run with their default options, except for the duality gap tolerance for mixed integer problems (MILPs), which we set to 1e-4. You can check the duality gaps for each solver in the problem details page corresponding to each problem.
All results on this website use the runtime measured by our benchmarking script. This may not be the same as the runtime of the solving algorithm as reported by the solver, and it may include things like time for input file parsing and license checks. See more details and join the discussion on whether to use reported or measured runtime here.
Some solvers returned errors when running on some benchmark problems. For more details, please see the tracking issue.

For more details, please see our Methodology page.