M3 Max Benchmarks: A MacStadium Preview - Part 2

Introduction

As a follow-up to our previous article with benchmarks for M3 and M3 Pro, this blog takes a look at M3 Max, Apple’s latest high-performance chipset. Much has been said about the M3 Max, with many articles stating that it comes very close to the performance of the M2 Ultra. We have tested this and expanded upon the tests performed previously to get a better sense of the M3 Series’ GPU performance in AI applications.  

Apple took a different approach when packaging the M3 Pro and M3 Max chipsets this year, with a reduction in performance cores in M3 Pro when compared to M2 Pro (6 vs. 8). As a result, we found that the M3 Pro had a marginal, albeit noticeable improvement in benchmark scores, less so when compared to M2 vs. M3. We had speculated that this was done to increase the differentiation between M3 Pro and M3 Max. Now the time has come to test our theory. 

In the previous generation, the 12-Core M2 Pro was effectively a cut-down version of the M2 Max, with the same number of compute cores, but with slightly reduced memory bandwidth and substantially fewer GPU cores. This made for little practical difference in compute power between M2 Pro and M2 Max, with the largest differentiator being the substantially improved GPU performance in M2 Max.  

All of that said, the Mac Studio with M2 Max is an excellent machine for local content creation applications but, due to its increased size, is not ideal as a hosted solution at MacStadium. We stock machines that support our users' workloads, including heavy CI/CD builds. Unfortunately, due to the lack of performance improvements in this area, MacStadium could not justify carrying a large stock of this machine, as the majority run complex CI/CD workloads. For customers with GPU-focused workloads, we offer the Mac Studio with M2 Ultra, which is effectively two M2 Max chipsets running in parallel, offering massive gains in GPU performance. If M3 Max lives up to the hype, it will be worthwhile to offer it in MacStadium data centers. 

With M3 Max, Apple has substantially increased the number of performance cores, going from 8 to 12, keeping only 4 efficiency cores (as opposed to the 6 in M3 Pro). When coupled with the single-core performance improvements found in M3, we expect to see a substantial improvement in M3 Max performance.  

Test Machines 

Our M3 Max test machine is a 14-inch MacBook Pro with M3 Max 16-Core, sporting 64GB of RAM and 1TB of SSD storage, in Space Black.  

 This is in addition to our previous M3 test machines: 

1. 14” MacBook Pro with M3 8-Core, 16 GB of RAM, 1TB SSD, Space Gray 
2. 16” MacBook Pro with M3 Pro 12-Core, 36 GB of RAM, 512GB SSD, Space Black 

This configuration was chosen to see how M3 Max compares to Mac Studio with M2 configurations, with the RAM matching our Mac Studio S2.M configuration with M2 Ultra in particular.  

Our Methodology 

To start, we ran the M3 Max through the same process that we used in our Mac mini benchmarks and our first round of M3 tests, using the same benchmark suites and methodologies. Since these are laptops, we ensured that the laptops were plugged in using the supplied power adaptor and charged to 100% before starting the benchmark. Additionally, we used Amphetamine (App Store link) to prevent the Macs from going to sleep during the test. As with the Mac mini tests, these tests are intended to capture best-case performance, with the best result out of three selected for each test, and a 10-minute cooldown between individual test runs to minimize the impact of thermals on the results. minimize the impact of thermals on the results.  

Additionally, to better gauge GPU performance, and to better quantify M3’s performance in the emerging field of generative AI, we added an additional GPU-focused test: 

Diffusers/Stable Diffusion

To assess M3’s performance with image generation, we used Diffusers by HuggingFace, from the Mac App Store to run Stable Diffusion 1.5. Our test consisted of the default prompt (“Labrador in the style of Vermeer”) using the GPU, and with the Safety Checker disabled (as it interfered with consistency in results). It’s worth noting that, on some Macs, Diffusers will choose to utilize the Neural Engine by default, rather than the GPU, as it can be more performant and efficient, however, to better compare machines, all tests were performed on the GPU. As this workload is shorter than other benchmarks, a minute was given to allow the machines to cool down between tests. Results are measured in seconds per image generated. The best result was taken out of 3 tests. 

Results 

All machines were updated to macOS Sonoma 14.1, and Xcode 15.0.1. All results for each category are ranked by performance, with the fastest machines first.

Single Core CPU

As expected, M3 Max performs in line with other M3 chipsets in Single Core workloads. 

Multicore CPU

In Geekbench 6 Multi-Core, the M3 Max posts a 35% gain over the M3 Pro and comes within a very small margin of the 24-Core M2 Ultra. This is groundbreaking performance, and realigns the different processor tiers that Apple offers, providing meaningful differentiation between M3, M3 Pro, and M3 Max chipsets.

In Cinebench R23, the gap between M3 Max and M2 Ultra is larger, landing squarely between M3 Pro and M2 Ultra with a 45% performance gain over M3 Pro. It’s important to note that Cinebench is purely CPU-bound, and focuses on traditional compute, while Geekbench is focused on the execution performance of certain real-world tasks. As a result, Geekbench benefits more from cross-generational improvements in application-specific logic than Cinebench. Considering that M2 Max historically scores about the same as M2 Pro 12-core (historical results here), this is still an impressive jump in performance, redefining the different tiers of compute performance available with Apple Silicon. 

GPU Compute

As with previous generations of Max chipsets, the GPU performance of M3 Max is a substantial jump up from the Pro chipset variant, offering a 110% improvement in performance over M3 Pro. 

XcodeBenchmark

As noted in Geekbench 6 multicore, well-optimized applications perform particularly well on M3 series chipsets, and the M3 Max XcodeBenchmark results reflect this. Coming within ~5 seconds of M2 Ultra, the gap between M3 Max and M2 Ultra is half of the gap between M3 Pro and M3 Max. As with previous benchmarks, while the M3 Max doesn’t exactly match the performance of M2 Ultra, it comes close enough to be a decent alternative and reflects a substantial change in strategy when it comes to the different chipsets available in Apple Silicon. We expect to see even greater gains in performance with M3 Ultra, which should launch alongside the M3 Mac Studio lineup in late 2024, going by Apple’s past release schedule.

Stable Diffusion/Diffusers

To get a better gauge of real-world GPU performance for hosted applications, we added image generation with Stable Diffusion using the Diffusers app, which leverages Apple’s CoreML framework for additional optimizations when compared to running PyTorch with Metal. As previously noted, M3 chipsets benefit greatly when using well-optimized frameworks, and the results show that this continues to be the case with GPU-accelerated workloads as well. Notably, M3 Max is just 8% slower than M2 Ultra. M3 Pro also fares better in this test than in Geekbench GPU, pulling slightly ahead of the M2 Pro, despite having 1 fewer GPU core.

Conclusion

M3 Max represents a realignment of Apple’s Apple Silicon product lineup, bringing substantial gains over earlier generations in both compute and GPU performance. Rather than merely being an M3 Pro with more graphical power, as with the M1 and M2 generations, the M3 Max sets a new standard for performance in its tier. It nips at the heels of the previous generation’s Ultra chipset, especially impressive considering the Ultra chipsets have historically been a combination of two Max chips.  

Part of this is due to the availability of well-optimized frameworks. If your workload heavily leverages native Apple frameworks, you will see even greater gains. When comparing less-optimized benchmarks, such as Cinebench R23 and Geekbench GPU Compute, to real-world benchmarks that use native frameworks, such as XcodeBenchmark and Diffusers, the performance gains of M3 are quite substantial. 

It will be interesting to see where Apple takes the Mac Studio lineup in 2024. With M3 Max providing substantially more power than previous Max chipsets, we could see a huge step up in Mac performance with M3 Ultra.  

In the meantime, the M2 Ultra still reigns supreme, offering the best performance of any currently available Apple Silicon chipset. And you can get M2 Ultra today as a hosted solution at MacStadium, in Mac Studio S2.M. 

Check out our current Bare Metal offerings and stay tuned for M3 machines in 2024! 

As a bonus, here are some of our favorite Vermeer-styled Labradors that we generated with Diffusers while testing: