There is obviously a lot of detail about how Rosetta 2 and Universal 2 are going to work, and we wanted to explore the reality of how these will impact the pro-audio sector. In this article with the help of an audio software developer, we try to explain some of the challenges as it pertains to audio software, Big Sur and Apple Silicon.
But before we get to the nitty-gritty, some background.
What is Apple Silicon?
As we observed in our coverage of the Apple WWDC 2020 Keynote address on Monday there was no mention of ARM processors. Instead, Apple is calling its architecture Apple Silicon, which they describe as the same, Apple-developed Ax chips, that have been in the iPhone and iPad for years. But even though Apple is calling it Apple Silicon, at their core, the processors are based on ARM architecture, adapted by the Apple chip design team to make them for the iPhone and iPad, adding their own features like the neural engine to handle the machine learning capabilities.
The announcements at WWDC are all about Apple applying this chip architecture to the Mac computer range. The aim is to make optimisations and improvements to the ARM architecture to ensure that the performance of an ARM is better than the current Intel Macs. To demonstrate this, during the WWDC keynote, Apple ran Final Cut Pro X and Adobe Lightroom on an Apple Silicon prototype, without revealing any specific benchmarks. From what we saw, the Apple Silicon-equipped Mac ran these applications very smoothly.
What Will Developers Need To Do?
As Apple explained in the WWDC keynote, to get their software to run on Apple Silicon, developers will need to recompile their existing Intel-based source code for the new architecture. Apple suggested that for most applications this should be a simple process, with most projects done in a couple of days. But don’t underestimate this. Look at Waves with their hundreds of plugins. If each plugin will take a couple of days, that is a lot of days just to prepare their existing catalogue.
One of the reasons we understand it will take a little time to do the conversion is that for best performance, developers may need to make adjustments to the way they use hardware resources. Apple recommends that developers should reduce dependency on hardware and, if possible, use higher-level technologies. Apple quoted Grand Central Dispatch as an example, which they suggested should use multithreaded apps instead of creating and managing the threads itself.
As we saw in the keynote this conversion process will use a new version of the Xcode development environment. Xcode 12 now creates Universal 2 Apps, which contain the code for both Intel and ARM processors.
However, there is some good news for developers unable to have all their software ported ready for the release of the first Apple Silicon-based Macs. The Intel-based software should still work on the new hardware platform because Apple have also developed Rosetta 2 that works in a similar way as Rosetta did when we switched from PowerPC to Intel chips.
Everything is Throttled Back To The Slowest Component
During the keynote, Craig Federighi, Apple’s senior vice president of Software Engineering explained that the code could be translated to the new architecture when the programs were installed. In a publically available document, About the Rosetta Translation Environment, Apple explains more about how Rosetta 2 will translate executables, and what Rosetta 2 can’t translate.
“Rosetta translates all
x86_64
instructions, but it doesn’t support the execution of some newer instruction sets and processor features, such as AVX, AVX2, and AVX512 vector instructions. If you include these newer instructions in your code, execute them only after verifying that they are available. For example, to determine if AVX512 vector instructions are available, use thesysctlbyname
function to check thehw.optional.avx512f
attribute.”
Our software developer explains that this means modern audio application and plugins, which rely on advanced optimisations will, at best, run on a less optimised path, which is itself being translated. For pro-audio workflows, this could be a performance gotcha. At worst, they could crash or not work at all under emulation if they make assumptions about what high-performance Intel processor instructions will be available (although a simple software patch could solve some fundamental programming bugs like that, so we may see ‘Rosetta 2 compatibility’ updates before some full-fat native Apple Silicon ports arrive).
All Or Nothing - No Mix and Match
“The system prefers to execute an app’s
arm64
instructions on Apple silicon. If a binary includes botharm64
andx86_64
instructions, the user can tell the system to launch the app using Rosetta translation from the app’s Get Info window in the Finder. For example, a user might enable Rosetta translation to allow the app to run older plug-ins that don’t yet support thearm64
architecture.”
What this means is that although you get to decide to run your DAW under Rosetta 2 or Natively on Apple Silicon (if it’s been updated to native) you will not be able to run Intel-based plug-ins under Rosetta 2 in a DAW that is running natively on Apple Silicon (and vice versa). Put even more simply, our developer explains that this means you will not be able to mix and match a Native Apple Silicon DAW with Intel-based plugins. This means all of your plugins will have to be able to run natively on Apple Silicon before you can use them in a DAW running natively. You may even find your plug-ins get updated before your DAW – but the same problem will arise!
This means you are going to run everything through Rosetta 2 (DAW and Plugins) until everything you need has been ported over unless there is some kind of process bridge tool made available, as happened with the 32-64 bit transition. That’s assuming audio plugins, iLok/eLicenser protection, drivers work under emulation at all – when it comes to the audio production world there are a lot of ifs, buts and maybes.
The End Of The Road For Some AU Plugins
In another article, Apple has made public Porting Your Audio Code to Apple Silicon, Apple discusses how to eliminate issues in your audio-specific code when running on Apple silicon Mac computers.
“For the
arm64
architecture, always use the Audio Component API to load Audio Units, codecs, and other code modules into your app. The Audio Component API is the modern way to search for loadable code modules, and it’s available in macOS 10.6 and later. Apps that target thearm64
architecture or link against the macOS 11 SDK cannot use the legacy Carbon Component Manager API to open Audio Units. If your app uses the Carbon Component Manager, plan to migrate off of it when porting your app.”
What this means is that this will be the end of the road for some old Audio Units plugins when host DAW starts to use the new macOS 11 SDK (Software Development Kit), which our software developer expects to happen because developers will understandably want to support Apple Silicon.
New Drivers Needed Too
In another publicly available document from Apple Porting Your macOS Apps to Apple Silicon, which covers how to create a version of your macOS app that runs on both Apple silicon and Intel-based Mac computers, in the Update GPU-Specific Code section our developer is surprised to see that Apple has not killed OpenGL on Apple Silicon. They speculate that Apple will eventually kill OpenGL when it sees sufficient migration away from it but for now, they seem to be giving it a bit longer. Why this matters is that this will help lots of audio plugins and DAWs that are have not been ported to over to Metal yet for their high-performance graphics. Apple goes on to say…
“Kernel extensions must support the native architecture. Kernel extensions run in the kernel, and the kernel always runs as a native process. You cannot run kernel extensions using Rosetta translation.”
This means that the developer will not be able to lean on Rosetta 2 if they want to use old drivers, they will need to be updated to use the latest DriverKit frameworks.
All Stop On Product Development Its Port, Port, Port…
Finally, for now, in the Apple document Xcode 12 for macOS Universal Apps Beta Release Notes, our developer explains that this document tells developers that they can begin the process of porting their Intel-based plugins and apps. In other words, it’s time to down tools if they want to support Apple Silicon from day 1 with a fully tested back catalogue. Whatever the boss had them working on will now have to wait because they only have got 6 months before enthusiastic early adopters of the new Macs start to complain, bitterly accusing developers of being lazy, because Apple said it only takes a couple of days to port something to Apple Silicon. “Thanks, Apple”.
Windows users will also be able to thank Apple for monopolising a lot of audio development hours that might otherwise have gone into the creation, or improvement of, tools that they could have benefitted from!
Remember GPGPU? It Could Be Back!
There was a time when we were all excited about the prospects of GPUs being used for audio. Affordable graphics cards were going to replace expensive DSP hardware. It never happened, possibly because moving audio between the GPU and system in small batches is so inefficient it defeats the purpose of the exercise, and many audio algorithms were just not suited to the technology anyway.
Apple Silicon systems have a heterogeneous memory architecture. That means the GPU and the CPU work from the same pool of memory without needing to transfer data back and forth. This is because Apple designs both the CPU and GPU, and can package them into the same chip using the same memory and bus. Everybody gets to dine at the same buffet table.
Our developer goes onto explain that whilst the general argument that recursive audio algorithms don’t map well onto GPUs still applies, there is a lot of vectorisation (i.e. work that can be done in batches) that is actually done in modern audio applications. Some of this could be suitable for the GPU, so although the panacea of offloading your entire plugin stack to the GPU will remain an unrealistic pipedream, a more restrained form of it could sneak its way into audio apps in future and begin to give Apple Silicon systems the edge in a way Intel + AMD/Nvidia based systems would struggle to compete with.
That said, our developer’s view is that where Apple goes the industry usually follows – so expect Intel to be taking notes!
Mac Pro Maybe The Last To Go Apple Silicon? But Could It Be A Multi-Processor Beast?
All the speculation is that Apple will start with the portable Macs first like the MacBook Air, MacBook and the MacBook Pro when it comes to producing new Apple Silicon powered Macs. Then it will be the Mac mini, and then 24” iMac, which we expected to be announced at WWDC leaving the large screen variants and then 2 Pro machines to the end, the iMac Pro and of course the Mac Pro towards the end of the 2 year transition period quoted in the WWDC keynote.
But when they do start work on the Mac Pro our developer suggests that Apple might consider using multiple Ax chips in the more top line Macs to brute force the performance, because maybe they will be able to manufacture their own chips for a fraction of the price they are paying Intel for their chips once suppliers have got some experience producing Apple’s custom designs and the economies of scale kick in.
What about some future Mac Pro with 16 processors each with 16 cores? The work Apple has already done to reduce power consumption whilst improving performance could allow them to do this in a desktop machine where power consumption is less of an issue. “Let’s see Intel and AMD match that on x64 (you’d struggle to cool the system or price it competitively)”.
Would that be a real differentiator for Mac vs PC systems? Who knows – it’s just speculation, but Apple will be sure to want to make this all feel worth the switch!