Editor’s take: Of course, Apple is designing its own AI chips for its data centers. This will give them an end-to-end, cloud-to-edge AI system that its competitors will not be able to match anytime soon.

A report from Bloomberg’s Mark Gurman last week indicated that Apple was going to put its own chips in the data center, and a few days later, the Journal confirmed this with its own reporting. The information was initially met with a high degree of skepticism, particularly from the Journal, which later changed its stance on the subject.

Many said that Apple was “not a cloud company” and did not see the point in them pursuing this path. This struck us as odd since we know Apple has been looking at this for years. Here, we want to lay out some of the reasons we think Apple would want its own data center chip.

Editor’s Note:
Guest author Jonathan Goldberg is the founder of D2D Advisory, a multi-functional consulting firm. Jonathan has developed growth strategies and alliances for companies in the mobile, networking, gaming, and software industries.

For starters, Apple is building a chip because they can. In a world of constrained leading-edge capacity, Apple is TSMC’s largest customer and can presumably get most of the capacity they want. If they actually had a good use for the chip, now is the time. Everyone else is scrambling, and Apple could conceivably build up a big head start.

It should also be clear that Apple is very much a provider of cloud services – they literally have a product called iCloud. Part of the confusion here rests in the fact that Apple says very little about what it does with its data centers. They never make it entirely clear how they deliver their cloud services, which seems to be some combination of Google Cloud, Azure, and maybe some AWS, but also their own data centers. When we talk about the “hyperscalers,” we silently assume that Apple is one of those ten largest data center operators. And if all the other hyperscalers are designing their own chips, it makes sense that Apple would as well.

The big question is what Apple is going to do with that capacity. This is 2024, so the simple summary is that, of course, they are going to do AI.

The more nuanced answer is where this gets more interesting. We have long bemoaned the lack of compelling consumer uses for generative AI, and we often add that the industry needs Apple to show everyone else what can be done. We are not convinced that Apple has, in fact, figured it out, but given all the rumors that next month’s WWDC is going to heavily focus on AI, we are intrigued.

So let’s assume Apple has come up with some impressive AI applications, or at least applications Apple thinks are impressive. We know that they have been working on AI for years – from Siri’s steadily improving speech recognition to under-the-hood machine learning improvements to photo and video images, to the simple fact that the A series phone processor has had a “Neural Engine” since 2018.

They brought transformer support to their M-Series chips before ChatGPT. They launched that support so soon after those models became widely popular that it seems likely they had been working on the idea for some time already. Apple has a capable AI team.

Moreover, it seems likely that team has built its own foundational model. Apple, of course, owns all of its critical software and a huge amount of data (i.e., they know what you do with your phone). We are not clear how Apple trained this model, but we suspect they used Nvidia like everyone else.

However, as Apple rolls out its AI apps, they are going to need to do a lot of inference. They will run some of that on-device (a.k.a. “the edge,” a.k.a. consumers pay for the capex and opex), but they will also run some amount of inference in the cloud.

From all the reporting, it is not entirely clear what chips Apple will be using in the data center. We have seen reports they will use their M2 CPU, their M4 CPU, or a brand-new chip called ACDC (Apple Chips in Data Center, which would be the clear winner in 2024’s chip naming competition). Whatever they end up using, they will benefit from having a common software layer connecting the cloud and edge inference.

Many companies have talked about having this kind of offering, but Apple is likely to be the first to put one in large-scale production. Having everything connected in this way offers meaningful power savings but also probably allows for a higher degree of privacy, a big marketing theme for the company. We imagine the pitch will go something like, “Have your phone run amazing AI, and only you will see your data.” So there are clear benefits to having their own silicon along the entire software chain.

That leaves one question: how will Apple position this in their marketing?

As we mentioned above, Apple never talks about their data centers, but now they might be willing to, as it conveys a message that benefits them. In fact, given the amount of press coverage coming out on the subject, we suspect that Apple may have deliberately leaked a bit of this to make sure we are paying attention.

When we boil it all down, it looks a lot like Apple has an interesting story to tell about the breadth of its AI offering. This is not just an AI Mac or an AI iPhone but a complete system offering some form of differentiated AI. If this is true, and we may be wrong, it will be one more example of Apple stealing a beat on the competition, because this is something that its competitors will struggle to match.