MIT Startup Lightmatter’s Photonic Processors Reduce Power Consumption in AI

The demand for computing power is skyrocketing thanks to rapid growth in the artificial intelligence and high-performance computing markets. 

Data centers worldwide are quickly approaching their energy limits and finding it increasingly challenging to operate modern-day supercomputers, which–to run the likes of ChatGPT–consume vast amounts of power and produce a lot of waste heat. Some organizations have already deployed clusters with tens of thousands of accelerators as the only means of keeping up with the demands of their AI applications. 

If nothing changes, it’s estimated that up to 20% of the world’s power will be used for AI inference before the end of the decade. 

Thankfully, the founders of the silicon photonics startup Lightmatter appear to have found a viable solution: using photons and electrons for data processing and transportation. 

Company Overview

Lightmatter is using photonic computing to revolutionize how computer chips communicate and calculate. Its products promise to satiate the ever-growing demands of AI and HPC workloads while driving energy efficiency and reducing electricity consumption.    

“The biggest companies in the world are hitting an energy power wall and experiencing massive challenges with AI scalability. AI advances will slow significantly unless we deploy a new solution in data centers,” said the company’s CEO, Nick Harris.

Harris and fellow co-founders Darius Bunandar and Thomas Graham spent four years developing Lightmatter’s photonic processor technology at the MIT Research Laboratory of Electronics and successfully launched the startup in 2017 after winning the MIT $100K Entrepreneurship Competition. 

The company has scaled rapidly since then; currently headquartered in Boston, Massachusetts, it employs more than 200 people across the U.S. and Canada and is valued at $4.4 billion. What are the secrets to its success? The way Harris sees it, Lightmatter’s first products, Envise and Passage, address the major problems associated with existing chip technology: reducing power consumption and elevating performance. 

How is Lightmatter Leveraging Silicon Photonics?

Photonic systems generate, control, manipulate, and detect light when it is in photon form. In Silicon photonics, photonic components are fabricated directly onto a silicon material base, which acts as a medium.

The resulting devices, which include photonic integrated circuits, offer several advantages over their solely electrical or optical counterparts, including reduced energy usage, lower latency, and high-speed transmission. In addition, silicon photonics devices can be made much smaller. 

Lightmatter is tapping into even greater benefits through optical computing, which uses light as a primary means for carrying out calculations. The company’s photonic circuits pass light through arrays of microscopic optical waveguides, which enable it to perform complex logic operations with impressive speed. Better still, thanks to a technique known as Wavelength Division Multiplexing (WDM), different wavelengths, i.e., colors,  can simultaneously transmit multiple channels of data over a single waveguide. “Orange” can be used to carry out one operation, for example, while “red” performs another. The more wavelengths, or colors, in use, the greater the power of the device. 

Lightmatter’s full product stack includes Envise, Passage, and Idiom. 

• Envise: Envise is the world’s first photonic computing platform, specializing in AI operations. It combines the capabilities of electrons and light to perform huge matrix multiplications of deep-learning models. With this product, data centers can operate more capable neural network models and deploy new features, all while minimizing their environmental footprint.

• Passage: As “the world’s fastest interconnect,” Passage is designed for large-scale computing operations. The product leverages light to link processors in much the same way that fiber optic cables send data over long distances, enabling high-speed chip-to-chip and node-to-node communications. The product works with off-the-shelf chips thanks to integrated transistor and photonics control technology. “With Passage, the world’s fastest photonic engine, we’re setting a new standard for performance and breaking through the barriers that limit AI computing,” said Harris. 

• Idiom: Idiom is a software solution that enables developers to run existing deep-learning models on Lightmatter’s hardware. Users select “Envise” as their target hardware and let Idiom do all of the work to convert Pytorch, TensorFlow, or ONNX files.

Who Are Lightmatter’s Biggest Competitors?

Although its early successes and financial backing set it apart, Lightmatter is not the sole player in the silicon photonics sector. 

Perhaps its biggest competitor is Celestial AI, which partnered with several hyperscalers to better understand the computer, memory, and network system infrastructure chokepoints. The company’s resulting product is its Photonic Fabric platform, which separates compute and memory to enable faster processing of extensive AI and more energy-efficient computing. Earlier this year, Celestial AI closed a $175 million Series C funding round.

Another startup, Optalysys, is developing encryption technology and photonic hardware to accelerate always-encrypted data, while Ayar Labs is accelerating data movement with its silicon photonics optical “chiplet.”

While Lightmatter has yet to name any existing customers, it claims to be enjoying massive demand from a range of cloud providers and AI companies. Harris himself foresees market domination. “Over the next few years, all of the GPUs in the world that are designed for AI training and inference or high-performance computing are going to be built on Passage,” he says.

What Does the Future Hold?

In October, Lightmatter announced its latest Series D funding round had raised $400 million, bringing its total funding since inception to $850 million. These latest investments will fund the manufacture and deployment of Lightmatter’s photonic chips in partner data centers, as well as the expansion of its teams across the U.S. and Canada.
As for further exploring the potential of photon computing? That’s also in the pipeline. “We’re going to continue looking at all of the pieces of computers to figure out where light can accelerate them, make them more energy efficient, and faster, and we’re going to continue to replace those parts,” Harris says. “Right now, we’re focused on interconnect with Passage and on compute with Envise. But over time, we’re going to build out the next generation of computers, and it’s all going to be centered around light.”