MIT Startup Lightmatter’s Photonic Processors Reduce Power Consumption in AI

The demand for computing power is skyrocketing thanks to rapid growth in the artificial intelligence and high-performance computing markets. 

Data centers worldwide are quickly approaching their energy limits and finding it increasingly challenging to operate modern-day supercomputers, which–to run the likes of ChatGPT–consume vast amounts of power and produce a lot of waste heat. Some organizations have already deployed clusters with tens of thousands of accelerators as the only means of keeping up with the demands of their AI applications. 

If nothing changes, it’s estimated that up to 20% of the world’s power will be used for AI inference before the end of the decade. 

Thankfully, the founders of the silicon photonics startup Lightmatter appear to have found a viable solution: using photons and electrons for data processing and transportation. 

Company Overview

Lightmatter is using photonic computing to revolutionize how computer chips communicate and calculate. Its products promise to satiate the ever-growing demands of AI and HPC workloads while driving energy efficiency and reducing electricity consumption.    

“The biggest companies in the world are hitting an energy power wall and experiencing massive challenges with AI scalability. AI advances will slow significantly unless we deploy a new solution in data centers,” said the company’s CEO, Nick Harris.

Harris and fellow co-founders Darius Bunandar and Thomas Graham spent four years developing Lightmatter’s photonic processor technology at the MIT Research Laboratory of Electronics and successfully launched the startup in 2017 after winning the MIT $100K Entrepreneurship Competition. 

The company has scaled rapidly since then; currently headquartered in Boston, Massachusetts, it employs more than 200 people across the U.S. and Canada and is valued at $4.4 billion. What are the secrets to its success? The way Harris sees it, Lightmatter’s first products, Envise and Passage, address the major problems associated with existing chip technology: reducing power consumption and elevating performance. 

How is Lightmatter Leveraging Silicon Photonics?

Photonic systems generate, control, manipulate, and detect light when it is in photon form. In Silicon photonics, photonic components are fabricated directly onto a silicon material base, which acts as a medium.

The resulting devices, which include photonic integrated circuits, offer several advantages over their solely electrical or optical counterparts, including reduced energy usage, lower latency, and high-speed transmission. In addition, silicon photonics devices can be made much smaller. 

Lightmatter is tapping into even greater benefits through optical computing, which uses light as a primary means for carrying out calculations. The company’s photonic circuits pass light through arrays of microscopic optical waveguides, which enable it to perform complex logic operations with impressive speed. Better still, thanks to a technique known as Wavelength Division Multiplexing (WDM), different wavelengths, i.e., colors,  can simultaneously transmit multiple channels of data over a single waveguide. “Orange” can be used to carry out one operation, for example, while “red” performs another. The more wavelengths, or colors, in use, the greater the power of the device. 

Lightmatter’s full product stack includes Envise, Passage, and Idiom. 

• Envise: Envise is the world’s first photonic computing platform, specializing in AI operations. It combines the capabilities of electrons and light to perform huge matrix multiplications of deep-learning models. With this product, data centers can operate more capable neural network models and deploy new features, all while minimizing their environmental footprint.

• Passage: As “the world’s fastest interconnect,” Passage is designed for large-scale computing operations. The product leverages light to link processors in much the same way that fiber optic cables send data over long distances, enabling high-speed chip-to-chip and node-to-node communications. The product works with off-the-shelf chips thanks to integrated transistor and photonics control technology. “With Passage, the world’s fastest photonic engine, we’re setting a new standard for performance and breaking through the barriers that limit AI computing,” said Harris. 

• Idiom: Idiom is a software solution that enables developers to run existing deep-learning models on Lightmatter’s hardware. Users select “Envise” as their target hardware and let Idiom do all of the work to convert Pytorch, TensorFlow, or ONNX files.

Who Are Lightmatter’s Biggest Competitors?

Although its early successes and financial backing set it apart, Lightmatter is not the sole player in the silicon photonics sector. 

Perhaps its biggest competitor is Celestial AI, which partnered with several hyperscalers to better understand the computer, memory, and network system infrastructure chokepoints. The company’s resulting product is its Photonic Fabric platform, which separates compute and memory to enable faster processing of extensive AI and more energy-efficient computing. Earlier this year, Celestial AI closed a