Credo Unveils OmniConnect at OCP to Fix AI Memory Wall

The Physics of the AI Memory Wall

The concept of the 'memory wall' has shifted from a theoretical nuisance to an urgent crisis for the semiconductor industry.

In the context of AI, it refers to the widening gap between the speed at which processors can calculate data and the speed at which memory can supply it.

During the training phase of an AI model, massive datasets are processed over weeks or months, making raw compute power the most valuable resource.

However, inference—the phase where a trained model makes decisions or generates text—requires rapid, repeated access to model weights.

If the memory cannot feed the processor fast enough, the expensive silicon sits idle, wasting energy and time.

Industry experts explained that this bottleneck is exacerbated by the shift towards disaggregated infrastructure.

Modern data centres often separate memory, storage, and compute, connecting them via high-speed networks.

While this offers flexibility, it introduces latency and power overhead that cripple inference performance.

Credo's OmniConnect proposes a scale-in approach to counter this.

By tightly coupling memory and compute within a high-bandwidth interconnect fabric, the solution reduces the physical distance data must travel.

This reduction in distance directly correlates to lower latency and significantly lower power consumption per bit of data transferred.

For European hyperscalers facing strict energy regulations, this power efficiency is not just a performance metric but a regulatory necessity.

Credo's technology utilises advanced signal processing to maintain signal integrity at these high speeds without a proportional increase in power draw.

• Memory wall limits processor utilisation.
• Inference requires rapid access to model weights.
• Scale-in architecture reduces data travel distance.

OmniConnect Specs and the Scale-In Advantage

The technical specifications of the OmniConnect solution focus on maximising bandwidth density while minimising the physical footprint of the interconnect hardware.

Unlike traditional scale-out architectures that rely on extensive top-of-rack switching, the scale-in model aggregates bandwidth within the server enclosure.

Sources familiar with the design confirmed that OmniConnect utilises a combination of active electrical cables and high-density connectors to achieve throughput speeds that rival optical solutions at a fraction of the cost and power.

The architecture is specifically tuned for the deterministic latency required by inference workloads.

In a typical AI cluster, thousands of GPUs must operate in unison.

If the interconnect introduces jitter or variable latency, the entire cluster slows down to wait for the slowest link.

OmniConnect addresses this with a deterministic protocol that ensures consistent data delivery times.

This capability is crucial for financial services and autonomous driving applications, where milliseconds can translate into millions of euros or critical safety margins.

The contribution to OCP includes the electrical and mechanical specifications for these connectors, allowing ecosystem partners to build compatible cables and retimers.

Analysts observed that this open approach contrasts sharply with the proprietary strategies often employed by larger American chipmakers.

By open-sourcing the design, Credo hopes to create a broad ecosystem of suppliers, driving down costs through competition.

The specifications also detail thermal management improvements.

High-speed interconnects generate significant heat, which adds to the cooling burden of data centres.

OmniConnect's low-power analogue front-end reduces this thermal output, easing the load on cooling systems.

In Europe, where cooling costs can account for nearly 40% of a data centre's operational budget, such efficiency gains are financially substantial.

• High-density connectors reduce physical footprint.
• Deterministic latency ensures consistent performance.
• Low-power design cuts thermal output.

European Hyperscalers Embrace Open Standards

The reaction from the European technology sector has been notably positive, particularly among operators of sovereign clouds and high-performance computing centres.

European organisations have long advocated for open standards to reduce reliance on proprietary vendors, often referred to as avoiding vendor lock-in.

The Open Compute Project, founded to facilitate this exact type of collaboration, provides the ideal venue for Credo's contribution.

Officials at major European cloud providers suggested that OmniConnect could become a de facto standard for next-generation AI inference racks.

The European Union's focus on digital sovereignty aligns perfectly with the adoption of open hardware designs.

By utilising an OCP-standardised interconnect, European data centres can source components from a diverse supply chain, mitigating geopolitical risks.

This is particularly pertinent given the recent disruptions in global semiconductor logistics.

Furthermore, the energy efficiency of the OmniConnect solution supports the EU's 'Fit for 55' climate package, which sets ambitious targets for reducing energy consumption in the digital sector.

Data centres are under immense pressure to demonstrate their green credentials.

Switching to interconnects that reduce power consumption by even a few watts per port can result in megawatt-scale savings across a large facility.

Industry experts pointed out that while training clusters often grab the headlines, inference clusters are far more numerous and consume the bulk of the operational energy in the long run.

Therefore, optimising inference infrastructure is the most effective way to improve the overall sustainability of AI services.

The contribution also paves the way for European manufacturers to enter the high-speed interconnect market.

With the specifications available openly, specialised engineering firms in Germany and France can develop compatible hardware, fostering a local industrial base.

This localisation of the supply chain is a strategic priority for the European Commission.

• Open standards reduce vendor lock-in risks.
• Efficiency supports EU climate targets.
• Local manufacturing opportunities arise.

Market Implications and the Competitive Landscape

Credo's decision to open-source its core interconnect technology represents a strategic gambit in a fiercely competitive market.

The company competes with giants like Broadcom and Marvell, which dominate the high-speed connectivity market with proprietary portfolios.

By donating OmniConnect to OCP, Credo effectively commoditises a layer of the technology stack that its competitors monetise heavily.

Analysts believe this move aims to expand Credo's total addressable market by accelerating the adoption of high-performance inference systems overall.

Even if Credo does not sell every single connector, the proliferation of the standard drives demand for the IP and components that Credo excels at manufacturing.

The market for AI interconnects is projected to explode in the coming years.

As inference workloads surpass training workloads in volume, the demand for scale-in solutions will outpace traditional scale-out networking gear.

Financial estimates suggest the market for AI-specific interconnects could exceed €15 billion by 2030.

Credo is positioning itself to capture a significant share of this growth by establishing the architectural rules of the game early.

Competitors may be forced to respond by either joining the OCP effort or doubling down on the performance advantages of their proprietary closed systems.

However, the trend in the industry is moving towards openness.

Customers are increasingly reluctant to invest in closed ecosystems that carry the risk of obsolescence or price gouging.

The OmniConnect contribution puts pressure on the industry to justify premium pricing for proprietary alternatives.

Sources in the investment community noted that Credo's stock reacted positively to the announcement, reflecting investor confidence in the open-source strategy.

The move signals a shift from selling boxes to selling influence.

In the technology sector, defining the standard is often more profitable than merely competing within it.

• Strategy competes with Broadcom and Marvell.
• Market for AI interconnects could hit €15bn by 2030.
• Open-source model pressures proprietary rivals.

What Comes Next for OCP and AI Infrastructure

Looking ahead, the immediate focus for Credo and the OCP community will be the ratification and refinement of the OmniConnect specifications.

This process involves rigorous testing and feedback from potential adopters to ensure the standard meets the reliability requirements of enterprise environments.

Engineers expect the first commercial products based on the OmniConnect standard to hit the market by late 2027.

These initial implementations will likely target high-density inference pods used by cloud service providers.

Once the technology proves itself in these controlled environments, it will likely trickle down to enterprise data centres and edge computing locations.

The success of this initiative will depend largely on the ecosystem that builds up around it.

Server manufacturers, cable makers, and chip designers must all align their roadmaps to support the new standard.

Industry observers will be watching closely to see which major OEMs announce support for OmniConnect in the coming months.

A commitment from a major server vendor like Dell Technologies or HPE would serve as a powerful validation of the technology.

Meanwhile, the software ecosystem will need to adapt.

Operating systems and orchestration software must be updated to recognise and optimise for the unique topology of a scale-in OmniConnect fabric.

However, the potential rewards justify the effort.

Breaking the memory wall is the final frontier in unlocking the full potential of artificial intelligence.

With this barrier removed, AI models can become larger, faster, and more ubiquitous.

For European industry, this means smarter factories, more efficient energy grids, and more responsive public services.

The contribution of OmniConnect to the OCP is more than just a product launch; it is a statement about the future direction of the entire computing infrastructure.

It signals a move towards openness, efficiency, and collaboration.

As the digital and physical worlds become increasingly intertwined, the plumbing that connects them—the interconnects—becomes the most critical technology of all.

• Ratification and testing phase begins immediately.
• First commercial products expected by late 2027.
• Major OEM support crucial for widespread adoption.