Those two jobs need different physics: Rebellions CEO says training and inference need different chips

Date:
Tue, 28 Jul 2026 15:05:00 +0000

Description:
Rebellions says were starting to see the emergence of an inference chip
market could this be when AI starts to become truly efficient?

FULL STORY ======================================================================Copy link Facebook X Whatsapp Reddit Pinterest Flipboard Threads Email Share this article 0 Join the conversation Follow us Add us as a preferred source on Google Newsletter Subscribe to our newsletter The AI race started off with a pretty clear direction bigger and better. The first waves were characterized by building bigger models, but its all change in the world of artificial intelligence and with enterprises, SMBs and consumers all finding use cases for the technology, the focus has shifted.

Now, AI firms and model developers are looking to realize a much tougher
goal. Efficiency. Cost per token, performance per watt, output per input, its all about driving maximum efficiency. One clear divide is between training
and inference. While training models still requires huge amounts of
resources, inference efficiency is starting to improve, and one company (Rebellions) now believes an opening for inference-first hardware could
create a new market. Latest Videos From TechRadar Watch full video here:

The companys racks are said to consume around 16-20kW, compared with around 120kW for leading GPU-based inference systems that, for many use cases, are sheer overkill.

Rebellions rack costs are also said to be around one-third of the price, making AI inference more accessible and helping enterprises to deploy AI more widely. You may like Tomorrows AI networks need to adapt to stay ahead of the inference curve Why Arm is set to help power the next generation of AI workloads and beyond InstaLILY CEO Amit Shah says future enterprise success relies on owning intelligence more than renting models from hyperscalers This hardware shift could be the start of truly efficient AI Memory is also
another battleground, whereby huge trillion-parameter models are testing the limits of todays hardware and the intertwined reliance on memory and compute. Something Rebellions says its looking to fix by working with the likes of SK Hynix and Samsung to align multiple roadmaps, instead of having to respond to shifts in architecture.

Ultimately, todays black-and-white chip manufacturing landscape is now evolving, and Rebellions sees two key changes happening simultaneously. Firstly, training and inference hardware is starting to differ more drastically. Secondly, aligning multiple hardware roadmaps across memory, compute and more will drive more efficiency not just across deployments, but in terms of bringing new products to market. Are you a pro? Subscribe to our newsletter Sign up to the TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed! Contact me
with news and offers from other Future brands Receive email from us on behalf of our trusted partners or sponsors By submitting your information you agree to the Terms & Conditions and Privacy Policy and are aged 16 or over.

I spoke to Rebellions CEO Sunghyun Park allows me to understand how and why inference and training hardware are starting to separate, as well as the importance of open standards and collaboration in the drive for all-round efficiency. The AI chip market seems to be splitting between training-first and inference-first architectures. Why is that happening, and why now? This split exists because training and inference are fundamentally different problems.

Training is how you build a model. It happens once, involves a small number
of organizations, and rewards raw computational flexibility because the workload keeps shifting as research moves forward. What to read next The AI ROI gap: Why enterprise intelligence is stalling at the infrastructure level The hidden operational costs of agentic AI AI in orbit: The next evolution of compute infrastructure

Inference is how you actually use a model: every query answered, every transaction processed, every decision an AI system makes in production. That happens billions of times a day across nearly every industry, and its where
AI moves from R&D into revenue.

Those two jobs need different physics. Training requires maximum FLOPS. Inference requires efficiency, reliability, and economics that hold up when youre serving users at scale.

The industry forced a training chip into that second job because thats what existed. Now that inference has become the larger, more urgent market, that compromise no longer holds. Enterprises and governments are asking how fast they can deploy. Thats why the conversation is splitting now. Where does Rebellions fit in that split, and what makes your approach to AI inference fundamentally different from your competitors? We built for inference, from day one. Most first-generation AI chip companies emerged from the 2016-2017 training boom and adapted their architectures for inference afterward.

We started in 2020 after that wave with inference as the only target, which meant designing around what production AI actually needs instead of retrofitting a training chip.

The numbers reflect that choice. Our racks draw 16-20kW versus roughly 120kW for leading GPU-based inference systems, about a sixth of the power, in a market where power is the binding constraint for most operators.

Acquisition cost runs around $10 million per rack versus roughly $30 million, about a third of the cost. Our chiplet-based architecture also scales out rather than betting that a single device can handle a models full size, which matters now that production workloads are trillion-parameter mixture-of-experts models instead of the few-hundred-million-parameter models the first generation was built around.

Its also why our architecture is memory-centric rather than compute-centric. The chiplet approach exists to keep memory close to logic as models scale,
not just to add cores.

And we have three years of production deployments behind that architecture, not pilots. Thats the hardest part to replicate: real workloads, running at scale, today. A year ago, everyone in AI was talking about chiplets. Now the conversation has shifted to memory. What changed? The chiplet conversation
was about architecture: breaking a chip into modular pieces that scale independently, rather than betting everything on a single monolithic die.

That mattered because it let the industry move past an assumption the first generation of accelerators made in 2016 and 2017, that a single device would always be big enough to run any model.

That assumption broke once mixture-of-experts and trillion-parameter models arrived.

The memory conversation is the layer underneath that. Once the architecture problem is solved, the constraint becomes physical: can you actually get enough high-bandwidth memory (HBM) to build what youve designed?

HBM is 3D-stacked memory, and how closely you can physically stack it to compute is as much of a bottleneck as raw supply.

Every AI accelerator company is competing for the same limited supply right now, and demand has outpaced what memory makers can produce. Thats the memory-logic co-design problem: architecture and memory supply are no longer separable decisions.

Were in a different position because our investor relationships were built around supply, not just capital. Our memory partners are also investors, and we co-design our memory architecture directly against their roadmaps rather than simply purchasing off them.

Our chiplet architecture also develops against our foundry partners process roadmap. When the rest of the industry was fighting for allocation, we
already had a seat at the design table through those relationships.

Thats memory-logic co-design, not just secured supply. The shift from
chiplets to memory tracks has moved the real constraint: from architecture to physical supply. Your stack runs on open-source frameworks like vLLM,
PyTorch, Kubernetes, and OpenShift, tools many enterprises already use. Does that make adoption relatively plug-and-play, or is that an
oversimplification? Its mostly true, but plug-and-play undersells how deliberate that was, and oversimplifies in one specific way.

We built entirely on open standards: vLLM, PyTorch, Kubernetes, and Red Hat OpenShift. Were one of only two chip companies in the PyTorch Foundation, and the only AI accelerator company fully integrated with OpenShift.

A developer who already knows how to run inference on existing infrastructure already knows how to run it on ours. Theres no proprietary runtime to learn and no migration project. That part really is close to plug-and-play.

The first generation of AI chip companies each built proprietary software stacks, and hundreds of millions of dollars went into software that didnt survive. We came to market once the open source ecosystem had matured and chose to build on it instead of forking it.

Where it oversimplifies is assuming that means zero integration work. Production deployments still require validating performance at your specific workload and scale, and that takes real engineering time, no matter how compatible the stack is.

What open standards remove is lock-in risk and retraining cost, not the deployment work itself. What are the biggest challenges organizations face when trying to run AI inference outside of hyperscaler platforms? Most organizations arent built like hyperscalers, and much of the available inference infrastructure assumes they are.

The first challenge is physical. Most enterprises, telcos, and governments already have data centers. They cant wait two to three years or spend $600 million-plus on new ones, and they cant retrofit for liquid cooling without major cost and disruption.

So the practical question is whether inference hardware runs on what they already have: standard racks, air cooling, and existing power budgets.

The second is sovereignty. Organizations increasingly want to bring compute
to where their data already lives, rather than move sensitive data to
wherever compute is hosted.

Thats partly regulatory, partly operational, but either way, cloud-only inference creates a dependency a lot of operators are no longer comfortable with.

We built specifically for that gap. Our systems run at 4-5kW per server on standard air-cooled infrastructure, no facility redesign required.

SK Telecom has run on our hardware for nearly three years, scaling from a small cluster to close to 100 racks and now processing 50 million API transactions a day, entirely inside their existing network.

KT Cloud runs real-time inference on highway CCTV systems nationally. Both show you dont need hyperscaler-scale infrastructure to run AI at hyperscaler-relevant volume. Cerebras' IPO, and potential listings from
others in the space, have put AI infrastructure in the spotlight. What do these IPOs signal about the market, and where might the hype be outpacing deployment reality? It tells the public markets that AI infrastructure is a durable, investable asset class, not just a venture-backed bet. That validation benefits the whole sector, including us: it says purpose-built AI silicon is real, differentiated from general-purpose GPUs, and worth independent capital.

Capital flowing into the sector is necessary, but it doesnt by itself determine who wins. The companies that define the next decade of this market wont necessarily be the most-funded ones.

Theyll be the ones with real fundamentals: production customers, deployment scale, proven economics, durable supply chain relationships. Thats a
different filter than fundraising size, and its the one that matters once public market scrutiny starts.

Weve been building toward that filter since 2020, with production deployments and supply relationships with our foundry and memory partners that we secured before the rest of the industry was fighting over the same allocation. The capital is a tailwind for everyone serious about inference.

Whether it gets deployed well is a separate question, and one the market will answer over the next few years. Looking ahead, what trends are you watching most closely across AI infrastructure and inference over the next 1224
months? The buildout happening inside existing infrastructure keeps accelerating. Most enterprises, telcos, and governments arent waiting for new data centers.

Theyre deploying inference into facilities they already have, and I expect that to become the bigger story even though it gets less attention than hyperscaler headlines.

Additionally, memory remains the physical constraint. HBM supply hasnt caught up with demand, and as models keep moving toward trillion-parameter mixture-of-experts architectures, that pressure increases rather than eases.

Companies with secured, strategic supply relationships will have a real advantage over the next two years, not just a cost one.

Chiplets are how we get there. Weve already mass-produced a highly advanced 4-chiplet package a level of integration Nvidia has struggled to reach.

Reporting this year indicated Nvidia had built and demonstrated a
four-chiplet Rubin Ultra design, then canceled it in favor of a dual-die architecture over manufacturability concerns: a four-die single package
pushes roughly 7.5-8x past reticle limits on yield and cost. Thats the foundation.

The next layer were building on is performance optimization of HMB3E (3D-stacked memory) in close collaboration with memory and compute
co-designed together from the start, not bolted on after the fact.

Also on my radar is the fact that as more companies in this space go public, capital will get valued against production fundamentals rather than funding rounds.

And the efficiency point matters: as inference gets cheaper per token, demand doesnt shrink, it expands, because new use cases become viable at lower cost. Thats been true of every computing platform in history, and I dont expect AI inference to be the exception. Follow TechRadar on Google News and add us as
a preferred source to get our expert news, reviews, and opinion in your feeds.



======================================================================
Link to news story: https://www.techradar.com/pro/those-two-jobs-need-different-physics-rebellions -ceo-says-training-and-inference-need-different-chips


--- Mystic BBS v1.12 A49 (Linux/64)
* Origin: tqwNet Technology News (1337:1/100)