IBM & NTT Explain How AI Works on Edge Computing

One trillion plus AI parameter models are already banging hard at our front door. These new artificial intelligence models, with unprecedented computing power, will become the norm in the coming months and years ahead.

While the technological advancements of new generative AI models are promising and expected to benefit most sectors and industries, the world still has a big AI-size-edge-infrastructure problem (we will break this down in a moment).

How will giant AI models operate on the edge of networks while offering low latency and real-time services? It’s a ‘shrinking giant’ problem.

In this report, Techopedia talks with NTT and IBM experts to understand an approach to solving fast, resource-intensive artificial intelligence without over-burdening a network.

Table of Contents Table of Contents

1. Key Takeaways
2. Is the Solution an Edge Infrastructure Combined with Smaller AI Models?
3. Speaking the Many Languages of Edge Computing
4. An AI Built for Your Edge
5. The Bottom Line
6. References

Show Full Guide

Table of Contents

1. Key Takeaways
2. Is the Solution an Edge Infrastructure Combined with Smaller AI Models?
3. Speaking the Many Languages of Edge Computing
Show Full Guide
4. An AI Built for Your Edge
5. The Bottom Line
6. References

Is the Solution an Edge Infrastructure Combined with Smaller AI Models?

Compute power is rapidly moving from the data center to the edge. Organizations expect edge computing to significantly impact all aspects of operations. Meanwhile, worldwide spending on edge computing is expected to be $232 billion in 2024, an increase of 15.4% over 2023 — largely driven by AI.

Techopedia spoke to Paul Bloudoff, Senior Director, Edge Services, NTT DATA, which focuses on edge AI — particularly smaller, more efficient AI models. These models are use-case specific and sized to be simple to deploy and run.

“When we look at bringing AI to the edge, we need to understand the business use case and what organizations are trying to accomplish. Not every deployment will need monolithic AI models to support real-life use cases.”

Bloudoff explained how IT and Operational Technology (OT) teams can benefit from small AI models running on lightweight edge.

For example, in factory floors, these AI solutions can enhance maintenance by breaking down siloes and bringing together a suite of information provided by the Internet of Things (IoT) to, for instance, track data on vibration, temperature, machine wear, and more.

Industrial predictive maintenance AI can save organizations thousands of dollars by reducing downtime.

“To accomplish this (use case), organizations will not need a million-dollar AI platform to process this kind of information.

“For many organizations, the resource investment to bring monolithic AI solutions to the edge is too complex and costly.”

Shrinking AI Giants into TinyAI to Drive Sustainable Development

Scentistific researchers already advocate for TinyAI models as the solution to the complex transition of data center AI models into edge computing.

Scientists argue that TinyAI models can be deployed in healthcare, agriculture, and urban development and contribute to the development of the United Nations Sustainable Development Goals. Tiny models are designed for specific use cases and, therefore, are more cost-efficient, and consume less power, driving sustainability targets.

Nick Fuller, Vice President of AI and Automation at IBM Research, told Techopedia that for edge workloads requiring low latency, inferencing on devices is more than highly desirable; it is essential.

“To this end, serving ‘small’ foundation models on such devices (robotic arms, mobile devices, cameras, etc.) facilitates on-device inferencing within the budget (memory, compute, accelerator) constraints of such devices.”

“Models, of course, can still be trained on-premise or on the cloud. To this end, lightweight AI models are very appealing to the edge market and especially to specific workloads where latency requirements are essential.”

Speaking the Many Languages of Edge Computing

Many developers fear that edge infrastructure is constrained in terms of computing power processing, data storage, and memory. Tehcopedi asked Bloudoff from NTT how the company approached this problem.

Bloudoff explained that one of the challenges organizations face when deploying solutions at the edge is the silos between machines and devices across their network — machines and devices from different manufacturers do not always communicate well with each other.

“Think of it as different individuals speaking different languages who are all providing data that needs to be collected, analyzed, processed, and transformed into action,” Bloudoff said.

“What’s powerful about an Edge AI platform is that its software layer automatically discovers devices and machines across an IT or OT environment through its smaller, more efficient language learning model,” Bloudoff added.

NTT’s Edge AI platform runs auto-discovery and unifies and processes the data to provide a comprehensive diagnostic report that can be used for AI-powered solutions.

“Once you break down silos and connect with stakeholders across different teams, you will have a better understanding of your processing power and memory needs.

“Bigger isn’t always better.”

Speaking to Techopedia, IBM Fellow Catherine Crawford added that the challenge is not just in how to move AI to the edge.

“There are multiple existing use cases that can leverage edge using smaller AI models where the technical challenges still exist and assets can be developed for distributed, secure Edge to Cloud continuous development AIOps.”

Crawford explained that there is ongoing interest and research in understanding how task-tuned, smaller genAI and foundational models can be developed and leveraged for edge use cases considering constraints like compute, memory, storage, and power (battery life).

“For instance, perhaps having multiple, tuned smaller models on-board devices with hierarchical inferencing algorithms will be sufficient for edge use cases.

“AI algorithms and corresponding models will continue to evolve and, with the focus in our industry on sustainability, these models will quickly evolve to take less resources, becoming more appropriate for edge systems.”

An AI Built for Your Edge

The 2023 Edge Advantage Report — which surveyed 600 enterprises across multiple industries — found that approximately 70% of organizations use edge solutions to solve business challenges. Still, nearly 40% worry that their current infrastructure will not support advanced solutions.

Bloudoff from NTT said that the company is well aware of the market′s concern and focuses on smaller, more efficient language models. These are easier to deploy and can run in real-time without demanding advanced edge hardware updates.

Smaller AI models represent enormous savings in edge hardware costs for industries and businesses across the world. By going tiny, AI on the edge can maximize the edge computing power already in play through optimization.

The company is also moving forward by implementing dozens of proofs of concepts with customers across manufacturing, automotive, healthcare, power generation, logistics, and other industries.

The Bottom Line

Supercomputers, quantum computers, and state-of-the-art AI racks hosted in massive data centers, are undoubtedly the tip of the spear of modern innovation. However, the real world works at the edge level.

From the smartphone in your pocket to the automation of machines in industrial environments, healthcare, and agriculture, modern edge networks connect us all.

As bigger, faster, harder, and stronger AI models roll out, NTT and IBM invest in small and tiny models. They believe it is the solution to the future of giant AIs in our edge world.