To enrich the Qualcomm Cloud AI 100 Extremely accelerator, the corporate has developed a software program suite for AI inference workloads
From an enterprise perspective, AI is all about placing information to work in a means that improves course of and workflow effectivity, and creates new income alternatives. The middle of knowledge gravity is on the edge the place linked units of all kinds produce a gentle stream of knowledge that probably incorporates priceless insights if solely it could possibly be successfully, shortly parsed and fed ahead into no matter course of or workflow the consumer has recognized. In the mean time, the middle of AI gravity is within the cloud, though broad trade discourse suggests edge AI is a precedence given the clear advantages round value, latency, privateness and different elements. The high-level concept right here is to carry AI to your information moderately than bringing your information to AI.
Qualcomm has constructed a compelling narrative round edge AI and it’s function in bringing to market merchandise that propel AI from a sequence of level options to a bigger system. Final month in the course of the Client Electronics Present in Las Vegas, Qualcomm had a spread of consumer-facing bulletins masking automotive, private computing and sensible dwelling tech; however additionally they had an attention-grabbing launch that speaks to enterprise adoption of edge AI options.
Through the present, the corporate introduced its Qualcomm AI On-Prem Equipment Resolution and Qualcomm AI Inference Suite which, when mixed, let enterprises “run customized and off-the-shelf AI purposes on their premises, together with generative workloads,” in response to a press launch. This, in flip, can speed up enterprise AI adoption in a means that reduces TCO as in comparison with counting on another person’s AI infrastructure property.
The mixed {hardware} and software program providing “adjustments the TCO economics of AI deployment by enabling processing of generative AI workloads from cloud-only to a neighborhood, on-premises deployment,” Qualcomm’s Nakul Duggal, group normal supervisor for automotive, industrial IoT and cloud computing, stated in a press release. On-prem enablement of a spread of AI-based automation use circumstances “reduces AI operational prices for enterprise and industrial wants. Enterprises can now speed up deployment of generative AI purposes leveraging their very own fashions, with privateness, personalization and customization whereas remaining in full management, with confidence that their information is not going to go away their premises.”
Industrial large Honeywell is working with Qualcomm to design, consider “and/or” deploy “AI workflow automation use circumstances” utilizing the brand new {hardware} and software program merchandise. Aetina, a Taiwanese edge AI specialist, “is among the many first OEMs to supply on-premises tools for deployments based mostly on the AI On-Prem Equipment Options;” that’s within the type of Aetina’s MegaEdge AIP-FR68. And, “IBM is collaborating to carry its watsonx information and AI platform and Granite household of AI fashions for deployment throughout on-prem home equipment, along with cloud, to assist a spread of enterprise and industrial use circumstances in automotive, manufacturing, retail and telecommunications.”
The home equipment leverage Qualcomm’s Cloud AI 100 Extremely accelerator card. Related specs embody:
- ML capability (INT8) of 870 TOPs
- PCIe FH3/4L kind issue
- 64 AI cores per card
- 128 GB LPR4x on-card DRAM
- 576 MB on-die SRAM
The inference software program suite consists of ready-to-use apps and brokers for chatbots, code growth, picture era, real-time transcription and translation, retrieval-augmented era (RAG), and summarization.
Click on right here for particulars on the on-prem equipment, and right here for extra on the inference software program suite. And for a higher-level have a look at edge AI, distributed inference and test-time AI scaling, give this a learn.
