AI comes to the Edge with SolidRun and Gyrfalcon's AI inference server
Do you need AI video smarts on the edge? Then, SolidRun, a developer and manufacturer of high-performance edge computing hardware, and application-specific integrated circuit (ASIC) chip manufacturer Gyrfalcon Technology has a server for you: The Arm-based, Linux-powered Janux GS31 AI inference server.
Primers
What's an AI inference server you ask? Once you've trained a neural network with machine learning to recognize, say, cars and spaces, it's learned lessons can be built into an application. That program can then infer things about new data based on its training. So, for example, an AI-empowered traffic cop might infer when someone's speeding or has run a red light.
Of course, if you're going to do anything about this in real-time, you need a computer on the edge rather than a second or two of latency away in a cloud datacenter. That's where the Janux GS31 comes in.
The Janux GS31 comes as a rackmount 1U server. At its heart, it uses a CEx7 LX2160A 16-core Arm Cortex A72 CPU. For its real processing power, it can use up to 128 Gyrfalcon Lightspeeur SPR2803 AI acceleration chips and 32 i.MX8M System on Chips (SoC)s. For fast memory, it uses up to 64GB dual-channel SO-DIMM DDR4 RAM.
This supports all major neural network frameworks. Specifically, it supports the open-source TensorFlow, Caffe, and PyTorch frameworks.
It's designed to support ultra-low latency decoding and video analytics of up to 128 channels of 1080p/60Hz video. This makes it well suited for monitoring smart cities and infrastructure, intelligent enterprise/industrial video surveillance applications, and tagging photos and videos for text-based searching.
The makers claim that it can outperform System on Chip (SoC) and GPU-based systems by orders of magnitude while using a fraction of the energy required by systems with equivalent computational power.
"Powerful, new AI models are being brought to market every minute, and demand for AI inference solutions to deploy these AI models is growing massively," said Dr. Atai Ziv, CEO at SolidRun in a statement. "While GPU-based inference servers have seen significant traction for cloud-based applications, there is a growing need for edge-optimized solutions that offer powerful AI inference with less latency than cloud-based solutions. Working with Gyrfalcon and utilizing their industry-proven ASICs has allowed us to create a powerful, cost-effective solution for deploying AI at the Edge that offers seamless scalability."