AIP-FR68

Description

The AIP-FR68 is a high-performance and cost-effective on-premises AI system designed by Aetina, featuring the advanced Qualcomm Cloud AI100 Ultra accelerator. It delivers up to 870 TOPS of processing power in a compact, energy-efficient form factor with 192GB DDR5 RAM and 2TB M.2 storage, making it ideal for edge AI deployment. The system supports large language models (LLMs) up to 70 billion parameters, is fully compatible with popular AI frameworks such as TensorFlow, PyTorch, and ONNX, and integrates seamlessly with inference servers like Triton and VLLM.

Equipped with the EdgeEye management platform, it offers remote control, AI card health monitoring, OTA updates, and intelligent fan adjustment, providing desktop-level performance comparable to high-end workstation and server setups without the need for complex data center infrastructure. Suitable for document processing, OCR, multimodal AI, speech-to-text, generative AI, and enterprise chatbots, the AIP-FR68 combines high performance, low power consumption, and smart device management in a single, versatile solution.

Technical Specifications

Key Features

• Up to 870 TOPS processing power for heavy workloads in machine learning, computer vision, and generative AI
• Support for LLMs up to 70 billion parameters with 128GB onboard memory on the AI card
• Full compatibility with popular frameworks such as TensorFlow, PyTorch, ONNX, and inference servers like Triton and VLLM
• Advanced device management via the EdgeEye platform for remote on/off control, health monitoring, and OTA updates
• Desktop-level design with data-center performance without the need for complex infrastructure

 Suggested Use Cases

• Document generation and summarization with models like Llama4 Scout
• Text recognition and OCR for document and form processing
• Intelligent enterprise assistants for data analysis, question answering, and audio processing
• Text-to-image generation with multimodal Vision-Language models
• Speech-to-text conversion with models like Whisper
• Code generation and specialized chatbots using models like CodeLlama and Starcoder

Competitive Advantages

• Performance comparable to systems with NVIDIA RTX 6000Ada or H100 NVL GPUs
• Lower power consumption with a compact design suitable for on-site deployment
• Support for advanced models like Llama 3.3, Falcon, Gemma, Whisper, Granite, and DeepSeek
• Model conversion via ONNX and QPC for rapid deployment in production environments
• Ideal for organizations seeking on-prem AI solutions without large-scale data centers

Related Models

• AIP-FR68S: Upgraded version with dual AI100 Ultra cards
• AEX-2UA1: Another model in the Aetina family with three AI100 Ultra cards for heavier workloads

AIP-FR68 vs Competitor Systems for Running LLMs up to 70B Parameters