ETD

• artificial intelligence
• cognitive architectures
• deep learning
• human robot interaction
• micro services
• robotics

This thesis presents a perceptual system for Abel, a social robot, to construct a "meta-scene"—a coherent representation of its environment. We built a Go-based middleware that coordinates real-time data flow between neural network-powered modules.

Key modules include a Voice Activity Detector, Speech-to-Text converter, Object and Subject Detectors, Depth Estimator, and Saliency Estimator. These operate asynchronously, contributing to a shared representation maintained by the server. A Python-based client library enhances flexibility and extensibility.

Compared to existing middleware, our system offers superior modularity, low-latency processing, and adaptability. Benchmarks confirm its efficiency in handling sensory data and enabling multi-modal perception.