Skip to main content

Components

RoIS 2.0 defines 17 basic HRI Components. This page shows how each maps to a physical robot and to a virtual avatar, and what OpenRoIS provides for it today.

Robots and Virtual Avatars Share Most Components​

Perception and speech components run the same models whether their input comes from a robot camera or a webcam. Of the 17 basic components, 13 keep the same interface and the same underlying models across paradigms, differing at most in their input source, output device, or coordinate system. Only 4 (Reaction, Navigation, Follow, and Move) need implementations specific to the paradigm. The table is an analysis of the component definitions, not a measurement of shared code.

ComponentPhysical robotVirtual avatarAcross paradigms
Person DetectionDetector on the robot cameraDetector on a webcam or virtual sensorIdentical
Person LocalizationDepth camera and trackerWorld position or webcam depthCoordinate system differs
Person IdentificationFace or re-identification modelSame modelIdentical
Face DetectionFace detectorSame detectorIdentical
Face LocalizationFace meshSame face meshIdentical
Sound DetectionVoice activity detectionSame detectionIdentical
Sound LocalizationMicrophone arrayMicrophone array or virtual sourceSource differs
Speech RecognitionSpeech-to-text modelSame modelIdentical
Gesture RecognitionPose modelSame modelIdentical
Speech SynthesisText to speech on a speakerText to speech with lip syncOutput differs
Audio StreamingMicrophone streamSynthesized audio streamSource differs
Video StreamingCamera streamRendered frames streamSource differs
System InformationBattery, pose, and healthFrame rate and avatar stateState differs
ReactionLEDs or gesturesAnimations and expressionsParadigm-specific
NavigationNavigation stack, for example Nav2Navigation meshParadigm-specific
FollowNavigation and trackingVirtual follow behaviorParadigm-specific
MoveVelocity commands to motorsAvatar transformParadigm-specific

Status in OpenRoIS​

Three things exist per component type:

  • The profile: the commands, queries, events and parameters of the type, as a constant in openrois.interfaces.components, generated from the normative XML profiles of RoIS 2.0. A component declares the profile of its type, and the engine serves it.
  • Typed messages: Python, TypeScript and C# models of the type's results, so an event such as person_detected is a structured message instead of a list of strings. See how the types are generated.
  • Implementations: published components that run against a platform.
ComponentProfileTyped messagesImplementations
System InformationAvailableAvailableReachy Mini, mock
Person DetectionAvailableAvailableReachy Mini, mock
ReactionAvailableAvailableReachy Mini
NavigationAvailableAvailablemock
Speech SynthesisAvailablePlannedReachy Mini
Person LocalizationAvailablePlanned
Person IdentificationAvailablePlanned
Face DetectionAvailablePlanned
Face LocalizationAvailablePlanned
Sound DetectionAvailablePlanned
Sound LocalizationAvailablePlanned
Speech RecognitionAvailablePlanned
Gesture RecognitionAvailablePlanned
FollowAvailablePlanned
MoveAvailablePlanned
Audio StreamingAvailablePlanned
Video StreamingAvailablePlanned

The implementations are in two places:

  • The Reachy Mini adapter, for the open-source Pollen Robotics Reachy Mini. It runs on the real robot and in simulation, and is the reference for a complete adapter.
  • The mock adapter of the repository, which simulates its components with no robot. The quick start runs it.

A component implements the part of its profile that its platform supports. Its own profile lists exactly that part, which is what an application reads.

User-Defined Components​

RoIS allows components beyond the basic 17, reusing the common messages and the profile mechanism. OpenRoIS supports them with the same decorators: pass a profile of your own to @component instead of a constant, as write components and an adapter shows. Service applications discover user-defined components through the profile exactly like basic ones.

Community Components​

Adapters for additional platforms live in their own repositories, separate from the core repository. A registry for them is part of the long-term roadmap.