The social signal interpretation (SSI) framework provides tools to record, analyze and recognize human behaviors in real time, such as gestures, mimicry, nodding, and emotional speech. The post patch based design pipeline sets components independently, allowing parallel and synchronous processing of sensor data from multiple input devices. In particular, SSI supports complete machine learning pipeline length and provides a graphical interface to help users collect training corpus and obtain personalized models. In addition to the large number of built-in component SSIS, developers are encouraged to extend the new capabilities of available tools. An easy-to-use XML editor for inexperienced users can draft and run pipes without special programming skills. SSI is to write and optimize computer system and run on multiple CPUs with C + +.

SSI includes the following processes:

Record:

Training:

Distinguish:

Supported sensors: