Multimodal cognitive processing using artificial endocrine system for development of affective virtual agents

In this paper a comprehensive architecture for emotional and affective process in a virtual agent is presented. By fusing video, audio and text emotion of the users as affective sources to the system, the virtual agent can appraise the mood of clients. To emulate the influence of the human hormones in the virtual agent, the proposed system employs Artificial Endocrine System (AES) in the aspects of moods and biological needs, by controlling the concentration level of the influential hormones. The agent affective processor engages AES, personality and mood modules to manage the internal state. Intelligent virtual agent would interact with clients according to its affective state circumstances. The proposed system presents a complete platform to capture emotional channels through the network to analyze and process them in an affective engine in order to determine the emotional quality of the response.