Affective and context-aware personal assistant

Motivated by my wish to create better context and user-aware recommendation systems, in 2001 I started looking at affective computing, a paradigm that takes into account users’ emotions as well as other context information and preferences. Mood (or affective state) is important to me in filtering through my personal preferences, so I decided to give the same capabilities to the digital personal assistants. The system was distributed, with a wireless mobile device (an Compaq iPaq at that point) playing the role of the mobile interface to a user and the inference engine, user profiles and other data residing on a server. The initial scenario implemented was in a museum, where suggestions for exhibits be visited changed based on the user’s mood: e.g., “when feeling blue I prefer to see more seascapes or I prefer this painter more”. The inference engine was built using a CSP (Constraint Satisfaction Problem) representation. Later prototypes extended this system into other scenarios, such as music and news recommendation based on mood and other context information. In 2002 the focus moved towards home environments and I worked in designing and building affect and context-aware services for single and multi-visitor scenarios. In all prototypes I mainly focused on adapting applications and services based on user information and not on emotion sensing. The current mood was selected by the end user through the mobile device interface.

The selection was then used together with some other recorded context information (e.g., time, place) and user’s profile to make certain recommendations, such as what events to attend, what movies to watch, etc. External information sources, such as news feeds, weather status, local events, were also used.

The theme of context-aware personal assistants continued to appear throughout my further work in Nokia and afterwards in various scenarios and projects.

Other project members: Jerome Chevillat (2001), Agathe Battestini (summer intern, 2002), Vincent Schickel-Zuber (intern, 2003).
Related publication: D. Pavel, J. Hemanus, “What’s mood got to do with this?”, NRC Advance Magazine, 2003.