A.I. Research

Head Pose Estimation Using a Cylindrical Face Model

Master thesis AI
Participants: Roeland Weve (supervised by Nicu Sebe)

Human-computer interaction (HCI) research is the study of interaction between user and computer. The aim of HCI research is to improve this interaction, making computers more intelligent and user friendly, so that in the near future computers will interact with people in a more natural way. To achieve this, the computer should be able to understand human behaviour and respond appropriately. If the computer can respond to human expressions made by the head and face, it can enable a user to control a computer without devices such as keyboards, mice and other interfaces that depend on direct physical contact.

The process of head pose recognition starts with the detection of the head in the first frame of the video sequence. For proper face candidate detection, a near-frontal view is needed. Once the head position is known, the head pose can be estimated during the next frames of the video sequence. During this stage the head pose estimation parameters can be used for many different purposes, like facial expressions or gaze direction. The aim was to develop a system which can continuously track the pose of the head in a video sequence. Therefore the detection of the head is not performed automatically but is initialised by hand.

Much research has been done on head pose recognition. However, it is hard to achieve a robust and accurately working system, due to varying illumination, (self-) occlusion, large rotations, fast rotation and translation, background information, skin color and real-time implementation.

The contribution of my thesis to the area of head pose estimation consisted of detailed theory on how to implement a cylindrical head tracking system, extensive experiments with different videos and a working cylindrical head tracker framework in Matlab.

Keywords: Computer Vision, Human-computer interaction, head tracking, Lucas Kanade algorithm, perspective projection, ray tracing, Matlab

Playing pong using your mobile phone as a bat

Game Programming
Participants: Thomas Mensink, Martijn Liem and Roeland Weve

The goal of this project was to create an innovative game, using AI techniques as well as intelligent game programming techniques to create a next generation game experience. Our idea was to create a mobile Java based phone game introducing the classic game of Pong with our own innovative touch. We would create the first game of Pong in which your phone would actually be the bat you control to smash the ball to the other side of the field. This means we should find a way to control the bat using the movements made with the phone, and create a room in which it would be possible to combine these movements with a game of pong in a logical way. The most intuitive way to do this would probably be to create a 'first person' kind of view in which you would look into a room and smash the ball away from yourself into the wall ahead. To capture the motion vector of the phone, we decided to use the camera mounted on most modern mobile phones to track the image motion. For this purpose, we decided to implement the TinyMotion algorithm in Mobile Java.

Keywords: TinyMotion, gaming, camera capture, J2ME (java)

Scientific Visualisation

Scientific Visualization & Virtual Reality
Participants: Michiel Nieuwenhuijsen and Roeland Weve

The research we performed consisted of three different experiments. Each experiment has its own relatively small dataset, which was made out of text or binary data in a Lattice Boltzman Grid. The global goal of all three experiments is to visualize the flow of these grids through Lattice Boltzmann Methods (LBM). LBM's are methods in computational fluid dynamics (CFD) used for fluid simulation. Unlike the traditional CFD's, LBM models the fluid consisting of fictive particles, and these particles propagate and interact with each other. These particles are positioned on the nodes of a discrete lattice, and jump from one lattice node to the next, according to their velocity. Then, the particles can collide and thus possibly get a new velocity. The three experiments that where performed consisted of visualizing the flow of two fluid streams through a single element of a SMRX static mixer reactor, visualize a dataset of a tornado (eye of the storm) and visualize the blood flow through the point of bifurcation in the carotid artery during various phases of one heartbeat.

Keywords: scientific visualization, Lattice Boltzmann grids, Python, VTK (The Visualization ToolKit)

PISYS: Person Identification SYStem

Design and Organization of Autonomous Systems
Participants: Thomas Mensink, Martijn Liem, Markos Mylonakis, Roeland Weve and Marinus Maris

Most ambient intelligence applications for home automation rely on successfully identifying people throughout the home. Typically, intrusive methods such as RFID tags, retina scans or face recognition are used for identification. In this paper, an innovative reasoning system for identifying persons is presented based on recognizing behavioral characteristics of the residents. This Person Identification SYStem (PISYS) exploits simple, non-intrusive sensing methods to reason about the presence of persons in each room of the house and calculates probability values with respect to their identities. The reasoning system is composed of a hybrid of two computational methods, namely Bayesian networks and the Max-Plus algorithm. These two methods complement each other's analysis with respect to person identification. A simulator was developed to generate training data and test scripts. Simulation results show that PISYS can successfully identify people in a variety of situations, including multiple people in the same room. Such functionality may become a crucial enhancement to existing home automation systems.

Keywords: Bayesian Network, Max Plus, non obtrusive, MySQL, Java

Tracking objects using Gabor filters

Profile project
Participants: Mark de Greef, Roeland Weve and Sjoerd Kerkstra

This paper discusses the use of texture as a means of tracking objects. More specifically, a framework which selects the most discriminating feature for tracking at every frame framework is extended to use texture information obtained from using Gabor filters. The theoretical principles of Gabor filter construction and use are discussed. Our hypothesis is that by adding Gabor filter features to the on-line feature selection framework, tracking performance will improve. This hypothesis is tested by contrasting the performance of a tracker using only color features to the performance of a tracker using Gabor features. Testing is done on specifically constructed animations as well as more realistic video. Test results illustrate situations were tracking using Gabor filtered features succeeds while tracking using color features only does not.

Keywords: Gabor filters, tracking, Mean Shift, on-line feature selection, Matlab

Mean-Shift Tracker

Multimedia Information Retrieval
Participants: Michiel Nieuwenhuijsen and Roeland Weve

In this report we discussed our findings on two algorithms on Object Tracking. These algorithms can be used to track objects in video sequences. This can be used in applications such as tracking a person in a surveillance video, or tracking a player in a football match. The two methods we discuss are Brute Force Object Tracking and Mean Shift Object Tracking. Both methods make use of histograms to compare different images, and the images can be represented in either rgb, or normalized rgb (RGB).

Keywords: tracking, Mean Shift, Matlab

Ontdekken van Impressionisten m.b.v. afstanden tot bekende Impressionisten

Bachelor thesis AI
Participants: Michiel Nieuwenhuijsen and Roeland Weve

Tijdens dit project hebben we een methode ontwikkeld voor het zoeken in documenten naar nog onbekende Impressionisten. Deze Impressionisten worden gevonden door te kijken naar de afstand tot bekende Impressionisten. De methode bestaad uit het zoeken naar personen in documenten, vergelijken of twee persoonsnamen bij dezelfde persoon horen, en het berekenen van een score om een bepaalde zekerheid te krijgen of een naam wel of niet tot het domein hoort. We hebben een aantal tests uitgevoerd op het domein van Impressionisme, maar ook hebben we gekeken of de methode toepasbaar is op andere domeinen. Te denken valt aan andere kunststromingen, personen uit een bepaald sportteam proberen te halen of mensen die eenzelfde soort beroep uitoefenen bij elkaar proberen te vinden.

Keywords: NiWe Distance, Informatie Extractie (IE), Named Entity, Python