AI learns to predict human behavior from videos
In a new study by researchers at Columbia Engineering, a computer vision technique for giving machines a more intuitive sense for what will occur next, by leveraging higher-level associations between people, animals and objects.
Assistant professor of computer science at Columbia, Carl Vondrick, said their algorithm was a step toward machines being able to make better predictions about human behaviour and therefore better coordinate their actions with our own. He added the results meant the opening of a number of possibilities for human-robot collaboration, autonomous vehicles and assistive technologies.
Learning by ‘watching’
The AI systems involved learn to predict hundreds of activities from handshaking to fist bumping, after analysing thousands of hours of movies, sports games and TV shows. When they can’t the system finds the higher-level concept that links them, in this case the word ‘greeting’. Columbia’s algorithm is the first to learn the capability to reason abstractly about future events, despite the fact that not everything in the future can be predicted.
The mathematical framework developed by the researchers enables machines to organise events by how predictable they are in future. An example is swimming and running as forms of exercise. The new technique learns how to categorise such activities on their own. This could move computers closer to being able to size up a situation and make a nuanced decision, instead of a pre-programmed action and is a critical step in building trust between computers and humans. If machines are able to understand and anticipate human behaviours, they could seamlessly assist people in daily activities.
The possibilities for improving people’s lives
Researchers add the next step was to verify the system works outside the lab and if it can work in diverse settings. If so, there are many possibilities to deploy machines and robots that may improve people’s safety, health and security. The group plans to continue improving the algorithm’s performance with the use of datasets and computers, plus other forms of geometry. The study is titled ‘Learning the predictability of the future’.
