Feb 8, 2021

Why using AI to monitor your workforce might backfire

ai surveillance
Paddy Smith
3 min
AI surveillance
Companies have long sought to get visibility into their employees’ whereabouts and productivity. AI can help with that, but there are pitfalls too...

Looking over your employees’ shoulders is as old as employment itself. And using technology to do it isn’t new either. But AI threatens to usher in a new era of Big Brother spying on employees. And it might bite back.

What’s happening in AI surveillance?

A high-profile example of AI surveillance would be Amazon’s announcement that it will use Netradyne AI cameras in its branded fleet (see the video here). Amazon is selling the technology on safety credentials – it can warn drivers about distracted driving and deliver real-time alerts about road conditions. But the cameras will also be able to document driver behaviour, and it’s likely that that won’t be limited to road safety.

Using AI for desk-based employees is also on the rise, with software capable of logging and reporting time spent in front of computers, keystrokes made and other metrics deemed to be indicative of productivity.

How can using AI surveillance backfire?

A central problem with using AI is its very nature. The algorithm will have been trained to look out for key metrics (eg. screentime) but this can be reverse engineered by either enterprising employees or by software developers who see a lucrative market in creating counter-AI programs. The ‘hack’ might be something as simple as a cardboard cutout of the employee put in front of the camera to fox the AI, but increasingly it’s likely that more sophisticated means of overcoming the principal objectives of the software will be developed.

Isn’t that against the law?

Actually, it’s more likely that the software will breach existing privacy and human rights laws, so even if you know an employee hasn’t been pulling his or her weight, proving it could be minefield of legal implications. Employees have a reasonable expectation of privacy in the workplace unless the employer has legitimate grounds and uses proportionate means. Installing AI before those grounds are founded risks breaching employees’ human rights. This is even tougher when employees are working from home, as they maintain a right to private life and family life.

GDPR and AI surveillance

Large-scale monitoring using AI is not something that can be bought off the shelf and plugged in. It’s likely employers would have to produce a data protection impact assessment report which might be costly to assemble and urge caution in the implementation. The EU’s GDPR regulations are particularly stringent and enforce transparency about what and how data is collected. Again, this creates an opportunity for employees to find ways to outsmart the software.

So will AI surveillance become common at work?

Yes, it’s likely it will be used, but probably with a light touch approach. And it may be hard to act on findings without further non-AI supporting information. For some businesses, the risks will outweigh the rewards. Most likely, as with Amazon, the AI will have to be presented as a safety measure. Its use to monitor behaviour and productivity may be limited.

Share article

Jun 17, 2021

Facebook Develops AI to Crackdown on Deepfakes

3 min
Social media giant, Facebook, has developed artificial intelligence that can supposedly identify and reverse-engineer deepfake images

In light of the large tidal wave of increasingly believable deepfake images and videos that have been hitting the feeds of every major social media and news outlet in recent years, global organisations have started to consider the risk factor behind them. While the majority of deepfakes are created purely for amusement, their increasing sophistication is leading to a very simple question: What happens when a deepfake is produced not for amusement, but for malicious intent on a grander scale? 


Yesterday, Facebook revealed that it was also concerned by that very question and that it had decided to take a stand against deepfakes. In partnership with Michigan State University, the social media giant presented “a research method of detecting and attributing deepfakes that relies on reverse engineering from a single AI-generated image to the generative model used to produce it.” 


The promise is that Facebook’s method will facilitate deepfake detection and tracing in real-world settings, where the deepfake image itself is often the only information detectors have to work with. 

Why Reverse Engineering? 

Right now, researchers identify deepfakes through two primary methods: detection, which distinguishes between real and deepfake images, and image attribution, which identifies whether the image was generated using one of the AI’s training models. But generative photo techniques have advanced in scale and sophistication over the past few years, and the old strategies are no longer sufficient. 


First, there are only so many images presented in AI training. If the deepfake was generated by an unknown, alternative model, even artificial intelligence won’t be able to spot it—at least, until now. Reverse engineering, common practice in machine learning (ML), can uncover unique patterns left by the generating model, regardless of whether it was included in the AI’s training set. This helps discover coordinated deepfake attacks or other instances in which multiple deepfakes come from the same source. 


How It Works 

Before we could use deep learning to generate images, criminals and other ill-intentioned actors had a limited amount of options. Cameras only had so many tools at their disposal, and most researchers could easily identify certain makes and models. But deep learning has ushered in an age of endless options, and as a result, it’s grown increasingly difficult to identify deepfakes.


To counteract this, Facebook ran deepfakes through a fingerprint estimation network (FEN) to estimate some of their details. Fingerprints are essentially patterns left on an image due to manufacturing imperfections, and they help identify where the image came from. By evaluating the fingerprint magnitude, repetition frequency, and symmetrical frequency, Facebook then applied those constraints to predict the model’s hyperparameters. 


What are hyperparameters? If you imagine a generative model as a car, hyperparameters are similar to the engine components: certain properties that distinguish your fancy automobile from others on the market. ‘Our reverse engineering technique is somewhat like recognising [the engine] components of a car based on how it sounds’, Facebook explained, ‘even if this is a new car we’ve never heard of before’. 


What Did They Find? 

‘On standard benchmarks, we get state-of-the-art results’, said Facebook research lead Tal Hassner. Facebook added that the fingerprint estimation network (FEN) method can be used for not only model parsing, but detection and image attribution. While this research is the first of its kind, making it difficult to assess the results, the future looks promising. 

Facebook’s AI will introduce model parsing for real-world applications, increasing our understanding of deepfake detection. As cybersecurity attacks proliferate, and generative AI falls into the hands of those who would do us harm, this method could help the ‘good guys’ stay one step ahead. As Hassner explained: ‘This is a cat-and-mouse game, and it continues to be a cat-and-mouse game’.

Share article