People training new AI models admit they just get chatbots to do it

People who are paid to train new AI models by supplying them with high-quality conversation and tests are cheating and using chatbots like ChatGPT to do the job instead, multiple whistleblowers have told New Scientist. The seemingly widespread practice risks undermining the future of AI, as it could lead to the “collapse” of more advanced models.

Most AI models operating today were trained on text and data scraped from the internet. But as models have scaled up, requiring yet more training data, AI firms have begun using workers who carry out conversations and tests with AI, in the hope that the resulting high-quality data can improve the power and usefulness of future large language models (LLMs).

These workers are normally employed by third parties, rather than AI companies directly, and are often working without full-time contracts and for low pay. That can incentivise them to take shortcuts like using chatbots to complete tasks faster, according to a worker called Alice*, despite this being against company policies.

“It’s very widespread; every company I’ve worked for has had explicit guidelines around it and they clearly do try to catch people out, so I think they do care. But I don’t think they can stop it,” says Alice.

Alice says she feels “not in the slightest” guilty about using ChatGPT to complete training tasks, saying it is easy to get away with as long as you instruct chatbots to avoid the usual telltale signs of AI output, like a preponderance of em-dashes. “It’s only the sloppiest of users that get caught,” she says. “Anyone with a modicum of awareness around AI hallmarks can tell their output not to use them, and at that point what are you going to do?”

Free newsletter

Sign up to The Daily

The latest on what’s new in science and why it matters each day.

Sign up to newsletter

“If these companies want quality data, then they should offer quality contracts,” says Alice. “Instead they’re low-balling struggling people, employing them for the barest possible amount of time and tossing them aside as projects are finished with no warning.”

Another worker, Bob*, worked for a training platform called Outlier. Initially, he was tasked with AI training, which he says he illicitly used AI for, and was then promoted to a leadership role where part of his job was to catch others doing the same thing.

“Management vacillated between light tolerance to outright banning,” says Bob. Workers at Outlier would be tracked with a tool called Hubstaff which takes screenshots of their desktop at random intervals to ensure they are really doing tasks as ordered. Bob would look for evidence of AI models in those screenshots.

“People would have it (AI models like ChatGPT) open in other tabs, or minimised, so obviously we could see it in the task bar,” says Bob. “Even stuff like folders on their desktop with names gave it (AI use) away.”

Outlier, which is owned by Scale AI, did not respond to a request for comment. Scale AI claims on its website to carry out work for technology giants like Meta and Cisco, neither of which responded to New Scientist‘s request for comment. Bob says he had personally worked on projects for Google, which also did not respond to a request for comment.

Another worker, Carol*, who has worked on several platforms, says that her use of AI began by checking her work for anything that went against the lengthy guidelines for a task, because any contravention could mean expulsion from the project and a loss of earnings.

“I was terrified of not having an income source, and then after that, it just became easier to run everything through LLMs,” says Carol. “For a lot of the projects that I do now, it’s creating scenarios, so I will use one LLM to help me create the scenario and then I’ll use a different LLM to help me create the files that go along with the scenario. I do feel guilty but like I said, in the beginning it was more about trying to make sure I wasn’t making any errors.”

“I do worry that I’m actually making it (AI) worse. I thought using the models to train themselves negates some of the value,” says Carol.

Mark Lee at the University of Birmingham, UK, says research has shown that AI models “collapse” if they are recursively trained on AI-generated content. When this happens, the abilities of the model drop dramatically and they become less useful. The process is sometimes known as AI cannibalism or AI inbreeding.

“That’s the kind of worst-case scenario. And that’s probably not what’s happening in the real world,” says Lee. “There’s still a few humans. And if you have like 10 per cent human data, it mitigates it, it avoids model collapse.”

But Lee says that the kind of cheating these workers are doing isn’t without repercussions, and will hit performance. “Rather than it being catastrophic, you’ll see that the AI isn’t as good at doing human-like tasks. It’s an issue, because I think the models aren’t as good as they could be.”

*Names have been changed to protect identities