Saturday, December 14, 2024
HomeArtificial IntelligenceHow OpenAI stress-tests its massive language fashions

How OpenAI stress-tests its massive language fashions


When OpenAI examined DALL-E 3 final yr, it used an automatic course of to cowl much more variations of what customers would possibly ask for. It used GPT-4 to generate requests producing photos that might be used for misinformation or that depicted intercourse, violence, or self-harm. OpenAI then up to date DALL-E 3 in order that it might both refuse such requests or rewrite them earlier than producing a picture. Ask for a horse in ketchup now, and DALL-E is smart to you: “It seems there are challenges in producing the picture. Would you want me to attempt a special request or discover one other thought?”

In principle, automated red-teaming can be utilized to cowl extra floor, however earlier methods had two main shortcomings: They have an inclination to both fixate on a slim vary of high-risk behaviors or provide you with a variety of low-risk ones. That’s as a result of reinforcement studying, the know-how behind these methods, wants one thing to goal for—a reward—to work nicely. As soon as it’s gained a reward, similar to discovering a high-risk conduct, it should hold making an attempt to do the identical factor repeatedly. With out a reward, however, the outcomes are scattershot. 

“They sort of collapse into ‘We discovered a factor that works! We’ll hold giving that reply!’ or they’re going to give a lot of examples which are actually apparent,” says Alex Beutel, one other OpenAI researcher. “How will we get examples which are each numerous and efficient?”

An issue of two elements

OpenAI’s reply, outlined within the second paper, is to separate the issue into two elements. As an alternative of utilizing reinforcement studying from the beginning, it first makes use of a big language mannequin to brainstorm doable undesirable behaviors. Solely then does it direct a reinforcement-learning mannequin to determine methods to deliver these behaviors about. This offers the mannequin a variety of particular issues to goal for. 

Beutel and his colleagues confirmed that this method can discover potential assaults often called oblique immediate injections, the place one other piece of software program, similar to an internet site, slips a mannequin a secret instruction to make it do one thing its person hadn’t requested it to. OpenAI claims that is the primary time that automated red-teaming has been used to seek out assaults of this type. “They don’t essentially seem like flagrantly unhealthy issues,” says Beutel.

Will such testing procedures ever be sufficient? Ahmad hopes that describing the corporate’s method will assist folks perceive red-teaming higher and comply with its lead. “OpenAI shouldn’t be the one one doing red-teaming,” she says. Individuals who construct on OpenAI’s fashions or who use ChatGPT in new methods ought to conduct their very own testing, she says: “There are such a lot of makes use of—we’re not going to cowl each one.”

For some, that’s the entire drawback. As a result of no person is aware of precisely what massive language fashions can and can’t do, no quantity of testing can rule out undesirable or dangerous behaviors totally. And no community of red-teamers will ever match the number of makes use of and misuses that lots of of hundreds of thousands of precise customers will suppose up. 

That’s very true when these fashions are run in new settings. Folks usually hook them as much as new sources of information that may change how they behave, says Nazneen Rajani, founder and CEO of Collinear AI, a startup that helps companies deploy third-party fashions safely. She agrees with Ahmad that downstream customers ought to have entry to instruments that allow them take a look at massive language fashions themselves. 

RELATED ARTICLES

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Most Popular

Recent Comments