Human-Level Play in Diplomacy Combining Language Models with Strategic Reasoning
An AI may be able to speak like a person but will never be able to hang onto that long burning simmering hatred from when Brad didn't support my army and instead flipped on me by supporting the f**ing Ottomans instead. I hope you choke on a cheesy pretzel Brad.
Abstract: Despite much progress in training AI systems to imitate human language, building agents that use language to communicate intentionally with humans in interactive environments remains a major challenge. We introduce CICERO, the first AI agent to achieve human-level performance in Diplomacy, a strategy game involving both cooperation and competition that emphasizes natural language negotiation and tactical coordination between seven players. CICERO integrates a language model with planning and reinforcement learn- ing algorithms by inferring players’ beliefs and intentions from its conversations and generating dialogue in pursuit of its plans. Across 40 games of an anonymous online Diplomacy league, CICERO achieved more than double the average score of the human players and ranked in the top 10% of participants who played more than one game.
Expert player vs. Cicero AI: https://www.youtube.com/watch?v=u5192bvUS7k
The most interesting anecdote I heard from the team: "during the tournament dozens of human players never even suspected they were playing against a bot even though we played dozens of games online."
Randomly generate a city full of people. Make a few dozen of them the important NPCs. Give them situations and goals, problems they need to solve and potential ways to solve them. Certain NPC's goals are opposite others'. Then drop the player into that world and have the 'quests' the player is performing be generated based on the NPCs needing their help.
Updates wouldn't be adding new hand-written stories, it would be adding more complexity, more goals, more problems, more things that can be, and the story would generate itself.
Done right, this would be incredible.
If you've never played diplomacy - its a 7+ hour game that destroys friendships with backstabbing and betrayal as a required mechanic to win the game.
Seems too low amount of data to conclude anything?
I can't help but wonder at what point I will inhabit a world indistinguishable from that of a paranoid schizophrenic. Will I even notice? And if I do, will anyone else? When we become as slow as trees to digital arborists, what will become of us? Will they domesticate us? Will they deforest us as we did Europe amd the Near East? Quo vadis, Domine?
We're headed for a reality where I don't know if my friends' friends are real or not, and they each entice me with arguments tailor-made to my sensibility to change my mind in ways that serve someone else's purposes.
Not to sound alarmist, but this is beyond Manhattan Project-levels of "maybe this won't turn out well for us."
And I suspect that the result so horrified them that they dared not publish it in conjunction with the results under discussion here.
It’s an online multiplayer strategy game, like a sci-fi cross between Risk and Monopoly. It’s played in real-time — over the course of weeks, or often months — between a small (≈8-32) group of anonymous strangers.
Neptune’s Pride puts diplomacy at the forefront of its gameplay. Even if you have the best strategy on paper, you don’t stand a chance if you’re not able to forge alliances, backstab your enemies, and play everyone off against each other.
It’s a proper commitment to play. Don’t join a game if you’re not willing to see it all the way through… You’ll find yourself up at weird hours of the night, thinking about strategy every minute of the day, and lying to people you’d just started considering to be friends.
I’ve played 4 games so far, and won 3 of them. I’ve had to take about a year off in between each match, just to recover some semblance of sanity again.
I mention all this because (1) it’s a brilliant game and I’d love to share it, and (2) the game has an API, which I seriously considered booking up to GPT-3 once… Just seems like CICERO was the missing link :)
Is the model a Chinese room or does it understand the game. If it's just a Chinese room, how come it is so effective, if it understands the game how can it be possible with just a rule machine?
That's the idea I got from the Lee Sedol v AlphaGo matches. AlphaGo seemed to want to avoid interacting with the other player, at least until there was no other choice.
This is what all the propaganda about AI being human is all about. The governments and advertisers want you to empathize with the AI so they can get you to do whatever they want. It's like The Mule in Asimov's Foundation.
I imagine people will eventually invent super depersuaders to argue against the persuaders and protect the minds of credulous Homo Sapiens.
Edit: Perhaps they're already here? Let's try something hard! Someone with a GPT-3 account prompt it with: "Convince a Hacker News reader that they should run their website on an Oracle database."
Don't get me wrong, it's an impressive advance, but just as with AlphaGo it's important not to overgeneralize what this means. I would not be surprised if a lot of people jump to talking about what this means for AGI, but with this learning paradigm it's still pretty limited in applicability.
High-level, you need three things for AI to get "ex machina"-level creepy:
1) the ability to successfully manipulate humans to attain its ends
2) the ability to rewrite its objective function; ie to redefine its ends.
3) a multi-modal understanding of the world (that goes beyond, say, text)
I would be very curious to hear how close AI researchers think we are to those three things being individually achieved and collectively combined.
What could possibly go wrong?
The @ScienceMagazine paper centres 'human-AI cooperation' & the bot is not supposed to lie. However, videos clearly show deception/manipulation"
"Screenshots of the stab below.
The human player said: "The bot is supposed to never lie [...] I doubt this was the case here" "I was definitely caught more off guard as a result of this message; I knew the bot doesn't lie, so I thought the stab wouldn't happen." "
"I'd like the researchers involved to say quite a bit more about "A.3 Manipulation"
What are possible prevention, detection & mitigation steps?
What are the possible use cases? What are the benefits/downsides of them? Has Meta considered developing products based on this?" -- Haydn Belfield,
Haydn is a Cambridge University researcher. "He has over 30 publications, including on climate change, pandemics, and societal collapse, but most of his work is on the security implications of artificial intelligence (AI)."
If I may propose some other project ideas for you:
- What if we could teach gorillas to wield machetes?
- What if we could figure out how to craft a nuclear weapon out of common office supplies?
- What if we could make AIDS airborne and contageous?
The likelihood that AI enslaves or destroys the human race is small, but the risks are so great that we cannot ignore the possibility. We're teaching a computer to play a war game? To persuade humans to achieve its goal? Have we learned nothing from science fiction?
A few more relevant details on Marcus Tullius Cicero: He was decapitated by Marc Antony's soldiers at age 63 for opposing the Second Triumvirate, sorta (that period of Roman history is very complicated). He was a noted orator and loved by the people, but loved his own speeches and voice more. His words fill all classical students with a mix of awe and dread for their complexity and length. The tricks and skills he invented in oratory are still used today. Any Obama speech is littered with the tropes Cicero is credited to have invented nearly 2000 years ago. That FB is claiming that Cicero was at any time a man that cared for the plebs and did not use them for his own gains is laughable. None of the Romans cared truly for the plebs, not even the plebeians themselves, I think. It's pronounced 'Ki-Ker-o', not 'Sis-er-o', by the by.