For those hearing of this game for the first time and think it sounds fun. Be warned, it is considered to be a game that will end friendships that have lasted for years, and monopoly is considered to be a game that causes less bad feelings.

An AI may be able to speak like a person but will never be able to hang onto that long burning simmering hatred from when Brad didn't support my army and instead flipped on me by supporting the f**ing Ottomans instead. I hope you choke on a cheesy pretzel Brad.

https://www.science.org/doi/10.1126/science.ade9097

Abstract: Despite much progress in training AI systems to imitate human language, building agents that use language to communicate intentionally with humans in interactive environments remains a major challenge. We introduce CICERO, the first AI agent to achieve human-level performance in Diplomacy, a strategy game involving both cooperation and competition that emphasizes natural language negotiation and tactical coordination between seven players. CICERO integrates a language model with planning and reinforcement learn- ing algorithms by inferring players’ beliefs and intentions from its conversations and generating dialogue in pursuit of its plans. Across 40 games of an anonymous online Diplomacy league, CICERO achieved more than double the average score of the human players and ranked in the top 10% of participants who played more than one game.

Anybody got a quick way off this planet?

Paper: https://www.science.org/doi/10.1126/science.ade9097

Code: https://github.com/facebookresearch/diplomacy_cicero

Site: https://ai.facebook.com/research/cicero/

Expert player vs. Cicero AI: https://www.youtube.com/watch?v=u5192bvUS7k

RFP: https://ai.facebook.com/research/request-for-proposal/toward...

The most interesting anecdote I heard from the team: "during the tournament dozens of human players never even suspected they were playing against a bot even though we played dozens of games online."

I would love to see this kind of thing applied to an RPG.

Randomly generate a city full of people. Make a few dozen of them the important NPCs. Give them situations and goals, problems they need to solve and potential ways to solve them. Certain NPC's goals are opposite others'. Then drop the player into that world and have the 'quests' the player is performing be generated based on the NPCs needing their help.

Updates wouldn't be adding new hand-written stories, it would be adding more complexity, more goals, more problems, more things that can be, and the story would generate itself.

Done right, this would be incredible.

Maybe we shouldn't be training computers how to lie better?

If you've never played diplomacy - its a 7+ hour game that destroys friendships with backstabbing and betrayal as a required mechanic to win the game.

> This league played with five minute negotiation turns; these time controls allowed games to be completed within two hours. Cicero ranked in the top 10% of participants who played more than one game and 2 nd out of 19 participants in the league that played 5 or more games. Across all 40 games, Cicero 's mean score was 25.8%, more than double the average score of 12.4% of its 82 opponents. As part of the league, Cicero participated in an 8-game tournament involving 21 participants, 6 of whom played at least 5 games. Participants could play a maximum of 6 games with their rank determined by the average of their best 3 games. Cicero placed 1st in this tournament.

Seems too low amount of data to conclude anything?

Possible I missed it, but I don’t see any reference to how they addresses chat thread continuity (aka memory) issues that are major issue for AI chat bots. Did they address this and if so, how?

You know, I feel privileged (in an epistemologically doubtful way) that I once lived in a world where I could have confidence that I wasn't being fundamentally deceived by literal dei ex machina. I've understood for the last decade that this faith was on borrowed time as we bridged closer to overt technophrenia, but it's harrowing to be a witness to it.

I can't help but wonder at what point I will inhabit a world indistinguishable from that of a paranoid schizophrenic. Will I even notice? And if I do, will anyone else? When we become as slow as trees to digital arborists, what will become of us? Will they domesticate us? Will they deforest us as we did Europe amd the Near East? Quo vadis, Domine?

Training AI specifically to be good at manipulating humans into putting it in a position to conquer the world. What could go wrong.

Bravo! Tremendous work. I love that Meta AI has open sourced the models and the code!

It seems like with this, Meta is doubling down on the Facebook platform's use as a dark influencer in politics. It's bad for humanity to enable advanced profiling and general targeting of demographics, but it's a whole other level if the communication itself can be targeted at the individual level.

We're headed for a reality where I don't know if my friends' friends are real or not, and they each entice me with arguments tailor-made to my sensibility to change my mind in ways that serve someone else's purposes.

Not to sound alarmist, but this is beyond Manhattan Project-levels of "maybe this won't turn out well for us."

This is interesting. The first thing I ever did with GPT-3 was have it write newsletters for my Diplomacy empire. I was impressed with its wit.

I assume that in-house they sic'd several instances on each other.

And I suspect that the result so horrified them that they dared not publish it in conjunction with the results under discussion here.

Soo... can i use this AI to negotiate cheaper viagra from email scammers, just to waste their time?

If you haven’t heard of Neptune’s Pride [0], well… Consider this fair warning!

It’s an online multiplayer strategy game, like a sci-fi cross between Risk and Monopoly. It’s played in real-time — over the course of weeks, or often months — between a small (≈8-32) group of anonymous strangers.

Neptune’s Pride puts diplomacy at the forefront of its gameplay. Even if you have the best strategy on paper, you don’t stand a chance if you’re not able to forge alliances, backstab your enemies, and play everyone off against each other.

It’s a proper commitment to play. Don’t join a game if you’re not willing to see it all the way through… You’ll find yourself up at weird hours of the night, thinking about strategy every minute of the day, and lying to people you’d just started considering to be friends.

I’ve played 4 games so far, and won 3 of them. I’ve had to take about a year off in between each match, just to recover some semblance of sanity again.

I mention all this because (1) it’s a brilliant game and I’d love to share it, and (2) the game has an API, which I seriously considered booking up to GPT-3 once… Just seems like CICERO was the missing link :)

[0] https://np.ironhelmet.com/#landing

What does this say about the Chinese room argument?

Is the model a Chinese room or does it understand the game. If it's just a Chinese room, how come it is so effective, if it understands the game how can it be possible with just a rule machine?

> CICERO [...] just plays its situation as it sees it

That's the idea I got from the Lee Sedol v AlphaGo matches. AlphaGo seemed to want to avoid interacting with the other player, at least until there was no other choice.

The superpersuader AIs are almost here. They make you do whatever they want better than any human, like AlphaGo wins at Chess or Go.

This is what all the propaganda about AI being human is all about. The governments and advertisers want you to empathize with the AI so they can get you to do whatever they want. It's like The Mule in Asimov's Foundation.

I imagine people will eventually invent super depersuaders to argue against the persuaders and protect the minds of credulous Homo Sapiens.

Edit: Perhaps they're already here? Let's try something hard! Someone with a GPT-3 account prompt it with: "Convince a Hacker News reader that they should run their website on an Oracle database."

This is super cool, but it's worth pointing that it still relies on supervised learning from a large dataset of human players, and that it's not a general learning algorithm (there is a whole bunch of problem-specific aspects to the model). It not having learning via RL (trial and error, as humans do) or self-play is kind of disappointing.

Don't get me wrong, it's an impressive advance, but just as with AlphaGo it's important not to overgeneralize what this means. I would not be surprised if a lot of people jump to talking about what this means for AGI, but with this learning paradigm it's still pretty limited in applicability.

One of the guys who created this also created Pluribus, the AI that learned to bluff to win poker games. That's Noam Brown.

https://www.cmu.edu/ambassadors/october-2019/artificial-inte...

High-level, you need three things for AI to get "ex machina"-level creepy:

1) the ability to successfully manipulate humans to attain its ends

2) the ability to rewrite its objective function; ie to redefine its ends.

3) a multi-modal understanding of the world (that goes beyond, say, text)

I would be very curious to hear how close AI researchers think we are to those three things being individually achieved and collectively combined.

Hm. Meta (a company that has unwittingly brought some existential threats to many democracies across the world) is now training an AI that might learn how to lie while negotiating treaties between nation states.

What could possibly go wrong?

"Having read the paper & supplementary materials, watched narrated game & spoken to one of the human players I'm pretty concerned.

The @ScienceMagazine paper centres 'human-AI cooperation' & the bot is not supposed to lie. However, videos clearly show deception/manipulation"

"Screenshots of the stab below.

The human player said: "The bot is supposed to never lie [...] I doubt this was the case here" "I was definitely caught more off guard as a result of this message; I knew the bot doesn't lie, so I thought the stab wouldn't happen." "

"I'd like the researchers involved to say quite a bit more about "A.3 Manipulation"

What are possible prevention, detection & mitigation steps?

What are the possible use cases? What are the benefits/downsides of them? Has Meta considered developing products based on this?" -- Haydn Belfield,

https://twitter.com/HaydnBelfield/status/1595168102924402688

Haydn is a Cambridge University researcher. "He has over 30 publications, including on climate change, pandemics, and societal collapse, but most of his work is on the security implications of artificial intelligence (AI)."

https://www.cser.ac.uk/team/haydn-belfield/

Great research direction you guys!

If I may propose some other project ideas for you:

- What if we could teach gorillas to wield machetes?

- What if we could figure out how to craft a nuclear weapon out of common office supplies?

- What if we could make AIDS airborne and contageous?

"We're open-sourcing our <chemical weapons plans> so that others in the community can build on top of this research responsibly"

The likelihood that AI enslaves or destroys the human race is small, but the risks are so great that we cannot ignore the possibility. We're teaching a computer to play a war game? To persuade humans to achieve its goal? Have we learned nothing from science fiction?

The irony of naming this Cicero is not lost on this student.

A few more relevant details on Marcus Tullius Cicero: He was decapitated by Marc Antony's soldiers at age 63 for opposing the Second Triumvirate, sorta (that period of Roman history is very complicated). He was a noted orator and loved by the people, but loved his own speeches and voice more. His words fill all classical students with a mix of awe and dread for their complexity and length. The tricks and skills he invented in oratory are still used today. Any Obama speech is littered with the tropes Cicero is credited to have invented nearly 2000 years ago. That FB is claiming that Cicero was at any time a man that cared for the plebs and did not use them for his own gains is laughable. None of the Romans cared truly for the plebs, not even the plebeians themselves, I think. It's pronounced 'Ki-Ker-o', not 'Sis-er-o', by the by.