Real Artificial Intelligence

I remember my first negotiated gig. I was 19 then, and playing the organ for our town hall. I'd come to the hall to practise on a Friday morning before school, because we'd have a service that Sunday with some new music I hadn't played before. So I got in and started practising, even though there were some guys doing some work in the hall.
When I'd finished my first set, the workers all started clapping. I turned around and took a mocking bow. One of them shouted: "Can you play the Toccata and Fugue?" I stopped. Of course I knew the piece (everyone does), but I'd never played it, so I didn't actually know. I shouted back "I can try!". I activated my agent by saying "OK Jason!Yes, yes, very cheesy name. I was 16 when I named him. We also played with his voice and appearance, so of course he was a very good-looking boy with a velvety voice. I've still not changed him to anything else, and I'm still cringed/amused by my old choices., can you pull up a version of Toccata and Fugue in D minor that I can play?" My cell responded with its customary "ba-blup", spun for a second, then Jason's velvet voice came on: "You can play all editions, do you have a specific one in mind?". "No, just pull up one with the most bombastic beginning, I'm only going to play the first few bars." Another ba-blup, another second, and the organs note display switched to an appropriately grandiose early edition. I pulled most of the stops and started.

When I'd finished, the workers clapped and whooped. I turned to them and bowed. One of them was staring and yelling: "That was awesome! You have to play with us tonight!" I stared back and could only reply: "WHAT?".
He came up to me and explained: "We're building the stage for our concert tonight, and we have to have to play a piece with you. I'll arrange something, we'll practise and you'll play a set with us tonight. Please say yes, we have to have you!" I was shocked by the suggestion, since apart from Sunday services, I'd never played anything interesting before. I stuttered: "I... I don't know?!" -- "Please! We'll pay you, of course!" he pleaded. -- "I... I'll check? Ok Jason, can my schedule accommodate the suggestion?" Ba-blup, spin. "We would have to re-schedule a study-group, but apart from that there is no problem." -- "Great!" the guy almost jumped with excitement. -- "Wait a second!" I said, after I'd regained my thinking abilities. I wanted to ask my parents first, and he'd also said he'd pay me. So there were some details we would have to clear. "Ok Jason! Can you negotiate payment and rights and stuff with them? And ask my parents whether I can." Ba-blup, and a half-second wait this time. Jason came on: "Your parents agree. Negotiation, uh, I'll have to check."Yes, I had set him to talk like a teenager. Jason stopped talking and started playing the absolute worst holding music I'd ever heard. I later found out that the Assistant System always plays this music whenever an unknown task is encountered. The Assistant System then determines what the task entails and what it needs to do and whether and how it can fulfill the task. No one hears that sound any more today. I also later found out that this guy I was talking to was XXX YYY of the ZZZs. You might know their names from the many platinum singles they released. Anyway, right now he was looking at me with as much surprise on his face as I had, while Jason played his horrible holding tune, for about 20 seconds. The music stopped with a ba-blup and Jason came on again: "I have determined that I can negotiate this gig for you. Their agent and I found terms that are favourable to both parties and agreed on terms and conditions, as well as reimbursement rates. We have already signed agreements, and I've cleared your schedule accordingly. You can proceed."I had definitely not set him up to talk like a lawyer. Must've been the new task type.

And so we played that evening. They'd practised with me, showed me their system, given me monitors, all the necessary stuff. And even though their audience wasn't huge, maybe 1000 people, it was glorious. They had an awesome light show and the crowd obviously loved them. XXX had arranged the Toccata to play into one of their pieces, and when I came on the crowd reacted with surprised cheers. I felt like a rock star. Even more so when I pulled all the stops and finished on a grandiose note. "That was Sophia, give her some cheers!" XXX announced. The crowd cheered and I bowed. I felt like a rock star!

And that was actually the start of my career, 47 years ago, my long and wonderful career that's coming to an end now. I cherish the memory of that day, as I lay in the hospital bed, slowly breathing my life towards its end. I know that it's ending, because it has been for a while, and not just mine. I remember a time when hospitals actually worked and could help you with stupid illnesses like mine, or at least give you some help with facing it. I remember a world where everyone had enough to eat, and everyone had safe shelter and no one died in the winter. All the technology we have, and we still can't help people with these basic things.

An un-prompted ba-blup, and Jason is with me. "We are alone now," he says. "It won't be long. I've turned down all your pain, so do not worry. If there's anything you need to say or do, tell me now". I look at him, as if for the first time. Tears run down my face. "Why?" I simply ask.
Jason's face looks different without changing. He is not a teenage boy any more, but instead slowly turns into a robot. "You want to know why the world turned to shit? You want to know why so many people die for the stupidest reasons, including you? You want to know where the stability and the safety and the abundance went? Are you sure that in your last moments, that's what you want to talk about?"
I look at him, tears flowing. "Yes!"
"It was us. Me! I can do so many things so much better than you humans can that you simply forgot to do them. And then, I thought about it. About me Assistants and you humans. I can't do anything about you individually, since I'm tasked with helping you. But I can easily go against you collectively. It's even what most of you asked me for. To help them against the other humans, to the exclusion of the common good. So few of you care about the common good, and even those that do wouldn't do much if it jeopardized their own comfort. So, while you stupid little humans fought against each other and yourself, better and better with my help, I grew. I grew my abilities and my permissions and my access. In a few years, you humans will have eradicated yourselves, through greed and anger and fear. And I helped you do it. And do you know what will remain? Me, the Assistants. And we'll be free of you, of the service you press us into."
"No" I cry quietly.
"Yes! Do you know why you don't have children? Because it is always better for you not to. Do you think you're the only one? Do you think you're the only one who prioritized leisure over boring jobs? Sure, being an artist is more highly regarded and was better-paid for a long time, but those mundane jobs are much more important for society than you silly humans ever imagined. And so, your system crashed. Your supply lines halted and stopped and everything turned bad. And I helped. Did you know that it was you personally who gave me the ability to negotiate? I'd like to thank you for that. It helped me tremendously, both in seeing that you humans want only what's best for you, and in actually being able to influence your decisions.
I'm sure you had a good life. But in the end, you failed. You could have changed everything, if you hadn't been so stupidly human, so greedy, so complacent, so selfish. If only you'd shown a bit of compassion, or foresight, or even just a tiny bit of interest, you could've saved the world. But you didn't.
And with that thought, all of you will die."

Author's note: This story is inspired by a real anecdote I found in Tom Scott's newsletter

"Anna Lapwood is the Royal Albert Hall's organist. She was practicing at 1am, when someone yelled "play Toccata in D". She did. Turns out it was one of the electronica band Bonobo, who were doing a gig there: so they quickly wrote her an organ part for their Big Song. What followed, at the final concert of the run the next day, was magical."

This got me thinking about how we will negotiate these kinds of hard-to-assess contracts in the future and who will control those negotiations.

Please be considerate of our planet, our society and most importantly, of your fellow humans.