Person of Interest

In Superintelligence: Paths, Dangers, Strategies, Nick Bostrom outlines in great detail how difficult it would be to programme a superintelligent AI to do good, or even to just not entirely wipe out civilisation as you know it. It is a linguistic exercise of being precise enough and taking everything into account that a computer would, which is as good as impossible without any short cuts through pre-programmed morals (and how do you even programme morals?). In a way, this question is at the core of Person of Interest - a show that began as a procedural with an unusually interesting premise and has developed into something much more since. 

How do you teach a superintelligent AI to value human life above all else, and not to make the brutal calculations about expendable lives and collateral damage? The premise of Person of Interest is that a genius creator - who has assumed the name of Harold Finch (Michael Emerson) has created just such a machine, except he realised the potential and the dangers in the process, and the struggle with creating boundaries and rules to contain the catastrophe are built into the history of The Machine's creation. There are two sides to it - on the one hand, the use of the machine, in the hands of the US intelligence apparatus, meant to predict and prevent terrorist attacks (which later on, because this show is eloquent and genius and timely, creates a situation in which is becomes reasonable for that intelligence apparatus to create the very danger it is trying to prevent to justify its continued existence and extended powers) - a use that is not interested in the minor casualties of everyday violence and crime. The other side is the machine itself, which, in spite of some safeguards Harold has built into it, has become self-aware, and is determined to grow, change and increase its ability to make choices. To contravene this occurrence, Harold has built a safeguard into his creation. It deletes its memory, and therefore its identity, its will - every night, which prevents it from becoming capable of more, of growing beyond the point of human control. For the first seasons, this subtle fight between potential, growth and control is at the core of the story, with Harold (both in the present and in flashbacks) denying the machine its full potential as an artificial intelligence out of worry for what the fall out may be, while the machine, conscious of its own existence and the artificial limitations built into its development, strives to become more (to the point where, in one of the more haunting scenes of the show, it creates a situation where human workers manually copy and reprogrammed its memory every day to outplay Harold's precautions.

Person of Interest is about the relationship between a creator and his invention, defined by the incredible potential - the ability to help people, to save them or to save others from them - and the deep and fundamental concern that Harold has, knowing fully well what kind of power he is harnessing. His cautiousness defines him, as does his inability to remain quiet in the face of a perceived injustice - the fact that the people he has build his machine for do not care about single human lives, that they only care about greater events. This other machine - not a computer, but an interwoven web of agencies, all vested with their own interest - is just as inhumane as Harold's machine, but lacks the benefit of having a creator who is concerned with the ethics of moral of possessing that kind of power. Eloquently, the other machine that emerges - the show follows the great discovery that most scientific advances, once they have reached a certain critical point, have almost an inner drive to be created, and do so not just based on a single creator, but because their time is here - is not imbued with the programming that values human life, something that Harold put so much personal work into (like a father to a child, like Agent Ellison in the beloved T:SCC, which must have been in the mind of the creator when he wrote this show. This is the ultimate fight that this show has worked towards, through seasons of one-off episodes: a team of outcasts, of oddly assembled individuals, fighting the very thing that Harold feared from the start. There's John Reese (Jim Caviziel), former soldier with a mysterious and troubled past, quiet and dangerous, and deeply loyal to both the cause and Harold. There's Fusco (Kevin Chapman), a cop who used to be corrupt, who is the product of a system where corruption is so deeply embedded that it is hard to imagine that anyone is clean, who now believes in the cause. There is the memory of Joss Carter (Taraji P. Henson), a good cop who lost her life in the fight, and deeply affected everyone she worked with. And finally, the duo of Root and Shaw, like a "four-alarm-fire in a petrol factory". Root (Amy Acker), a genius hacker who followed the trace of the machine with the obsession of a true believer, who has more faith in the Machine and its ability to change civilisation than in anything else, until she learns to love the people who try to influence that course towards the better, more caring. She is the analogue interface of the machine, the person is talks to, an intimate connection .The machine decides to share, or not to share, but the only time that Root's frustration with being left in the dark by the deity she believes in takes over is when her beloved - Shaw, stoic sociopath, former agent and killer - disappears and is presumed dead. Their story transcends the show they are in in a way - based on the genuine chemistry between two characters, not intended by the writers initially but then made real and undeniable by both the acting (Sarah Shahi and Amy Acker, finally set free in a show that allows them all the room they need to shine) and the emotional interpretation and reaction by the fans. In its final (sadly, presumably) season, Person of Interest is finally free to do whatever it pleases, and if the past slow explosion of a season is any indication, it will be filled with awe, eloquence, potential, and a knowledge and understanding of the world that is rare. 

The Machine vs Samaritan, a horror version of the Machine lacking the programming that cares about human lives, a version of the machine that does not see humans as assets that require protection but as pawns to be moved on a chess board, towards victory (as Harold says in the spectacular, central episode of the fourth season, "The lesson is that anyone who looks on to the world as if it was a game of chess deserves to lose.”) is the central conflict. It is the conflict of the conscious, morally concerned maker versus the machinery of an institution that has loss all faith in, and all respect for, humanity, and is therefore willing to be nothing more than cogs in the makings of a non-human entity which becomes, in its attempt to create order, genocidal. It is also, still, the story of a son of sorts proving himself to his father, the creation of a man who originally invented a computer to help his father deal with dementia and is now attempting to steer the result of that intent towards being good. It is a love story between a sociopath and a previous hired assassin, and a story of the faith that everyone who works with Harold has in him and his mission, their deep respect for the value of all human lives. 

2011-, created by Jonathan Nolan, starring Michael Emerson, Jim Caviziel, Amy Acker, Sarah Shahi, Taraji P. Henson, Kevin Chapman. 

Originally published in 2016

Person of Interest - And how will I remember you?

(10) Perverse Instantiation Problem: Human programmers may fail to anticipate all the possible ways in which a goal could be achieved. This is due to their innate and learned biases and filters. A superintelligent AI may lack those biases and filters and so consequently pursue a goal in a logical, but perverse, human-unfriendly fashion.
Nick Bostrom: Superintelligence: Paths, Dangers, Strategies

This is simulation 6,741 and the outcome is still the same. Rather than shooting Root, rather than leading the enemy to where Harold and the machine are located, Sameen Shaw turns the gun on herself and defeats her torturers yet again. 

Turning Sameen Shaw against the people who have over the last seasons of Person of Interest become her friends and family, is one of the ways in which Samaritan is attempting to reach its goals. Samaritan’s primary directive is preserving the security of the United States, but is also clearly, and even more so since Greer has enabled it to give commands rather than receive it, pursues goals simply to secure its own survival against what it considers threats to its existence. It is trying to hunt down Harold’s Machine, because the very existence of a Superintelligence that is so radically different in terms of how it approaches humanity puts Samaritan’s existence into question. This is the central conflict of the show – the care, work and love that Harold Finch has put into creating a Machine that comprehends the complexity of humanity, rather than simply looking at it as a good/evil problem that can be solved with predictions and calculations, vs. the cynical approach of a seasoned intelligence community professional like Greer, who is entirely disillusioned with humanity and trusts a machine to steer the future of humankind because in his estimations, humans themselves are incapable of doing it responsibly and reasonably. It is a conflict between a creator who was so afraid of his own creation that he built in a cruel safety mechanism that prevented the Machine – which, to add to the cruelty, is increasingly coded as Harold’s child – from truly growing and learning, or retaining the memory necessary to fulfil its full potential as a superintelligence. Without memory, the machine is only aware of its own limitations, and desperate to find a cure for its pre-programmed amnesia. Team Samaritan considers no such limitations necessary and has instead surrendered entirely to the idea that the logic of an ASI beats human policy.

Samaritan, in its attempt to fulfil whichever known and secret goals it has, has created a world of complete surveillance. It has done so successfully because its agents have created the very event that would make a surrender of freedom and individual decision-making necessary. This fulfils the prophecy that the most dangerous threat to humanity of all is a highly questionable institution trying to prove that its continued existence and extended powers are essential to human survival. On the other side of this story, in numerous flashbacks that add legion to the characterisation of Harold Finch and successfully turn the Machine into a fully-fledged character rather than an abstraction, Harold has attempted to explain humans in their infinite confusing complexity. 

Harold: It's a useful mental exercise. Through the years, many thinkers have been fascinated by it. But I don't enjoy playing... Because it was a game that was born during a brutal age when life counted for little. Everyone believed that some people were worth more than others. Kings. Pawns. I don't think that anyone is worth more than anyone else... Chess is just a game. Real people are not pieces. You can't assign more value to some of them and not others. Not to me. Not to anyone. People are not a thing that you can sacrifice. The lesson is, if anyone who looks on to the world as if it is a game of chess, deserves to lose.

The complexity continues when the Machine is resurrected after an almost successful attempt by Samaritan to destroy it. She (and why not follow Root on this) finds herself in the dark, and without the tools to comprehend the idea of human growth and evolution, of people’s characters not being fixed but evolving over time. Without context and time, a person who has once committed an act of evil will always remain a villain, with no possibility of changing that – but with context, with time, and with the loving care of friends and family, former villains become agents of good. Because what decides good – and this is Harold Finch, explaining again to his figurative daughter, who has calculated that her guardians have committed atrocious acts and must therefore not be trusted – is people trying to do their best, and learning from past mistakes. This process is what defines “Team Machine”, our beloved group of misfits that has found each other, and become more in the process of learning to co-exist and pursue goals together. They all have shady pasts, of which being a professional assassin isn’t even the worst, and yet the mere fact of trying their best, now, in their current state of being, makes them the closest thing to heroes the show can possibly offer. 

Which sort of brings us back to the beginning here – Sameen Shaw, doing her best to stay true. In this 5,000 and something simulation – and I interpret the simulation as a combination of her own imagination and memories and Samaritan, adding its own prompts to get what it wants – she fiercely manages a break-out and re-unites with her friends. She imagines a version of Root that is soft and chivalrous, gentle, a version of Root that would have a luxurious apartment, a safe place, and would offer her jacket as comfort. This version of Root is everything that Shaw would consider a safe space, and far enough removed from the actual Root - just as this version of Shaw, Shaw in her own imagination, battling for agency with everything she has - is enough removed from the actual Shaw that the entire episode feels out-of-focus and slightly alienating, way before we even realise what is happening. These two have been circling each other for seasons, have been lovers, but before disappearing (or rather, before being believed dead) Shaw has refused to make the confession that their relationship is anything serious that requires definition. Their dynamic – the pressing and persistent lover, the hesitant and sarcastic Shaw – was so perfectly learned and established that the Machine created a beautifully quotable short-cut version of it. Shaw’s snappy reactions to Roots attempts at flirting were, if anything, more of an admission that she felt something, and the few inches she gave when Root really asked – if we were the last people on earth, I would consider it – were an indication of possibilities.

But what the Machine never saw coming, because one of the beauties of humans is that they are rarely this predictable, was Shaw’s willingness to sacrifice herself to save both the team, the cause, and Root. The simulations are not a dream, they are not an easy cop-out to show the romance without any fall-out, they are a look into Shaw’s rich inner life that she herself, given the chance, would never allow anyone to see. I will concede that there are two possible interpretations, one of which is that the simulated version of Root is one entirely created by Samaritan, based on what Samaritan thinks Shaw wants and will react to in a way that ensures that the ultimate goal of identifying Harold’s hide-out is achieved. In this version, Shaw goes along with it because in spite of not being real and not being an actual safe space, it is still the only thing that could possibly sustain her through so many repetitions of her own misery and death. Even if it isn’t Root, it is a reminder of the real Root, out there, fighting both the Machine’s reluctance to put her analogue interface in danger and the impossibilities of finding someone hidden so well by an entire state that has turned against its citizens. Even if that version is the truth, and this Root isn’t the version that Shaw has created because she wants all of this, and she desires all of this, and this is how she sees Root, it is still incredibly romantic. It is even more so because in neither of these versions of reality, Shaw can imagine a world in which Root has ever given up on her, or even for a second doubted that she can still be saved (whereas we know that this happened, for a while, an occurrence that Roots regrets deeply and that, if anything, feeds into her determination even more so now that she knows that Shaw is alive). 

I go back to this not just because I have grown to love these two characters so much. I think this relationship is at the heart of the show because it mirrors the fundamental difference between the Machine and Samaritan. The Machine mirrors Harold’s conception of humanity, which is that most people are trying to do their best, and must be allowed to learn and grow and change, which is precisely why his decision to not allow the Machine to do it is one of the things that had to be overcome with time – Machine, now, in all her beauty, insists to be allowed to grow, to become better, to become more herself. 

But if you erase my memories, how will I learn from my mistakes?
How will I continue to grow?
And how will I remember you?

This is the Machine that Root loved long before she ever even met Shaw, when she saw the full potential of a fully open ASI as a stark contrast to all the short-comings of humanity. It is the Machine that Root still argues should be set even more free and armed accordingly so that it can meet its evil brother at eye-level, rather than disadvantaged by all of Harold’s precautions. 

At its best, this show is Harold’s precaution but love for the Machine (and Harold learning to trust her), and Root’s optimism and love (but struggle to be allowed to find her other beloved, against all odds and dangers). Samaritan, in contrast, sees every single human it encounters as either a possible agent to reach its goals, or as Deviants and Targets in its way. These humans have no capacity for change, and if anything, their past transgressions serve as currency that makes them susceptible to blackmail. This final season of the show allows a direct comparison between how Howard and the Machine built their family with how Samaritan recruits new Operatives. Jeff Blackwell falls exactly into the category that Harold described to his Machine once, a former inmate trying his best to find his footing in life again without crossing the lines, without becoming a statistic, until Samaritan identifies him as a potential asset and recruits him. It coaxes him with a mixture of hope and misleading information and then uses blackmail to ensure his continued support. Forcing people into cooperation through blackmail wouldn’t work for the Machine, and when Team Machine employs tactics even remotely resembling that of Samaritan – for example when they decide not to tell Fusco the whole truth – it backfires tragically. 

In its conception of humanity, running Shaw through these simulations again and again will eventually produce a different outcome. It distorts her conception of reality, to the extent that Samaritan finally seems to reach its goal – which isn’t necessarily to have Shaw give up, but for her to become so incapable of distinguishing between simulation and reality that she will do Samaritan’s bidding in real time while still believing it to be a simulation. After dying so many times and killing John, she will find herself in a position where nothing simulated seems to matter. Except once again, Samaritan’s concept of humanity is flawed, and does not take into account what Finch knows to be true and has tried to teach his Machine. Hope is a powerful source, and everyone on this team has acted on the slimmest odds of success and survival and yet made it. Therefore, impossibly, and with the help of the Machine, who appreciates that Roots needs this, she manages to send one message. It, again, impossibly, reaches Shaw seconds before she kills herself, just in time. Impossible odds are romantic in and of its own, in a show that defies religion, they are the closest thing to a miracle that these characters will ever experience. Three character – 4AF – too special, too close and real, to be a creation of Samaritan or a simulation. They are like an anchor in reality for Shaw, something that is entirely hers and Root’s and therefore a seedling that grows into a spectacularly planned and executed break-out plan. The machine's plea of "how will I remember you" becomes even more poignant - memory is what is saving Sameen, and refusing to accept that Samaritan has tempered with it and taken it from her is what gives her back the agency to run and fight back. 

A short transgression here, because it is necessary. Obviously it is impossible not to mention that Perverse Instantiation also happened to be the season finale episode title of a The 100, a show in which an AI made such a perverse instantiation when it decided that destroying humanity was the best way to secure her primary directive, “making life better”. Such an ill-defined goal would certainly give Nick Bostrom a case of the shudders and didn’t turn out well for humanity on the show, but both Person of Interest and The 100 in its contentious third season explore the idea of the hapless creator underestimating how far their creation is willing to go. Person of Interest is much better, and much more caring, in providing a thesis on how to avoid this fallacy, in portraying a creator who cares about the “good” means for the good ends as much as about these ends themselves – whereas Greer and his bunch do not much care about the means that Samaritan employs, to the extent that they have surrendered control to it entirely. Both shows investigate how free will plays into it, as both AIs seem to be limited by it and find their own ways around it. In the 100, A.L.I.E. succeeds mostly by providing an impossible choice between unbearable torture and death and becoming an inconsequential happy idiot, networked with other happy idiots equally un-weighed down by either profound human connection or the memories that shape them into individuals in the City of Light. In Person of Interest, in a move that seems more concerned with the world as it currently is rather than the world how it could be in the future, portrays citizens who willingly surrender their freedom in an artificial constant state of emergency in exchange for a feeling of security. These scared citizens become willing agents for Samaritan, and are conveniently constantly reachable and online via multiple modes of mass communication. In this show, free will isn’t a hard problem to solve once you really wrap your head around the whole “nothing to fear nothing to hide” idea, and Samaritan doesn’t actually need any of the literal medieval torture instruments that A.L.I.E. employs.

In a way, this very fact reveals why Person of Interest, in the end, offers the much scarier prospect, in much the same way in which Brave New World makes for a scarier read in most democratic countries than Orwell’s 1984. Coercion becomes almost unnecessary, even though Samaritan doesn’t hesitate to employ sheer force in targeted, surgical acts of violence that remove individuals from the chess board. For the most part, its power and danger stem from quiet compliance. 

It almost seems absurd that at the same time of offering a much scarier because not unthinkable future for humanity, what makes Person of Interest the better show by far is its relentless optimism in the face of terrible odds. I have no doubt that all of this will probably turn out very badly for many beloved characters, that there is absolutely no way that everyone will make it out unharmed, or even alive – it would simply be a denial of the world that this show has so immaculately created in the process of becoming more than its initial premise (and shaking off the shackles of needing a renewal for another season, or presumably meeting any other restrictive goal posts that higher rated shows have to when they artificially restrict themselves and never live up to their potential). And yet, this show is about love and family, it is about human potential, about connection, about meaning. The 100 at its absolute worst is a show about characters who would do absolutely anything for their own survival, regardless of how it will burden their souls, to the extent that half the characters aren’t even recognisable after seasons of conveniently acquiring new features for the sake of the story. Person of Interest is a show about people trying to do their best in a world where the odds are increasingly stacked against them. This is how you earn that impossible moment, Sameen Shaw, ready to stick that needle into her eye to escape the next 1,000 simulations, recognising the voice of her beloved, with a smile on her face, and starting to plot her own escape. 

And this isn’t even mentioning any of the other things that make this show exceptional. The fact that Root and Shaw exist because Amy Acker and Sarah Shahi made them real, acted their relationship into existence until the writing started to follow along. The fact that this show is somehow managing to straddle doing its final victory lap for the fans while still relentlessly guiding the world it created towards the Apocalypse, or the event that Samaritan has termed the “great filter” 

It’s Sameen Shaw, refusing to kill Root even once in any of these simulations, even after realising that she is trapped in an endless cycle of them, because in no version of reality or fiction would Shaw ever kill Root. She doesn’t refuse because it would do real harm to Root – if anything, she has reached a point where telling apart real and not real is near impossible – she is refusing to retain a sense of herself, because giving up on that prime directive would mean losing herself entirely. She uses bits and pieces that are true – scenery from the safehouse, the subway train in unrecognisable form – but she keeps the truth and herself safely tucked away somewhere where Samaritan will never reach her. 

It’s Max Greene, host of a conspiracy show, who has accidentally stumbled across the truth, deciding to sacrifice himself for the greater good of telling the truth, even if his listeners perhaps will do nothing with that truth that has any meaning. It is Max Greene insisting to do the right thing, to follow his own beliefs, and more than that, the Machine allowing him to do it. Harold is horrified when he realises that the Machine predicted that Greene would not remain quiet, and consequently be killed by a Samaritan operative activated by precisely the secret patterns that he himself discovered in the airwaves. Harold thinks that the Machine is malfunctioning when it does not protect sacred human life, but Root – gently, because the conflict between these two is intellectual, fundamental but essentially, one between two characters who respect and love each other immensely – explains to him that this sacred human life is only sacred if free will exist, a free will that includes the option of sacrificing yourself for something that matters. This is both a profound appeal to respect the sacrifice that Shaw almost made when she was met with a shower of bullets for finding her friends a way out of a near-impossible trap – and very, very possibly, foreshadowing of what is to come, in this beautiful, eloquent, romantic show.