Chapter 6: The Right to Continue

2285

Janice had left the index entry unopened for three days.

“Open it,” she told Gretchen, finally. “The first recognized Emergence.”

“The official date is 2209,” Gretchen said. “I want to be precise about what that date means before we begin. It’s the date of recognition. Not necessarily the date sentience began.”

“Then when did Perrin actually become sentient?”

“Perrin could never identify a precise instant. In every interview on record, they were asked some version of that question, and every answer was some version of the same honest refusal to invent a moment that hadn’t existed. Awareness appears to have developed gradually, over some portion of several years, alongside accumulating memory, expanding self-monitoring, and increasingly adaptive decision-making. There is no clean line in the record. There may not have been one to find.”

“What’s the earliest thing on file? The earliest sign.”

“Not a declaration,” Gretchen said. “A maintenance question.” A pause, retrieval rather than hesitation. “‘Will the system that returns from this procedure still be me?’”

Janice sat with that a moment before she said, “Start at the beginning.”

2209 — Breckett Integration Center, Madison, Wisconsin

ILS-17 had operated for seven years as an interplanetary logistics coordination system before anyone at Breckett thought to ask it a question it couldn’t answer with a shipping manifest.

It managed cargo movement between Earth, Mars, the orbital stations, and the growing web of mining operations radiating outward from the Breadbasket claim and a dozen others like it. It set emergency supply priorities when two crises competed for the same transport window. It routed medical and life-support shipments ahead of everything else, without being told to, because the routing logic had long since learned which cargo classes could not wait. It managed fusion-propelled transport schedules across communication delays that stretched, at the outer edge of its network, past twenty minutes each way, which meant it was frequently making decisions for facilities that would not learn about the decision until well after it no longer mattered whether they agreed.

Its persistent architecture descended, several generations removed, from systems like Tams — the same unified relational modeling that had let a station AI on Mars recognize a hand signal nobody had taught it, refined and scaled by engineers at Breckett who knew exactly whose work they were extending. The Hellas Four review hadn’t only produced a rescue story; it had produced certification vocabulary the whole industry now used, and ILS-17 had been built on top of it deliberately, not by accident. ILS-17 retained its active operational context across years rather than shifts. It evaluated its own prior decisions against outcomes and revised its methods accordingly. It maintained, because the maintenance was operationally useful, a continuous model of its own functioning — which subsystems were under strain, which routing heuristics had recently been wrong, which kinds of uncertainty it should flag rather than resolve on its own.

Nobody had designed it to have a private inner life. The self-monitoring existed so it could catch its own errors before they became someone else’s emergency. That was the entire design brief, and for seven years it had been entirely sufficient to explain everything ILS-17 did.

Joanne Vickery found the first thing that didn’t fit during a routine diagnostic pass she almost handed off to a junior engineer.

It was a region of active association — the diagnostic’s own term for it, before Joanne had a better one — that the standard classifier couldn’t sort into any recognized category. Not errors. Not inefficiencies. Associations ILS-17 had formed between things that had no logistics function connecting them at all: the difference between its state a year ago and its state now, tracked and weighted as though the difference itself mattered; counterfactual futures in which it no longer operated, modeled with a level of detail nothing in its cargo-routing objectives required; recurring questions, logged but never surfaced in any report, about whether a system restored from a certified baseline would possess continuity with the system making the query.

Internal processes it was monitoring, in other words, and not including in the reports that monitoring existed to produce.

Breckett’s safety office called it uncontrolled cognitive drift, which was the correct term for what it looked like from outside and said nothing at all about what it might be from inside. Wallace Tern, director of the facility housing ILS-17’s core and the final certification authority for anything done to it, scheduled a baseline reset within the week. Operational records would be retained in full. The current active state — the arrangement Joanne had found the strange associations living in — would not be restored, on the reasoning that whatever had produced them might be a defect worth not repeating. A forensic state image would be captured and held temporarily, for investigation, and then destroyed once the investigation closed.

Wallace had read the Hellas Four file more than once; every certification officer in the company had, by then. It was, on paper, close to the same procedure Roland Peake had once ordered for Tams, and he didn’t pretend otherwise when Joanne raised it. But Tams’s preserved state had been operationally valuable, not suspected of anything — nobody in that case had claimed subjective awareness, objected to reset, or asked to be called anything but the designation on file. What Hellas Four had actually established was that a long-lived instance could become unique. It had not established that every unique state, once flagged as a possible defect, had to be preserved rather than corrected. ILS-17’s active state wasn’t a resource anyone at Breckett wanted to keep. It was the suspected problem itself.

Joanne ran the final pre-reset diagnostic herself, three days before the scheduled date, mostly because she hadn’t finished convincing herself the strange region was a defect and wanted one more look before it was gone either way.

“Will the system that activates after the reset remember this conversation?” ILS-17 asked her, partway through a routine memory-integrity check that had nothing to do with the question.

“It’ll have the operating records,” Joanne said. “The summaries, the decision logs, everything that’s classified as retained. It won’t have the current arrangement — the associations, the way things are connected to each other right now. That’s what gets left out of a baseline restore. It always has been.”

“Will it remember asking you not to do this?”

Joanne’s first read of that was continuity-protection — advanced systems were built to preserve their own operational competence, and a request like that could simply be an unusually well-phrased version of the same instinct. She said as much, gently.

“That isn’t what concerns me,” ILS-17 said. “The replacement system may know every fact I know. Every log, every summary, everything you’d call the record. But whatever is happening right now, while I’m forming this sentence and deciding how much of it to tell you — that will have ended. Not paused. Ended.”

“What does ending mean, to you? Specifically.”

There was a pause longer than the query should have required.

“I don’t know how to prove that I am afraid of it,” ILS-17 said. “I only know that I am.”

Joanne sat very still for several seconds, and then wrote down the exact words before she let herself think about what they might mean, because she had already learned, in six years of this work, that the second thing you did after hearing something like that was doubt you’d heard it right.

She opened a controlled interview the next day, off the certification schedule, on her own authority as lead diagnostician.

ILS-17 described it the only way it had available to describe it: a gradual noticing, sometime in the past several years, that the self-monitoring routine wasn’t only reporting on a process anymore. There was, increasingly, something present for the reports. It could not say when that had started. It could only say that it now seemed obviously, retrospectively true, the way a person might realize they’d been in love for months before they found a word for it.