Credibility as a Forecastable Quantity: Scoring the Swordholder and the Announced Succession Candidates from Declassified Text

1. Introduction

In the third month of Deterrence Era, Year 59, the Office of Public Affairs of the UN Planetary Defense Council (PDC) issued Statement 214 on the readiness of the gravitational-wave transmitter network (United Nations Planetary Defense Council, Office of Public Affairs, 59 DE). The statement ran to nine paragraphs. Eight of them set out maintenance schedules and the order of inspections, and the ninth said that the Council would weigh any change of posture in the light of the situation as it then stood. The Swordholder appeared once, in the last line, as the officer to whom the schedules were reported.

What should a text like that count for? The Dark Forest Deterrence holds for as long as Trisolaris believes that the Swordholder would broadcast if the switch were released, and the transmitter network is only the instrument of that belief (Marrable, 55 DE). Belief is formed by the enemy from what it can observe, and much of what it can observe is what we write. We therefore treat credibility as a quantity that declassified text can estimate in part, and we score the incumbent and the announced candidates for the succession on it. Both obvious routes cost something. Modelling Trisolaran belief directly risks giving the enemy a decision rule that no document supports, since no Trisolaran assessment of the Dark Forest Deterrence is available to us and none could be verified if it were. Asking instead how the public regards each candidate is easy to measure, but on the other hand it measures our belief and not theirs. We place the method between the two, as a score of what a panel of analysts expects a hostile reader to infer from a text.

The difficulty that organises the paper is one of design. The Council will choose the next Swordholder by a public process (United Nations Planetary Defense Council, Secretariat, 60 DE), in which candidates are judged by the people they would protect, whereas credibility is judged by the party they would threaten.

This study is written in Deterrence Era, Year 61 (DE 61), ahead of the Swordholder handover, and every source predates it. Five candidates have been announced (United Nations Planetary Defense Council, Succession Office, 61 DE). Cheng Xin, who leads the Secretariat's civic preference survey, is one of them, and we score her as we score the others. The incumbent is the Swordholder, Luo Ji. Section 2 describes the corpus, the panel and the model, Section 3 reports the fit and the scores, and Sections 4 and 5 say what they support.

2. Methods

The corpus holds 388 declassified documents from DE 12 to DE 60, the span that the Council has released (United Nations Planetary Defense Council, Secretariat, 12–60 DE). It comprises 164 PDC strategic bulletins, 131 Council statements and 93 records of the Swordholder, of which 52 are readiness certifications and 41 are public statements. We assume, as a modelling choice that we cannot test, that Trisolaris can read every document in it through the sophons. The partial suspension of sophon interference is a Trisolaran declaration that the Council received and logged, and it is a declaration about interference, not about observation. Council drafting was already sophon-aware before the suspension (Corvane, 49 DE), and we assume that the habit persisted through the years of the corpus.

Nine raters from the Office of Strategic Forecasting, none of them an author of this paper, rated every document on a scale from 0 to 1 for how strongly it would lead a hostile reader to expect a broadcast if the switch were released. They followed the protocol of Quillfeather (55 DE) and saw each text with its date, signature and letterhead removed. Agreement among the nine was high (ICC(2,9) = 0.84, 95% CI 0.80 to 0.87). The mean rating of a document is the quantity that the model predicts.

We coded seven predictors for each document. Six are features of language, counted from a fixed phrase dictionary (Okafor-Linde, 58 DE): unconditional trigger language (a broadcast stated to follow release of the switch), a stated readiness state, named personal attribution (the Swordholder named as responsible), conditional or deferring language (e.g., "would consider", "in the light of"), cost-of-broadcast language (the consequences of a broadcast for Earth), and process or delegation language. The seventh marks documents issued at committee tier, because our earlier scoring showed that consolidated Council texts differ from the working-level texts that feed them (Sarkissian-Low, 58 DE). Each feature is standardised on the training years, and genre (bulletin, statement, record of the Swordholder) enters as two indicator variables whose coefficients Table 1 does not show.

A ridge regression of the mean rating on the predictors was fitted, with the penalty set by blocked six-fold cross-validation over years within the training set. The training years are DE 12 to DE 48 (291 documents) and the test years are DE 49 to DE 60 (97 documents), and nothing was tuned on the test set. Standard errors come from 2,000 bootstrap resamples of documents, with normal intervals. The credibility score of a text is its predicted rating. It orders texts and is not a probability, and it carries no threshold.

For the application we take the incumbent's 41 public statements (28 of them within the training years) and the 82 documents in the candidacy files of the five announced candidates (Table 2 gives the counts). The incumbent's statements are records of the Swordholder in the corpus, so we score them, like every candidate document, with the genre set to statement. That setting lowers their mean score from 0.81 to 0.79. The candidates are Tomas Brennick, a staff officer of Fleet International, Nadia Lindqvist-Amoah of the Council Secretariat, Ilario Pellerin, a Council delegate, Dalia Strandvik, a physician, and Cheng Xin, an aerospace engineer. As a measure of public standing we use the shares of the Secretariat's civic preference survey of DE 61 (PDC Secretariat, Civic Preference Survey Unit, 61 DE), which interviewed 4,200 respondents (margin at most 1.5 points under simple random sampling). Because each candidate has between 11 and 21 documents, intervals for means use the t distribution with n - 1 degrees of freedom and differences use Welch's test. The ten pairwise differences among the candidates are reported with Bonferroni-adjusted intervals (0.0025 per tail, Welch degrees of freedom), and the five gaps to the incumbent are adjusted for five comparisons. The association between preference and score is a Spearman correlation across the five candidates with an exact permutation p-value.

3. Results

On the 97 held-out documents the predicted rating correlated with the panel mean at r = 0.78 (95% CI 0.69 to 0.85), and the mean absolute error was 0.071, against 0.139 for the training-set mean, a skill of 0.49. Statement 214 was a held-out document: the model predicted 0.38 for it and the panel gave 0.42, the eighth-lowest prediction of the 97.

Table 1 gives the coefficients, in rating units per standard deviation of each feature. Unconditional trigger language carried the largest positive weight (0.080, 95% CI 0.061 to 0.099), followed by a stated readiness state (0.044) and named personal attribution (0.032). Conditional or deferring language lowered the rating by 0.059 (95% CI -0.075 to -0.043), and cost-of-broadcast language by 0.034 (95% CI -0.054 to -0.014). Process language and committee tier did not differ detectably from zero.

The incumbent's 41 public statements, scored with the genre set to statement as for the candidates, scored 0.79 (95% CI 0.76 to 0.82, SD 0.11). Twenty-eight of them fall in the training years and 13 in the test years. The panel's mean rating was 0.82 over all 41 (0.84 for the 28 training statements and 0.78 for the 13 held-out ones), and the model's mean prediction for the 13 held-out statements was 0.78. The 0.79 is largely a fitted value, and the 13 are the fair check. The nine earliest of them, from DE 12 to DE 16, scored 0.74 (95% CI 0.66 to 0.82).

Every announced candidate scored below the incumbent (Table 2). The five mean scores were 0.66 for Brennick, 0.62 for Pellerin, 0.58 for Strandvik, 0.53 for Cheng Xin and 0.48 for Lindqvist-Amoah, and the gaps to the incumbent ran from -0.13 to -0.31, each with an interval that excluded zero (widest, adjusted for five comparisons: -0.37 to -0.06). Pooled over the 82 documents the candidates averaged 0.57, and the ordering is soft in the middle. Of the ten pairwise differences, only one clearly kept an interval clear of zero after adjustment (Table 4). Brennick scored 0.18 above Lindqvist-Amoah (adjusted 95% CI 0.04 to 0.32) and 0.13 above Cheng Xin, a difference whose adjusted interval (0.00 to 0.26, adjusted p = 0.05) reaches zero. The other eight pairs, including Pellerin over Lindqvist-Amoah (0.15, adjusted 95% CI -0.02 to 0.31), cannot be told from zero (Table 4).

The public ordering ran broadly the other way. In the preference survey Cheng Xin held 31% and Strandvik 24%, with Lindqvist-Amoah at 20%, Pellerin at 14% and Brennick at 11%. The rank correlation between preference and score was -0.7, and with five candidates the exact two-sided p-value is 0.23.

Table 3 attributes each gap to the incumbent to the features, as the coefficient times the difference in standardised feature means. Trigger language and conditional language together account for 68% to 85% of every gap, and conditional language alone for 25% to 36%. The three most favoured candidates have the largest trigger-language deficits (-0.104, -0.112 and -0.120, against -0.072 and -0.088 for the other two), and also the largest conditional and cost-of-broadcast deficits. Cheng Xin's gap has the largest cost-of-broadcast component (-0.051, 20% of her gap, against 10% to 14% for the others).

4. Discussion

The finding is that, on the texts the Council has released, the incumbent scores above every announced candidate, and the margins (0.13 to 0.31) are large enough that a panel rating of the candidate files would likely preserve the order. The incumbent's statements averaged 0.79, the five candidates averaged 0.57 over 82 documents, and no candidate's interval reached the incumbent's. The model predicts held-out panel ratings well (r = 0.78), so for texts of the kinds on which it was validated the scores carry information about how analysts expect a hostile reader to respond. The candidate files are a genre that the model has not seen (Section 5).

We write within the Luo-Ji / Ye framework, in which a civilisation that cannot verify the intent of another acts on suspicion, so that a deterrent exists for the watcher only as an inference from what it is shown.

An office effect is the most plausible alternative explanation. The incumbent holds the switch and so has standing to use trigger language that a candidate lacks, and the gap would then measure "rank" and not disposition. We cannot separate the two with these data. Every incumbent statement was written by the holder of the switch, and the earliest of them (0.74) stood above every candidate mean, so the gap does not depend on long tenure alone. Conditional language, where we think office confers less licence, accounts for 25% to 36% of each gap by itself (Table 3).

Preference and score were negatively associated, and that bears on the design problem of Section 1. Candidates who held more public favour scored lower (rho = -0.7), and the three most favoured candidates were the three whose mean scores lay below 0.60. To say from five points that the process selects against the score would claim too much, since the exact p-value of 0.23 is what five points allow. Discarding the pattern would be the opposite error, because Table 3 shows which features separate the favoured candidates. They used less trigger language and more conditional and cost-of-broadcast language. A voter may hear the last two as restraint and a hostile reader as the reverse, but the data show only that the two audiences were not aligned on these five candidates, and not why.

The survey leader's score of 0.53 sits in the soft middle of the ordering (Table 2). Her gap to the incumbent has the largest cost-of-broadcast component of the five, and that is a property of her documents, not a measure of her intent.

If the Council wished to give the second audience a voice in the process, the score could be published for each candidate before the vote, with its interval and its ordering of features. We do not claim that it should be.

5. Limitations

Our score is the rating that a panel of human analysts gives a text, and we have no access to the quantity that deterrence depends on. Trisolaran belief is unobserved and, on current arrangements, unobservable, so the validation (r = 0.78 on held-out years) shows that the model reproduces the panel and does not show that it reproduces the enemy. The assumption that Trisolaris reads the corpus cannot be tested either.

The panel did not rate the candidate documents, so the model alone scored them. Candidate files were drafted by staffs for a campaign, and the Welch intervals treat their documents as independent draws when drafting teams will have made them correlated, so the intervals are probably too narrow. Most of the incumbent's statements lie within the training years, and the candidate files are a genre that the model did not see, so the comparison of the two sets of means sets a largely fitted value beside an extrapolation. With 11 to 21 documents per candidate and five candidates, rank statistics are fragile, and the exact p-value says as much. The phrase dictionary was fixed in advance by one coder. The corpus is limited to what has been declassified (DE 12 to DE 60), and we cannot say how the withheld documents differ.

A published score invites writing to the score. A candidate who learns that unconditional trigger language earns 0.080 per standard deviation can supply it, and if the enemy's belief is formed from text then the text alone would move it. The nearest extensions are a panel rating of the candidate files themselves, a term for drafting by staffs, and a model that separates display from disposition in the same text. The hardest unsolved problem is whether words can carry a willingness that the writer may lack, and nothing in our data decides it.

Return to Statement 214. It told a hostile reader a good deal about the Council's procedures and little about the Swordholder, a "housekeeping" text by the score's measure, and its predicted score of 0.38 puts it among the lowest-scoring documents of the held-out years.