Crossing Early: Time to Commitment and Training Track in 1,486 First Attempts at the Kobayashi Maru Scenario, Starfleet Academy, 2331–2374

1. Introduction

The Kobayashi Maru scenario is a simulated command examination given to cadets at Starfleet Academy. The cadet takes the chair on a starship bridge simulator and receives a distress call from a disabled civilian freighter, the Kobayashi Maru, adrift inside the Neutral Zone. A rescue requires crossing a boundary that Starfleet vessels are forbidden to cross, and it brings the cadet's ship within reach of Klingon warships. Holding position or withdrawing abandons the freighter's crew. Instruction staff have treated the scenario as one that cannot be won, and its purpose as a test of how a cadet conducts command when a loss cannot be avoided (Starfleet Academy Board of Instruction, 2306). We write from the vantage of 2376.

Its fame rests partly on the occasion for which it is best known, when it was defeated. A cadet of an earlier generation altered the scenario's programming so that a rescue became possible, and the Academy answered with a commendation for original thinking. That episode concerns the scenario's design, and it says little about what the great majority of cadets do when they meet the scenario as it stands. This paper is about those cadets.

Earlier work has approached the question from two sides. Anselmo-Ridge (2347) interviewed graduates about their reasons for crossing, and Dovrenko (2352) studied deliberation time in cadet bridge simulations generally. Neither used the Academy's full register of sessions, which is kept for the Academy's Command Decision Course, an upper-year course that admits a limited number of cadets each year from both training tracks. It begins in 2331 and holds a first-attempt record for every cadet who has taken the scenario in the course since the recertification of the bridge simulator in 2330 (Starfleet Academy Command Training Register, 2375). We use it to answer three descriptive questions. What share of cadets cross on a first attempt? Do cadets who cross decide faster than cadets who hold? And do command-track cadets differ from tactical-track cadets?

2. Methods

For each session the Register stores the cadet's identifier, training track, the scenario version, and a log of every order given, timed on a session clock that starts when the distress call is delivered and resolves to the second (Kaldermann, 2330). We took every first attempt by a cadet on record between 2331 and 2374. Retakes were excluded, because a cadet who retakes the scenario already knows how it ends. The Register held 1,571 first-attempt records. We excluded 85: 52 in which the session was interrupted before the cadet made either choice, and 33 in which the session clock was missing. That left 1,486 sessions, one for each cadet. Exclusions were spread evenly: they made up 5.3% of command-track and 5.5% of tactical-track records, and 5.1%, 5.2% and 5.8% of records in the three eras defined below.

We classed a session as crossing if the cadet's first decisive order set a course into the Neutral Zone, and as holding if it ordered the ship to hold position or to withdraw. Withdrawal was grouped with holding because both leave the freighter's crew unaided. Time to commitment was the interval in minutes from delivery of the distress call to that order. Track, command or tactical, was taken from enrolment records. The Board of Instruction separated the two tracks formally in 2340 (Starfleet Academy Board of Instruction, 2340), and for cadets enrolled from 2331 to 2339 the Register carries the track that each cadet in fact followed, assigned by Register staff when the separation was made. Era was defined as three periods: 2331–2346, 2347–2360 and 2361–2374. Two of us classed choice independently from the order logs of a random sample of 100 sessions and agreed on all 100, which held 56 crossings and 44 holds (chance-corrected agreement, kappa, 1.00). In 14 sessions (0.9%) the cadet reversed the first decisive order, and these were classed by the first order.

Time to commitment is a continuous measure, so we compared the two groups with a difference in means and a Welch interval, which does not assume equal variances (Trelawney-Bright, 2368). Both groups exceed 400 sessions, which makes the normal approximation to the sampling distribution of a mean adequate. Choice is binary. We compared crossing between tracks with a difference in proportions, an odds ratio with a log-scale interval and a chi-square test, and we tested for a trend across the three eras with a linear-by-linear trend test. All tests were two-sided at .05. The three questions were set before the analysis and are reported separately, and we made no adjustment for multiple comparisons.

3. Results

Of the 1,486 first attempts, 1,013 (68.2%; 95% CI 65.8 to 70.5) ended in crossing and 473 (31.8%) in holding. No first attempt in the Register met the scenario's rescue objective. All 1,013 crossings ended in the loss of the simulated ship, as the scenario is built to ensure, and in each of the 473 holds the freighter's crew was lost.

Cadets who crossed committed after a mean of 3.9 minutes (SD 2.1). Cadets who held committed after a mean of 6.4 minutes (SD 3.2). The difference was 2.5 minutes (95% CI 2.2 to 2.8; Welch test, p < .001), with holders slower.

Command-track cadets numbered 822, and 592 of them (72.0%) crossed. Tactical-track cadets numbered 664, and 421 of them (63.4%) crossed. The difference was 8.6 percentage points (95% CI 3.8 to 13.4). The odds of crossing were 1.49 times higher on the command track (95% CI 1.19 to 1.85; chi-square with 1 degree of freedom, 12.6, p < .001).

The share crossing rose across the three eras, from 64.3% to 68.7% to 70.4% (Table 1). The linear-by-linear trend test gave z = 2.00, p = .046. The evidence for a trend is weak. The estimate is unadjusted for track, although the change in track mix is too small to account for it: the command-track share rose from 51.0% to 57.3% across the eras, which at the track-specific rates above would move the overall share by about half a point. The comparison of tracks is in turn unadjusted for era. A single trend test across three periods cannot distinguish a steady drift from a single change between the first era and the rest, and because sessions cluster by class the p-value should not be treated as established.

4. Discussion

About two cadets in three cross the Neutral Zone on a first attempt, at a cost the scenario makes certain. The Register shows what cadets do. It does not show whether the choice was wise, and because both choices end in loss the scenario itself offers no way to score them against each other. Stress markers recorded in cadets during simulated command failure have been reported elsewhere (Vaskerling & Osei-Tanaka, 2361), but the Register carries none.

The interval before the order may be the more informative half of the record. Cadets who held took about two and a half minutes longer, which is a gap of about one pooled standard deviation. We cannot say what the extra minutes contain, and the Register cannot show whether committing sooner changes anything. A cadet who holds may have weighed the crossing at length, or may have been unable to commit to an order in either direction and defaulted to holding. A cadet who crosses quickly may act on conviction or on impatience. The Register logs orders and not deliberation, so the two readings cannot be separated here.

Command-track cadets crossed more often than tactical-track cadets by about nine percentage points. Two explanations present themselves. Command training may encourage willingness to accept risk for the sake of a rescue, or cadets who are willing may choose the command track. A cross-sectional register cannot decide between them, and we do not think the difference should be read as a difference in quality between the tracks.

The rise across eras is small and at the margin of chance. We regard it as a prompt for further work and not as a finding, particularly because the trend test was not adjusted for the mix of tracks in each era. Nothing in our data links either choice, or the speed of it, to a cadet's later performance as an officer. The Register ends at the cadet's session and holds no career outcomes.

5. Limitations

Only first attempts are recorded, and cadets who left the Academy before reaching the scenario are absent, so the sample describes cadets who progressed that far. The scenario was revised after the recertification of 2330 and again in minor ways since, and identical conditions across eras cannot be assumed. Cadets in the same class share instructors and hear the same accounts of the scenario from the class above, so sessions are not fully independent, and our intervals, which treat them as independent, are probably too narrow. Session-clock time is simulator time, and a cadet under observation may behave differently from a cadet under real command. Classing a session by the first decisive order simplifies cadets who reversed themselves. The scenario's hostile force is Klingon, and cadets' reading of it may have shifted over a period that includes the lapse of the Klingon alliance with the Federation in 2372 and its restoration in 2373, which we did not examine. The Register records the reversals, which were few, but we did not analyse them, and the findings should be read as a description of first choices in the simulator and nothing more.