Engineering journal
Why 15 of 18 Drone Missions Passed but the Controller Still Failed Acceptance
AeroLoop's scheduled gains improve mission completion, while separate outage and recovery gates still fail. Retained flight data makes that distinction inspectable.
Source: https://louijiecompo.com/writing/why-fifteen-of-eighteen-drone-missions-still-failed-acceptance/
About 3 min read
A better count is useful evidence. It is not a reason to change the question after an experiment. AeroLoop's horizontal approach study records 15 passing missions out of 18 with scheduled gains, compared with 12 out of 18 using fixed gains. The study still reports accepted: false.
The retained validation record (opens in a new tab) explains why. Mission completion, comparison against intact feedback, and sustained recovery answer different questions. Passing one cannot silently stand in for the others.
Keep three questions separate
The study uses nine final seeds, each with intact feedback and two-second feedback outages, for each controller mode. It combines 24 new final flights with 12 retained baseline flights. Development seed 83 is excluded from the final cohorts.
| Original measure | Fixed gains | Scheduled gains |
|---|---|---|
| Missions passing their gates | 12/18 | 15/18 |
| Same-mode outage pairs passing | 7/9 | 8/9 |
| Sustained recovery windows passing | 14/18 | 15/18 |
A mission result asks whether that flight meets its original bounds. A same-mode pair compares an outage flight against its intact reference. Recovery asks whether the disturbance settles within the defined window. These are separate denominators, not three ways to report the same success rate.
The scheduled controller still has support-position failures. In the no-outage case, seed 503 improves from about 0.428 m to 0.338 m, crossing the original 0.350 m limit. Seed 401 improves from about 0.661 m to 0.457 m and still fails. Both observations matter: improvement and acceptance are different claims.
Compare the same physical phase
The later landing-response lab (opens in a new tab) groups all four recordings for each seed: fixed and scheduled gains, with intact and outage feedback. It adds synchronized inspection, common-phase windows, control-effort analysis and exportable results over the same 36 flights. It adds no measurement or changed acceptance criterion.
A clock window can contain airborne motion in one recording and ground contact in another. Common pre-contact and armed windows help expose that difference; unavailable windows remain unavailable. A reduction in integrated error over a mixed-phase window does not by itself identify a better airborne controller.
The lab also distinguishes squared acceleration demand from physical energy. Its horizontal wind/thrust work accounting uses recorded forces and displacement. The unexplained residual is not measured friction, battery energy or a complete mechanical-energy balance. Outage contrasts remain descriptive rather than causal.
Repair validation without relabeling the experiment
PR #28 (opens in a new tab) repaired test imports that failed in a clean checkout. It changed test setup, not the frozen flights. The merged response-lab revision subsequently passed its integrated CPU check. That resolves the publication hold without pretending the measurements were rerun.
The candidate remains opt-in. Small fixed cohorts, synthetic feedback, simplified dynamics and open yaw refinement limit interpretation. The engineering lesson is to keep failed gates visible while investigating why an experiment improved.
The AeroLoop case study exposes the retained numeric comparison. Its older interactive flight keeps its own provenance; it is not playback of this new response-lab bundle. Continue with the earlier contact diagnosis to see how the next experiment was framed.