QuartumSE Strategic Analysis & Next Steps¶
Date: November 3, 2025 Phase: Phase 1 (Foundation & R&D) - Target completion: Nov 2025 Analyst: Claude Code
Executive Summary¶
QuartumSE has successfully completed initial smoke tests on both simulator and real IBM quantum hardware (ibm_fez). The project is in the final stretch of Phase 1 with several critical experiments remaining before the Phase 1 gate review.
Current Status¶
✅ Completed: - Core SDK infrastructure (estimator, shadows v0/v1, reporting, provenance) - Simulator validation: SSR 17.37× on 3-qubit GHZ (exceeds 1.2× target) - Hardware smoke test on ibm_torino (Oct 22, 2025) - Hardware smoke test on ibm_fez (Nov 3, 2025) - 7.82s execution
⚠️ Critical Outstanding for Phase 1: - Extended IBM hardware validation with SSR ≥ 1.1× across multiple runs - Cross-workstream starter experiments (C/O/B/M) - need first data drops - Patent theme shortlist finalization
Phase 1 Exit Criteria Analysis¶
| Criterion | Target | Current Status | Gap |
|---|---|---|---|
| SSR on simulator | ≥ 1.2× | ✅ 17.37× | None - exceeded |
| SSR on IBM hardware | ≥ 1.1× | ⚠️ Needs validation | Multiple runs needed |
| CI coverage | ≥ 80% | ⚠️ Unknown | Need to run pytest |
| End-to-end IBM run | 1+ backend | ✅ Completed | None |
| Patent themes | Top 3 | 📝 In progress | Drafting continues |
| Cross-workstream data | First drops | ⚠️ Not started | C/O/B/M experiments |
Available IBM Quantum Backends (as of Nov 3, 2025)¶
| Backend | Qubits | Pending Jobs | Recommendation |
|---|---|---|---|
| ibm_fez | 156 | 77 | ⭐ BEST - Lowest queue |
| ibm_marrakesh | 156 | 298 | Good backup |
| ibm_torino | 133 | 485 | Previous smoke test |
| ibm_brisbane | 127 | 3175 | Avoid - heavy queue |
Recommendation: Use ibm_fez for all near-term experiments due to minimal queue time.
Critical Experiments for Phase 1 Completion¶
Priority 1: Extended GHZ Hardware Validation (S-T01/S-T02)¶
Status: Smoke test complete, need extended validation Goal: Achieve SSR ≥ 1.1× consistently across ≥10 hardware trials Next Action:
# Run extended GHZ with v0 baseline (200 shadows)
python experiments/shadows/S_T01_ghz_baseline.py \
--backend ibm:ibm_fez \
--variant st01 \
--shadow-size 200 \
--seed 42
# Run noise-aware version with MEM (S-T02)
python experiments/shadows/S_T01_ghz_baseline.py \
--backend ibm:ibm_fez \
--variant st02 \
--shadow-size 200 \
--seed 43
Priority 2: H₂ Chemistry Workstream Starter (C-T01 / S-CHEM)¶
Status: Script ready, not executed Goal: First chemistry data drop with shadow-based Hamiltonian estimation Target Metrics: - Energy accuracy: 0.02–0.05 Ha - Uncertainty reduction: ≥30% vs grouped measurement - SSR ≥ 1.1×
Next Action:
# Modified H₂ experiment with reduced shots for fast validation
python experiments/shadows/h2_energy/run_h2_energy.py
# Note: Script needs update to use ibm_fez and reduced shadow_size (200-500)
Priority 3: Other Cross-Workstream Starters¶
Status: Scripts available, not executed Goal: Generate first data drops for Phase 1 closure
Available Experiments:
1. Parallel Bell Pairs (S-BELL): experiments/shadows/parallel_bell_pairs/run_bell_pairs.py
2. Random Clifford (S-CLIFF): experiments/shadows/random_clifford/run_random_clifford.py
3. Ising Chain (S-ISING): experiments/shadows/ising_trotter/run_ising_chain.py
Recommended Execution Plan (Next 2-4 Days)¶
Day 1 (Today - Nov 3)¶
- ✅ Completed: Smoke tests on simulator + ibm_fez
- Execute: Extended GHZ validation (S-T01) on ibm_fez
- Execute: H₂ chemistry starter (C-T01) on ibm_fez
Day 2 (Nov 4)¶
- Execute: Noise-aware GHZ with MEM (S-T02) on ibm_fez
- Execute: Parallel Bell pairs experiment
- Analyze: Compare S-T01 vs S-T02 metrics
Day 3 (Nov 5)¶
- Execute: Random Clifford benchmarking (S-CLIFF)
- Execute: Ising chain experiment (S-ISING)
- Run: Full pytest suite to measure CI coverage
Day 4 (Nov 6)¶
- Aggregate: All experiment results and manifests
- Compute: Final SSR metrics across all hardware runs
- Prepare: Phase 1 completion report
- Update: CHANGELOG.md and phase1_task_checklist.md
Risk Assessment¶
High Risk¶
- Queue saturation: ibm_fez could become saturated if others discover it
- Mitigation: Monitor queue with
quartumse runtime-status, have ibm_marrakesh as backup - SSR target not met: Hardware noise might prevent SSR ≥ 1.1×
- Mitigation: Use MEM (v1), increase shadow_size if needed, try multiple backends
Medium Risk¶
- Long execution times: Chemistry experiments with 4000 shots may take hours
- Mitigation: Reduce to 200-500 shots for initial validation, scale up later
- MEM calibration overhead: Each experiment needs confusion matrix calibration
- Mitigation: Reuse cached calibrations where possible
Low Risk¶
- Token expiration: IBM Quantum token could expire
- Mitigation: Already configured in .env, easy to refresh
Phase 2 Preparation Checklist¶
While completing Phase 1, prepare for Phase 2 transition: - [ ] Draft provisional patent themes (VACS, Shadow-VQE, Shadow-Benchmarking) - [ ] Prepare arXiv preprint outlines based on Phase 1 data - [ ] Plan IBM hardware campaign #1 (blocked time windows) - [ ] Design Shadows v2 (Fermionic) experiments - [ ] Outline Shadows v3 (Adaptive) prototype
Key Metrics Dashboard¶
Simulator Performance (Nov 3)¶
- GHZ(3): SSR = 17.37×, CI Coverage = 100%
- GHZ(4): SSR = 731,428,571×, CI Coverage = 100% (outlier due to near-zero baseline error)
- GHZ(5): SSR = 0.08×, CI Coverage = 88.89% ❌ (failed target)
Hardware Performance (Nov 3)¶
- Backend: ibm_fez
- Test: 3-qubit GHZ, 100 shots, v0 baseline
- Execution: 7.82s
- Results:
- ZII: -0.03 (expected: 0.0) - good
- ZZI: 0.54 (expected: 1.0) - shows hardware noise
- ZIZ: 0.99 (expected: 1.0) - excellent
Note: Need extended validation with comparison to direct measurement to compute SSR.
Immediate Next Action¶
Execute the H₂ chemistry experiment with optimized parameters:
# Create run_h2_quick.py with reduced shots for fast validation
# Backend: ibm:ibm_fez
# Shadow size: 300 (reduced from 4000)
# Version: v1 (noise-aware + MEM)
# Expected runtime: ~10-15 minutes
This will provide the critical chemistry workstream data drop needed for Phase 1 completion while keeping execution time manageable.
Success Metrics for Today's Session¶
✅ Simulator smoke test passed ✅ Hardware smoke test passed (ibm_fez) ⏳ Extended GHZ validation (in progress) ⏳ H₂ chemistry starter (next) ⏳ Cross-workstream data drops (following)
Target: Complete at least 2-3 hardware experiments today to advance Phase 1 closure.