Choose an existing workflow with consequences
A new CSV-to-chart demo is easy to make look successful. An existing workbook tests more: queries, formulas, hidden sheets, prior periods, links, add-ins and review conventions. Bring a sanitized copy of the model that your team actually uses, with enough history to expose a period-roll error.
Our public case provides 37 monthly close packs and an October task. A new warranty account requires a classification decision; a payroll accrual already in the ledger must not be posted twice. The output is both an Excel close and a six-slide PowerPoint review. You can use the case as a repeatable test without exposing customer records.
State the definition of done before selecting a provider. “Create an XLSX” and “refresh the existing Excel model and preserve the charts and VBA” are different requirements. A Linux file library may meet the first; the second needs a runtime that supports the actual application behavior.
Inspect the application boundary
Confirm the Windows version, installed Office version, desktop session, and execution identity. Test a simple native workbook operation before attempting the full task. A successful PowerShell command does not prove that it ran in the same interactive desktop where Office is signed in.
Microsoft describes failure modes for unattended and non-interactive Office automation. A Windows VM does not remove those application constraints. Ask how the environment exposes visible desktop execution, handles dialogs and allows a person to take over. Evaluate the behavior with your specific app and add-ins.
For Panes, connect through MCP or an SDK, choose the Office-installed template documented in the machine setup, and use the signed-in desktop for Office calls. Office installation and Office entitlement are separate checks. Do not clone a user’s signed-in environment across unrelated customers.
Test persistence with a witness
Write a unique checkpoint file, save the document, record the expected application state, then stop and resume. Read the checkpoint before making further edits. This separates retained state from a script that quietly regenerates an output after restart.
Time at least three boundaries separately: the start request, the usable desktop, and the first verified Office result. A machine can be running before Excel is ready. Count failed and unattempted cases alongside completed timings, and distinguish repeated tests on one retained computer from independent machines.
Include recovery in the test. What happens after the agent loses its connection halfway through an update? Can it inspect the current stage and continue without appending the same ledger twice? Your workflow should leave a stage receipt and a saved working copy, so a retry has evidence to use.
Score what the buyer receives
Use a small scorecard tied to files rather than claims. Native opening is one check; numerical reconciliation is another. A management deck should preserve editable data when required. A screenshot should accompany the file, not stand in for it.
Cost comparisons need the same boundary. Record setup time, time while waiting, actual running intervals, model charges and any storage or backup charges separately. Do not multiply a short recording’s duration by an hourly rate and call it the cost of the entire job.
| Boundary | Evidence to request |
|---|---|
| Workbook integrity | Original closed pack preserved; formulas, tables and names retained. |
| Financial result | Independent source reconstruction; unresolved exceptions listed. |
| Native behavior | Excel refresh/recalculation and PowerPoint opening in the target desktop. |
| Deck handoff | Editable charts, embedded data, correct rendered slides. |
| Recovery | Checkpoint survives stop/resume; retries do not duplicate work. |
| Cost | Billed running intervals, compute size and separate model/setup costs. |
Compare current capabilities fairly
As checked November 15, 2026, E2B’s documentation describes Linux VMs and a Linux desktop product. Daytona’s Windows documentation describes Windows VMs, desktop control and retained memory state, with access requiring Tier 3 or higher and a support request. These are documentation facts, not results from our own side-by-side benchmark.
Panes supplies Windows computers with file transfer, desktop command execution, screenshots, input and browser viewing. The right comparison is whether each option passes your task and operational requirements. Do not award a reliability or speed win to a provider that you have not run under the same test.
Start with the public workflow, inspect the files and validators, then adapt the task to your own acceptance criteria. If the job needs only file generation, keep that path simple. Use native Windows where the actual application is part of the work.