Molina
A tree of branchesOne history rises from the bottom to the save, where it splits into many branches. One branch, drawn in coral, ends in a broken promise.the history so farthe savea broken promise

Don't know why your program fails? Molina shows you exactly how it happened.

A customer's saved data is gone after a power cut. One engineer says that can't happen, the customer says it just did, and nobody can make it happen again.

Molina settles the argument with an experiment. On my own machine I run the container image your engineers ship, and make its disk say "saved" before it is, cut its power, or cut its servers apart, at points I choose.

Nothing is installed in your environment and nothing is added to your tests.

If your software breaks a promise, like "no saved write is ever lost", you get a replay code that names the failure, a recording your engineers can check, and the shortest list of failures that still breaks it. You get it as a folder of plain files you can read and check, and one web page that opens in a browser.

A disk said data was safely stored when it was only in memory. After a power cut, 63 of the 85 saved writes the application had been told about were gone.

An illustration, not to scale.

A disk that reported truthfully, cut at the same point, lost none. On one Postgres table, I set the disk to report writes as stored when they were not, so the run says nothing about a Postgres defect. Three runs lost 63, 64 and 63.

See more

In a three-node Akka.NET cluster (a library for servers working together), the others removed the node I cut off in 2 of 2 runs. The main run's report says it removed every member, itself included.

See more

All results

Figures as of .

In five of my runs, the replay matched the original exactly. Replay is not yet measured on your software until I run it.

See more

An illustration.

Akka.NET servers with their connection cut

In 3 of 3 pairs, run and replay agreed on all 1,476 network messages, on one replay code, one machine and one build.

Full record

A .NET program with four racing threads (parts of a program updating one number at once)

Restored 96 times, 90 counted, none differed. On an ordinary computer it gave 50 different answers in 50 runs.

Full record

Who this fits right now.

A fit

  • A database, or servers working together, if your engineers ship Linux container images.
  • An engineer with a few hours a week, and someone who can say yes.

Not yet

The work.

You and I agree on one promise and the failures to try, on one service or a small cluster.

If it breaks, I replay the failure on request until I delete your files, 30 days after the report.

If it holds, you get a recording. I cannot promise to find a bug. A clean report is still a result: what was explored held up, and nothing more.

An example result

Start with a call.

Bring one failure you couldn't reproduce, or one promise your software must keep. No prep.

You get a straight yes or no on whether Molina can run your software today and a proposed scope. Work after the call is priced case by case.

Before any image or log reaches my hardware, you and I agree on how they are kept, deleted and read.

I'm Will Ehrendreich, and I run every engagement myself.