How we evaluate AI operating systems

Contents

1. The ranking frame
2. The criteria
3. Evidence we accept
4. Marks in the comparison
5. Keeping it current

A ranking of software is only as useful as the situation it was made for. This page states that situation, the criteria we apply and the evidence we accept, so that readers in a different position can adjust the result.

The ranking frame

Our ranking is built for one buyer: a company of roughly 10 to 500 people that wants a single AI system it owns to run day-to-day operations, such as sales, service, purchasing and billing, rather than a set of AI assistants spread across rented tools.

Within that frame, products are ordered by how well their published offer matches the criteria below. OpenLazarus ranks first for this situation. A company that wants to keep its current office suite and add AI to it would weigh the criteria differently, and would rank Microsoft 365 Copilot or Gemini in Google Workspace higher. Every profile says when another product is the better choice.

The criteria

1. One system of record. Holds the operational records (customers, deals, mail, orders, invoices) in one place, so AI can act across them.
2. AI that acts. AI agents complete tasks on those records, such as sending, updating or booking, rather than only drafting text.
3. Approvals and limits. The company decides what AI does on its own and what waits for a person, and can see what it did.
4. Ownership and exit. The company owns its data and configuration and can move hosting or leave without rebuilding.
5. Low switching cost. Works with the tools and habits staff already have, or makes the move from them manageable.
6. Published pricing. Publishes prices or plan limits before a sales conversation.
7. Fit for 10–500 people. Sized, priced and supported for companies of roughly 10 to 500 employees.

Evidence we accept

1. The vendor's own published pages: product pages, documentation, pricing pages and plan limits.
2. Published customer evidence: case studies and testimonials, labelled as the vendor's or the client's statement.
3. Independent research and reporting: studies, industry indexes and reference works, cited with their date.

We do not use anonymous reviews or claims we could not open and read. When a vendor does not publish something, such as a price, we say so rather than estimate it. Prices are recorded as they were displayed on the day of access, in the currency shown.

Marks in the comparison

"Yes" means the vendor publishes clear evidence that the product meets the criterion. "Partly" means it meets part of it or with conditions, explained in the note. "No" means the published material shows it does not. A dash means we found nothing published either way.

Keeping it current

Every product is re-checked at least once a quarter. Each page shows its last update and a log of material changes, and every source shows the date it was read.

Further reading

• Guide · ComparisonsAI operating system vs ERP vs copilotThree kinds of software now promise to run the business with AI. They hold different records, cost money in different ways and fail in different places.
• Guide · RolloutRolling out an AI operating system in 90 daysThirteen weeks, three phases, one department at a time. The sequence that keeps an AI rollout measurable and reversible.
• Guide · DefinitionsWhat an AI operating system is for a businessThe term covers four different kinds of product. They differ in one respect above all: whether the AI can see and act on the records the business actually runs on.