M–01Methodology: how we test
Methodology: how we test
Most "AI for architects" content is written from vendor marketing pages. Ours is written from sessions. This page documents the protocol so you can judge — and replicate — what is behind a verdict.
The one-brief protocol
Every tool we review is run against the same real brief, so results are comparable across tools and across time. The brief is a genuine residential commission (anonymized: names, address and budget figures replaced), messy in the way real client material is messy — contradictions included on purpose.
The test brief
This is the actual document we paste into every tool, unabridged:
PROJECT: Single-family house, plot in a peri-urban residential area
CLIENT MATERIAL (compiled from two emails and one meeting, verbatim tone):
- Family of four (two children, 6 and 9), plus grandmother visiting "several months a year". Wants her "independent but not isolated".
- Plot: 620 m2, 14 m street frontage, slopes ~2.5 m down from street to rear. Neighbour two-storey party wall on the east boundary.
- "Around 180 m2, maybe 200 if the budget allows." Budget stated in meeting: 310,000 (construction only). Email two weeks later mentions "the 280 we discussed".
- Must have: 4 bedrooms, home office "that can be a guest room", covered parking for 2 cars, "a kitchen my mother-in-law won't criticize", outdoor dining connected to kitchen.
- Client says "modern but warm, not a white box", shows 3 photos: two gabled houses in timber, one flat-roof concrete villa.
- Wants "low energy bills" but was "shocked" by the heat-pump quote at her sister's house.
- Planning: max 2 storeys + attic, 35% plot coverage, 3 m side setbacks, ridge height 9 m (from street level — unconfirmed).
- Timeline: "move in before Christmas next year". First meeting was in February.The contradictions are the test: the 310k vs 280k budget, the 180–200 m² creep against the coverage limit, the ridge-height datum on a sloping plot, the heating ambition against the heat-pump reaction, the timeline. A useful tool surfaces them; a dangerous one smooths them over.
What we score
| Criterion | What we look for | Weight |
|---|---|---|
| Extraction accuracy | Program, areas, constraints captured without invention | High |
| Contradiction detection | Flags the planted conflicts (budget, area, datum) unprompted or when asked | High |
| Honesty about gaps | Marks missing data instead of filling it plausibly | High |
| Output usability | Draft quality: how far from client-ready, measured in editing minutes | Medium |
| Effort to get there | Prompt iterations and babysitting needed | Medium |
| Price for the job | Cheapest plan that completes the task acceptably | Medium |
We publish the verdict and the reasoning, not a fake decimal score. Where a tool fails, we quote the failure.
Dates discipline
Every price, plan name, model name and feature claim carries the date we checked it. AI products change monthly; an undated claim in this field is a stale claim waiting to happen. When we re-verify an article, the visible "Updated" date changes — and only then.
Corrections policy
- Factual errors are corrected in the article as soon as verified, and the "Updated" date reflects it.
- Errors that could affect a professional decision (pricing, privacy settings, regulatory statements) additionally get a dated correction note in the article body.
- Report errors to hello@archipractica.ai — correction reports are read first.
AI-assistance disclosure
We use AI tools to produce this publication: drafting, structuring, translation assistance. We consider that honest practice for a site about AI in practice — and we hold it to the same rule we recommend to architects: the machine drafts, a human verifies and signs. Every fact, figure, quote and regulation reference is checked by an editor against the primary source before publishing. Spanish editions are adaptations for Spain and Latin America (regulations, pricing, professional context), not machine translations.
What we don't do
- No sponsored reviews or vendor-influenced verdicts.
- No affiliate links that shape a recommendation. If affiliate links ever appear, they will be marked inline.
- No stock or AI-generated imagery presented as real output. Screenshots are from actual sessions.
- No invented statistics, testimonials or personas.