QA Like Glass: It Doesn't Have to Be Seen — But You Have to See Through It

A pane of glass has a strange property.

When it works, you look straight through it. You see the street, the garden, the rain, the light in the building opposite — but not the glass.

You notice the glass the moment something appears on it. A crack. Dirt. A stuck insect.

A bug.

Software quality behaves the same way. Nobody celebrates the fact that an order was saved exactly once, permissions held, a form didn't drop the user's input, and the new release didn't break a six-month-old process.

None of that reads as an achievement. It reads as "how the product is supposed to work."

So quality usually becomes visible through failure.

And so does QA.

When a serious defect ships, everyone wants to know how it got through. When nothing ships broken, almost nobody asks which questions were raised early, which risks were removed before implementation, or how many problems never happened because a control was designed well.

From there it's a short step to a dangerous conclusion: QA work is invisible, and QA people should make peace with it.

But transparent is not the same as invisible.

Good QA isn't glass because nobody knows it's there. It's glass because it lets everyone else see the real state of the product without distortion.

Good QA doesn't try to be seen. It tries to be seen through.

Invisible preventive work creates an understandable pressure: QA wants to show it's useful.

So it starts counting.

Tests written. Scenarios executed. Bugs found. Cases automated. Hours spent testing. Dashboard percentages that prove something is happening.

All of it can be useful operational data. None of it, on its own, tells you whether the product is good enough — or whether the team understands its risks.

Consider:

• A thousand green tests can be checking things that don't matter.
• One red test can be protecting the most important financial transaction in the system.
• A high bug count can mean excellent testing — or a misunderstood spec, a weak technical design, or feedback that arrived far too late.
• A low bug count can mean a solid product — or shallow testing. Once QA is judged mainly by the volume of its visible activity, the glass turns into a billboard. Still in plain sight — but you can no longer see through it.

The real value of QA isn't the volume of artifacts it produces. It's the quality of the information the team can decide on.

Visibility says: something is happening.

Transparency says: we know what is happening, why, what the result means, and where our certainty ends.

The difference is not cosmetic:

Transparent QA doesn't have to produce more information. It has to produce information that is timely, understandable and usable.

This isn't just a nice principle. DORA's research builds on Westrum's model of organizational culture and shows that environments with high trust and good information flow predict better software delivery performance and better organizational outcomes. Westrum describes good information as information that answers the receiver's questions, arrives in time, and is delivered in a form they can actually act on. (DORA: Generative organizational culture)

Transparency, then, is not a report count. It's the system's ability to deliver the right information to the right person while they can still do something with it.

In 1969, Edsger W. Dijkstra formulated one of the most important limits of testing: tests can show the presence of defects, never prove their absence. (E. W. Dijkstra: Notes on Structured Programming)

That doesn't make testing weak or pointless. It means a test result is evidence with a defined scope — not a universal certificate of truth.

A green test says only this: under specific conditions, a specific check ran and produced the expected result.

How much that's worth depends on whether it:

• verifies behaviour that actually matters;
• matches the current version of the product;
• uses credible data and a credible environment;
• is stable and reproducible;
• asserts the right outcome, not merely the absence of a technical error;
• and whether we understand what it doesn't cover at all. Without that context, green is just a light on a dashboard.

Quality itself is also broader than "we didn't find a bug." ISO/IEC 25010:2023 defines a product quality model of nine characteristics and their sub-characteristics, used among other things to define requirements, test objectives, acceptance criteria and quality measures across the lifecycle. (ISO/IEC 25010:2023)

In other words: a product can compute correctly and still be slow, insecure, confusing, unmaintainable, or unfit for its real context of use.

QA that shows only a list of functional defects offers a view through a very narrow strip of glass.

One of the most durable organizational illusions is that development builds the product and QA then "adds quality" to it.

It doesn't.

QA can expose a contradiction, produce evidence, challenge an assumption, propose a control, improve testability, or stop a decision that has no basis. It cannot inject quality at the end of a process into a product that was designed without it.

DORA consistently frames quality, availability, reliability and security as shared responsibilities. In a healthy information culture, risks are shared, collaboration crosses role boundaries, and failure leads to inquiry into the system rather than a search for someone to blame. (DORA: Generative organizational culture)

That's why "how did this get past QA?" usually starts in the wrong place.

Better questions:

• Which assumption turned out to be wrong?
• Which signal were we missing?
• Why did feedback only appear in production?
• Was the risk known but poorly communicated?
• Or was the decision made without the evidence it needed? This isn't diluting accountability. It's sharpening it.

If "QA owns quality," everyone else can put accountability down. If quality is a shared outcome and QA owns the architecture of evidence, questions and feedback loops, then it's clear what each participant has to do.

QA then isn't a customs officer standing in front of production. It's the architect of transparency across the whole flow of change.

1. The promise

First it has to be clear what the product promises, to whom, and under what conditions.

Not "the feature works," but something like: a user in this role can complete this goal, data stays consistent, the operation is traceable, and failure won't cause unacceptable impact.

Without an articulated promise you can't reason about quality. You can only run checks.

2. The evidence

Every important claim about quality should have a traceable basis.

That can be an automated test result, exploratory testing, static analysis, a security check, production monitoring data, or a combination.

What matters is that it's clear what the evidence supports, how it was produced, and whether it's still current.

"Tested" is a meaningless status if we don't know what, how, when, where, and against which expectation.

3. The gap

Transparent QA doesn't only show what we know. It shows what we don't know, just as honestly.

A missing environment. Unrealistic test data. An unverified migration. A third-party dependency. A scenario that can't be automated reliably. Output produced by a tool or an AI that hasn't been independently verified yet.

An acknowledged gap is not a QA failure. A hidden one is.

4. The risk

Not every defect is worth the same, and not every uncertainty justifies stopping a release.

QA's job is to translate a technical finding into the language of impact: who is affected, what it can cause, how likely it is, whether it can be detected or corrected, and what options exist to limit the damage.

That's what turns a bug report into input for a decision.

5. The decision

QA can recommend, warn and evidence. But business or product risk must be accepted by someone with the authority — and the accountability for the consequences.

"QA approved the release" blurs that line. A more precise formulation sounds like this:

Critical scenarios A, B and C were verified on build X in environment Y. For area D we lack sufficient evidence due to constraint Z. We assess the remaining risk as high, given the potential impact on user group E. The product owner chose to accept the risk with mitigation F.

That's not bureaucracy. That's transparency.

QA is sometimes visible mainly as an obstacle: the team wants to ship, QA says no.

But a veto without shared criteria isn't quality management. It's a power struggle wearing a process costume.

Google's Site Reliability Engineering offers a useful counter-example. The error budget turns the conflict between change velocity and reliability into an agreed, measurable frame.

Product and operations jointly set a target service level. Measured reliability then shows how much acceptable unreliability is left. While budget remains, the team can experiment more. When it's spent, stability takes priority.

Google explicitly notes that 100% reliability is usually the wrong target, and that error budgets help take politics out of risk negotiation. (Google SRE: Embracing Risk)

That's exactly the difference between a wall and a pane of glass.

A wall says: "You're not shipping, because QA disagrees."

Glass shows: "This is the agreed limit, this is the measured data, and this is what the next decision will cost."

Quality architecture shouldn't replace human judgement. It should make sure that judgement doesn't rest on impressions, hierarchy, or whoever is loudest in the room.

A dashboard can support transparency. It can just as easily fake it.

Test pass rate is the classic example. "99%" looks precise, but without context it tells you almost nothing.

• Did one cosmetic test fail out of a hundred?
• Or the one test out of a hundred that protects a critical process?
• Were hundreds of other checks never executed at all?
• Are these results from today, or from the last release? A good metric doesn't just answer how many. It helps decide what to do next.

DORA, for example, recommends using its delivery performance metrics primarily to track the same application over time, and stresses that they measure a team outcome, not an individual. One person's commit count may say more about a slow review process than about their ability. (DORA: How to empower software delivery teams)

The same discipline belongs in QA. Instead of competing on test or bug counts, it usually makes more sense to watch — depending on context:

• how quickly usable feedback appears after a change;
• what share of critical risks has current evidence;
• how many production changes require a fix, rollback or other unplanned work;
• the impact of defects that reach users;
• how many tests are flaky and generate false alarms;
• how long it takes to tell a real problem from a broken test;
• how many decisions were made despite an explicitly documented evidence gap. These aren't universal quality scores either. They're lenses. Each reveals a different part of reality — and each distorts if it becomes a target in itself.

The hardest thing to prove about QA work is the thing that didn't happen.

When a question asked at the right moment changes a design and prevents a future defect, there's no incident to add to a list of wins.

When a contract test catches an incompatible change before deployment, the customer never hears about it.

When good diagnostics cut root-cause analysis from three hours to ten minutes, the outage gets the headline — not the months of observability work.

The economic impact of poor quality is real, though hard to generalise. A widely cited study prepared for NIST estimated the cost of an inadequate software testing infrastructure to the US economy at $59.5 billion a year, with roughly $22.2 billion recoverable through feasible improvements — and around 60% of the estimated impact borne by users.

It's fair to add that this is a 2002 study built on case studies and extrapolation. Presenting that figure as today's universal price of software defects would be dishonest. Its durable insight is simpler: the cost of a problem doesn't stop with the team that created it. (NIST: The Economic Impacts of Inadequate Infrastructure for Software Testing)

So transparent QA shouldn't promise an exact price for every defect it may have prevented. It should show which prevention mechanism exists, which risk it covers, and whether it actually works.

Organizations routinely mistake the record of work for the work itself.

Git shows who changed a file. A test management tool shows who executed a scenario. A ticket shows who reported a defect.

None of those systems necessarily captures the person who knew why a particular check exists, who could connect an unremarkable symptom to an old incident, or who recognised in time that a green result wasn't trustworthy this time.

There's a technical term for this: the bus factor — how many key people a project can lose before it seriously stalls because knowledge was concentrated.

An analysis of 133 popular GitHub projects estimated that 65% of them had a bus factor of at most two. That result shouldn't be transplanted onto every corporate team — these were open-source projects, and the estimate was derived from repository data. Still, it's concrete evidence that knowledge concentration isn't just a management anecdote. (Avelino et al.: A Novel Approach for Estimating Truck Factors)

The more important finding comes from elsewhere. Bus Factor in Practice surveyed 269 engineers and analysed data from 13 JetBrains projects. The authors point out that commits alone can't identify everyone who holds critical knowledge.

That knowledge is also created and passed on in code review, in meetings, in discussions, while writing tests and documentation, and while supporting tooling. A person can be essential to a project while looking almost invisible in the repository. (Jabrayilzade et al.: Bus Factor in Practice)

In QA, the effect is especially strong. After an experienced person leaves, a team may only then discover that:

• nobody knows the real reason behind several odd regression tests;
• nobody can distinguish routine noise from the first signal of a serious problem;
• nobody maintains the link between a business rule, its test data and production telemetry;
• nobody knows which green results are credible and which have merely learned to pass;
• and decisions that felt obvious were in fact propped up by one person's experience. That's a very late and very expensive way to discover someone's value.

The goal, though, isn't to cultivate an indispensable hero. It's to surface their knowledge before they leave: share ownership, record the reasoning behind decisions, convert repeatable experience into controls, pair, write usable runbooks, and make sure more than one person knows the critical parts of the system.

If a system collapses when one person leaves, that's not only proof of their value. It's proof that the organization left essential knowledge invisible for too long.

Someone's departure should not be the first test of their contribution.

Not by shrinking, going quiet, and accepting that nobody will see the work.

And not by proving value through drama, finding counts, or owning the last word before a release.

A professional QA practitioner:

• doesn't hoard knowledge, but converts it into shared rules, examples and controls;
• doesn't collect tests as trophies, but maintains evidence that matches real risk;
• doesn't just report defects, but explains impact and the degree of uncertainty;
• doesn't present an assumption as a verified fact;
• doesn't use a green dashboard as a substitute for judgement;
• builds short feedback loops instead of a late, dramatic testing finale;
• helps product, development and operations take on their share of quality;
• and builds a system that still works when the QA person isn't in the room. That last one may be the highest mark of professional maturity — and it's also what people quietly fear. If I hand over my knowledge and remove the dependency on me, don't I make myself redundant?

No. What becomes redundant is manually relaying the same information. The person's value moves up a level: from repeatedly executing checks to designing a system that can create, run, evaluate and improve trustworthy checks.

A tester can be visible at every test run. A quality architect is recognised by the fact that the whole system produces better information without their constant intervention.

It would be naive to claim that doing good work is enough and the organization will notice its value on its own.

It won't.

Preventive, systemic work has to be communicated deliberately. Not as content-free self-promotion, but through traceable change:

• which risk is now detected earlier;
• which decisions now rest on better data;
• where feedback loops got shorter;
• which recurring defects were removed by a process or architecture change;
• how dependency on manual intervention was reduced;
• and where debt, uncertainty or uncovered areas remain. Your work doesn't have to be loud. But its state must not be a secret.

Transparency protects more than the product. It protects QA itself — because it lets leadership tell the difference between a team that "found nothing" because it tested shallowly, and a team that found nothing serious because it helped build quality in earlier.

The better QA works, the less it resembles a standalone inspection station.

Requirements become verifiable before code exists. Developers get feedback in minutes. Critical rules live in the pipeline. Production signals complement pre-production tests. Risks have owners. Decisions have reasons. Ambiguity doesn't get swept under a green status.

In a system like that, the number of dramatic last-minute findings can fall.

That's not evidence QA has lost its purpose. It can be evidence that its influence spread far enough that problems stopped waiting for a phase called "testing."

Good QA can therefore become less visible as a separate activity and far more transparent as a property of the whole system.

That's the difference between disappearing and growing up.

A crack isn't the moment glass becomes useful. It's the moment we finally notice it.

A bug isn't the moment QA starts creating value. It's just the most visible proof that a gap opened between expectation and reality.

QA's value is created much earlier: in the right question asked in time, a sharpened expectation, a testable design, a short feedback loop, credible evidence, and honestly named uncertainty.

QA doesn't have to be visible in every ticket, every meeting, or under every decision. But it must always be possible to trace what we know, what we know it from, what we don't know, which risk we're accepting, and who decided.

We don't want QA standing in front of the window, pointing at itself.

We want QA that lets everyone see through.

Invisibility can be a consequence of quality working well. Transparency is its obligation.

• ISO/IEC 25010:2023 — Product quality model
• E. W. Dijkstra — Notes on Structured Programming (EWD 249)
• DORA — Generative organizational culture
• DORA — How to empower software delivery teams as a business leader
• Google SRE — Embracing Risk
• NIST — The Economic Impacts of Inadequate Infrastructure for Software Testing
• Avelino et al. — A Novel Approach for Estimating Truck Factors
• Jabrayilzade et al. — Bus Factor in Practice