When AI Gets It Wrong, Who Answers?

By: Zuflin Nazli

Artificial intelligence can support better decisions, but it cannot inherit human accountability. In matters involving safety, employment and human dignity, someone must still be willing to own the final decision.

Artificial intelligence is becoming remarkably capable. It can draft reports, analyse large amounts of data, identify patterns and offer recommendations before a person has finished a cup of coffee. Yet AI has one advantage that no human decision-maker has ever enjoyed: when it is wrong, it does not have to face the family of an injured worker, answer to someone who has lost a job or explain to a board why a warning was ignored.

The machine produces an answer. People live with the consequences.

That is why the most important question about AI at work is no longer simply, “What can it do?” We must also ask, “When it gets something wrong, who will answer for it?”

AI is already changing work

This is not an argument against AI. Used well, it can reduce repetitive work, help people find information faster and reveal risks that might otherwise be missed. It can support doctors, engineers, managers, teachers and public officers. In many workplaces, it will become as ordinary as email or a spreadsheet.

The change is already under way. A 2025 study by the International Labour Organization found that one in four workers worldwide is in an occupation with some degree of exposure to generative AI. The same study did not conclude that one in four jobs would disappear. Because human input remains necessary, the ILO said most jobs are more likely to be transformed than made redundant.

That transformation will often look ordinary. A clerk may use AI to prepare a first draft. An engineer may use it to search technical records . A manager may ask it to summarise performance data. The final task still belongs to a person, but the starting point, the pace of work and the information placed before that person have changed. If the AI leaves out an important fact or gives undue weight to the wrong one, the human decision may already have been pushed in the wrong direction.

That distinction matters. Transformation can bring progress, but it can also change who makes a decision, how that decision is checked and where responsibility sits when the outcome is harmful.

The danger begins when a useful tool is quietly treated as an authority. A system may sort information, highlight a concern or suggest an action. None of that means it understands the full situation. AI works from data, rules and patterns. It does not carry professional duty. It does not understand the weight of a human life, the effect of a dismissal on a family or the moral meaning of a decision.

Confidence can make this problem worse. AI systems often present an answer in clear and polished language, even when the information is incomplete. People may assume that a fast, detailed response must also be reliable. In practice, the answer may reflect an outdated document, a weak assumption or a pattern that does not fit the case in front of us. A professional must still ask where the answer came from, what it may have missed and whether it makes sense in the real world.

A complete form does not mean a safe job

Consider a permit-to-work system in a high-risk workplace. An AI tool checks the form and finds that every required field has been completed. The job description is present. The hazards have been listed. The isolations are recorded. The names and approvals are in place. On the screen, the permit appears complete.

But the plant does not exist on the screen.

At the worksite, an actual valve may be in a different position from the drawing. A temporary connection may have been installed. The weather may have changed. Another team may be carrying out work nearby. The operating condition may no longer match the information used by the system. A document can pass every digital check while the hazard remains very much alive.

This is where false confidence becomes dangerous. Once a green tick appears on a screen, people may become less willing to look again. The digital approval begins to feel more reliable than the evidence in front of them. Yet the system has not seen the leaking flange, heard the unusual vibration or noticed that the person doing the job does not fully understand the instruction. Those details still require human eyes, practical knowledge and a conversation at the worksite.

AI may help to review the permit. It may flag missing information or compare the document against earlier jobs. Those are valuable functions. But it cannot replace the competent person who goes to the location, verifies the isolation, speaks to the people doing the work and stops the task when something does not make sense.

This principle extends far beyond industrial safety. AI may screen job applicants, assess performance, prepare work schedules, recommend medical priorities or identify people for investigation. If the data is incomplete, outdated or biased, the output may look objective while repeating an old unfairness at computer speed.

The accountability gap

A serious problem appears when everyone involved is able to step one pace away from the decision. The employee says the system recommended it. The manager says the dashboard showed no problem. The technology team says the model only provided advice. The vendor points to the terms of use. The organisation then discovers, usually after harm has occurred, that nobody truly owned the risk.

This is the accountability gap : technology influences the decision, but responsibility becomes blurred among the people who designed, bought, configured, approved and used it.

The gap cannot be closed after an incident by adding another sentence to a procedure. Ownership must be settled before the system is used. Someone must decide what data is acceptable, what level of error can be tolerated, when a human review is compulsory and who has the power to stop the system. If these questions have no clear answers before deployment, the organisation is relying on hope rather than governance.

We have seen similar behaviour long before AI. People have hidden behind procedures, committees, audits and computer systems. AI simply provides a newer and more convincing place to hide. “The system says so” is now becoming “the AI recommended it”.

But a recommendation is not a transfer of responsibility. If an organisation chooses to use AI, it also chooses the duty to understand its limits, monitor its performance and correct its failures. Leaders cannot take credit when AI improves productivity and then blame the algorithm when the result causes harm.

Human oversight must be real

Many organisations will respond by saying that a human remains “in the loop”. That sounds reassuring, but it can mean very little. A person who is given ten seconds to approve an automated recommendation is not exercising meaningful judgement. Neither is an employee who has been trained to follow the system but fears being questioned for disagreeing with it.

In such cases, the human is present only to provide a signature. If the recommendation is accepted, the organisation credits the technology. If it fails, the person who clicked “approve” may carry the blame. That is not oversight. It is a digital rubber stamp followed by convenient accountability.

Real human oversight requires authority, time, information and competence. The reviewer must be able to understand why a recommendation was made, examine the evidence behind it, challenge the output and reject it without punishment when the situation demands it.

There is also a risk that constant dependence on AI will weaken human capability. If workers become button-pushers who merely accept what appears on a screen, they may slowly lose the practical knowledge needed to detect when the system is wrong. Efficiency gained today can create fragility tomorrow.

This matters especially for younger workers. Many experienced professionals developed judgement by doing the slower work first: reading full reports, checking drawings, visiting the site, making small decisions and learning from mistakes. If AI removes every basic task in the name of efficiency, organisations may also remove the training ground where future experts learn how the work actually fits together. We cannot expect good judgement from people who have never been given the chance to build it.

Malaysia has already recognised the need for responsible adoption through its National Guidelines on AI Governance and Ethics, commonly known as AIGE. ASEAN has also issued guidance on AI governance and expanded it to address generative AI. These efforts are important. However, a guideline creates value only when organisations turn its principles into everyday decisions, named responsibilities and actions that can be audited.

Four safeguards every organisation needs

First, organisations must decide which decisions AI may support and which decisions it must never make alone. Any matter involving life, safety, employment, legal rights or serious harm should require meaningful human judgement.

Second, every high-impact use of AI must have a named human owner. Not a department, committee or generic job title, but a person with the authority and duty to ensure that the system is being used properly.

Third, people affected by an AI-assisted decision need a clear way to challenge it. A worker should be able to question an automated performance rating. A job applicant should not be trapped by an invisible screening rule. A frontline employee should be able to reject an unsafe recommendation without being accused of resisting technology.

Fourth, organisations must keep a reliable record of what happened: the data used, the system’s recommendation, the human review, the final decision and the reason for it. If nobody can reconstruct the decision later, accountability is already too weak.

These controls also need regular testing. A system that worked well during a pilot may become less reliable when the work changes, the data shifts or users begin to rely on it in ways the designers did not expect. Organisations should examine actual decisions, including the ones that caused no visible harm, and look for repeated errors, unfair outcomes and cases where employees ignored warning signs because they trusted the system too quickly.

These safeguards may slow a decision by a few minutes. In some cases, they may prevent a decision entirely. That is not a failure of innovation. It is the cost of using powerful technology responsibly.

The final decision must still have a human name

We do not need to fear machines simply because they are becoming more capable. We should be far more concerned about people becoming less willing to think because a machine has produced a confident answer.

AI can read thousands of pages, calculate faster and detect patterns that human eyes may miss. But it does not know duty, courage, guilt or dignity. It cannot carry accountability because accountability is more than producing the correct output. It is the willingness to stand behind a decision, explain it and accept the consequences.

The future of responsible AI will not be secured by better technology alone. It will depend on whether organisations preserve the space for human judgement and whether leaders are prepared to own the decisions made under their authority.

For every decision that can change a life, end a career or expose someone to danger, there must still be a human name at the end of the process – someone prepared to say: “I reviewed it. I understood the risk. The decision is mine.”

Editorial references

• International Labour Organization, Generative AI and Jobs: A 2025 Update, 20 May 2025.

• Government of Malaysia, National Guidelines on AI Governance and Ethics (AIGE).

• ASEAN Secretariat, Expanded ASEAN Guide on AI Governance and Ethics – Generative AI.

About the author

Zuflin Nazli is an occupational safety and health practitioner with nearly three decades of experience in operational safety, leadership and workforce capability development. He writes on work, safety, technology and society.

The views expressed are the writer’s own and do not represent any organisation.