The Albanese government is being warned it is sleepwalking into another robodebt scenario due to a lack of transparency and legislation around the use of algorithms and artificial intelligence in government decisions. The warning comes from Dr Catherine Williams, chief executive of independent thinktank the Centre for Public Integrity, who launched the centre's "Computer Says No" report on Thursday.
The report examines the use of automated decision-making (ADM) tools by the government and calls for urgent reform. It raises concerns that Australians may not know whether ADM has been used in decisions affecting them, potentially leading to another scandal similar to robodebt.
What the Report Found
The Centre for Public Integrity's report, "Computer Says No," highlights the increasing use of automated decision-making tools across government services. It found that there is a lack of transparency and oversight, which could lead to errors and injustices similar to the robodebt scheme.
One key concern is the government's integrated assessment tool (IAT), which is used to assess the level of home support older Australians are entitled to. The IAT has been found to routinely under-assess people's needs and package entitlements, with aged care assessors banned by the government from overriding incorrect algorithmic decisions.
Dr Williams said the report was developed amid the centre's increasing concern about the use of an algorithm as part of the IAT. "We have been watching the evidence emerge in relation to the aged care algorithm tool, and becoming increasingly concerned about the lack of transparency and oversight," she said.
Why This Matters
The use of automated decision-making in government services can have serious consequences for individuals. If algorithms are used without proper oversight, they can lead to incorrect decisions that affect people's access to essential services and support.
The robodebt scandal, which involved the automated calculation of welfare debts, led to widespread criticism and a royal commission. The government has been urged to learn from that experience and ensure that new technologies are used responsibly.
The report's recommendations aim to prevent another such scandal by increasing transparency and accountability in the use of ADM.
Key Recommendations
The report makes eight recommendations, including:
- Mandatory audit and public register of all ADM use by government agencies.
- Independent oversight body with powers to monitor, audit and investigate the government over its use of ADM.
- Legislation to govern the use of ADM and ensure transparency.
- Notification to individuals when ADM is used in decisions affecting them.
- Right to appeal decisions made by ADM to a human reviewer.
- Training for public servants on the ethical use of ADM.
These measures are designed to build public trust and ensure that the use of technology in government decision-making is fair and accountable.
Comparison of ADM Oversight
The table below compares the current situation with the recommended reforms.
| Aspect | Current Situation | Recommended Reform |
|---|---|---|
| Transparency | Limited public information on ADM use | Public register of all ADM systems |
| Oversight | No independent body | Independent oversight body with investigative powers |
| Individual rights | No right to human review | Right to appeal to a human decision-maker |
| Accountability | Unclear who is responsible | Legislation to assign responsibility |
What Happens Next
The government has not yet responded to the report's recommendations. However, the warning comes amid growing scrutiny of the use of AI and algorithms in public services. The robodebt royal commission highlighted the dangers of automated decision-making when not properly managed.
Advocates argue that without urgent reform, the government risks repeating the mistakes of the past. The Centre for Public Integrity is calling on the government to act swiftly to implement the recommendations.
As reported by Albanese warned not to underestimate Pauline Hanson's threat, the government is facing pressure on multiple fronts, and this report adds to the challenges it must address.
FAQ
What is automated decision-making (ADM)?
ADM refers to the use of algorithms and computer systems to make decisions or assist in decision-making, often without human intervention. In government, it can be used to determine eligibility for services or benefits.
Why is the Albanese government being warned about ADM?
The government is being warned because of a lack of transparency and legislation around ADM, which could lead to another scandal like robodebt. The Centre for Public Integrity is concerned that Australians may not know when ADM is used in decisions affecting them.
What is the integrated assessment tool (IAT)?
The IAT is an algorithm used to assess the level of home support older Australians are entitled to. It has been found to routinely under-assess people's needs, and assessors are banned from overriding incorrect decisions.
What are the report's key recommendations?
The report recommends a mandatory audit and public register of ADM use, an independent oversight body, legislation, notification to individuals, a right to appeal to a human, and training for public servants.
The warning over ADM comes as the government also faces legal challenges from whistleblowers. As covered in Robodebt whistleblower sues Australia over job threats, the fallout from robodebt continues to affect those who spoke out.
Transparency in government use of technology is essential to maintain public trust. The asbestos in kids' sand: why parents weren't warned incident shows how a lack of communication can erode confidence.
The government must act on these recommendations to avoid another robodebt. As Australia student visa crackdown: colleges warned not to game rules demonstrates, enforcing rules fairly is crucial.
The robodebt saga has had lasting impacts, as seen in Robodebt victim's mother denied compensation in $475m settlement. It is vital that the government learns from these mistakes.
Whistleblowers play a key role in exposing issues, as highlighted in Robodebt whistleblower sues over job threat allegations. Their courage should be protected.
The Robodebt whistleblower sues over threat to her job case underscores the need for strong protections. Without them, others may be deterred from speaking out.
As the Robodebt whistleblower lawsuit: key evidence and threats revealed shows, the details matter. The government must ensure its use of AI is transparent and accountable.