When an App Becomes the Law
I still think about ArriveCAN.
Not because it was merely a badly designed app. I disliked it from the beginning, but what troubled me was what the app represented: the merging of technology, government authority and personal freedom.
I was coming home to Canada from Mexico during the pandemic. I was not sick. I was trying to get home.
Instead, I found myself navigating quarantine requirements, testing requirements, government instructions and an increasingly complicated digital system. On my second return from Mexico, the experience was particularly upsetting. The way I was treated felt hostile to me—so much so that, at the time, it felt racist. I remember thinking that something had gone badly wrong when coming home to my own country could feel like this.
There was also the mandatory hotel quarantine system. I remember dealing with a nurse whose manner was frightening and threatening rather than helpful. I had a panic attack. Eventually, I received a $3,500 federal COVID-related ticket.
I am not writing this to relitigate the pandemic or argue that governments should never have imposed public-health measures. I am writing about something different:
What happens when those measures are delivered through technology that people are required to trust, while the consequences of mistakes can fall entirely on the individual?
I had studied interaction and app design at Emily Carr University. For six months, five days a week, three hours a day, we studied much more than how to make an app work. We studied users, scenarios, accessibility, different platforms, security and what happens when systems fail.
That training stayed with me.
ArriveCAN never felt like a well-designed system to me. And years later, official investigations revealed problems that went far beyond my personal frustration.
The Auditor General found that the federal government released 177 versions of ArriveCAN between April 2020 and October 2022, often with little or no documentation showing that the releases had been properly tested. The audit estimated the application cost approximately $59.5 million, while noting that poor financial records made the exact cost impossible to determine.
Then came something that should concern anyone who believes government technology must be reliable.
In 2022, an ArriveCAN software defect incorrectly told approximately 10,200 fully vaccinated travellers that they had to quarantine. The Privacy Commissioner subsequently found that the government had failed to take all reasonable steps to ensure that personal information used in an administrative decision was accurate. The complaint was found to be well-founded.
Think about what that means.
A computer system was providing instructions connected to quarantine—something with real consequences for people's lives—and the information behind those instructions was wrong.
The Privacy Commissioner noted that affected people could experience significant psychological stress from the confusion. More importantly, the inaccurate information was being used for an administrative decision directly affecting individuals and was being communicated to public-health authorities for enforcement purposes.
That is no longer simply a question of whether an app is annoying.
It is a question of power.
The federal Auditor General also found serious problems with the way ArriveCAN was contracted and managed. Essential information was missing from contracts, invoices often lacked adequate details about the work performed, and there was little documentation explaining how GC Strategies received the initial non-competitive contract.
The government later acknowledged that ArriveCAN had been used for more than 60 million digital submissions.
That is an enormous amount of personal information flowing through a system that, according to the subsequent audits, had significant weaknesses in testing, documentation, contracting and oversight.
And this is where my concerns about technology become broader than ArriveCAN.
Today we are debating artificial intelligence. People worry about algorithms making decisions about us, automated systems monitoring us, databases following us and governments or corporations possessing enormous amounts of personal information.
Those concerns can sound futuristic.
But we already experienced a version of the problem.
ArriveCAN wasn't artificial intelligence. It wasn't an autonomous machine deciding who could move freely. Human beings created the rules and government agencies were responsible for enforcing them.
But technology became the mechanism through which those rules reached millions of people.
And when the technology was wrong, ordinary people were the ones who had to deal with the consequences.
That is the lesson I took from ArriveCAN.
Technology should never become a substitute for accountability.
If a government requires people to use a digital system, there must be a reliable alternative when the technology fails. There must be a way to correct errors quickly. There must be human beings who can actually listen. And there must be accountability when mistakes affect people's freedom, finances, health or ability to travel.
Most importantly, people should not be expected to blindly trust a system simply because it has been given an official government logo.
I knew ArriveCAN was a bad experience for me.
What I didn't know at the time was how many weaknesses would eventually be documented by Canada's Auditor General and Privacy Commissioner.
That doesn't prove every suspicion I had was correct.
But it does prove something important:
The questions were legitimate.
And perhaps that is the lesson we should carry into the age of AI.
Before we give technology more power over our lives, we should ask the question my design education taught me to ask years ago:
What happens when it gets it wrong?
Today, I see the same question appearing in something as ordinary as a parking meter.
You used to park your car, put coins into a machine and go about your business.
Now, in some places, you may be expected to have a smartphone, a working app, an internet connection, a payment method and the ability to navigate the technology correctly.
When did putting a few coins in a parking meter become a technology problem?
And what happens when the app fails?
Who is responsible when you did everything you were supposed to do, but the system says you didn't?
These may seem like small questions compared with ArriveCAN. But they are connected by the same principle:
Technology should serve people. It should never become the thing people are forced to serve
10 Reflective Questions
When did convenience become compulsory?
What happens when technology designed to make life easier actually makes life harder?
Should an app ever become a requirement for exercising an ordinary freedom?
What happens when the system makes a mistake—but the consequences fall on you?
Who is accountable when nobody can explain why the computer made the decision?
What happens to people who don't have a smartphone, a data plan, or the ability to navigate complicated apps?
Should there always be a non-digital alternative when government or essential services require technology?
How much personal information should we surrender simply to park a car, cross a border, or access a service?
Are we testing technology thoroughly enough before making millions of people depend on it?
Before we give AI even more power over our lives, have we learned anything from ArriveCAN?
#ArriveCAN #DigitalRights #TechnologyAndFreedom #GovernmentAccountability #PrivacyMatters #AIAndSociety #HumanCenteredDesign #AppDesign #DigitalPrivacy #TechnologyEthics
