Would ‘explainable AI’ force companies to give away too much? Not really.

Here is an argument for allowing companies to maintain a lot of secrecy about how their data mining (AI) models work. The claime is that revealing information will put  companies at a competitive disadvantage. Sorry, that is not enough of a reason. And it’s not actually true, as far as I can tell.

The first consideration when discussing transparency in AI should be data, the fuel that powers the algorithms. Because data is the foundation for all AI, it is valid to want to know where the data…

Source: The problem with ‘explainable AI’ | TechCrunch

Here is my response.

Your questions are good ones. But you seem to think that explainability cannot be achieved except by giving away all the work that led to the AI system. That is a straw man. Take deep systems, for example. The IP includes:
1) The training set of data
2) The core architecture of the network (number of layers etc)
3) The training procedures over time, including all the testing and tuning that went on.
4) The resulting system (weights, filters, transforms, etc).
5) HIgher-level “explanations,” whatever those may be. (For me, these might be a reduced-form model that is approximately linear, and can be interpreted.)

Revealing even #4 would be somewhat useful to competitors, but not decisive. The original developers will be able to update and refine their model, while people with only #4 will not. The same for any of the other elements.

I suspect the main fear about revealing this, at least among for-profit companies, is that it opens them up to second-guessing . For example, what do you want to bet that the systems now being used to suggest recidivism have bugs? Someone with enough expertise and $ might be able to make intelligent guesses about bugs, although I don’t see how they could prove them.
Sure, such criticism would make companies more cautious, and cost them money. And big companies might be better able to hide behind layers of lawyers and obfuscation. But those hypothetical problems are quite a distance in the future. Society deserves to, and should, do more to figure out where these systems have problems. Let’s allow some experiments, and even some different laws in different jurisdictions, to go forward for a few years. To prevent this is just trusting the self-appointed experts to do what is in everyone else’s best interests. We know that works poorly!

Car repossession: Big Data +AI tools are not value-neutral

Does recent technology inherently favor capitalists over workers?

There is a lot of concern about AI potentially causing massive unemployment. The question of whether “this time will be different” is still open. But another insidious effect is gaining speed: putting tools in the hands of large companies that make it more expensive and more oppressive to run into financial trouble. In essence,  harder to live on the edges of “The System.”

  •  Cars with even one late payment can be spotted, and repossessed, faster. “Business has more than doubled since 2014….”  This is during a period of ostensible economic growth.
  • “Even with the rising deployment of remote engine cutoffs and GPS locators in cars, repo agencies remain dominant. … Agents are finding repos they never would have a few years ago.”
  • “So much of America is just a heartbeat away from a repossession — even good people, decent people who aren’t deadbeats,” said Patrick Altes, a veteran agent in Daytona Beach, Fla. “It seems like a different environment than it’s ever been.”
  • “The company’s goal is to capture every plate in Ohio and use that information to reveal patterns. A plate shot outside an apartment at 5 a.m. tells you that’s probably where the driver spends the night, no matter their listed home address. So when a repo order comes in for a car, the agent already knows where to look.”
  • Source: The surprising return of the repo man – The Washington Post

Continue reading

It will be very tricky to test and regulate safety of self-driving cars

My friend Don Norman wrote an op-ed this weekend calling for an FDA-like testing program before autonomous cars are put on the roads in the US. Clearly, some level of government approval is important. But I see lots of problems with using drug testing (FDA = Food and Drug Administration) as a model.

Here is an excerpt from a recent article about testing problems with Uber cars, which were the ones in the recent fatal accident. After the break, my assessment of how to test such cars before they are allowed on American roads.

Waymo, formerly the self-driving car project of Google, said that in tests on roads in California last year, its cars went an average of nearly 5,600 miles before the driver had to take control from the computer to steer out of trouble. As of March, Uber was struggling to meet its target of 13 miles per “intervention” in Arizona, according to 100 pages of company documents obtained by The New York Times and two people familiar with the company’s operations in the Phoenix area but not permitted to speak publicly about it.Yet Uber’s test drivers were being asked to do more — going on solo runs when they had worked in pairs.And there also was pressure to live up to a goal to offer a driverless car service by the end of the year and to impress top executives.

So Uber car performance was more than 100 times worse than Waymo cars?!

Continue reading

Hollywood as a model for academic research

Academia has a problem: the value, necessity, and practices of collaboration are increasing, but the system of giving credit is inadequate. In most fields, there are only 4 levels of credit:

  • None at all
  • “Our thanks to Jill for sharing her data.” (a note of thanks)
  • First Authorship (This is ambiguous: it may be alphabetical.)
  • Listed as another author

In contrast to this paucity, modern empirical paper writing has many roles. Here are a dozen roles. Not all of them are important on a single paper, but each of them is important in some papers.

  • Intellectual leadership.
    • Source of the original idea
  • Doing the writing
    • Writing various parts, e.g. literature review
    • Doing the grunt work on the stat analysis. (Writing and running the R code)
    • Doing the grunt work of finalizing for publication. (Much easier than it used to be!)
    • Dealing with revisions, exchanges with editors, etc.
  • Source of the data.
    • Funder of the data
  • Raised the funding;
    • Runs the lab where the authors are employed
    • Source of the money: usually an agency or foundation, but sometimes the contracting author is listed as a coauthor.

Continue reading

Irrational tax cuts won’t raise long term economic growth

I recently received the following on a Dave Farber’s “Interesting People” list, a collection of techies with interest in Internet policy issues. Why discuss it now, since the tax bill has been passed? It is important for all to realize how much the Republicans in Washington no longer believe in basing their decisions on  reality  (“facts”). It is very hard to believe this, but the evidence is now overwhelming, and the consequences will continue to be grave. I wrote the following quick response.

It seems completely reasonable and even desirable to take actions such as lowering corp taxes,  lowering taxes on productivity and reducing regulation to get the economy growing at the 3-4%  range.

Continue reading

Fraudulent academic journals are growing

Gina Kolata in the NY Times has been running a good series of articles on fraudulent academic publishing. The basic business model is an unholy alliance between academics looking to enhance their resumes, and quick-buck internet sites. Initially, I thought these sites were enticing naive academics. But many academics are apparently willing participants, suggesting that it’s  easy to fool many promotion and award committees.

All but one academic in 10 who won a School of Business and Economics award had published papers in these journals. One had 10 such articles.

Continue reading

What snakes are growing in the Gardens of Technological Eden?

Two emerging technologies are revolutionizing industries, and will soon have big impacts on our health, jobs, entertainment, and entire lives. They are Artificial Intelligence, and Big Data. Of course, these have already had big effects in certain applications, but I expect that they will become even more important as they improve. My colleague Dr. James Short is putting together a conference called Data West at the San Diego Supercomputer Center, and I came up with a list of fears that might disrupt their emergence.

1) If we continue to learn that ALL large data repositories will be hacked from time to time (Experian; National Security Agency), what blowback will that create against data collection? Perhaps none in the US, but in some other countries, it will cause less willingness to allow companies to collect consumer data.

2) Consensual reality is unraveling, mainly as a result of deliberate, sophisticated, distributed, attacks. That should concern all of us as citizens. Should it also worry us as data users, or will chaos in public venues not leak over into formal data? For example, if information portals (YouTube, Facebook, etc.) are forced to take a more active role in censoring content, will advertisers care? Again, Europe may be very different. We can presume that any countermeasures will only be partly effective – the problem probably does not have a good technical solution.

3) Malware, extortion, etc. aimed at companies. Will this “poison the well” in general?

4) Malware, extortion, doxing, etc. aimed at Internet of Things users, such as household thermostats, security cameras, cars. Will this cause a backlash against sellers of these systems, or will people accept it as the “new normal.” So far, people have seemed willing to bet that it won’t affect them personally, but will that change. For example, what will happen when auto accidents are caused by deliberate but unknown parties who advertise their success? When someone records all conversations within reach of the Alexa box in the living room?

Each of these scenarios has at least a 20% chance of becoming common. At a minimum, they will require more spending on defenses. Will any become large enough to suppress entire applications of these new technologies?

I have not said anything about employment and income distribution. They may change for the worse over the next 20 years, but the causes and solutions won’t be simple, and I doubt that political pressure will become strong enough to alter technology evolution.