Software, Design Defects Cripple Health-Care Website – WSJ.com.
Poor software design is still common. I notice the developer was Experian, a private company. Outsourcing the web system for the Affordable Care Act was the right idea, but looks like they picked a weak company.
It will be interesting to get a post-mortem in a year or two. I hope someone writes it up for the New Yorker. It should make a good case study on software product development.
System is down…
A recent exchange on Mathbabe’s blog about the meaning of Big Data led me to some insights about where decisions need human judgment and analysis, and where we can turn decisions over to automated data mining. For example, serving up “you might also like X” in a web store will work a lot better than estimating how many people have flu. Why?
Here’s what I wrote. (Not clear if her WordPress interface picked it up.)
Cathy, big data in your sense does not work widely. If you say that “no human judgment is needed,” this is approximately equivalent to “the relationships do not need to be supported by causal theory, just by raw correlation.” This works great in certain domains. But the underlying correlations have to be changing relatively slowly, compared to the amount of data that is available. With enough data for “this month,” an empirical relationship which holds for multiple months can be data mined (discovered) and used to make decisions, without human judgment.
But many of the world’s important problems don’t have that much stability. For example trying to use searches to track the spread of an annual flu, at the state-by-state level, won’t be very reliable without human judgement. The correlation between search terms and flu incidence in 2012 is not likely to be the same in 2013. One reason is that news cycles very from year to year, so in some years people are more frightened of the flu than other years, and do more searches. Consider the following experiment: use the “big data relationships” from 2010, to track the incidence of flu in 2014. It won’t work very well, will it?
On the other hand, if you could get accurate weekly data about flu incidence, the same methods might work much better. Using the correlations between search terms and flu in November might give reasonably accurate estimates in December.
Automated systems based on data mining are a form of closed-loop decision systems. (Closed loop basically means “no human in the loop.”) Closed-loop feedback works great under certain conditions, and very poorly under others. A key difference is whether the system designer has sufficient (accurate) knowledge about the system’s true behavior.
Once again “it all comes back to knowledge.”
This is the “entry page” for my paper on the slow adoption of better flying methods in WW 2. Please link to this page, rather than to the actual PDF, which I will be updating. Here is the paper itself. (July 19 version)
In the late 1930s, US military aviators in the American Army and Navy began using aviation checklists. Checklist became part of a new paradigm for how to fly, which I call Standard Procedure Flying, colloquially known as “flying by the book.” It consisted of elaborate standardized procedures for many activities, checklists to ensure they key steps had been done, and quantitative tables and formulas that specified the best settings, under different conditions, for speed, engine RPM, gasoline/air mixture, engine cooling, and many other parameters. This new paradigm had a major influence on reducing aviation accidents and increasing military effectiveness during World War II, particularly because of the rapidly increasing complexity of military aircraft, and the huge number of new pilots. Continue reading
I have just finished a working paper called NOT FLYING BY THE BOOK: SLOW ADOPTION OF CHECKLISTS AND PROCEDURES IN WW2 AVIATION. It tells how, in 1937 shortly before World War 2, the American air forces invented a much better way to train new pilots, and to fly complex aircraft and missions. What they invented is now used all over the world, by all licensed pilots and military aviators. But during the war, even American pilots resisted switching to the new way of flying. The only full-speed adopters were the strategic bombing forces attacking Germany and Japan. The US Navy, despite being one of the 1937 inventors, did not fully make the switch until after 1960!
Precise flying was a matter of life or death.
Solar Industry Anxious Over Defective Panels – NYTimes.com.I had not solar panel quality was becoming such an acute issue “so soon.” Judging by this NYT article, many Chinese-branded PV panels are not reliable. This article sounds straight out of the book that Barry Naughton pointed me to, Poorly Made in China. The performance degradation data on well-made panels is pretty encouraging: 0.5% per year is typical, but the key is well made. There are many manufacturing shortcuts and quality problems that will lead to failure of electrical connections after a few thousand temperature cycles, for example. (Think night/day in Colorado!)
testing solar panels
Power inverters, which are straight power semiconductor products, apparently may also be unreliable. https://www.greentechmedia.com/articles/read/3-Reasons-Why-Chinese-Solar-Inverters-Cost-Half-of-American-Inverters
It will be interesting to see what this problem with Chinese panels leads to in trade/market share. California and other states that subsidize PV should only pay for systems that pass good certification – for both performance and safety. For obvious reasons, testing long-lifetime behavior of electronics is very tricky.I wonder if we will see a repeat of the “solar water heating” fiasco of the 1980s, when lots of houses put pool heaters on their roof that started to leak and ended up getting ripped out. When the economics of a project are based on a 20 year life, and it only lasts 5 years, that is a colossal fail. If it catches on fire, as described in the NYT article, that is another situation entirely! What is the typical guarantee for homeowners in California?
Physics Proves It: Everyone Should Shoot Granny-Style | DiscoverMagazine.com.
I’m writing about the slow and uneven adoption of Standard Procedure Flying (SPF) during World War 2. SPF is superior, especially for newer pilots (which was who fought the war from 1943 onward, after the experienced ones were killed). So why didn’t everyone use it?? Analogy: free throw percentages in basketball are higher with underhand throws; but nobody uses it! Rick Barry is a legend who used it; but not even his sons followed him.Barry makes free throws into a “standard procedure.”
B-17 Throttles (Photo credit: rkbentley)
On Sunday I gave a capstone talk at the Production & Operations Society meeting in Denver. I oriented my talk toward a comparison of health care now, with aviation’s transition to Standard Procedure Flying in the 1940s and 50s. BOHN POMS Standard procedure flying 2013e
As in medicine now, experienced expert flyers who did not use standard procedures were still better than newly trained pilots who did. And there was resistance to the changes. But aviation had a couple of advantages in making the transition: New pilots who did not learn SPF died quickly, usually in accidents. And the old experts got rotated out of combat positions (United States Army Air Force), or eventually got shot down no matter how good they were. (Germany)
Art and Science in Technology - Roger Bohn's Blog 