Accelerated Learning by Experimentation
In most technologies and most industries, experiments play a central role in organizational learning as a source of knowledge and as a check before changes are implemented. There are four primary types of experiments: controlled, natural, ad-hoc, and evolutionary operation. This paper discusses factors that affect learning by experimentation and how they influence learning rates. In some cases, new ways of experimenting can create an order of magnitude improvement in the rate of learning. On the other hand, some situations are inherently hard to run experiments on, and therefore learning remains slow until basic obstacles are solved. Examples of experimentation are discussed in four domains: product development, manufacturing, consumer marketing, and medical trials.
Keywords: Learning, Experimentation
Full published version: Bohn Accelerated Learning by Experimentation. in Learning Curves: Theory, Models, and Applications edited by Mohamad Y. Jaber, CRC Press, 2011.
Preprint version, through SSRN: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1640767.
This is a rewritten version of my 1987 paper about manufacturing. Although I never published that paper, it was very influential in the 1990s, including becoming the (plagiarized) framework of a book and several articles by Stefan Thomke. This book chapter expands and extends the original concepts, including new material from Michael Lapré.
Gina Kolata in the NY Times has been running a good series of articles on fraudulent academic publishing. The basic business model is an unholy alliance between academics looking to enhance their resumes, and quick-buck internet sites. Initially, I thought these sites were enticing naive academics. But many academics are apparently willing participants, suggesting that it’s easy to fool many promotion and award committees.
All but one academic in 10 who won a School of Business and Economics award had published papers in these journals. One had 10 such articles.
Two emerging technologies are revolutionizing industries, and will soon have big impacts on our health, jobs, entertainment, and entire lives. They are Artificial Intelligence, and Big Data. Of course, these have already had big effects in certain applications, but I expect that they will become even more important as they improve. My colleague Dr. James Short is putting together a conference called Data West at the San Diego Supercomputer Center, and I came up with a list of fears that might disrupt their emergence.
1) If we continue to learn that ALL large data repositories will be hacked from time to time (Experian; National Security Agency), what blowback will that create against data collection? Perhaps none in the US, but in some other countries, it will cause less willingness to allow companies to collect consumer data.
2) Consensual reality is unraveling, mainly as a result of deliberate, sophisticated, distributed, attacks. That should concern all of us as citizens. Should it also worry us as data users, or will chaos in public venues not leak over into formal data? For example, if information portals (YouTube, Facebook, etc.) are forced to take a more active role in censoring content, will advertisers care? Again, Europe may be very different. We can presume that any countermeasures will only be partly effective – the problem probably does not have a good technical solution.
3) Malware, extortion, etc. aimed at companies. Will this “poison the well” in general?
4) Malware, extortion, doxing, etc. aimed at Internet of Things users, such as household thermostats, security cameras, cars. Will this cause a backlash against sellers of these systems, or will people accept it as the “new normal.” So far, people have seemed willing to bet that it won’t affect them personally, but will that change. For example, what will happen when auto accidents are caused by deliberate but unknown parties who advertise their success? When someone records all conversations within reach of the Alexa box in the living room?
Each of these scenarios has at least a 20% chance of becoming common. At a minimum, they will require more spending on defenses. Will any become large enough to suppress entire applications of these new technologies?
I have not said anything about employment and income distribution. They may change for the worse over the next 20 years, but the causes and solutions won’t be simple, and I doubt that political pressure will become strong enough to alter technology evolution.
It has always been clear that Musk does not understand high-volume manufacturing. Building rockets is very hard, but building 100,000 cars is very hard for a different reason! His predicted ramp rate was absurd. In the last 6 months, I think he has started to realize this.
Tesla has little chance of hitting its 5,000 weekly output during the fourth quarter. The chief reason: Its current production line can’t build vehicles at that rate unless it runs two 10-hour shifts seven days a week, which is
According to the article, Tesla has also deliberately ignored much of the accumulated wisdom about how to ramp in auto production. That might be OK for his second high-volume vehicle.
More details on Tesla’s ramp plans:
Tesla now has 2 choices, both bad:
- Go ahead and start building and shipping as fast as possible. The result will be multiple problems that require expensive hardware recalls.
- Add another 6? months to the schedule to run the as a pilot line for learning, rather than for volume. Expect zero salable output during that period. (As one of the comments said, they can give/sell those cars to employees.)
Added December 27: Tesla “still in manufacturing hell.”
Here is a comment on that article: Musk needs to face up to having made a MAJOR mistake when he skipped some steps in the original manufacturing ramp-up.
He is probably also making another major mistake at present: adding new machines to the manufacturing process, before he has the existing machines working perfectly. This seems logical to people with no manufacturing experience, but it does not work. For one thing, it diverts his key resource, which right now is manufacturing engineers.
Warning: this post is entirely opinion about American politics.
Bret Stephens had an interesting op-ed in the NY Times recently. On first reading, it was great. Then I went through the comments, and realized it was quite one-sided. (He is a conservative, over from the WS Journal.) So I wrote the following letter to the editor.
In his column of Sept. 24 Mr Stephens sharp eye noticed, and sharp tongue castigated, only the left’s fundamental error in today’s discussions: judging arguments based on the speaker’s identity. But even more destructive is the fundamental error found primarily on the right: judging arguments based on the desire to believe them. That Congressman R believes something, no matter how strongly, does not make it true, nor a valid basis for setting policy.
I am at a university that emphasizes science and engineering, and teaches little about Mr. Stephens’ Great Books. But we teach our students that objective reality exists, and that it matters. We base our arguments on empirical evidence. And if evidence is insufficient, we look for more.
Here are a few examples of facts that are somehow viewed as controversial: making contraception and information more available to teenagers reduces unwanted pregnancies, and abortions. (See Colorado for a large-scale proof.) Vaccinations reduce disease. Cutting income taxes of the rich will do little to stimulate the economy when the economy is near full employment. Pumping gases into the atmosphere creates a “greenhouse effect.” There is room to disagree about what actions to take as a result of these facts, but not about the facts themselves.
I have elsewhere argued that America (and other parts of the world) are retreating from Reason back to Faith, reversing the Enlightenment of the 1600s. If this continues, the consequences for our country will be dire. But that is a longer discussion.
Ever wanted to play sound through multiple audio devices on your Mac OS X system? It cannot be done with the normal Mac controls, but to my surprise there is a decent sound mixer built into the base Mac OS.
Step by step instructions: Play sound on multiple devices, including Internal Speakers, on OS X | Best Mac Tips
I have this setup for my 90 year old mother, so we can all watch TV at once:
- HDMI from Mac to our Panasonic TV
- Headphones plugged into the earphone jack on the Mac
- Closed captions turned on for TV
The result is that we can turn her volume way up, while we listen over the TV’s internal speakers. The headphones, even at maximum volume, may not be quite loud enough for her. In that case, I will add a $25 earphone amplifier into the system.
Still missing: I cannot find Bluetooth headphones that are loud enough for her.
Second, I don’t know of anything similar for her phone.