Making more sense with numbers, part 8

I'm not a professional statistician; I am a professional who uses statistics in the course of my work. Increasingly I'm drawn to Bayesian approaches. Various people have asked me what Bayesian statistics is; when I was asked for the elevator speech version recently, I was stumped. I'll try to make up for it here.

Statistical problems have three parts: the setup, the calculation, and the presentation of results. By my understanding, Bayesian and classical (frequentist) statistics differ in all three.

In the setup, Bayesian statistics starts with the development of a probabilistic model and a set of prior probabilities for the parameters of interest. Classical statistics seems to start with the development of a null hypothesis (what if there is no effect from whatever intervention being considered) and an alternative hypothesis. There's a difference in how one considers information one has before the data collection starts. Some have taken Bayesian approaches to task for the sometimes subjective form of those prior probabilities, but others have pointed out that classical approaches also have their subjective moments in assuming that the particular nature of the classical assumptions apply in a particular situation. Some point out that one can pick prior probabilities in a way that doesn't rely on subjective assessments; those tend to be the weakly informative priors you can read about. I'm intrigued by this part of the difference, but it's not the telling difference for me.

In the calculation, the classical statistical approach relies on selecting the appropriate test to decide if one should accept or reject the null hypothesis or to calculate confidence intervals for parameters of interest. As some have pointed out, this is not always an easy task, and the tests are not always easily matched to complex problems. With unique problems, one may have to modify the problem to match the method or invent new methods to match the problem.

The Bayesian approach relies on basic probability models, which makes it easier to develop an approach that meets the specific problem at hand. This is a telling difference for me.

There is a problem. Except for the simpler cases (for example, see the original original Making sense with numbers), it's often hard to carry out the integration involved in making the calculations. Markov chain Monte Carlo (MCMC) approaches make that much more approachable, but they're not things one carries out on the back of an envelope.

Finally, there's the presentation of the data. This, too, is telling for me. While the classical approach gets tied up in explaining precisely what it means to reject the null hypothesis or what a confidence interval means, the Bayesian result means exactly what most of us likely think when we hear a statistical result: it states the probability of a particular event we care about happening.

I'm still looking for a short, easy-to-read but complete elevator speech from a statistician on the topic that's consistent with some of what Andrew Gelman writes (I think he has some excellent writing on the subject, but I'm not sure I've found anything that fits the elevator speech model). In the meantime, Bayesian Statistical Inference for Psychological Research may help some begin to understand, even as it's somewhat old chronologically. Some might enjoy Why we (usually) don't have to worry about multiple comparisons. shows a simple but powerful application of Bayes Theorem, although it's rather more simple than what one would recognize today as Bayesian analysis.

Objections to Bayesian statistics actually does contain an elevator speech about Bayesian inference, even if it is a bit mathematically concise: "'Bayesian inference' represents statistical estimation as the conditional distribution of parameters and unobserved data, given observed data."

It's a bit longer than an elevator speech, but Dr. David Lucy of Lancaster University does have a short introduction to Bayesian methods that may help; it's part of his CFAS415a course materials.

If you've got a great but simple introduction that can explain the difference between Bayesian and classical inference well, please add it to the comments here! Thanks.

Gender and diversity

At a recent Bainbridge Graduate Institute Intensive, I found a short note affixed to my nametag. It was placed there by their Diversity & Social Justice Committee, and it read, "Men who want to support women in our struggle for freedom and justice should understand that it is not terrifically important to us that they learn to cry; it is important to us that they stop the crimes of violence against us." (Andrea Dworkin)

I would write more, except that I think Dworkin said it well.

A systems take on math and science education?

Richard Hake, Emeritus Professor of Physics, Indiana University, recently posted an article describing how US colleges and universities are gradually coming to the view that they can't simply blame US secondary schools for the quality of math and science education incoming college students have, for the teachers and administrators of those secondary schools are themselves almost all products of the US college and university system.

This seems like closed-loop (feedback) thinking in action. Check out his Mobilization for Math/Science Education - Role of Higher Education.

Recognizing one's errors

Justin Kruger and David Dunning of Cornell University published Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments.

My first reaction is that no one here is subject to the Dunning-Kruger effect; we're brighter than the bank robber described in their first example. My subsequent thought is that I may be letting us off the hook too easily; perhaps we're all subject to the Dunning-Kruger effect in the right domain. None (well, perhaps darn few) of us are highly competent in everything, but we still may be tempted to make pronouncements in knowledge domains where our expertise lags that of our peers. That conclusion is scarier. Knowing oneself is apparently not easy.

Read their article to get some ideas how to test our thinking, and compare that to my earlier postings on scepticism. This sounds related to the idea of confirmation bias, or maybe it's similar to the Lake Wobegon effect.

How do we get around this problem? As best as I can see, life-long learning plays a key role, for it fits with their prediction 4. I suspect careful observation and reflection can help, too, for that might help us recognize our abilities.

Your thoughts?

Thanks to RealClimate for pointing out the article via the Wikipedia article.

Causality

When we evaluate something, we typically are trying to understand and make claims about causal relationships. When we create a system dynamics model, we are mapping and modeling causal relationships. But how do we tell what relationships are causal and which are correlational?

Thanks to a recent pointer on the evaltalk mailing list, here's Sir Austin Bradford Hill's “The Environment and Disease: Association or Causation?” Hill gives nine considerations to ponder.

For a rather shorter read, see xkcd's take on causality. Be sure to see the alt tag.

System dynamics applied to music

One of the project teams from last year's system dynamics class in the Information School at the University of Washington will be presenting their work at the International System Dynamics Society Conference this summer in Albuquerque, New Mexico. Look on the tentative schedule for "Exploring the Dynamics of Music Piracy" by Trond Nilsen, Brian Houle, Douglas Kuzenski, and Arpan Sheth, or check out their abstract, paper, and models.

Congratulations Trond, Brian, Doug, and Arpan! For the rest of you, check out their work. Perhaps it will shed light on a subject you've talked about.

Creating sustainability in complex ecosystems

I recently had the privilege of teaching a course in system dynamic for Willamette University's Sustainable Enterprise certificate program. The course lasted two days, with a follow-up two-hour web seminar. We focused on qualitative system dynamics, but we treated it at a somewhat more rigorous level than many such courses, I think.

I'm writing because of one particular lesson I learned—we all learned. Early in the course, we used a simulation game to help people have a common, shared experience of interacting in a challenging system environment.

As with many such games, the expected result is that people fail in making the system work. Typically, the debrief is used to help people understand the ways of thinking that led them into trouble and to prepare them for the material that's to come.

Unexpectedly, this class managed their challenges quite sustainably. While their skill wrecked the planned flow of that part of the session, I was really pleased to see their skill in action. We spent some time talking about what made them successful and how that might carry over to real-world situations. Their insights were useful enough that I wanted to share them (with the students' permission) with a larger audience: you.

I first asked what made them succeed in the game and what provided the most challenges.

Goals were the first. While the game tells them the goal they should have, they rapidly realized that focusing on the stated goals would lead to ruin, and so they decided to set a much longer-term goal.

Communications was the second factor. After the first round, they began to spend most of their time huddled in the center of the room, talking animatedly through their decision-making processes instead of working in isolated teams.

They noted that delays provided a key challenge. As they worked to establish trust in the social system they had set up, they were both trusting other teams' commitments and verifying that they were indeed living up to their commitments. That takes time: commitments made today may not show up for quite a while.

Those delay effects were complicated by the natural delays in the system. Without revealing the game we used, I will say that the dynamics of the game included natural delays between actions and results that complicated decision making.

Some noted this seemed analogous to the situation OPEC finds itself in. They rely on mutual agreement to limit production as a way to manage prices. If anyone in OPEC breaks that agreement, the system can collapse. OPEC's problems are complicated by uncertain demand and uncertain prices, factors that had no analogy in our game.

Math skills created another success factor, which some may find surprising. A subset of the players rather immediatedly began developing quite a useful understanding of their system based on a mathematical model they developed. Once others saw that their results were accurate, everyone became driven by the data. Without some in the group being able to pull that off, they would likely not have succeeded.

Interestingly, trust and math worked together. At one point, the analyst team made a numerical error and then made an especial effort to communicate that they had made that error to others so that the others would be able to differentiate that error from a breaking of the trust relationship. Apologies were key. Information and the lack of information thus played a key role in the group's success. Even then, it took time for the others to regain their trust in the analysts' team.

Playing into this was the lack of external shareholders. Everyone on the teams had a serious take in the workings of the game; no one was in it just for the "money." Similarly, there were no new entrants into the field who might have upset the cartel relationship they had crafted.

I then asked them what they'd advise people in the real world.

Collaboration was the first clear answer. Work together across groups to align goals and actions.

They then said, "knowledge is power." After a bit of reflection and revision, they revised that to "timely, transferrable, actionable knowledge is power."

They felt it was important for everyone to be clear on a vision.

They would encourage people to watch their egos and to be visibly trustworthy.

At one point, in an attempt to test the strength of their commitment (okay, as an attempt to derail their commitment), I as facilitator announced I was the government and was giving them something they really didn't want. (To be accurate, that idea came from Anne Murray Allen, the executive director of the program, who was running the simulation computer.) For a while, I felt as if I were about to experience the French Revolution, as some rather emotionally argued for standing up to government and refusing my help, a bit of resistance I wasn't accepting.

As a result, their last bit of advice was to "Don't trust the wisdom of government, of the private sector, ... of either." In other words, test the data and the reasoning yourselves instead of blindly accepting what others say is good for you.

This was an intense and very exciting two-day workshop. I think those in the class learned a lot; I know I learned as they taught themselves and me (and now perhaps you) how to make sustainability work.

Perhaps I'll see some of you there next year.

M. King Hubbert on growth

I've written about growth from time to time; perhaps you'd be more interested in what M. King Hubbert said in 1974 in Hubbert on the Nature of Growth.

If you'd like to read more on Hubbert's model, see Evaluating Hubbert's Peak and Improvements? by Ben Witten.

Welcome to Pegasus Communications

Today I'm welcoming Pegasus Communications and their new Leverage Points Blog! They've done much to foster systems thinking over the years, from publishing The Systems Thinker, the free Leverage Points, and many of the major books at all levels of systems thinking and system dynamics to putting on the annual Systems Thinking in Action conference. Check them out.

And thanks, Janice and the others at Pegasus Communications, for listing Making Sense With Facilitated Systems among the august company in your "In the Loop" list.

Bringing Outlook into the Internet age

Until recently, I had only used Outlook for a short period in 1998-99. It was okay, but I only used it inside a corporate context. The feature I most enjoyed was the calendaring and appointments function. I've tried several email clients since then and settled on Gnus as the most useful I've found.

Over the years, I've participated in a number of mailing lists, and I've noticed that some people struggle with what others regard as basic courtesy: failing to trim excess quotations out of replies and top posting are things many seem to get complaints on. I've wondered why that was so hard to get right. (If you're wondering what's the problem with top posting, see the example at the bottom of the Top Posting section on Wikipedia, starting with "A: Because it messes up the order....")

Now I'm back using Outlook in one part of my professional life, even as I continue with Gnus in another. I begin to see why people struggle with some of the basics. It's quite hard to do the basics well in Outlook, including trimming quotes and bottom posting. Seeing real email addresses involves extra work, and the Outlook text editor is limited in its power. After using Gnus for years, I get the impression of Outlook as a tool with limited capability even compared to simpler tools such as Thunderbird. The only advantage I see to Outlook is its appointment tracking, and one can do that in multiple ways today including with the free Google Calendar.

Yet I realize most Outlook users have little say in which client they use. I recently found a tool that seems reliable and does help offset Outlook's weaknesses: Outlook QuoteFix (there's also a version for Outlook Express, although I've never used it nor OE). If you use Outlook and communicate with people on mailing lists or with people who don't use Outlook, check it out. It's been quite unobtrusive so far and lets me treat email either in the Outlook fashion or in ways I've grown comfortable with over the past 20+ years of using Internet email.

PS: Gmail didn't do it so well, either, the last time I tried it; it put the cursor at the top of replies. In researching this article, At least it's not too hard to do it manually in Gmail. I found Gmail bottom posting in replies, which seems to promise help.

Cause for optimism: women on the move

I'm sometimes leary of people claiming that either women or men are better than the other in terms of what they contribute. Perhaps that's because I've heard too many uncredible claims about what men could do that women couldn't (fortunately, I hear those far less today than when I was younger). I also once participated in a most excellent Multi-Cultural Awareness Workshop put on by Sanchez-Tennis Associates that, among many other things, helped me see the strengths (and the weaknesses) of any cultural group, so I'll praise the contributions of various groups while trying never to pit one group against another.

Tonight I heard a video essay by Richard Rodriguez on The News Hour called Women on the Move. Sometimes the news one sees about the environment and about energy can be a bit depressing, but Rodriguez's essay seemed like cause for optimism. Perhaps that couples with what I've been reading from Tom Peters. Seeing more and more women active in all levels of society is indeed good news.

The (un)Sustainable Commentator on growth

Just to keep the question series on growth going, here's what Wayne Maceyka is saying on The (un)Sustainable Commentator.

Check out Wayne's blog, too, and his extensive list of links in the right-hand column.

Nancy White on Sharepoint

I've long said one can work online successfully in almost any environment; it's the approach, not the tool, that makes the difference. Yes, I have my preferences, but they vary a bit by the need of a group, and I've only rarely encountered a tool that just didn't seem to work anywhere I wanted to be.

If you don't know her, Nancy White is one of the leaders in the online facilitation space (she started and still moderates the onlinefacilitation yahoogroup, and she teaches an excellent and very intensive course in online work) as well as a number of other fields. I've known her for about a decade, enjoyed working with her a number of times, and come to trust her judgment highly.

She recently posted Tom Vander Wall Nails My Sharepoint Experience, which claims "SharePoint is a silo builder, not buster." I've never even seen Sharepoint, but I do sense that her words are worth considering when you're considering a tool to foster community.

Check out other parts of her blog, too; you might find you like it.

Incidentally, David Woolley's Thinkofit Web Conferencing Review has long been a classic place to see what online tools exist.

Questions on growth: a follow-up

After all of our good questions about growth, we've gone silent, so I was about to change topics. Then I saw The Growth Bubble on Tom Fiddaman's MetaSD and its link to Thomas Friedman's The Inflection Is Near?

It's a Friday night after a long week, so I'll leave it to you to make the editorial comments.