Is education always behind the times?

When I was at the university, I majored in electrical engineering and math. Encouraged by Michael Morrison, my physics major roommate at the time, I took an excellent English course taught by Dr. David Minter that was intended for English majors. Normally I would have been leary of my chances at a decent grade, but this was the start of pass-fail options at my school.

We read and wrote a report on a major novel a week, as I recall. One of those was The Education of Henry Adams. Recently, I picked it off the shelf and re-read it. It's amazing how much more sense it made, now that I have a few more life experiences. I was glad to have read it at the time; I was glad to read it again this time.

Within the last few months, I watched L'armée des ombres (Army of Shadows). One of the lessons I drew from both of these is that our educations don't prepare us well for the world in which we find ourselves. The Resistance fighters had to kill a traitor, their very first person to kill. Adams, prepared mentally and culturally for the eighteenth century, had to prepare himself for the beginnings of the twentieth.

Perhaps the lesson is that our educations, as important as they are, always teach us about the problems of the last generation. Our challenge is to apply those and more insights and wisdom to the needs of today. Our challenge is to learn from our education how to learn ourselves, how to complete our education, how to rise to the challenges we face. Taken at face value, my education prepared me for a world that's no longer visible. Perhaps the key thread that ran through my education (and perhaps yours) was the concept that education was not about learning something. Education was about learning how to learn what you needed to learn for the future.

For us, I suspect the needs of the day include figuring out how to live together on a planet that seems increasingly small and learning how to live in an age that is facing climate change, the end of oil, and a transition to equilibrium (or so we can hope).

What is the education of me? What is the education of you? Can we help each other in this cause?

Recognizing one's errors

Justin Kruger and David Dunning of Cornell University published Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments.

My first reaction is that no one here is subject to the Dunning-Kruger effect; we're brighter than the bank robber described in their first example. My subsequent thought is that I may be letting us off the hook too easily; perhaps we're all subject to the Dunning-Kruger effect in the right domain. None (well, perhaps darn few) of us are highly competent in everything, but we still may be tempted to make pronouncements in knowledge domains where our expertise lags that of our peers. That conclusion is scarier. Knowing oneself is apparently not easy.

Read their article to get some ideas how to test our thinking, and compare that to my earlier postings on scepticism. This sounds related to the idea of confirmation bias, or maybe it's similar to the Lake Wobegon effect.

How do we get around this problem? As best as I can see, life-long learning plays a key role, for it fits with their prediction 4. I suspect careful observation and reflection can help, too, for that might help us recognize our abilities.

Your thoughts?

Thanks to RealClimate for pointing out the article via the Wikipedia article.

Causality

When we evaluate something, we typically are trying to understand and make claims about causal relationships. When we create a system dynamics model, we are mapping and modeling causal relationships. But how do we tell what relationships are causal and which are correlational?

Thanks to a recent pointer on the evaltalk mailing list, here's Sir Austin Bradford Hill's “The Environment and Disease: Association or Causation?” Hill gives nine considerations to ponder.

For a rather shorter read, see xkcd's take on causality. Be sure to see the alt tag.

The (un)Sustainable Commentator on growth

Just to keep the question series on growth going, here's what Wayne Maceyka is saying on The (un)Sustainable Commentator.

Check out Wayne's blog, too, and his extensive list of links in the right-hand column.

Questions on growth: a follow-up

After all of our good questions about growth, we've gone silent, so I was about to change topics. Then I saw The Growth Bubble on Tom Fiddaman's MetaSD and its link to Thomas Friedman's The Inflection Is Near?

It's a Friday night after a long week, so I'll leave it to you to make the editorial comments.

The chicken or the egg?

Perhaps you've been wondering what in the world system dynamics is good for.

System dynamics can help you answer the question, "Which came first: the chicken or the egg?", which you can only do if you look at both the chicken and the egg at the same time.

I was reminded that I wrote that back in 2003 when I saw Tom Fiddaman's SD on Long Waves, Boom & Bust. Click on my name "Where are we in the long wave?" in that post to the the thread in which it occurs.

The metric system and math skills

I once suggested that eliminating the use of the customary system of units in this country would help us learn and compete. Now Richard Slettvet, a local teacher, has made the same point in A logical way to improve math scores.

Is it time to act? Who will you engage?

Feel free to tell others about this post.

Cool tool

If you like making sense of (or with) numbers and use Linux, check out Qalculate!. The screenshots give you an idea of its power and ease of use.

Number 500

I made my first blog posting in May 2004. Then I waited until October of that year to make my second. It started a trend of including environmental issues, not in addition to business issues but because I think environmental issues are business issues. Finally, in March 2005, I began blogging on a more regular basis.

Now, not quite five years after I started, I've reached my 500th posting. I've enjoyed getting to know some of you along the way, and I've enjoyed sharing ideas and getting your comments, insights, and feedback. I invite you to share this link with others and to continue to share your ideas with me. Thanks!

Good graphs

Doing graphs well is important for communicating information (you do use graphs, don't you?). Rafe Donahue has published Fundamental Statistical Concepts in Presenting Data: Principles for Constructing Better Graphics. I think it's well worth our time to read and heed.

Thanks to Andrew Gelman for the tip.

Questions on growth: a pre-evaluation

You've helped me accumulate quite a list of questions about growth, and I thank you. Here they are once again:

1. What if (aggregate) growth went to 0% forevermore? What would it mean to you, to your business, and to your personal life?
   
   
2. Do you think there's reason to believe growth could stay at 0% or below for a very long time? Is that of necessity either good or bad, or does its value depend upon our reaction?
   
   
3. Is it a good idea to try to keep growth positive? Why? Are there any downsides? Are there indeed any limits to growth, either in terms of annual growth rates or the overall size of anything?
   
   
4. As attractive as growth may be in the current worldview, systems ideas would indicate that the longer we try to keep growth going once we've exceeded the carrying capacity of the system (the planet), the worse the eventual and inexorable fall and the lower the eventual sustainable standard of living. If you favor continued growth, how will you overcome those seemingly inviolate systems limitations?
   
   
5. If the systems theories play out in the real world, how do we reasonably make the transition from our current state to a new, equilibrium state in ways that attend to people (social justice, the ability to procure what we need for life, the ability to make a difference or find purpose, etc.) and the natural environment (sustainability, the depletion of nonrenewable resources)?
   
   
6. Can we make such a transition a good thing and not a painful thing?
   
   
7. What do we owe our descendants? For how far into the future do we bear responsibility?
   
   
8. If these systems ideas have merit, what changes in mindset (in worldview) do we need to survive emotionally as well as physically? For but one example, negative growth has long meant failure for people leading businesses, but that could be the way of the future. Can we realistically change our mindsets and our systems so that satisfying needs (instead of generating growth) defines success?
   
   
9. In case these systems ideas don't apply in this situation (e.g., if technology can once again save us even in the face of decreasing energy supplies and rising population), can we design robust actions that work well in either eventuality? How do you answer Tom Fiddaman's questions about the sufficiency of technology?
   
   
10. Can the current economic "engine" be morphed into one based on such a radically different paradigm?
    
    
11. If we change to a different paradigm that's not built on growth, can we figure out how to get money into the hands of all who need it? Or will our reaction to low or zero growth be to trim people out of companies to keep the organizations viable while building unemployment?
    
    
12. Is an economy built on lending inexorably drawn to growth for survival?
    
    
13. How we to encourage understanding that quality of life can still improve while quantity of consumption decreases?
    
    
14. In light of the current economic crisis, how can we best protect the economy's life-supporting functions such food production, health care and ecosystem services amidst the chaos that will undoubtedly trim the less important financial and luxury markets?
    
    
15. What new national and international policies and institutions do we need to design in order to prepare for a transition to a steady state, or true cost, economy that recognizes the need for investments in natural and social capital as well as financial?
    
    
16. What would constitute the analogy of complex relationships those with "imaginary" components?
    
    
17. Will a state of zero net growth become a state of dynamic economic equilibrium, and will this new state actually make markets MORE efficient, and effective at elevating the state of the common man?
    
    
18. Why is our world so hung up on growth to begin with? How did growth get into our DNA? Has it always been there or is it just since the invention of the steam engine?
    
    
19. What do you do if evolution favors individuals or groups who aspire to growth?
    
    
20. What if the US and EU go green and China and Russia don't?
    
    
21. What if growth had to be -X% per year for Y years in order to reach a sustainable steady state (in material throughput)? How might social systems accommodate that peacefully?
    
    
22. What if technology has limited potential?
    
    
23. What would an evolutionary landscape that favored sustainability look like?
    
    
24. Can wealth can go up while the material flow goes down? How?
    
    

I've numbered them this time for easy reference. If you want to see who contributed each question, refer back to the original posting.

Now the next step: how would one go about answering these questions?

Before getting to the process, though, I'm curious how you think we would recognize a good answer. I don't expect that we'll all agree, but, rather than getting into a bunch of statements about our respective positions, I think we might learn more by first thinking about the criteria by which we'll evaluate potential answers.

I suspect our answers to that may be all over the map. As food for thought, I offer up the questions critical systems heuristics offers about motivation, control, expertise, and legitimacy. You can find one article by Bob Williams and Martin Reynolds on Bob's Web site: go to Systems Stuff and scroll down to the article called Critical Systems Thinking. You can see another introduction by Werner Ulrich here. He lists the questions starting on page 11, but you probably need to read the earlier pages to understand what it all means.

Once we have some idea how we'll evaluate potential answers (and how we think we should evaluate them), then we can think about picking approaches, methods, methodologies, or processes we think might be of use for each of these 24 questions.

So what do you think? How will you evaluate the answers to these questions? How do you think you should evaluate them?

Questions on growth

From time to time, I've posted some of my musings about growth. I had a related email discussion recently with a colleague, and one of my emails to him contained questions that I'd like to ask you, too.

We know the world's economies are currently experiencing hard economic times. Yet I wonder if thinking of this in terms of business cycles is the best way today. Jay Forrester has written, "Our greatest challenge now is handling the transition from growth to equilibrium".

If you think of the world in terms of business cycles, getting from bad times to good is a largely waiting game, assuming you've saved enough in the good times to carry you through the bad. Before refrigeration and long-distance transportation of food, that happened on an annual basis, too: you had to save enough food from this year's harvest to survive to next year's. Admittedly, some organizations are good at gaining ground during these periodic "yellow flag" situations, but many of us will just try to survive until the good times return.

If Forrester and others are correct and we're really moving from a long period of growth into an even longer period (possibly forever, at least in practical terms) of relative equilibrium, then dealing with those (equilibrium) times may require a fundamentally different mindset (as Cynthia McEwen and John Schmidt would remind us), not just a changed process.

• What if (aggregate) growth went to 0% forevermore? What would it mean to you, to your business, and to your personal life?


• Do you think there's reason to believe growth could stay at 0% or below for a very long time? Is that of necessity either good or bad, or does its value depend upon our reaction?


• Is it a good idea to try to keep growth positive? Why? Are there any downsides? Are there indeed any limits to growth, either in terms of annual growth rates or the overall size of anything?


• As attractive as growth may be in the current worldview, systems ideas would indicate that the longer we try to keep growth going once we've exceeded the carrying capacity of the system (the planet), the worse the eventual and inexorable fall and the lower the eventual sustainable standard of living. If you favor continued growth, how will you overcome those seemingly inviolate systems limitations?


• If the systems theories play out in the real world, how do we reasonably make the transition from our current state to a new, equilibrium state in ways that attend to people (social justice, the ability to procure what we need for life, the ability to make a difference or find purpose, etc.) and the natural environment (sustainability, the depletion of nonrenewable resources)?


• Can we make such a transition a good thing and not a painful thing?


• What do we owe our descendants? For how far into the future do we bear responsibility?


• If these systems ideas have merit, what changes in mindset (in worldview) do we need to survive emotionally as well as physically? For but one example, negative growth has long meant failure for people leading businesses, but that could be the way of the future. Can we realistically change our mindsets and our systems so that satisfying needs (instead of generating growth) defines success?


• In case these systems ideas don't apply in this situation (e.g., if technology can once again save us even in the face of decreasing energy supplies and rising population), can we design robust actions that work well in either eventuality? How do you answer Tom Fiddaman's questions about the sufficiency of technology?
  


• What other questions do you have?

Let me be clear: aggregate growth of 0% does not imply that there is no dynamism, no growth in the economy. Our need for certain products and services will increase, even as our need for others will decline. I'm only thinking of aggregate growth in posing these questions.

I have lots of questions and only tentative answers. Right now, I really am interested in seeing your questions, as I've seen the power of listing questions before trying to provide answers.

Later, we'll get to considering answers and how we might test those answers for reasonableness and usefulness.

Questions?

Sustainable Energy without the hot air

You may have noticed it's sometimes hard to get good data on issues of current importance. We read and hear adjectives, but we too rarely hear numbers. When we do, they're often presented in ways that are not conducive to clear understanding. I've written about that from time to time, for grounding decisions in good data seems to be a fundamentally important skill.

I've also written about the environment, for I think we do and will face challenges of the sort our ancestors never had to address (There: more descriptive phrases! Relief is on the way.).

Today Andrew Gelman pointed to David MacKay's free book Sustainable Energy -- without the hot air as an example of a book that brings data to the fore of the discussion about sustainable energy. In general, he likes the way the data is portrayed, although he doesn't attempt to vet the book for its content. While I haven't yet double-checked any of the numbers, I have begun to read the book, and I find the data clearly, cogently, and interestingly put (quite a change from William Farr's advice to statisticians). I like that he seems to use a significant number of clear time series ("behavior over time") graphs, and the time horizons are long enough to see useful patterns developing. So far, it appears as if this work will help me put events in perspective; I'll be curious to see what I learn and what reactions I have as I finish the book.

While I'm reading it, I encourage you to get a copy, too, and see what you think. Perhaps we'll all learn something important, both about living on the planet successfully and about presenting data effectively.

Tabbloid!

So you'd like to get the news delivered to your door daily, but you'd really like to pick and choose what news that is. Perhaps, as I've suggested before, you'd like a broader range of stories, perhaps seen from viewpoints in your country and in others. Perhaps you'd like more emphasis on one field than you get in your local newspaper.

Now you can get that delivered to your inbox for free from Tabbloid, an HP development. It's simple, and you get a nice, letter-sized PDF document on the schedule of your choice. This would be great for public transit commuters: print out your personal Tabbloid before leaving for work, and you have custom news of the day you can finish by the time you arrive at your place of work (or at least by the time you arrive home).

There are risks. Just by taking some interesting selections from my current RSS feeds and generating a sample issue, I got a 24-page document that took about 800kB. I know there are other feeds I'd like to include, and I know there are some feeds I selected I'll probably delete soon (I don't read them all daily; I simply skim for useful material when I have time). If I can print for US$0.05 per sheet, that 24-page document will cost me US$1.20 a day—more than I would spend for the local paper even if I bought it at a newstand (duplex printing gets it down to US$0.60, but that's still more than the US$0.50 newstand price of my local paper).

Spending that much daily could drive me to cancel my newspaper subscription, which is part of a dynamic that moves revenue from the news media to the paper (and ink) industry. None of that revenue gets back to the writers of the news; what effect will that have?

Printing my Tabbloid would consume a ream of paper in about a month of workdays. That generates a lot of waste; even with recycling, that doesn't sound like a great idea.

Don't forget that a piece of paper doesn't have hyperlinks, either. Subscribing to a feed whose articles consist heavily of links won't be of much use here.

There are a few features I wish Tabbloid had:

• It would be nice to be able to arrange the order of feeds in the final product, perhaps by dragging and dropping the list of feeds on Tabbloid.
  
• It would be nice to be able to create Tabbloids and then put them on hold for a while or for specified periods. For example, I might want an international news Tabbloid I would only read occasionally, or I might want a different Tabbloid on the weekend than on weekdays. I can probably do the latter with judicious use of additional email accounts, but I'm not sure about the former.
  

Nonetheless, I'll try it for a few days and see what I think. It is a creative idea from HP, and it could be helpful for commuters who lack the time to catch up on their RSS feeds but have time sitting in buses and trains.

Perhaps this is an alternative to advertising-supported news delivery: instead of paying for Tabbloid through ads, perhaps we're paying for it through printing supplies.

Perhaps I can be selective enough to make a much shorter paper (and I won't be printing it for now, at least).

What do you think?

Making more sense with numbers, part 7

Tim O'Reilly tweeted about Florence Nightingale: The passionate statistician. While I'm not a fan of pie charts, Nightingale's work here is impressive, especially given the date. We can probably ignore William Farr's advice about statistical writing, though.

By the way, I got off on my numbering in this series, giving several postings the number 4. You can now find the real Making more sense with numbers, part 5 and Making more sense with numbers, part 6, as well as earlier postings in the series.

Heretics, skeptics, and cynics: your ideal business partners!

Art Kleiner has written The Age of Heretics, celebrating those who are loyal to our organizations but see reality somewhat differently. Günter Grass wrote Aus dem Tagebuch einer Schnecke, celebrating, among other things, skeptics and questioners. Now the TP! Wire Service points to Working best: Cynicism not always workplace hindrance by Bill Repp of the Organization Development Group.

For more on the topic, see The importance of a focus on disconfirmation, Skepticism revisited, and Skepticism, numbers, and making sense.

Secret tool for online meetings

Have you ever sat in an informal (no slides) Web-based meeting and tried to keep up with what's going on? When the meeting was over, have you wondered what was decided?

Here's a small secret: I've found it helpful to use FreeMind, a free mind-mapping tool, to take notes during meetings. I often share Freemind through the Web-based meeting tool so that it works as a virtual flip-chart: everyone can see what I'm recording, people can suggest corrections, and I can hide parts easily when we're addressing other issues. When the meeting is over, I can convert it to PDF or HTML or any of a number of other formats and distribute meeting minutes with no additional work.

That's very related to Bernie DeKoven's technography (be sure to watch the video to understand how this works).

Try it sometime; you might find you like it.

Dealing with risk and uncertainty

Times are uncertain. Risks seem high. We may feel that the price of missteps is high; we know it's hard to decide what steps to take.

In situations such as this, how do you make decisions in and for your organization? How do you plan effective actions? How do you solve the inevitable problems that arise?

The German psychologist Dietrich Dörner, author of The Logic of Failure, has made a career of studying why people make mistakes and what we can do to improve. One of his key pieces of advice is to use computer simulation to get insight about the situations we face so that we can make better decisions in real life.

Perhaps today's uncertainties are your signal that the time is right to apply more systemic approaches in your work and to ground your planning, problem solving, and decision making with simulation that takes into account factors important to your business. Perhaps it's time to test and rehearse your plans before you implement them.

That's what we've been discussing here, and that's how I help others. If you're concerned that your standard approach to business may need augmentation in today's world, perhaps I can help you, too. Drop me an email or give me a call. There's no obligation—only opportunity.

Road Maps to Peace

Yesterday I blogged about what I see as one of the two big problems we face: how we deal with the load we're placing on the planet. I think the other big problem is how we deal with disagreements among those of us who live on this planet. As the load we place on the planet increases, I think the importance of figuring out how to get along will increase, too.

Some of you may know of Rick Steves as a travel guide who also writes newspaper columns, produces TV and radio shows, owns a travel agency, and sells travel gear. He also has a more serious purpose in life, or, perhaps, as I've written about Bernie DeKoven and sustainability, his more serious purpose is inherent in what he does in his travel business.

On September 6, he hosted Lord Alderdice in a discussion called Road Maps to Peace. Lord Alderdice played a key role in helping people work to end the "Troubles" in Northern Ireland, and he has continued his work by helping others to reduce conflict in their regions.

Rather than me taking the time to summarize Lord Alderdice's message, listen to him directly. The recording is about an hour long. I encourage you to listen carefully, with an open mind, to the end, for there are many good ideas. Often we hear people expressing opinions (or express them ourselves) about how peoples should get along; Lord Alderdice is talking about what has worked in helping peoples to get along together. I found ideas that I can apply in interpersonal relationships as well as in thinking about larger political issues we help decide at the ballot box or in dialog with our elected officials as well as in working with or inside organizations.

One message he gives (but not the most important—I'll let you listen to find the essence of his ideas) is that we should all listen to news from other parts of the world so that we gain a broader perspective. That's a message I've given here before. I've long thought we should strive to have personal friends and professional contacts in multiple parts of the world, for I suspect we'd be more likely to work things out and less likely to go to war with countries where we have those connections.

As with my previous message, I really would appreciate your comments on these ideas.

The Marblehead Letter

I've written about my musings on growth a number of times, even as I worried that my ideas might be controversial.

Now I've seen the Marblehead Letter, written by executives at a SoL conference in 2001, and I think those of you reading this blog might find it worthy of your time. Read both the full letter—it's only two pages long, and I think it states its questions better—and the summary, which hints at some of the signatories.

Note that the letter has questions, not answers, and note that the letter comes from people high in the ranks of major organizations.

I discovered this letter by reading Presence: An Exploration of Profound Change in People, Organizations, and Society, courtesy of InBubbleWrap and 800ceoread. For various reasons, that's been a hard book for me to get through, but I'm persevering (and those of you who have a copy can tell the page I've reached by this blog posting). Perhaps I'll blog more about it when I finish it.

What do you think? It's comforting to know that others are considering similar questions to the ones I've been raising. Question 3 is exactly what I want to work on, but reading it brings two thoughts:

• You can't address that one question in a vacuum; you have to consider their other questions and still more (for example) in the process. It is a systems issue on multiple levels.
  
• I wonder if they didn't go far enough in question 3. They want to reconceive growth. I wonder if and how and under what conditions overall sustained growth is possible and good for us. If, in aggregate, it is not (and I have yet to see evidence that the systems mantra of "there are always limits to growth" is false), I want to help find a new and successful way forward. While we have to address the long-term situation, I'm more interested in helping us figure out how to make the transition from growth to sustainability, whether on an organizational, societal, or personal level.
  

Those of you acquainted with some of the literature on growth will realize that a stable system doesn't mean there is no growth. For example, in a business sense, some technologies, products, or services outlive their usefulness, and their companies shrink or perhaps go out of business. Other technologies, products, or services are needed in increasing amounts, and their companies grow. Equilibrium in the aggregate doesn't require equilibrium in the details.

While I'm optimistic we'll figure out a way to deal with this, I still think the issue of growth is an integral part of one of the two major problems we face as a people. From what I read, we may well have exceeded the carrying capacity of the planet. If that be true, then these are important times, for the way we respond can likely have a major effect on the response of the systems in which we live, and the recovery of a system from overshoot can be harsh.

I really would welcome your comments on this.

Two big problems

I've written before about what I see as the two big, high-level problems we face (it's the weekend—I'll let you search for the articles). This week, I plan to publish articles on both. Monday's article on one is mostly written, and I'm beginning to address the second.

Check back next week!

Twittering away my life

Well, I'm not that into it, but I have started twittering. If you're a Twitter user, sign up to follow me, and I'll likely follow you, then, too.

Stocks, flows, and the President's weight

I've noted before the importance of thinking appropriately about stocks and flows. Janice Molloy of Pegasus Communications just wrote "A Weighty Take on Stocks and Flows" for the August 2008 issue of The Systems Thinker, using stocks and flows to communicate the message of a New York Times column by Gail Collins. It's a good tale; if you subscribe to The Systems Thinker, check it out, or check out the original column.

It was fun talking through the implications of these ideas with Janice, creating a few simple models together to clarify Collins' message, and producing the diagrams that Janice used. There is a message hidden in the fun, though: be sure you understand what they really mean when someone says they'll reduce the rate at which something is growing. It may not be all that good a deal.

Looking from the outside in...seven months and a trip later

Last January, Henrik Müller posted an article called "Amerika steht mit dem Rücken zur Wand" ("America stands with its back to the wall"). Now he's made a trip through the USA and written again in "Amerikas schwärzeste Stunde" ("America's Blackest Hour").

If you read German, check out the article, and let me know what you think. If you don't, here's the short version:

• The USA faces as serious a challenge as we've faced in years, perhaps as serious as the Great Depression.
  
• The difference between this challenge and many of the others since the Depression is that individual people are now impacted in real ways.
  
• While we have some number (he doesn't really guess how many) of tough years ahead of us, we have a way in our culture of getting through tough times successfully. He thinks our situation parallels that in Germany from 1993 through 2005, but he thinks we'll be more successful in breaking out.
  

If you do read German, feel free to add key points you think I missed. You can also read his comments in Google's translation.

What do you think? Maybe more importantly, where do you think we should be headed? What should business look like? How about the economy? How about society? How should we get there? Perhaps a return to 2006 isn't what we want or need for 2010. Feel free to add your ideas below.

Prediction, system dynamics, and Future-Fusion

Recently, I made the claim that we're better off focusing on adapting to the present than predicting the future. I've made similar claims in the past, too. I've even given one example in which predictions serve a useful purpose.

That's all a bit simplistic, of course. Even system dynamicists could be said to predict the future in a way: we show behavior over time we feel is more likely to occur (although we may warn people away from point predictions based on a behavior over time graph). In other words, I might suggest that your current policies could produce a boom and bust effect in your business, but I wouldn't want you to draw the conclusion that your business will grow another 172.3% by June 15, 2009 before taking a tumble that afternoon.

Because we all would like to know the future, I've experimented with blending system dynamics and Bayesian analysis to quantify the probability of a particular behavior pattern, for example. Of course, that probability is conditioned on both the historical data and the model being correct, which is a loophole big enough for a good-sized locomotive to run through: models are always incorrect. Still, I think this approach may give more useful insight in certain cases.

Now Kshanti Greene of Stottler Henke Assocates, Inc. has shown me a Bayesian tool they've developed called Future-Fusion, and I've been exploring it a bit. They are using Bayesian networks and the power of groups to get a better handle on what the future holds. Much as Data360 looks at the past, Future-Fusion attempts to look at the future. As of this writing, they've created four test areas which you can explore: the 2008 US presidential election, the Iraq war, corporate strategy, and energy. Try it out: learn how to use the system, see current predictions, and add your own (I think you only have to create a free account if you want to add your own predictions). Perhaps you'll learn something, and perhaps they will, too.

Kshanti has pointed out a recent addition to Future-Fusion that may intrigue some of you: time. They've enhanced their technology to allow limited dynamic execution of a network model, which begins to narrow the gap between Bayesian networks and system dynamics from the Bayesian network side, much as what I've tried has narrowed it from the system dynamics side. To try that out, go to the energy model, select a prediction (e.g., "Reduced SUV sales"), click "view graph," note the numbers, and then click "Next Time Step."

I think this is all still experimental in many ways, but it's a good opportunity to learn a bit about this technology by trying it out on real-life issues. I'll be curious what you discover.

Why are we here?

Why are we here in business, that is?

You can find lots of answers, and "making money" seems to crop up frequently, at least in informal conversations. Some of you know I used to work at Hewlett-Packard. At least at the time, it was governed to a large degree by "Bill and Dave stories": tales of how Bill Hewlett and Dave Packard responded in certain situations.

Today, thanks to Tom von Alten's note on Corporate blogging, I discovered Anna Mancini's blog called From the HP Archives… (she's the HP archivist) and this Dave Packard Quote.

Why are you in business? Why is your company in business? What is your contribution to society?

What is progress?

It's easy to think progress is measured by GDP, trade balances, or the number of things we have; that's what we read and hear about in the news. Yet there's an undercurrent that suggests such views have it all backwards.

The Glaser Progress Foundation has a program area devoted to measuring progress. Go there to see a video or hear an audio of a 1968 speech by Robert Kennedy suggesting that GDP measures all the unimportant things or to research articles they've assembled.

Thanks to Joost Bonsen's Maximizing Progress for the link. Thanks, too, to Cliff Havener and his Meaning : The Secret of Being Alive. I read that years ago, and I'm pretty certain he makes the point that Lord Kelvin was wrong: all the important things—love, peace, faith, art, ...—share the attribute that they can't be measured by numbers. I've looked, though, and can't find the reference; if anyone can provide me the page number, I'd appreciate it.

Making more sense with numbers, part 4

This could be called Monty Hall and cognitive dissonance. John Tierney just published And Behind Door No. 1, a Fatal Flaw, a brief review of the Monty Hall problem and a report on its potential application to psychology, including its potential to invalidate some prior claims about subjects such as cognitive dissonance.

I'll leave the psychological arguments to others; the point is that thinking carefully isn't always as easy as it seems. If you're not convinced, read the start of that article down to "Before I tell you the answer, I have a request," and then write down your answer before proceeding. Then try out the online version to see if you got the right answer, to get a visceral feel for the game, and to see the reasoning.

Once you get the hang of those, try out Monty Hall’s Other Problems.

Do you now think you've got the hang of it? Just to confuse things a bit more, read Behind Monty Hall's Doors: Puzzle, Debate and Answer?, Tierney's 1991 report of playing the game with Monty Hall. By the end, you may have an even deeper appreciation of the challenge of making sense with numbers.

And if you wonder what this might have to do with business, remember that the impetus for Tierney's column was Yale economist M. Keith Chen's application of the Monty Hall problem to psychology. Are similar gotcha's waiting for us in business?

PS: Yes, there is a Making more sense with numbers, part 3.

The importance of a focus on disconfirmation

Here's a lesson from John Sterman's Business Dynamics: Systems Thinking and Modeling for a Complex World section 1.3.7: we gain little to no new insight by observing cases where data supports our hypotheses. We gain much from testing cases where data might disconfirm our hypotheses.

For more on that, see Raymond Nickerson's Confirmation Bias: A Ubiquitous Phenomenon in Many Guises, Bob Dick's Rigour and relevance in action research, the Skeptic's Dictionary entry on confirmation bias, Wikipedia's entry on the same subject, or one of my prior essays on skepticism.

System dynamics, black swans, and the management of business

I'm currently reading Nassim Nicholas Taleb's The Black Swan: The Impact of the Highly Improbable. While I intend to tell you more of what I think when I'm finished, I have an early impression, based on stories such as what he calls "Hume's problem" (or the turkey problem). That's a problem in which everything seems to be getting better and better, only to change direction suddenly and drastically for the worse. In his example, the turkey sees life as a daily succession of friendly humans offering food, only to have it cut short in a manner seemingly quite out of character for life as the turkey has perceived it. (As Taleb points out, it all makes eminent sense to the butcher.)

I think that's part of the reason for system dynamics as yet another tool for thinking and working. As Geoff Coyle points out in his System Dynamics Modelling: A Practical Approach, top management is concerned about things such as the consequences of actions, the likely future, and robustness against uncertainty (p. 15). One of the basic parts of the system dynamics approach is to challenge preconceived notions of the extent of the system causing the current situation: are we looking over a broad enough time span, are we including enough of the actors and actions, and are we paying attention to feedback effects (what Taleb calls recursive effects on p. xxii), where something we do today might come back and affect the situation we face tomorrow?

While there are no guarantees, that unfortunate turkey, had she had good training in system dynamics (or a competent system dynamicist at her side), might have been inspired to look at life over a 5-10 year time span, not just the few months she had experienced. That might have surfaced the fate that led to her demise as part of a regular pattern (albeit one that occurred rarely compared to her lifespan). Had she looked not only at the friendly human feeding her and the other turkeys eating with her, she might have noticed the butcher looking eagerly over the fence from time to time and asked about his role in her life. Had she realized the implications of those observations, she might have decided not to become quite so friendly with her "caretaker," she might have decided not to eat nearly as much (if she were scrawny, might her fate have been different?), and she might even have encouraged the other turkeys to join her in an escape attempt.

Now I don't think that the use of system dynamics conveys infallibility; in fact, that's why I'm reading Taleb's work, to figure out more places my insights may be fallible so that I can make them more robust.

Taleb advocates tinkering as a way to make progress; I see system dynamics as a way to tinker faster and think more effectively in support of your (and my) goal of more effective action.

While my comments may be out of the main focus of Taleb's thesis (system dynamicists tend to focus on the deterministic, not the random, even as they seek to help you be able to respond better in the presence of the random), I don't yet see them in contradiction. I offer them to you in the hopes they are of use to you. Now it's my (and your) task to try to disconfirm them; the longer we can't, the greater the likelihood there's something worth attending to!

If you want to tinker faster with the situation you find yourself in but don't want to risk your business each time you tinker, let's talk.

Thanks to Andrew Gelman for his posts that led me to Taleb's work.

Looking from the outside in...in English

Realizing that the majority of those reading this blog may not read German, I put together a quick summary of Henrik Müller's arguments to which I pointed last Friday.

In his most recent article, he claims that, in a somewhat healthy economy, we have three feedback loops that would stabilize our economy and dampen out our current problems:

• People and the government would spend more to stabilize consumption.
  
  
• Government would borrow more in order to support its temporarily increased spending.
  
  
• The Fed would lower rates to encourage consumption (and, presumably, investment).

He claims all three are at their limits here. He quotes an OECD number that says our savings rate is -1.0%, and housing values are dropping, so we have nothing left to spend.

He says our Federal budget deficit is only 3% of the GDP, and our debt, at 60% of the GDP, is 60% beneath the norm in Europe, so we could increase the debt to try to pull us out. Unfortunately, because we save so little, the only people who can buy that debt are foreigners.

Finally, while the Fed has room to lower the rate, he sees banks as ready to absorb any excess cash rather than loan it out, and he worries about inflationary pressures that may present, thanks in part to an ever-weakening dollar.

In the current political scene, he sees candidates pushing protectionist agendas and hope, while he sees our real hope as lying in global product and capital markets. In fact, the only good news he sees is that the devalued dollar has increased exports and that foreign governments seem ready to invest huge sums in US banks, and he's worried that we don't see that for the good news it is.

What do you think? If you read German (especially if you read it natively), what important points do you think I missed from the two articles?

Looking from the outside in...

One of the things that doing enough system dynamics work teaches one is the benefit of perspective. Sometimes when you're in the middle of something, it's hard to see the forest for the trees. Standing back a ways and, just for a moment, trying to drop any emotions that are tied up in one's current situation can give one better insights.

That's what system dynamics modeling can often do: change a situation in which you're an intimate part to a situation you and your colleagues can look at with a bit of perspective. It also gives you the ability to test ideas on the model before you test them on the real situation.

When we can't get that perspective ourselves, either because of time limitations or because we can't figure out how to do it successfully, reading or hearing what others say about us can sometimes provide us similar perspective. Sure, those outsiders may not understand our situation as well as we do, at least in the details, but they may help us find a better perspective into which to place our more detailed understanding.

If you live and work in the USA, you've no doubt read much about our economic situation recently. I've suggested before that it's healthy to see how others see us. Recently Henrik Müller of the German business magazine manager magazin posted an article entitled Amerika steht mit dem Rücken zur Wand ("America stands with its back to the wall"), a follow-up to an earlier and more data-filled Nach der Orgie ("After the Orgy"). If you read German, or if Google Language Tools suffices, I encourage you to read these articles. He may not be correct in all his assessments, but he may give a better perspective than the headlines in the nightly news about sub-prime mortgages, foreclosures, and other problems. Certainly his earlier article does something I like to see: he gives graphs of at least some key data over a five-decade span, which is more useful for seeing patterns and gaining perspective than merely seeing what has happened this century.

How do your mental models compare with his? What are you doing in your company to adapt? What should you be doing?

A problem in policy or implementation?

There's a discussion about different approaches to solving organizational problems on a mailing list to which I belong. I posted the following follow-up:

Problems in policy implementation may be due to problems in policy design.

(In this context, a policy is a set of rules or guidelines by which we make decisions. )

While it's taken totally out of context here, I think it's very consistent with Deming's ideas, with the lesson that problems in the user manual for a product may really be a problem with the design of a product, with what I've learned as a manager leading change, and with what I've observed as a consultant: if you get the "system" designed well, the implementation may well become significantly more straightforward.

So if you're not getting the results you want out of your organization and if you're tempted to think the problem lies in the people implementing your policies, think again, just to be sure. It's possible that the problem lies in your policies.

That's actually good news, for it means the problem lies in an area you really do control, one where you really can make meaningful, effective changes.

What do you think?

Top postings of 2007

In the last 12 months (to be precise, from last December 28, the day after the Top postings of 2006 entry through December 26, 2007), you have chosen ten top postings on Making Sense With Facilitated Systems as ranked by unique pageviews in Google Analytics.

As I noted last year, there are potential statistical problems with this list. Those who read my blog every day using the main URL don't get counted; both last year's and this year's tallies were made from those who landed on specific URLs as reported by Google Analytics (but excluding visits I may have made). That may be okay; those who linked to specific pages may have cared more about them. Recent entries have a more difficult hurdle, as they haven't been around as long to be viewed. The dates don't quite line up with the calendar year, although I suspect that makes little difference in the results. If you know of a better way, let me know.

10. For some time now, I've been using an open source simulator for my system dynamics work because it seems to help me think more effectively. That doesn't mean I've given up on commercial tools; I still use iThink for creating interactive environments, and I will be teaching IMT 586 at the University of Washington using Vensim PLE (and I may be using it in professional applications, as well). Last April, I combined my interest in the arts with my interest in this new approach to system dynamics in a public article about marketing program for symphony orchestras. You selected TAFTO 2007, the pointer to that article, as number ten on the list.
    



9. I've written several articles about data and numbers. Making more sense with numbers part 3 offered an easy process to plot data you receive in email or reports.
   



8. The words we use can be vitally important in helping us think productively about key business, organizational, and social challenges. In A systems language for business, number eight on the list, I described one team's evolution towards a better language for discussing business issues, thanks to a course they took from me in system dynamics modeling and simulation.
   



7. Good data helps us ground our thinking in reality. Still more on data, a pointer to several online sources of data, captured the number seven spot.
   



6. Growth can create problems (witness any of the bubbles that have occurred over history), but where are good examples of successful companies that intentionally don't grow? Number four on the list is Small Giants: the American Mittelstand?, pointing to a book that answers that question.
   



5. Sometimes old technology still has utility; sometimes it still attracts interest. At number five, Technology comes full circle, a description of my continuing use of a slide rule in my work, certainly fits that description. For those who are interested, it points to a source for new slide rules.
   



4. When I first started work as an engineer, PERT charts were done using mainframe computers or hand-drawn charts. Today, project management has become a profession with a certification process, and automated tools with graphical user interfaces have long since replaced tables of numbers and dates. Your sixth-most-popular entry was Critical chains: a decade later, my revisiting of Eliyahu Goldratt's critical chain theory that linked to Tom von Alten's revisiting of his views on the approach.
   



3. Productivity is obviously important to you. Your third most popular posting of the year was a surprise to me: If you can say it, it's done, an entry about the array programming language J.
   



2. Barry Richmond has a deserved place as an educator and thinker on system dynamics and systems thinking. I posted a link to an article he wrote about systems thinking and followed up with "Scientific thinking" the modern way, a differing view on the application of modern scientific thinking in system dynamics. That was your second favorite posting from 2007.
   



1. The 2007 posting you viewed the most was the series Making musical sense by email, showcasing a conversation between music critic, composer, author, professor, and consultant Greg Sandow and me that used a system dynamics model to explore the aging of audiences for symphony orchestra concerts in the USA. Now I'm curious: was its popularity because of the topic (music), the approach (a somewhat novel approach to using system dynamics), or the fact it was a real conversation between two people? Let me know.
   

All of those postings were made in 2007. It wouldn't be fair to finish this list without noting that some postings from prior years did rank higher than some of these. Here's the all-time top ten list of postings from Making Sense With Facilitated Systems as measured by your viewings in the last twelve months:

15. TAFTO 2007 (2007)



14. Making more sense with numbers part 3 (2007)
    



13. A systems language for business (2007)
    



12. Still more on data (2007)
    



11. Small Giants: the American Mittelstand? (2007)
    



10. Technology comes full circle (2007)
    



9. System Dynamics for Cheapskates (November 2006)
   



8. Critical chains: a decade later (2007)
   



7. If you can say it, it's done (2007)
   



6. "Scientific thinking" the modern way (2007)
   



5. Making musical sense by email (2007)
   



4. System dynamics with MCSim (November 2006)
   



3. In praise of the lazy employee (April 2005)
   



2. System dynamics and program evaluation (June 2005)
   



1. Making sense with numbers (November 2006)
   

That list includes the top ten postings written in 2007 plus the five entries written in prior years that were at least as popular as the top ten 2007 postings.

As 2007 draws to a close, I want to thank you who read Making Sense With Facilitated Systems and to invite you to continue with me in 2008. If you have suggestions or feedback for this blog, contact me.

I would be honored to be of service to you or your organization in 2008. If you're trying to make sense of tough business or organizational challenges, curious how I might be able to help, or just want to talk about some of the issues you face or that I write about, get in touch.

Testing the speed limits

In the Fall 1990 Sloan Management Review, Christoph-Friedrich von Braun published "The Acceleration Trap," calling into question our focus on shorter time to market. He later published the related The Innovation War (Prentice-Hall International Series in Industrial and Systems Engineering) (a book still on my reading list).

Tom Peters blogged about it (briefly). Brice Dattée and Dr. David FitzPatrick published The Acceleration Engine: Pattern of Technological Development, a mathematical exploration of the topic from a slightly different perspective. Barry L. Bayus of the University of North Carolina at Chapel Hill wrote an interesting review of his book. Eugene Garfield wrote another review. Alexander Kandybin and Martin Kihn quoted von Braun's work in The Innovator's Prescription: Raising Your Return on Innovation Investment in strategy+business.

His words haven't been accepted universally, as anyone in a wide range of industries may attest to once they leave work at 9 or 10 in the evening. On the more literate side, Preston Smith wrote From Experience: Reaping Benefit from Speed to Market. It was the subject of a debate (Die Innovationsfalle) at the 2001 CeBIT.

If von Braun is right, there's another risk to growth besides running out of natural physical resources: there's running out of time. It's analogous to an addiction: we need to keep getting more and more of the substance in question (reducing time to market, in this case) to remain satisfied. If we can't maintain our "supply," we crash and go into withdrawal.

In this case, the risk von Braun pointed out was the limit to how short one can make product cycle times and the risks to any financial success that's built on steadily decreasing time to market. Perhaps we can eventually cycle through product generations faster than our customers will accept them (do you want to replace the computer you bought yesterday with a new generation today and then do it again tomorrow?). Perhaps we'll begin to hit physical limits to speed (zero time to market would seem to be a very hard limit to exceed). If we try to break through that limit, whatever it is, and fail, we lose the business benefits we've been sustaining based on constantly improving time to market in the past.

I'm sure many of us are tempted to say, "We don't know if there's a limit or not; we should push forward as hard as we can, and we'll let the real world tell us if there are limits." Unfortunately, if von Braun is correct, hitting those limits won't mean a leveling off; it will mean a crash, and that could have a ripple effect that none of us will enjoy.

Does this mean I'm against reducing time to market or cycle time in general? No. There are many places in our organizations where reducing delays can help, likely including the delay from a customer perceiving a real problem to being able to obtain a product or service to address that problem. Without thought and testing, though, it's hard to make generalizations.

I perceive that time to market reduction is like growth: neither serves as an eternal, prime goal, but each may have its place in at the right time and in the right situation. I do encourage you to do your own reading, think about it, and draw your own conclusions.

How do you know if it's the right time and situation for you? If you'd like to explore ways to discern whether now is such a time and this is such a situation for your organization and how you might create policies that further your goals, let's talk.

A rash generalization

Friend and colleague Jay Forrest and I talk about what I find to be interesting ideas from time to time. Stimulated by an idea he once told me, I've assembled this rash generalization:

• Income (living wages, one hopes) gets made off the production that's both necessary to support the status quo and possible because of the status quo.
  
• Riches can be made off the transitions from one status quo to the other.
  
• Some disasters are made off the mistaking of a transition (from point X to point Y) for a never-ending trend (from point X ever upwards at a constant CAGR) and the effort we expend to try to make it so even after we've passed that situation's limits to growth.
• Other disasters are made when we envy the growth others exhibit and try to force our steady-state situation to match their growth.
• Disasters may take time to materialize. Part of the art of business is recognizing turning points and responding appropriately.

What do you think? Where is your business?

Making more sense with numbers, part 4

In the spirit of helping us all make better sense of data we read, I encourage you to read Mark Liberman's Thou shalt not report odds ratios in his Language Log if you write about data. If you read reports containing data (including the newspaper), read it, too, to help decipher what you read.

It's a somewhat long article, but you'll probably get the message by the end of the first example. (There is a possibly useful pointer to odds ratios and risk ratios on Wikipedia at the end of the article.) If you want another view on the same subject, see Odds ratios should be avoided when events are common, a letter by Douglas Altman, Jonathon Deeks, and David Sackett in BMJ. For an opposing view, see Stephen Senn's response.

If you're not writing for a highly technical audience and making it clear (perhaps through context) what you mean, I agree with the first and second articles.

Thanks to Jeremy Miles for the pointer.

Those curious about the title of this posting can read part 3 and find earlier parts.

You've got to see this graph!

One of the ways we make sense of situations is in how we portray data. I'm a fan of carefully crafted graphics, often trying to follow the lead Edward Tufte sets in his books and workshops.

That's why you have to see this graph on Statistical Modeling, Causal Inference, and Social Science. It ... ah, Phil says it better than I could; go take a look.

When you come back, note that the message is not to copy this design into the next five graphs you do (or at least the ones you show) but to have the courage to show the data in creative ways, breaking a few rules along the way if that helps to convey your information with clarity and integrity.

What he said

See Gill South's report about Systems Thinking, System Dynamics: Managing Change and Complexity by Kambiz E. Maani and Robert Y. Cavana.

The advice?

• Take the time to think.
  
• Patterns, not individual datapoints (problems), are key.
  
• If you want to change an event (fix a problem), you have to change the process (structure) that created the pattern of which the event is a part.
  

As some might say, it's all about context. Others might suggest it's about getting leverage.

If you're facing a challenge in your organization, are you looking at an individual event, or is there a pattern? What's the structure (process) that created that pattern? What can you do to change the structure so that the pattern changes in a way you want? Would you like to talk about it?

Structure influences behavior

That's part of a message I try to convey: structures tend to create patterns, and events are usually part of patterns. If you have to fix (change, eliminate) an event, don't focus on the event. Rather, see if the event is part of a pattern, and focus on the structures that fix that pattern. Then you'll see the events become fixed.

That's the reason I apply system dynamics to organizational problems: it helps us find the pertinent structure.

In It's the invisible structures that get you, Andrew Taylor says it eloquently.

A better way to show data?

We all know that measured data comes with some uncertainty. Perhaps it's measurement error; perhaps it's sampling error. We even expect to see it mentioned explicitly in political polls, but I rarely see it published in business and financial reports. There are likely many reasons for that omission, only one of which is the difficulty of presenting the uncertainty concisely and informatively.

Thomas Louis and Scott Zeger recently published Effective Communication of Standard Errors and Confidence Intervals with a proposed approach to indicating such uncertainties. On the one hand, I like it. It makes a nice, neat display of two, three, or five numbers that can describe a statistic nicely, and it's relatively easy to understand and to incorporate into your reports (the authors give three lines of LaTeX code you can use directly). On the other hand, as Andrew Gelman notes, graphs are still better (thanks to Andrew for the pointer).

What's a person to do? I have some suggestions:

• If you've got data in tables, strongly consider figuring out a graphical approach that conveys your information clearly and effectively instead of using a table, as easy as that might seem. In addition to the ideas in the paper Andrew references, consider boxplots among the potential candidates.
  
  If you don't have time to create useful graphics, consider whether your audience has the time to make sense of your tables. There are usually more of them than there are of you; taking 10 extra minutes to save 20 other people 2 minutes each sounds like a good trade-off (plug in your own numbers).
  
  Of course, if you're doing a balance sheet or income statement, you probably need the numbers at least once, although graphics may still help to convey your message.
  
  
• If you've got isolated bits of data that you're using in flowing text, consider using Louis and Zeger's approach.
  
  It's easy to do in LaTeX and pretty easy in OpenOffice.org's Write. It seems a bit harder in Word (I have Word 2000) because I don't see a way to have subscripts on subscripts, but you can select a smaller font on some numbers.
  
  Unless and until this becomes a standard idiom, you'll probably need an explanatory note somewhere in your report.
  
  
• Consider sparklines as a way to convey graphical data (time series graphs or histograms) in flowing text. Sparklines can be generated with a number of different approaches for a number of different document formats; if you'd rather, you can generate them online.
  
  

Perhaps the real answer is that we now have yet another way to portray data, from which we can pick and choose to fit our current needs.

Scenario planning

Scenario planning is a business process we got largely from Royal Dutch Shell. If you just read The Art of the Long View: Planning for the Future in an Uncertain World, you should know things haven't stood still. Martin Börjesson has a very good reference site worth checking out.

If there's one thing I've learned about scenario planning, it's that it's a literary and creative process, not a mechanistic one. Those scenarios that offer the most value, that help me think the most, are also those that qualify as true stories, not as mechanical selections of various events.

Making more sense with numbers, part 4

Now that you've got an easy way to capture numbers out of emails and documents, how do you get numbers back into emails?

Graphs are great, but perhaps you don't want to use attachments? Check out Gnuplot's dumb terminal mode as a way to create plain text graphics. If you keep it simple, you can convey decent graphical information with plain text (as long as your recipients use a non-proportional font in their email client for plain text emails—a very good idea, anyway). I tested this approach in a public discussion and found some liked it and some didn't.

Perhaps you really do want to include a table of numbers or numbers and words. If you're working in J, it's pretty straightforward to create the table you want and then use J's clipfmt and wdclipwrite verbs to create something you can simply paste into your email or other document.

If you're using J, you can create your (text or other) graphics in Gnuplot, if you prefer, or you can create them in J directly.

Incidentally, this note and its predecessor have addressed specific cases of the more general problem of getting data into and out of J, a problem I think lots of newcomers to J discover early on. It's easy to do powerful calculations in J, but manually transcribing the data from another window into J or from J into another document loses all the benefits. The J Wiki has a page called Interfaces that might help. I've found the Text Files page quite helpful in getting data out of plain text files. Any statisticians reading this might find the interface to R useful.

Making more sense with numbers, part 3

One of the early mantras one hears in statistics is "Plot the data." When I first heard it, it was followed by "by hand"; I suspect that part gets elided these days. Still, the advice is good. It's often easier to make sense of a list of numbers if you can visualize them.

Most of the time, that takes time we don't have. When we get an email or a report with a table of numbers, we know that plotting the numbers means grabbing a piece of graph paper (does your office supply cabinet even stock graph paper anymore?) or opening up your favorite spreadsheet, copying numbers, and drawing a graph. I rarely take the time.

Last week, I got yet another email with a table of numbers showing how something had changed over time. I was curious, so I wrote a short J script (now edited into a one line script) to turn the clipboard into data and another to plot the data.

Voilá! Now I had an easy and quick way to grab and plot data. I tried grabbing data out of an OpenOffice.org Writer document, and it worked, too. Grabbing data out of a Writer table was almost as good; my script lost the shape of the table, but that's easy to fix.

What's more, when you've got it in J, you can also apply various J statistical routines to the data, or you can pass it to R for more advanced statistical processing.

Yet another simple productivity tool, yet another reason to learn J as a tool for thinking and doing, yet another way to make sense with numbers.

I don't really care if you use J or some other tool; just pay appropriate attention to the data you handle. I just happen to think J is a powerful tool for this task (and for many other tasks). If you're learning J, check out the J lab called "An Introductory Course in J" by Henry Rich (thanks to Kip Murray of the University of Houston for pointing that out recently on the J Programming forum. Kip notes that Henry's lab covers a lot of territory very clearly but with a steep learning curve. If you are just seeing J for the first time, check out the J Primer.).

Interested readers might also be interested in tables2graphs.com and Using Graphs Instead of Tables.

So, if you have a table in email that looks like

Year Amount
2000 150
2001 200
2002 250
2003 225
2004 260
2005 254

and you'd like to graph it, one J program is

require 'format misc files plot'
sd=: > @: (". each ) @: |: @: clipunfmt @: wdclipread

Just copy the numbers, and type

plot ;/ sd''

to see your graph. I'll let you figure out how to add options and how to deal with multi-column data tables (it's easy).

Why is this part 3? Because there already has been a first and a second making sense with numbers, of course.

Jane Jacobs

Some time ago, I wrote briefly about Jane Jacobs and her The Death and Life of Great American Cities. I found it in a list of recommendations from Andrew Gelman, which makes me want to go back and review the other books on his list that I haven't read.

I liked three things in particular about this book:

• her lessons about cities
  
• her detailed and interesting descriptions of her observations
  
• her very early use of ideas of complex adaptive systems
  

The first was simple: I had never really thought about the functions sidewalks and side streets play, and I had never thought about how a mix of uses during the day plays into keeping a city safe. I had not understood why some recommendations for urban renewal seemed to work so poorly nor what might be done about it. She made all that clear. I'll trust what she says, for I don't live in a city environment to be able to experience it first-hand.

The second was more important to me. Her book was, in a way, one long series of low-level observations, coupled closely with reasonable and reasoned inferences she would draw from those observations. She never got far away from the observations, so it seemed easy to verify her thinking. Using the ladder of abstraction metaphor, she seemed to stay on the lower rungs, and that made her thinking and her arguments more powerful.

The third surprised me. I didn't really expect an early 1960s book about city planning to dive into complexity theory, but she did it at the end of the book, after building up a remarkable story, and she did it in a way that was quite approachable. If you're curious, you can see an excerpt from "The kind of problem a city is," the last chapter of her book, at Katarxis No. 3.

If I took away lessons from this experience, they would include:

• Observe.
  
• Attend to outliers as well as central tendencies; attend to diversity as well as averages.
  
• Make sure inferences are based on observations, and make the chain to the observations as short and as transparent as reasonable.
  
• Explore new ideas and new theories, for some of them make help make better sense of observations. This admittedly may cause tension with the previous lesson.
  
• Be interesting, which comes in large measure from being interested.
  

I like to give links to other sources you can explore, but there are so many options in this case. P.J. Tayor published Jane Jacobs (1916-2006): An Appreciation in Environment and Planning A. Jacobs gave credit to Warren Weaver in her work on complexity; you can read the part of his work she references in The Rockefeller Annual Report, 1958 (start on page 23 of the PDF). I recommend this highly. If you liked Weaver's article and want to read more about making sense of complex situations in the social sciences, F.A. von Hayek's Nobel Memorial Lecture The Pretence of Knowledge might well belong on your reading list.

But, more than anything, read Jacob's The Death and Life of Great American Cities. It's worth it.