Friday, September 25, 2009

What did Schellnhuber say this time?

Some of you may recall that Hans Joachim Schellnhuber and his research team produced a very good article on potential tipping points in the environment. I happened to find an interview with him in Der Spiegel entitled 'Industrialized Nations Are Facing CO2 Insolvency.'

What do you think?

Labels: , , , ,

Wednesday, September 23, 2009

Growth from a German perspective

Der Spiegel just published Can Economies Function without Growth?, which directly relates to some of the conversations we've held here in the past.

What do you think?

Labels: , , , ,

Tuesday, June 09, 2009

Creating sustainability in complex ecosystems

I recently had the privilege of teaching a course in system dynamic for Willamette University's Sustainable Enterprise certificate program. The course lasted two days, with a follow-up two-hour web seminar. We focused on qualitative system dynamics, but we treated it at a somewhat more rigorous level than many such courses, I think.

I'm writing because of one particular lesson I learned—we all learned. Early in the course, we used a simulation game to help people have a common, shared experience of interacting in a challenging system environment.

As with many such games, the expected result is that people fail in making the system work. Typically, the debrief is used to help people understand the ways of thinking that led them into trouble and to prepare them for the material that's to come.

Unexpectedly, this class managed their challenges quite sustainably. While their skill wrecked the planned flow of that part of the session, I was really pleased to see their skill in action. We spent some time talking about what made them successful and how that might carry over to real-world situations. Their insights were useful enough that I wanted to share them (with the students' permission) with a larger audience: you.

I first asked what made them succeed in the game and what provided the most challenges.

Goals were the first. While the game tells them the goal they should have, they rapidly realized that focusing on the stated goals would lead to ruin, and so they decided to set a much longer-term goal.

Communications was the second factor. After the first round, they began to spend most of their time huddled in the center of the room, talking animatedly through their decision-making processes instead of working in isolated teams.

They noted that delays provided a key challenge. As they worked to establish trust in the social system they had set up, they were both trusting other teams' commitments and verifying that they were indeed living up to their commitments. That takes time: commitments made today may not show up for quite a while.

Those delay effects were complicated by the natural delays in the system. Without revealing the game we used, I will say that the dynamics of the game included natural delays between actions and results that complicated decision making.

Some noted this seemed analogous to the situation OPEC finds itself in. They rely on mutual agreement to limit production as a way to manage prices. If anyone in OPEC breaks that agreement, the system can collapse. OPEC's problems are complicated by uncertain demand and uncertain prices, factors that had no analogy in our game.

Math skills created another success factor, which some may find surprising. A subset of the players rather immediatedly began developing quite a useful understanding of their system based on a mathematical model they developed. Once others saw that their results were accurate, everyone became driven by the data. Without some in the group being able to pull that off, they would likely not have succeeded.

Interestingly, trust and math worked together. At one point, the analyst team made a numerical error and then made an especial effort to communicate that they had made that error to others so that the others would be able to differentiate that error from a breaking of the trust relationship. Apologies were key. Information and the lack of information thus played a key role in the group's success. Even then, it took time for the others to regain their trust in the analysts' team.

Playing into this was the lack of external shareholders. Everyone on the teams had a serious take in the workings of the game; no one was in it just for the "money." Similarly, there were no new entrants into the field who might have upset the cartel relationship they had crafted.

I then asked them what they'd advise people in the real world.

Collaboration was the first clear answer. Work together across groups to align goals and actions.

They then said, "knowledge is power." After a bit of reflection and revision, they revised that to "timely, transferrable, actionable knowledge is power."

They felt it was important for everyone to be clear on a vision.

They would encourage people to watch their egos and to be visibly trustworthy.

At one point, in an attempt to test the strength of their commitment (okay, as an attempt to derail their commitment), I as facilitator announced I was the government and was giving them something they really didn't want. (To be accurate, that idea came from Anne Murray Allen, the executive director of the program, who was running the simulation computer.) For a while, I felt as if I were about to experience the French Revolution, as some rather emotionally argued for standing up to government and refusing my help, a bit of resistance I wasn't accepting.

As a result, their last bit of advice was to "Don't trust the wisdom of government, of the private sector, ... of either." In other words, test the data and the reasoning yourselves instead of blindly accepting what others say is good for you.

This was an intense and very exciting two-day workshop. I think those in the class learned a lot; I know I learned as they taught themselves and me (and now perhaps you) how to make sustainability work.

Perhaps I'll see some of you there next year.

Labels: , , , , , , ,

Thursday, February 26, 2009

Is climate change irreversible?

I don't know the answer, but Susan Solomona, Gian-Kasper Plattner, Reto Knutti, and Pierre Friedlingstein make that claim in Irreversible climate change due to carbon dioxide emissions from the February 10, 2009 Proceedings of the National Academy of Sciences. If you read it, you'll understand their definition of irreversible and their conclusions.

Labels: , ,

Thursday, October 30, 2008

More not good news

A bit over three years ago, I published Not good news; what do we do about it?, a story prompted by the Millennium Ecosystem Assessment.

Today, Grist News published Planet Ahead, a pointer to the World Wildlife Fund's 2008 Living Planet Report. In short, we're continuing to live beyond our (or the planet's) means, and, at current trends (not consumption levels, as reported in Grist according to my interpretation), we'll need two planet's worth of resources by the mid 2030s.


While they don't reveal the details of their model, it seems by their admission (p. 22) that it does not incorporate feedback effects. Thus it may miss behaviors that could prove dominant and perhaps more damaging over time.


One challenge is that the degradation is largely invisible to many of us. For one, many of the factors they measure are not bought or sold, so normal market forces don't make them visible nor work to manage their use. For another, the degradation is largely confined to tropical regions, yet I suspect most of us reading this live in temperate regions and don't really observe those changes.

Yet, amidst that gloomy projection, there is cause for optimism. They speak of "sustainability wedges" as an approach to help us have our cake and eat it, too, or at least to grow energy services and reduce greenhouse gas emissions.

The report is 48 pages long, so I've just skimmed it. I think it's well worth the time at least to skim it to become better informed on the current state of the world and current trends.

What does that mean for us in business? What does that mean for us as citizens? What are your thoughts?

Labels: , , ,

Thursday, October 23, 2008

Food policy

I heard part of a piece about food policy on Fresh Air recently, but I forgot about it until I read Marilyn Holt's A Locavore Manifesto by Michael Pollan in her Amicus Agraria. She points to Michael Pollan's thoughtful and thought-provoking letter to the US President-Elect called "Farmer in Chief" in the New York Times. Yes, it's long; yes, it's worth your time.

You can see prior postings about food in A Prius for the pantry?, No such thing as a free (or even cheap) lunch, and Food and fuel in the news.

Postscript: Thanks to a tweet from Garr Reynolds, I found the related Eat fast, eat until full to get super-sized.

Labels: , ,

Tuesday, September 09, 2008

Road Maps to Peace

Yesterday I blogged about what I see as one of the two big problems we face: how we deal with the load we're placing on the planet. I think the other big problem is how we deal with disagreements among those of us who live on this planet. As the load we place on the planet increases, I think the importance of figuring out how to get along will increase, too.

Some of you may know of Rick Steves as a travel guide who also writes newspaper columns, produces TV and radio shows, owns a travel agency, and sells travel gear. He also has a more serious purpose in life, or, perhaps, as I've written about Bernie DeKoven and sustainability, his more serious purpose is inherent in what he does in his travel business.

On September 6, he hosted Lord Alderdice in a discussion called Road Maps to Peace. Lord Alderdice played a key role in helping people work to end the "Troubles" in Northern Ireland, and he has continued his work by helping others to reduce conflict in their regions.

Rather than me taking the time to summarize Lord Alderdice's message, listen to him directly. The recording is about an hour long. I encourage you to listen carefully, with an open mind, to the end, for there are many good ideas. Often we hear people expressing opinions (or express them ourselves) about how peoples should get along; Lord Alderdice is talking about what has worked in helping peoples to get along together. I found ideas that I can apply in interpersonal relationships as well as in thinking about larger political issues we help decide at the ballot box or in dialog with our elected officials as well as in working with or inside organizations.

One message he gives (but not the most important—I'll let you listen to find the essence of his ideas) is that we should all listen to news from other parts of the world so that we gain a broader perspective. That's a message I've given here before. I've long thought we should strive to have personal friends and professional contacts in multiple parts of the world, for I suspect we'd be more likely to work things out and less likely to go to war with countries where we have those connections.

As with my previous message, I really would appreciate your comments on these ideas.

Labels: , , , ,

Wednesday, August 20, 2008

Prediction, system dynamics, and Future-Fusion

Recently, I made the claim that we're better off focusing on adapting to the present than predicting the future. I've made similar claims in the past, too. I've even given one example in which predictions serve a useful purpose.

That's all a bit simplistic, of course. Even system dynamicists could be said to predict the future in a way: we show behavior over time we feel is more likely to occur (although we may warn people away from point predictions based on a behavior over time graph). In other words, I might suggest that your current policies could produce a boom and bust effect in your business, but I wouldn't want you to draw the conclusion that your business will grow another 172.3% by June 15, 2009 before taking a tumble that afternoon.

Because we all would like to know the future, I've experimented with blending system dynamics and Bayesian analysis to quantify the probability of a particular behavior pattern, for example. Of course, that probability is conditioned on both the historical data and the model being correct, which is a loophole big enough for a good-sized locomotive to run through: models are always incorrect. Still, I think this approach may give more useful insight in certain cases.

Now Kshanti Greene of Stottler Henke Assocates, Inc. has shown me a Bayesian tool they've developed called Future-Fusion, and I've been exploring it a bit. They are using Bayesian networks and the power of groups to get a better handle on what the future holds. Much as Data360 looks at the past, Future-Fusion attempts to look at the future. As of this writing, they've created four test areas which you can explore: the 2008 US presidential election, the Iraq war, corporate strategy, and energy. Try it out: learn how to use the system, see current predictions, and add your own (I think you only have to create a free account if you want to add your own predictions). Perhaps you'll learn something, and perhaps they will, too.

Kshanti has pointed out a recent addition to Future-Fusion that may intrigue some of you: time. They've enhanced their technology to allow limited dynamic execution of a network model, which begins to narrow the gap between Bayesian networks and system dynamics from the Bayesian network side, much as what I've tried has narrowed it from the system dynamics side. To try that out, go to the energy model, select a prediction (e.g., "Reduced SUV sales"), click "view graph," note the numbers, and then click "Next Time Step."

I think this is all still experimental in many ways, but it's a good opportunity to learn a bit about this technology by trying it out on real-life issues. I'll be curious what you discover.

Labels: , , , , , , , , , , , ,

Thursday, July 31, 2008

Systemic problems in US healthcare?

Wow!

If you know Tom Peters and his work, you know his focus on WOW!

This is different, though. This is my "Wow" at the intensity of his recent postings, intensity that trumps even his usual high-energy attitude.

Start with Ten Years in the Making!, his take on the state of health and healthcare in the USA today. Go read it the attachment, and then come back (that might be tomorrow; it's 288 slides long!).

I have not double-checked his references, but most of his slides provide references to other's work (and, if you're not familiar with his style, no, they aren't boring bibliographies).

Then check out The Healthcare14: U.S. Healthcare Trauma in 2008 and his tongue-in-cheek Proposed Hospital "Organization Chart."

There are other reports, among them the 1999 Institute of Medicine report To Err is Human: Building A Safer Health System, and there are and have been projects designed to make things better.

While I sense that Tom Peters isn't a big systems fan, it sounds as if a systems approach might help. The IOM and Tom Peters both described patterns of behavior. Patterns are usually stimulated by structures in the organizations we create.

To make a long-term change, we had best find the ways to change the structures appropriately. Focusing on the patterns alone (or, worse, simply the bad events) is literally treating the symptoms, not the disease. Treating the symptoms might make things better in the short term, but we shouldn't be surprised when they return if the disease remains present. That's where the systemic approaches come in: they can help us find the structures at work creating our problems, and they can help us test our theories about proposed solutions. That doesn't always mean the same approach, and it doesn't always mean simulation. It does mean deriving insights from system principles and applying them effectively to solve real-world problems.

If you're in healthcare, do these issues ring true? What do you see?

Labels: , , ,

Monday, June 16, 2008

Climate change made understandable

There's lots about climate change that's confusing to those of us who aren't climatologists. What is climate, what is weather, and how do they interact? What are the prime causes of climate change, how well do we understand them, and which causes are more important? What will climate change mean to me?

I happened to discover Reports to the Nation: Our Changing Climate, written by Professor Dennis Hartmann, interim dean of the new University of Washington College of the Environment, and I thought some of you might find it useful, too. While it was published in 1997, it seems to explain many of the key factors. Certainly more up-to-date reports can help, but this one seems both good and easy to read.

Labels: ,

Friday, April 04, 2008

Grow or die

The title is a rough quotation from an interview I read by a CEO recently. It doesn't matter who it was; it's a common business notion, I sense.

I understand the sentiment, really I do. There is a real fear that, if we don't grow, we'll be overtaken by those who do grow. Or that we'll become stagnant and stale.

Yet I'm mindful that, if we all grow, we'll surely die (or at least suffer), too. Read Limits to Growth, if you're uncertain about my statement. Note that "limits to growth" in that book is not a statement of an environmentalist's hope; it's a statement of fact. We will face limits to growth. We can choose the nature of those limits (or at least we have had the chance), but stop growing we will. You can't exceed the carrying capacity of an environment forever.

So, if it's true that we all (or most all) want to grow (our companies, our houses, ...), and if it's true that we will face (and are already facing) real limits to that growth, what do we do?

I've written about growth a number of times, but I admit that I don't have all or even many of the answers yet. Perhaps I'll find out more in the next few months, for I'm co-teaching a systems thinking course at Bainbridge Graduate Institute.

If you're not familiar with BGI yet, their vision is "To infuse environmentally and socially responsible business innovation into general business practice by transforming business education," and they've got a good reputation in this area.

As I did when I taught system dynamics at the University of Washington, I suspect I'll learn a lot here, this time with a distinct focus on sustainability and business. I'm really looking forward to this experience. (And, as I did last time, I will refrain from telling you anything that goes on in class unless I have explicit permission, but I may tell you a bit about how I grew.)

Labels: , , , , ,

Thursday, March 06, 2008

Tipping points

A few years ago, thanks to a lead by John Sterman, I posted a note about a Proceedings of the National Academy of Sciences (PNAS) article that attempted to quantify the carrying capacity of this planet. The concept of carrying capacity is important; if you try to exceed the carrying capacity of an ecosystem, you will eventually be brought back. What's even more worrisome is the potential that you might eat up carrying capacity while you're in overshoot: if you do that, the equations suggest you'll eventually suffer a collapse.

There's another concept that's discussed both in the systems literature and the popular press: the tipping point. That's the claim (loosely) that systems may reach a point where a slight addititional change will lead to a qualitatively different state. Often the definition includes something about irreversibility (you can't go back) or at least about the difficulty of reversing a change due to the reaching of a tipping point.

Now John Schellnhuber and his colleagues have published Tipping elements in the Earth's climate system in the February 12, 2008 issue of PNAS. They've also published appendices, not in the printed version, that provide their formal definition of a tipping point, evaluate other potential tipping elements, and describe how they elicited the information for the main article.

Those of you interested in tying systems concepts to the real world (hopefully that's just about anyone doing systems work, although some may be more interested in some applications than others) might find this of real interest. Those of you wondering about the risks of climate change might find it informative.

For a report on Schellnhuber's lecture at the annual meeting of the AAAS, see John Schellnhuber’s Third Industrial Revolution, a New Approach to Addressing the Hazards of Global Warming by Julia Whitty in Mother Jones.

Labels: , , ,

Wednesday, December 12, 2007

The voice of a child ...

I write here about the environment from time to time, for I think there is a significant likelihood that we will face important transitions in our lifetimes or the lifetimes of our offspring that will impact us in multiple ways: personally, in the way we live, and professionally, in the way we produce what we need to live and earn what we need to acquire those things we need to live. (If TIME is right, we may not have to wait that long for those transitions.)

Presentation Zen posted You can learn a lot from "a child", a speech by Severn Cullis-Suzuki at the 1992 Earth Summit in Rio De Janeiro. It touched on the big issues I perceive we face: figuring out how to make life sustainable on this planet, and figuring out how to live together. Watch it.

I also write about presentation skills from time to time. As Presentation Zen reports, Al Gore called this "the best speech at Rio."

Labels: , , , , ,

Friday, November 02, 2007

A manager's job

Here's a philosophical question for you on a Friday. What is a manager's job?

Is it to lead a group? To direct people's actions? To manage and control the progress of an organization? To make decisions? To solve problems? To follow directions from above?

While all of those may fit in the typical manager's day, I think the foremost responsibility of a manager is to create and manage human and organizational systems that will get the correct things done.

As managers, we have the responsibility to get things done through our organizations (as a one-person company, I'm counting myself as a "manager without portfolio"). If we believe the mantra that events are part of patterns and patterns are caused by structure, then our task is to create the structures that will lead to the sort of patterns and events we seek.

If we focus on individual problems and decisions, we're focusing on events. That will lead us into the perpetual task of trying to address more and more events, and we risk being overwhelmed. We're perpetually pushing against the tendencies of the system that's there.

If we make it our business to create and maintain an organizational system that gets the right work done, we create a "machine" (a system) that gets the work done for us with less effort and fewer problems. That's how I helped an organization reduce its budget variance by 95%, and that's part of what I meant by praising "lazy employees." While you're working on this system, remember that it has business, technical, and people dimensions, which all demand attention!

So if you are a manager, your real role is that of a (human and organizational) systems engineer.

Labels: , ,

Thursday, September 20, 2007

Evolution and organizations

Andrew Taylor has a thought-provoking piece on The Artful Manager today. It's a reminder to keep the goal in mind.

Labels:

Friday, August 03, 2007

Jane Jacobs



Some time ago, I wrote briefly about Jane Jacobs and her The Death and Life of Great American Cities. I found it in a list of recommendations from Andrew Gelman, which makes me want to go back and review the other books on his list that I haven't read.

I liked three things in particular about this book:


  • her lessons about cities
  • her detailed and interesting descriptions of her observations
  • her very early use of ideas of complex adaptive systems


The first was simple: I had never really thought about the functions sidewalks and side streets play, and I had never thought about how a mix of uses during the day plays into keeping a city safe. I had not understood why some recommendations for urban renewal seemed to work so poorly nor what might be done about it. She made all that clear. I'll trust what she says, for I don't live in a city environment to be able to experience it first-hand.

The second was more important to me. Her book was, in a way, one long series of low-level observations, coupled closely with reasonable and reasoned inferences she would draw from those observations. She never got far away from the observations, so it seemed easy to verify her thinking. Using the ladder of abstraction metaphor, she seemed to stay on the lower rungs, and that made her thinking and her arguments more powerful.

The third surprised me. I didn't really expect an early 1960s book about city planning to dive into complexity theory, but she did it at the end of the book, after building up a remarkable story, and she did it in a way that was quite approachable. If you're curious, you can see an excerpt from "The kind of problem a city is," the last chapter of her book, at Katarxis No. 3.

If I took away lessons from this experience, they would include:


  • Observe.
  • Attend to outliers as well as central tendencies; attend to diversity as well as averages.
  • Make sure inferences are based on observations, and make the chain to the observations as short and as transparent as reasonable.
  • Explore new ideas and new theories, for some of them make help make better sense of observations. This admittedly may cause tension with the previous lesson.
  • Be interesting, which comes in large measure from being interested.


I like to give links to other sources you can explore, but there are so many options in this case. P.J. Tayor published Jane Jacobs (1916-2006): An Appreciation in Environment and Planning A. Jacobs gave credit to Warren Weaver in her work on complexity; you can read the part of his work she references in The Rockefeller Annual Report, 1958 (start on page 23 of the PDF). I recommend this highly. If you liked Weaver's article and want to read more about making sense of complex situations in the social sciences, F.A. von Hayek's Nobel Memorial Lecture The Pretence of Knowledge might well belong on your reading list.

But, more than anything, read Jacob's The Death and Life of Great American Cities. It's worth it.

Labels: , , ,

Wednesday, July 25, 2007

Complex adaptive systems, society, and, oh, yes, Wikipedia

For an interesting case study of how some stuff works on Wikipedia, see Sean Silverthorne's HBS Cases: How Wikipedia Works (or Doesn't). You can even check out the actual page in question and the case study itself.

Thanks to the TP! Wire Service for the pointer.

In case you haven't explored Wikipedia thoroughly, check it out. In addition to the main starting page, there are Wiktionary, Wikibooks, the Wikiversity, Wikinews, and more. There are even versions in quite a few languages, including one often (predominantly?) used where I once worked and which I can still read, at least mostly. Speaking it would be (and always was) something else.

Labels:

Thursday, June 07, 2007

Why are we headed there?

Yesterday I posted a rather dark story about what we could be facing. As with any system dynamics model, it's showing the likely effect of the modeled policies; it's not predicting the future. (There is a difference; call if you'd like to chat about it.)

If that message is so dark, why don't we see much action to change our course? (Admittedly, we're beginning to see more action now.) The Oil Drum published Living for the Moment while Devaluing the Future, an essay exploring just that question. For a more academic approach, see Larry Karp's Global Warming and Hyperbolic Discounting, an article referenced at the bottom of The Oil Drum article (now on my reading list—I've only had time to skim it so far).

I think these ideas are important to understand and explore as we try to craft a "soft landing" from our ecological overshoot.

I think they may also be important to us in business. We get used to constant discount rates, because that's what we use. Do our customers and our bosses (they do have similar roles in our lives) really think that way, or do they do hyperbolic discounting, too? Would it it make a difference which they use? Would that difference imply we should act differently than we do?

Labels: , , , , , ,

Wednesday, June 06, 2007

Where are our policies leading us?

A policy is a set of guidelines or rules by which we make decisions. We have certain policies by which we work and live, even if we don't always make those explicit. If we see a pattern of decisions, decisions that seem cut out of the same mold, that's likely evidence of a policy.

François Cellier of the Institute of Computational Science of ETH Zürich, Switzerland has published an article, Ecological Footprint, Energy Consumption, and the Looming Collapse, at The Oil Drum that examines the potential effects of our policies towards growth. It's a high-level view, to be sure, but sometimes those offer great insights. Be sure to read both the article and the accompanying slide set (the article isn't that long; it's the 333 comments that take up most of the length).

I think this is a very important discussion. That's why I think it's important for each of us to be skeptical about such claims. It's not because I think he's wrong; his analysis, at least so far, seems good. It's not a call to wait for "proof," for, as John Sterman points out, we're not really waiting; we are doing things to the environment every day. It's not a call to ignore the claims, for that's not being skeptical; it's a call to test them and then to act based on what we determine. It's not a call for depression; Cellier does show a way forward (especially in the slides).

By suggesting we be skeptical, I may give the impression I think we can ignore this for a bit. I want to re-emphasize that the IPCC and others have given some pretty clear signals that the time to act is now (actually, the time to act was some years in the past; the next available time to act is now).

What does this mean for our businesses and for business in general? I think it means figuring out what to do to ensure the sustainability of our businesses and our economic system in the face of the challenges the best science says await us. The key lesson from "Out of Gas: A Systems Perspective on Potential Petroleum-Fuel Depletion" was that we not wait too long to attend to signals we get, for our systems have inertia, and we can't, as much as we might wish, always change direction instantaneously. Pay attention to Cellier's description of easy and difficult problems starting on slide 38; the signals may not be as we'd normally expect. Sometimes we can't wait to feel the wind from an impending storm; we have to rely on forecasts from meteorologists to know when to board up windows in the face of an approaching hurricane.

We can also apply that message to more typical business decisions. Do we discover we will need to add (or remove) capacity well in advance, so we can react smoothly, or do we make such discoveries only when the market begins to complain loudly? How do we figure out whether our latest initiative is about to make real progress or it's about to fail and we should abandon it and change course?

Reacting too soon can lead to the Chicken Little trap: if we respond too quickly, we diffuse our energies by responding to simple noise; if we respond too slowly, we're trapped. One of the lessons I've learned is that feedback models (of the sort I've sometimes discussed here, also called system dynamics models) can help us find what things to monitor so that we have a clearer picture to guide our decisions.

What do you think?

Labels: , , , , , ,

Wednesday, March 14, 2007

Some energy fundamentals

I've written before about the risk of running out of petroleum as a viable energy source. Obviously, I'm not the only one, and many an idea has been broached regarding potential replacements.

Don Lancaster's Guru's Lair News newsfeed turned up his now-several-years-old article called Some Energy Fundamentals today. It might be worth your while to read, if you figure the transition to the replacement for gasoline is going to be easy. While I haven't checked all his calculations and statements, it seems more scientifically based (or engineering-based) and less alarmist than many such essays. It doesn't paint a rosy picture, though. I'll be curious regarding the critiques any of you might provide.

Labels: ,

Thursday, November 16, 2006

System dynamics and program theory (evaluation)

I just returned from a meeting of the Program Evaluators Northwest, where seven of us related what we had observed at the recent Evaluation 2006 in Portland. I learned from lots of what I heard, but one idea really hit home.

Bill Leon of Geo Education & Research spoke about the King County Guidebook to Elements of Successful Programs. In a way, it is a meta-meta-evaluation, as it summarizes the results of meta-evaluations of effective, youth-serving programs so that local non-profits can compare their work to that of the best non-profits across the country. In practice, they found element 3, "Program Design Based on Theory and Research," to be an often-missed element, one they'll emphasize in new programs.

Program theory isn't nearly as intimidating as it sounds; it's simply your explanation of the actions you chose to achieve your goals and the rationale behind your decisions. That's awfully close to one use of system dynamics models: capturing, exploring, and exposing for review your dynamic hypotheses about why a system works as it does.

Would it help important or risky programs to include a system dynamics model as part or all of their program theory?

That's certainly not an appropriate approach for all programs. Some programs may not warrant the effort, although small models, which often provide some of the most insight, may not take that much effort to develop. Some may not operate over a long enough time horizon for many feedback loops to develop, and feedback is the raison d'être of system dynamics. Some may be better viewed through another systemic approach. Encouraging a singular approach to anything opens us up to missing important issues which that approach can't address.

But I can't help but wondering if system dynamics could make an important contribution in the development of solid program theories for many programs precisely because it forces one to be absolutely clear about one's theories and because it enables one to test those theories in ways few other approaches do. While the result of a system dynamics simulation isn't guaranteed to be correct, a well-done, useful model can help people see both the assumptions built into the program and the likely ramifications of those assumptions.

I'm still thinking about this one. If you're involved in program evaluation or program design, what do you think? Might this have utility? If not, why not? (I'd like to sharpen my thinking on this topic.) If so, what are the limits to its utility? What have I missed? Follow up with a comment here, or drop me an email.

By the way, there is a guidebook to the guidebook, Program Assessment and Improvement Plan: Using the Guidebook to Elements of Successful Programs, which you may find useful to make the afore-mentioned guidebook more approachable.

Labels: , ,

Friday, April 01, 2005

Octosystems

I like finding articles that show multiple "systems thinking" approaches to help us develop multiple lenses through which we can view and act in the world. I just stumbled across Allenna Leonard and Stafford Beer's THE SYSTEMS PERSPECTIVE: METHODS AND MODELS FOR THE FUTURE, which covers eight such approaches. While I've only skimmed it, I like what I see so far.

Labels: ,

Tuesday, March 29, 2005

Making the world a safer place

We'd all like the systems (both the computerized ones and the harder-to-see organizations of people, processes, and tools that also compose systems we use) that protect our lives to work well and never fail, whether they are the safety features in nuclear power plants or the clinical procedures and equipment in our hospitals. Yesterday's comp.risks had two postings about failures in computerized medical systems, reminding us that many of our traditional beliefs about what causes failures in such complex systems aren't consistent with research.

In 1990, James Reason wrote the excellent and very readable Human Error, describing how errors are made in practice and describing how they can be reduced. Richard Cook and the Cognitive Technologies Laboratory have created a Web site with many short (and some longer) articles helpful to those designing such systems. Nine Steps to Move Forward from Error is food for thought for people thinking about how to make improvements after a significant system failure has occurred.

What's that got to do with us? Some of us manage people who create or use such systems; we need to know what good, current research tells us. Sometimes the pressures we put on others may create the problems we're trying to avoid.

Some of us may design such systems. We especially need to know what the research is discovering, for we will make many decisions and recommendations that affect safety and performance.

Some of us may simply use such systems. Normally, we may not need to know much of this. When systems fail, though, we may become part of a concerned and vocal populace that cries out for action. Knowing what research suggests about dealing with the aftermath of such failures helps us advocate for actions that will help us, not actions that may create even bigger problems in the future.

These concerns don't only apply to life-critical systems; we're also involved in business and organizational systems daily. The penalty for failure in those systems may not be someone's death, but it could involve people's livelihood.

By the way, comp.risks is a good way to stay abreast of the risks associated with the development and use of computer systems.

Labels: , ,