QUALIFICATIONS
I have only spent three days reading and thinking about the content of your draft. It is highly detailed and the issues are most complex. You stimulate many, many thoughts and my comments would undoubtedly be of much greater potential value if I could spend longer on the draft and developing my comments but unfortunately I cannot do this. Being away from my office, I am also unable to refer to materials, such as the Gland Report and my Stockholm report as well as my files, to refresh my memory. You should also understand that I was only a distant viewer of the Commission while it did its work. My closer involvement was from the late fall of 1996 through to the first meeting of the Commission in 1998 and then again in early 2001 through the final meeting of the Forum.
Besides this involvement in the establishment of the WCD and the final meeting of the Forum, I bring several perspectives to your report that might be potentially useful. I have been involved in the design, implementation and facilitation of multistakeholder and various forms of stakeholder involvement processes for more than 20 years and for even longer have worked on water resources issues in Canada and internationally. My experience is as a practitioner, teacher and researcher. I teach courses on negotiation, facilitation and mediation in sustainability planning and governance. In recent years I have offered a course entitled Innovations in Multistakeholder Processes, which is primarily taken by the graduate students who want to undertake research with me in these areas. Many of them undertake studies designed to assess experience with MSPs. Much of my own writing is focused on the assessment of MSPs. Two particularly relevant examples can be found through links on my web page. One is an assessment of my experience in establishing and chairing a unique MSP, the Fraser Basin Management Board during 1992-94 http://www.interchg.ubc.ca/dorcey/chcwra/fccwra.html. The other is an assessment of the experience in Canada with the design, implementation and assessment of citizen involvement processes in sustainability governance (including particularly MSPs) over the last 50 years http://www.interchg.ubc.ca/dorcey/trends/draft2.html.
VALUE OF YOUR REPORT
When I was reviewing the literature in 1997 to advise on the design of the WCD, I wish I had had accounts like yours for earlier commissions such as Brundtland’s. At the time there was relatively little information around addressing the breadth of assessment questions that you have considered. It is exceptionally valuable to have this detailed account of the WCD and for you to have done it while events are still relatively fresh in people’s memories and you could also observe some of the events unfolding. We are desperately short of assessments of MSPs and badly need to emphasize what we can learn from experience.
I am obviously biased but I do believe that the processes leading up to the establishment of the WCD and the process itself are pathbreaking examples of MSPs and that, all things considered, they have been extraordinarily productive. It is therefore particularly valuable to have your account and assessment of the experience.
Just as the WCD process was an heroic venture in itself, so was your assessment of it. You too faced a task that was daunting in its scope, complexity and controversy. Likewise, you had limited time and, I suspect, very limited resources relative to what was needed. Both have to be looked at in this context and highly commended for all that they have achieved given the constraints upon them.
As is often the case in work such as yours, it tends to generate more questions than it answers. In many places your discussion stimulated me to scribble questions in the margins reflecting my desire to know so much more about the specifics and details. But to expect this of you would be unrealistic. Rather this should be seen as one of the contributions of your report, a potential that future researchers will pursue.
ANTICIPATING THE REACTIONS (MAXIMIZING THE VALUE, MINIMIZING THE DAMAGE)
As you recognize, your report will generate all kinds of reactions and responses. Just as has been the case throughout the WCD process, different stakeholders will see your work in different ways and because of its importance will comment on it and exploit its content to advance their own agendas. It is remarkable how closely the questions about your report parallel those that you are addressing about the WCD!
My concern is that the value of your work and the conclusions that relate to advancing and expanding use of MSPs not be lost under the weight of potential negative comments. I will therefore concentrate the rest of my comments and suggestions on ways in which you might anticipate and strategize to mitigate negative reactions. I focus particularly on general issues and leave it to others, who are much better informed than I, to give you feedback on the “accuracy” of your account and interpretations of events. I will indicate where I think your assessment is vulnerable and then suggest how this might be addressed recognizing that you probably have very limited time and resources for making revisions.
CONSTRAINTS
As indicated above there were major constraints on what you were able to do in your assessment. I would encourage you to preempt criticisms and put your work in the most favourable light possible by speaking explicitly and fully about them and their implications at the beginning of the report. Many of the potenial criticisms I’ll suggest below canbe mitigated at least in part by your discussion of constraints and their consequences.
FRAMEWORK OF ANALYSIS
There are some significant ways in which the framework of analysis for the study might be strengthened so as to provide a clearer rationale and focus for the specific questions that are being addressed in the study. Chapter 2 does an excellent job of putting the WCD in the context of the history of global commissions and governance. However, while it includes a general consideration of their evolving use of MSPs, it does not provide a comparable consideration of MSPs as such. For this the report primarily relies on the short section in Chapter 1 (pp. 3-5) and one general reference, Hemmati et al. (footnote 4).
There had already been a great deal of experience with MSPs before 1996 when the idea of the Gland meeting first began to be discussed. The majority of this experience had been in North America with diverse processes from the national to the local level. When George Green first approached me about the Gland meeting, it was because of my experience with these processes in Canada and because of the perceived potential to extend them into the global arena. The accumulated experience with MSPs and their facilitation was a major consideration among those who were designing the Gland meeting and this carried over into the design and implementation of the WCD. Achim Steiner who played such a pivotal role in design and implementation of both was highly experienced and skilled in facilitating such processes.
Thus it is hugely important in establishing the context for assessing the WCD as an innovative form of governance to highlight the status of the art and science of MSPs that influenced its design and implementation. Some of this experience was explicitly referred to in my introduction to the Gland and Stockholm reports (if I recall correctly; remember I don’t have a copy with me to check). Beyond this you should have some information on this from the interviews with myself, Achim and others. But more fundamentally you can refer to the published literature at the time (my recent paper that is available from my website summarizes some of the major points and provides references to seminal work that preceeds Hemmati et al.)
I believe it is important for you to consider at least three questions. First, what did the literature in 1997 indicate were the key principles that should guide the design and implementation of MSPs (i.e. what were the normative ideals and how did these translate into criteria)? You can then relate these to the ones used in the Gland and WCD processes. Most importantly, you can relate them to your own choice of assessment criteria. Notably, while there are many ways of expressing it, your criteria don’t include any consideration of what some call “cost-effectiveness” and others “efficiency”. Criteria of independence, transparency and inclusiveness, along with other key considerations such as the breadth of consensus, each have to be related to the other dimensions of costs and benefits (broadly defined) (e.g. more time, money and other resources generally enables more of each of the other criteria.)
Second, apart from what people were proposing as principles for MSPs, what was the state of practice in 1997? This field was (and still is) no different from many others in that the state of practice often varies considerably from espoused principles. There are all kinds of reasons that range across the extremes of being willing but unable through to being unwilling and deceiptful. Particularly important for your study is a sense of what was feasible in practice as opposed to the ideals of principle (e.g. how well did earlier MSPs succeed in being inclusive and representational even with the best will in the world?)
Third, what was perceived in 1997 to be the importance of convening and facilitation to the relative success of MSPs? As you will see in my recent paper (and as mentioned in the Gland and Stockholm reports), I have argued that each was viewed as crucially important to the productivity of MSPs. By “convening” I mean who was involved and how in bringing the MSP into being and establishing its mandate (e.g. the Gland process was unique as a means of convening and establishing the mandate for a global commmission). By “facilitation” I don’t just mean the individual involved in doing this but the varied individuals and potentially institutions/organizations involved in providing this function. I believe these are two components of an analytical framework for assessing the WCD that are critically important in assessing the overall performance and productivity and which don’t receive enough attention in your present draft.
METHODS
There are a number of methodological weaknesses in the draft, some of which can be addressed by adding information, others of which you probably can only mitigate in part by recognizing them.
Good practice in qualitative research such as your assessment study involves being more explicit about (i) the investigators’ backgrounds, (ii) the rationales for the methods used and (iii) the experience with implementing them. This transparency gives greater understanding, credibility and legitimacy to the assessment.
<paraindent><param>right,left</param>What is the background and experience of the four authors? What are their biases? How does this relate to the goals and objectives of the organizations conducting the assessment? What special precautions were taken to control for biases?
What and why were the particular methods chosen for collecting data, analysing it and writing it up? What precautions were taken to ensure consistency of approach among the four investigators? What strategies were employed to enhance the validity/credibility of your conclusions by triangulation (e.g. testing for corroborating evidence through interviews with multiple individuals, and comparing interview results with written documents)? How were interviews conducted and recorded? What was the form of the raw data and how did you collate, organize and analyze it (e.g. were interviews tape recorded, then typed up and their content analyzed using software such as Nudist)? How did you determine your conclusions (i.e. how did you assess your evidence and different perspectives among the investigators on this, in determining where the weight of the evidence lay)? How were recommendations developed? How did you select and respond to reviewers comments?
What actually happened in practice (i.e. how did things change from what you intended in the design of the assessment)? How well did the strategies and methods work? What additional insight can you give the reader by describing your experiences and impressions in carrying out the study?
I would recommend putting this information up early so that it conditions the reading of the report. Likewise I would put the key considerations with respect to methods early in the Executive Summary.
