Four different kinds of interventions in the project have been observed from a social learning perspective. We used participant evaluations and our own observations during these workshops to assess the impact of these four kinds of intervention. How do they enable to deal with the ambiguity provoked by frame diversity? And do they foster constructive ways to connect knowledge from different backgrounds?Organizing interactive workshops: evaluations and observations
To assess the impact of organizing interactive workshops, rather than relying on written communication, we have assembled evaluations and observations from 6 project workshops. We report the results thematically hereafter.
Participants in interactive workshops valued them for several reasons:
In this sense, the workshops contributed to the relational side of connecting the people using the different frames: getting to know each other and mutual expectations, finding a place and a role in the project, starting to work as a group.
That the participants referred to the more interactive parts of the workshops as the more fruitful ones adds to this picture. Working in smaller groups was experienced as the most helpful to explore and define new concepts.
Also working with an open format ("market") to stimulate intensive interactions was valued:
From our observations it was also clear that small group activities during the workshops generated much more lively and in depth discussions than the usual presentation plus questions format.
According to the experiences of the participants, open communication contributes to the value of interactive workshops.
One of the participants reported that his way of framing adaptive systems was changed through a workshop ("my mental model about adaptive systems evolved"). From our observations, an interactive discussion episode during a workshop about the difference between current water management, integrated water resources management and AM allowed an exploration and redefinition of these concepts among the participants in ways that would be very hard to achieve through other means.
As a downside, frustration was also expressed with respect to the high investment of time and resources that workshops require. It seems like there is never enough time in a workshop:
Furthermore, the important relational effects a workshop can have, impacts only the people who were there, and the results that are generated depend on the group of people that went through that process together.
Facilitation: evaluations and observations
Evaluations and observations from 4 project workshops served as a basis for assessing the impact of facilitation.
At the end of the kick-off meeting of the project, the difficulty of taking the double role of presenting a lot of information and at the same time facilitating the discussion was identified in a debriefing between the organizers and observers of the meeting. Presenting calls for clear explanation and information flow from presenter to public. Facilitating the meeting calls for stimulating participation, checking comprehension, following up on comments and explicating (and checking) the goal of the meeting and what we expect as outputs. It’s not an easy job to combine both these roles in one person, and presenters seemed to have difficulties with it.
This was one of the reasons why a next workshop was specifically designed and facilitated for exploring the different frames of reference of the participants and dealing with them in constructive ways, in order to reduce confusion and increase mutual understanding. This took the form of inviting participants to prepare a 2-slide presentation on their view of the workshop’s central topic. These presentations were then given at the start-up of the workshop. In this way, a number of differences in conceptions came to the fore, which might otherwise have remained implicit. The way this phase of the workshop was designed and facilitated allowed people to better understand the background of the others around the table and use that knowledge throughout the further discussions. Comments of the participants when evaluating the workshop show they valued the influence of the facilitators on the design of the workshop as well as the role they took during the workshop.
Group model building: evaluations and observations
By contrast, in another workshop where only the presenters' role was taken and important differences in interpretations and views between the presenter and the other participants emerged but remained unaddressed, the following comment was given at the end of the day: "Did we have to meet for this ? We could have read this. I expected more guidance."
Sometimes one person is able to fulfill both the roles of participating and facilitating. In that case process skills are combined with a good scientific understanding, the latter increasing the credibility of the facilitator. This situation was present in a workshop that was organized to change the dynamic of a rather polarized discussion between two research groups. A third group of researchers was brought in. They had their own alternative approach and thus contributed to the discussion at the content level. At the same time interventions on the procedural level were made by this third party, leading to shared action plans and thus improving the interaction process. The example shows that the facilitator’s role can be taken in an informal way if this is accepted by the group. One of the group members confirmed the effectiveness of this setting: "So in the meeting yesterday we brought in X, who hadn't been part to that, I mean, been part of some of the discussions. A new one can and could immediately see how it could work, could see a role for themselves. The rest of us were willing to allow this."
To assess the impact of group model building, we analyzed observations and evaluations from the 2 first project workshops where UML was intensively used for building up joint conceptual frameworks.
Several graphical notations for the graphical visualisation of the framework were discussed at project workshops and bilateral meetings. UML (unified modelling language) was finally adopted as the modelling notation for the project framework on transition towards AM. UML is not a running model itself. It is rather a notation system that allows documenting or specifying knowledge about objects, relations and associations, workflows and processes, responsibilities, information flows, interfaces, etc. UML supports different views or frames on the same part of the world. All ‘views’ or diagrams share the same terminology and their level of detail depends on the degree of information that is required to understand a certain problem. Due to this approach a diagram is a view into a model presented from the aspect of a particular perspective (e.g. a stakeholder), it provides a partial representation of the system, and it is semantically consistent with other views.
Using UML in group discussions helps to make mutual assumptions explicit, because everybody attempts to translate his or her concepts into a common language. In selecting aspects, labelling them, drawing the relations and labelling the relations, differences between participants’ frames can emerge and can be discussed. It also helps to keep the attention focussed on the developing diagram and it results in a tangible output of the discussion (one or more diagrams).
Making the diagrams helps to identify where knowledge of a system or process is incomplete. Information about some elements or links may be missing.
It’s important to embed the making of diagrams in a larger process and clarify the goals of making them (knowledge representation, integration, facilitating discussions, ...). Otherwise, people may not see the point of making the diagrams, or at the other extreme, people may start using UML indiscriminately. Keeping track of the developed UML diagrams and limiting their number by prioritizing will not be an easy task.
In terms of finding a mutually workable representation, the top level representation (the “matrix”) itself may be the biggest challenge, since this captures world views on a high level of abstraction. In some cases, starting from representations of concrete subsystems may be easier (e.g. starting from the most important issues in a specific case).
A workable equilibrium between technically correct UML and easily understandable UML should be found. The best diagrams may be those where “UML-experts” watch the formal correctness, while “UML-laypersons” assure it is easily understandable for non-experts as well.
Using concrete case contexts: evaluations and observations
A characteristic of UML as a formal language is that the visual lay-out of elements and relations in a specific view is technically meaningless: as long as the entities, attributes and relationships remain the same, you can rearrange the visual representation as you like without affecting what it means in UML. People are used however to derive meaning from the visual aspects of a diagram (above-below, left-right, close-distant, ...), and this may affect the meanings that are connected to the diagram. The advantage is that UML does not rely of these often implicit and possibly diverging meanings of the visual aspects of the diagram. The disadvantage is that people may still read those meanings into the visual aspects of the diagram and make diverging conclusions.
A general concern with respect to the integration of different theories into a conceptual model, is the difficult distinction between (1) connecting concepts that represent different parts of reality, and (2) connecting concepts that categorize differently the same part of reality. The problem is that our concepts to some extent define what we take to be the reality. The more cautious approach may be to allow for parallel representations of parts of reality where or when necessary (different ways of framing the issue), and try to identify overlapping parts and look for complementarities where possible.
To analyze the impact of using concrete case contexts we used evaluations and observations from 3 project workshops. The third workshop was explicitly designed to profit as much as possible from the use of concrete case contexts.
The idea of using concrete case contexts when confronted with diverging theoretical frameworks, is that the meaning of a concept, theory or method can be easier understood when we look at specific contexts and illustrate there what the concept means. The concrete case context can then provide the necessary common ground to discuss concepts that come from different backgrounds.
In one of the facilitated workshops, case presentations were deliberately used for clarifying concepts related to AM. Two people who were each very familiar with a specific case, prepared an overview of the situation. The other participants were then invited to draw upon these inputs, and the presenters, to clarify and discuss the various concepts. A number of evaluations which were voiced during and at the end of the workshop referred to this approach, evaluating it as motivating and helpful.
However, concerns were also raised that scientific rigor may suffer when focusing on concrete cases.
In another project workshop, efforts to integrate different approaches for vulnerability assessment didn't succeed. A direct comparison of different research methods seemed to be too sensitive. Probably the position of both research groups as well as the differing contexts in which these methods have been applied, made a comparison very difficult. The decision to choose one “stylized situation” on which the different approaches under investigation would be applied, created some commonality to easier interpret the differences in the results. This approach uses case contexts not in their full complexity and detail, but through a simplified representation. The stylized situation still refers to a specific situation in a specific case and is thus different from an abstracted theoretical model.
In the studied project, very different frames about uncertainty exist among scientists coming from different research traditions. In a workshop on uncertainty, concrete situations in which uncertainty was experienced by water management practitioners were collected through dialogue sessions with decision-makers in several case study contexts. By means of a few open questions they were invited to tell about specific uncertainty related situations they had experienced, and these were summarized as short stories or vignettes. The most striking illustrations of different types of uncertainty were selected from the stakeholder dialogues and presented at workshop among scientists. Scientists from different fields discussed in break-out groups how they would deal with the uncertainties in specific case situations. In the discussion participants could easily refer back to these case situations, which provided a common focus for the group discussions. The cases allowed them to present and explain better their concepts and the approach they would take by applying them to the case examples. In the evaluations of this workshop, participants referred to this way of working in positive terms.