A research project supported by the European Commission under the Fifth Framework Programme and contributing to the implementation of the Key Action “Sustainable Management and Quality of Water” within the Energy, Environment and Sustainable Development. Contract no: EVK1-CT-2001-00090
Overview of the project
Integrated catchment management has arisen because managing environmental processes independently does not always produce sensible decisions when the wider view is taken. However, the problem for those charged with integrated management is the complexity of the process they are attempting to manage. They are therefore turning to decision support systems. The models used in these systems tend to address single issues. However, the catchment manager needs to understand all the possible impacts of pursuing any given policy. Implicit in this is a requirement both to understand and to be able to model not only the individual catchment processes but also their interactions. This project is about making possible the construction of whole catchment models in order to facilitate the integrated catchment management called for in the Water Framework Directive.
The objective of this project, therefore, is to develop, implement and prove a European Open Modelling Interface and Environment (OpenMI) that will simplify the linking of models and hence allow catchment managers to explore the likely outcomes of different policies. This will be achieved through ten work packages:
The project will build upon existing experience of the members in the physical and IT domains as well as model linking. One of the primary design objectives of the OMI is to facilitate the migration of existing models to the new standard so that they are more widely accessible. This will enhance user acceptance for the standard, since it will enable them to work with already familiar models and tools.
The tangible benefits that the project will deliver are:
- The simplification of the model linking process leading to an improved ability to model process interactions, the ability to use appropriate model combinations and the ability to swap in and out different models of the same process and hence facilitate sensitivity analyses and benchmarking.
- The ability to represent feed back loops and process interactions that are key to integrated management and the development of sustainable policies.
- The availability of a communication standard for modelling leading to increased choice for model users, an increased market for suppliers and increased opportunities for the creation of Small Medium Enterprises (SME).
- A reduction in development time for decision support systems.
- The contribution to the implementation and evolution of different EU policies.
OpenMI in integrated water management
The inevitable consequence of the rising aspirations of the world's population is to increase the level of competition for scarce natural resources. In former times, when the pressure on the environment was at a lower level and there was a large natural buffer in the system, it was possible to consider problems largely in isolation. Then, the effects of any given decision were usually local. Now, this is no longer the case. An apparently beneficial decision in one area of policy or operation can have major and often less desirable repercussions elsewhere, in both the natural and man-made environments. Integrated catchment management is intended to address this issue by creating regulatory and other organisations whose remit is to consider many, if not all, the competing uses for our environmental resources and attempt to devise and implement sustainable polices for their use.
Implementing integrated catchment management presents many challenges because it involves making highly subjective value judgements about matters that are not directly comparable, for example, reducing river pollution versus the need to maintain employment. The complexity of environmental processes and the ways in which they interact compound these problems. Indeed, the problems are so complex and require such a breadth of understanding, much of which we do not yet possess, that integrated catchment management is beyond the capacity of most normal people to deliver, hence the current need to develop decision support systems to assist in the process.
In this context, a decision support system (DSS) comprises one or more models and their associated data. The models are used to predict the likely outcomes of pursuing different policies for given scenarios and thus contribute to the decision making process.
Decision support systems are having to become ever more sophisticated as we attempt to model and so understand more and more of the likely consequences of following any particular policy, for example, the socio-economic implications of river regulation. It is not yet feasible to have a single model of all the processes taking place within a catchment. The reality for many years to come is that model linking will be used to simulate complex processes.
Models, however, have different strengths and weaknesses and during the various phases of a modelling exercise, it may be appropriate to replace one model by another. Different situations require different combinations of models. Although, individual organisations have addressed the linking problem, there is no generic solution. Current solutions tend to be confined to the temporal dimension and do not address the spatial aspects of linking. Model linking is therefore either confined to the products of a single supplier or requires a major software development exercise. The absence of an open modelling environment therefore makes it difficult to capitalise on the huge past European investment in model development.
The objective of this project, therefore, is to develop, implement and prove a European Open Modelling Interface and Environment that will simplify model linking and address the problems all the problems involved. The purpose behind the objective is to make possible the integrated water management that the Water Framework Directive seeks to achieve.
Model linking is at a very early stage. The reason for this is that in many countries there has not been the legislative requirement or organisational structures to implement integrated water management. Consequently, the need and the funding to solve the problems have not been there. The costs of developing a generic solution to such problems are really only justified when it can serve a large user base. Until relatively recently, that base has not existed. As importantly, the computing power and fast communications to support linked modelling have not been widely available.
The first models generally addressed a single issue and were usually self-contained - see Figure 1. Model linking is a relatively recent practice. So far, it has been achieved either by running models serially, using the output of one model run as the input of the next - see Figure 2 - or by writing a very specific shell programme in which all the models are embedded - see Figure3.
A good example of the former is contained within a recent large coastal zone research programme. Here, the interactions of the rivers, estuaries, coastal waters and the deep oceans were modelled by taking the outputs of the river models and feeding them into the estuarine model and so on down the chain. This simple approach side-stepped many of the potential problems such as interfacing 1-D, 2D and 3D models working to different spatial and temporal concepts, scales and resolutions. It was merely necessary for the modellers to ensure a common understanding of the variables and agree an interchange file format. However, the approach was not able to represent many of the feed back loops, which, in other situations, might have been important.
Figure 1. Self contained model with no facilities for linking
Figure 2 Linking models serially by file transfer
Where feed back has been important then, generally, modellers have stripped the models' codes out of their original shells and reassembled them in a new shell. The models have had their time steps synchronised and they have been run in parallel. Models of the interaction of river flow, chemistry and ecology are perhaps the best illustration of this approach. In this situation, flow and chemistry affect plant and animal growth, which in turn affects flow and chemistry. Running the models separately would completely fail to represent this interaction - see Figure 3.
Figure 3 Model linking to represent feed back loops
A number of frameworks into which alternative models can be plugged have been developed but most have not progressed beyond the conceptual design stage and only a few have been implemented. So far, the frameworks have represented a specific situation. Each slot for a model has been for a particular type of model and the models used in that slot have generally had to work on similar principles. In most cases, these frameworks have been designed to run on a single machine. Few attempts have been made to model environmental situations where the models are geographically distributed running on different machines. The problems of joining two models together and producing scientifically sensible results has been challenging enough, without adding the complexities of attempting to do it through firewalls, across networks and operating systems, not to mention national frontiers. However, sufficient progress has been made to convince the members that a generic solution to linkage is now feasible and to justify their own investment in the project.
The major innovation that this project seeks to bring about is the introduction of 'plug and play modelling' in the same sense that it is now possible to plug most peripherals into most PCs either directly or via a network. To achieve this goal, similar problems have to be overcome. They fall into three classes: technical, standards and creating the collaborative environment in which users and developers benefit from the success of the work. The innovative aspects of the programme lie in all three areas.
The challenge is to analyse and characterise the generic nature of the information that passes between models, databases and other tools together with the situations that lead to a need to exchange data. Additionally to the links between models of the physical world such as hydrological and water quality models, links between these models and socio-economic models will be also considered. If a generic solution can be found, then it should be possible to conceive of a set of conceptual 'plugs, sockets, adapters, converters and cables' that will enable the information to be passed. However, these devices will only be useful if the sending and receiving models and databases have a common understanding of the information being exchanged, hence the need for standards. Just as different items of electrical equipment receive inputs and generate outputs in a whole range ways, so too do models. For example, some need to produce data on a daily time step while others use monthly or annual data. Some use raster spatial data while others are vector based. Therefore, there is a need for the equivalent of electrical adapters, transformers and converters to handle the equivalent problems of connecting incompatible plugs and making voltage, frequency and other conversions which equate, in modelling terms, to unit of measurement, scale, resolution, frequency, code, language and many other conversions and translations.
Our belief that a common interface is now feasible is based on the convergence of the members' individual approaches and our observation that the global trend in modelling and database design is towards the 'object oriented' paradigm. This provides the generic elements of objects, properties, methods and events in terms of which all model data and modelling activities can be described. The main technical problem is therefore one of developing a suitable class library that all modellers can use to pass information whatever its type. If the end result conforms to current and forthcoming industry standards, then it will be usable with most of the current programming languages used for modelling.
The result will be a standard Open Modelling Interface and Environment available to all modellers, which, if they choose to use it, will enable their models to be linked to any other model, database or tool conforming to the standard.
The communication standard opens the way for a number of advances that will help to facilitate the development of future DSS. A key aim in creating the communication standard is to encourage DSS designers to breakdown their systems into distinct modules, each with a clear function: user interface, model, database or etc. Not only is this good design practice anyway, but, if it can be achieved, then the way is opened up for drag and drop DSS construction. For each module, the inputs and outputs must be defined. The user will drag the modules from a series of menus to a construction area. Linking will be achieved by a further drag and drop process similar to the process by which tables are related in relational databases. Links between models and the database and models and the user interface can be achieved in the same way - see Figure 4.
Figure 4 Drag and drop modelling
Having created such an environment, it is easy to see that a demand for a range of tools will develop. An obvious example is a device to facilitate the calibration, validation and operation of the linked model. Others are the adapters and converters to enable models operating to different scales, resolutions and on different principles to communicate.
In such a scenario, exchanging one model for another for sensitivity testing and benchmarking should be relatively simple.
A technical solution alone, however, is of little use if the environment for its wide adoption has not also been created. An important element of the programme is therefore the involvement of all the key European producers and a range of users within the project.
Contribution to EU Policies
Contribution to the implementation of the European Water Framework Directive
A major requirement of the Water Framework Directive and an integral part of integrated water management is the production of river basin management plans. These plans often have an international dimension and their production requires co-operation between the institutions and agencies of many countries.
Models play a key role in the development of many of the plans. As the number of process interactions that need to be considered increases, so too will the need to link more and more models together. Equally, as it becomes necessary to link more together, so it will become less and less likely that one supplier will be able to provide all the models. Hence, builders of linked models will need to obtain their models from different suppliers. If this is not to become a prohibitively expensive process then a standard model interface is required.
The Open Modelling Interface will increase the competitiveness of EU suppliers, increase the choices open to model users and stimulate designers to produce new and innovative products.
Currently, many models are developed within the academic sector. Because they are stand-alone models and cannot be easily linked to other models, they frequently die at the end of the research project within which they were developed. An open modelling interface creates a whole range of opportunities for entrepreneurial researchers to take their models out into the market place and either licence them to the major players or create their own small medium enterprises (SME's).
Models are expensive and users become locked in to particular vendors. An open modelling interface removes this problem and users will be able to pick and choose the models most appropriate to their needs.
The OMI will make a significant contribution to the effectiveness of EC research
In addition to greatly facilitating the formulation of water policy, a generic 'plug and play' mechanism should also remove many of the barriers to co-operation at the scientific level. The ability to create models of process interactions more easily should both allow us to extend our knowledge of those interactions and improve their representation in models, hopefully improving the effectiveness with which the Water Framework Directive is implemented. Duplicate developments of model code can be more easily avoided, as suitable existing models can be integrated into the modelling system being constructed. The standard could also improve the accessibility of river basin data and hence make the comparison of models simpler.
HarmonIT will facilitate the type of integrated modelling required by EU and therefore will contribute directly to the EU's ability to honour its International Agreements.
The EU is a signatory number of international agreements which either directly or indirectly refer to the water environment. The outcomes from HarmonIT will enable the EU to meet its commitments under these Agreements. The Earth (Rio) Summit sets specific integrated catchment management targets, while many of the renewable energy sources specified under the Kyoto Protocol, have an impact upon catchments and eco-systems.
HarmonIT will provide a framework to facilitate rapid uptake of new integrated catchment models, DSS and environmental management systems in the Accessing states.
HarmonIT will make a direct contribution to EU Environmental Policy.
HarmonIT will assist in the exchange of environmental information technologies between members states and facilitate the implementation of existing legislation . The project outcomes will increase the EU's capacity to produce tools with which environmental concerns can be integrated into a broader range of EU policies. It will assist member states, regions and local communities implement Natura 2000 objectives, and provide a facilitating mechanisms to improve EU industry's ability to assess, model and improve environmental performance.
The outcomes from HarmonIT will facilitate achievement of the objectives of the draft Sixth Environment Action Plan (2001-2010). In particular, it will enable the EU, member states, regional and local communities to assess, monitor and improve actions in relation to climate change, bio-diversity, human health, and in the management of natural resources and waste.
HarmonIT will fulfil a number of the priorities of the European Research Area.
The project is information technology based, targeting a defined area of the international market. The project objectives are set against a medium to long term time frame, while specifying immediate short-term deliverables. Much of the project and its outcomes will be undertaken through large-scale fast communications networks. The partners in the projects are world leaders in their respective fields and through collaborative work over the past ten to twenty years have established a proto-Network of Excellence. The HarmonIT project will capitalise upon and feed back to national programmes in the area of IT harmonisation and will draw in expertise and advice from throughout the world.
Description of work
- The work plan has been divided in ten work packages. The key objectives of the work plan and their relationship to work packages (WP) are:
- To review the state of the art with respect to frameworks for model integration (WP1).
- To analyse the needs of integrated water management for linked models and hence identify the user requirement (WP2). The requirement will concentrate particularly on the shared needs of EC members, the problems that occur in modelling trans-national catchments and the facilities that will give maximum benefit to both model users and producers in terms of increased choice and market size.
- Based on the requirement produced in WP2, to establish a standard for model linking (WP3) consisting of:
- a) A specification for the domain boundaries
- b) A specification for the software architecture for model integration
- c) A specification for a set of basic standard tools related to model integration
- To develop the detailed design for the linkage mechanisms and component interfaces (WP4).
- To develop the detailed designs for the generic tools (WP5).
- To implement and test the detailed designs (WP6).
- To prepare guidelines to enable model developers to utilise the new standard (WP7).
- To prove the concept by migrating a selection of model components to the standard architecture - including components developed by organisations which are not taking part in the definition of the standard (WP6 and WP8).
- To further prove the concept by linking model components from different sources and proving that the integrated models are working (WP8).
- To disseminate information about the new standard to potential users through publications, conferences and workshops (WP9).
- To establish a plan which will ensure that the Open Modelling Interface will continue as a living standard after the end of the project (WP9).
The co-ordination of the work of the project team (progress, deliverables) is the responsibility of WP10. The organisation of the work packages is illustrated in the following diagram. Where necessary, feedback relationships between work packages have been incorporated in the work plan.
Work Package 1. Review of current practice
- To review current practice with respect to the use of linked modelling in integrated water management, model architectures and techniques for linking models.
- To identify relevant standards and how they are maintained
- To identify the limitations of current techniques and procedures.
- To identify priorities.
- To provide an input to Requirements Analysis (WP2) and Architecture (WP3)
Work Package 2. Requirements analysis
- To specify the functional capability of the Open Modelling Interface and Environment with respect to the linkage mechanism and an initial set of tools to support and monitor its use.
- To make a preliminary identification of the models to be migrated.
- To achieve positive acceptance of the proposed functionality by the potential user community.
Work Package 3. Architecture
- To develop a generic conceptual model for the Open Modelling Interface and Environment that will enable the concepts of the system to be explained to users at all levels and which meets the requirement defined in WP2.
- To develop the high level system architecture by which the conceptual model will be implemented. The architecture will define the main components, their relationship one to another, the information that will flow between them, the mechanisms by which they will communicate, the means by which control will be exerted.
- To achieve positive acceptance of the model and architecture by the potential user community.
Work Package 4. Detailed design - model linkages and component interfaces
- To extend the architectural design, i.e. the result of WP3, into a clear, well-documented and detailed specification document, covering the data models, data definitions, linkage mechanisms, and interface definitions for all components of the Open Modelling Interface and Environment.
Work Package 5. Detailed design - generic tools
- To prepare the detailed design specifications for the selected set of tools identified in the Architecture document (WP3) so that these tools may be implemented in WP6.
In contrast to WP4, which specifically deals with the interfaces of components and the general framework for model linking, this WP deals with the detailed functionality and design of tools for creating and monitoring links and for managing the linked models.
Work Package 6. Implementation and migration
a) To transform the detailed design specifications prepared in previous work packages into operational code.
b) To migrate a selection of available existing simulation models used in water management to the IT framework developed in a.
Work Package 7. Guidelines
- To develop guidelines for the use of the Open Modelling Interface and Environment by model developers and users.
- To publish the guidelines in formats suitable for both the web and printing.
Model developers are all those who are involved in the conception, design, prototyping, production of operational versions and the commercialisation of models.
Model users are those who will be using the Open Modelling Interface and Environment to link models together or substitute one model for another and those who will be running models in order to assist the decision making process.
Work Package 8. Proof of concept
- To apply the new mechanisms to one or more real integrated water management situations requiring the linking of models in order to ascertain the strengths and weaknesses of the approach.
- To test (in increasing order of complexity):
- Simple linking by file exchange
- The ability to build simple linked models from the components produced by the project, where the models are drawn from different organisations.
- The ability to handle scale, resolution, frequency and unit of measurement differences between models
- Swapping one model for another where the model dimensionality is changed, e.g. swapping a 1 dimensional model for a two dimensional model.
- The ability to represent feed back loops
- To test the Guidelines from WP7.
Work Package 9. Dissemination and exploitation
- To gain the modelling community's acceptance of the Open Modelling Interface and Environment (OMI)
- To allow the modelling community to benefit from and contribute to the OMI during its creation
- To enter into dialog with the major international standards agencies
- To prepare the modelling community for the OMI so that they can plan for its incorporation into their work
- To plan for and implement the introduction of the OMI within the participants' own organisations
- To develop the participants' marketing strategies for exploiting the OMI first within Europe and then more widely
- To put in place the organisation to support the OMI into the future.
Work Package 10. Coordination
- To co-ordinate the work of the project team and ensure that the deliverables are produced on time and to budget.
- To co-ordinate the work of this RTD with that of the work of the Concerted Action and other RTDs in the cluster.
- State of the Art Review, by Chris Hutchings
- Generic Framework : a broadly supported Dutch modelling environment, by Michiel Blind
- Requirement Analysis Final Report (approved), by Jan Gregersen
- OpenMI Proof of Concept Workshop, by Roger Moore
- Ensemble Kalman filtering with OpenMI, by Peter Gijsbers
- The future of the OpenMI, by Roger Moore
- Update on the Harmonit Project, by Roger Moore
- Update on the Harmonit Project, by Roger Moore
- HydroInformatics Conference presentation, by Jan Gregersen
- OpenMI architecture - the details (Hydroinformatics conf 2004), by Peter Gijsbers
- IEMSS Conference paper, by Jan Gregersen
- Linking Models for the Water Framework Directive, by Roger Moore
- HydroInformatics 2002 paper, HarmonIT project, by Peter Gijsbers
- HarmonIT - who & what, HydroInformatics 2002 presentation, by Peter Gijsbers
- HarmonIT - An open Modelling Interface and Environment, by Roger Moore