Testing of the Guidance on Risk Assessment of LMOs

Roadmap
Stacked genes
Abiotic stress
LM mosquitoes
LM trees
Monitoring
General

ROADMAP FOR RISK ASSESSMENT OF LMOS

PRACTICALITY

Parties

Austria: Practicality was considered in regard to the targeted purpose. Specifically the Roadmap was regarded as an instrument to introduce a basic concept for risk assessment, i.e. indicating important elements of a risk assessment framework (policy), rather than a document presenting very detailed guidance for individual case-specific risk assessments. As a reference document relevant for the development of an appropriate overall approach to risk assessment by risk assessors it is considered to be of very high overall practicality. Specifically the points to consider included in the Roadmap connect to aspects which need to be complemented by further guidance available/developed at the level of implementation to address practical details of assessments. In case of our testing we considered that highly consistent additional guidance is available at the EU level (EFSA (2010): Guidance on the environmental risk assessment of genetically modified plants. EFSA Journal 8 (11): 1879). Line 178: The above consideration should be underlined in the indicated ways of use of the Roadmap. We suggest to include respective formulations: 'The Roadmap may be useful as a reference for designing and planning risk assessment approaches and identifying the need for development of further guidance by risk assessors. It may also be of help for risk assessors when conducting risk assessments and as a training tool in capacity-building activities.'

Belarus: 178-179 - The Guidance is worked out in such a way that it is not prescriptive in character and is devoid of particular recipes and prescriptive examples for assessing risks. The Guidance is rather difficult to be used as a training tool for unexperienced experts therefore. It most likely is a good tool for already trained experts. Also we propose to arrange terms in glossary in alphabetic order. Each term should be described as full as possible in clear manner. So, we would like to suggest the following text for lines 178-179: '...reviewing risk assessment and as basic tool for developing training modules on risk assessment process (procedure)'. 193 - We propose to make the following correction: '...acceptable and/or manageable...' 199 - We would like that for simplicity of perception such concepts as 'protection goals' and other ones were not only interpreted in the section 'Use of Terms' but also particular examples or references to the examples illustraiting well the concepts were given just after the definition in the text. 453-456 - We think it i necessary to include information wherther target gene is expressed in a certain part of a plant (e.g. in tubers) or in all the plant parts with indication of the expression level. It is desirable to include figures illustrating instrument readings, for example, the gene expression level, insertion sites, copy number, PCR results, verifying stability and integrity of insertion. It is desirable also to give references to analogous scientific research and tests, if any, or to indicate them as 'know-how'. 519-522 - It would be fruitful to give an example of quantitative determination of uncertainty. The same thing could be made in the text of lines 580-582. 2030-2031 - In our opinion, it is unfortunate wording of the term 'antagonism'. We suggest to use the term which expresses an ecological sense of this term relevant to LMO and biodiversity, namely: 'Antagonism - relationship of two (or more) species of alive organisms when individuals of one species negatively influence on individuals of other species as competition for use of the same ecological niche resulting in displacement (partly or in full) of competitors by one species'. 2064-2065 - It is desirable to clarify the definition 'cumulative effect'. For example, the cumulative effect is the effect (impact), enhanced as a result of a joint or repeated influence of LMOs or their products on the receiving environment. Effect of accumulated exposure. [translation from original in Russian] 209-211 - This guidance is not binding and lacks specific instructions for and prescriptive examples of conducting risk analysis. Therefore it is rather difficult to use this guidance as an educational medium at the early stages of training of experts. It is probably a good tool for already trained experts. We think that for training purposes a special manual (a training module) should be prepared based on this guidance with a detailed explanation of all terms used. Consequently we propose to present lines 21--211 as follows: '... when conducting or reviewing risk assessments and as a basis for developing training modules when conducting capacity-building activities'. 213-217 - For a clearer understanding of the text by Russian-speaking specialists we propose to introduce into it the following amendments: '... relevant to the risk assessment of all types of LMOs and their intended uses within the scope and objective of the Protocol. This document has been developed based largely on received and gained experience to date concerning environmental risk assessment of LMOs as crop plants. 229 - To introduce the following editorial correction: '... estimated risk is acceptable and/or manageable…' 230 - To introduce the following editorial correction: '... uncertainty. Risk assessments serve a basis for decision-making regarding LMOs'. 237 - To make it more understandable one would want that such notions as 'protection goals' (цели защиты) etc.' were not only explained in more detail in the section 'Use of terms' but were accompanied by concrete examples or links to examples well illustrating these notions right after they are used in the text. 262 - To introduce the following editorial correction:'... information to be considered in the risk assessment, as well as the means to identify and describe degrees of uncertainties…' (the phrase 'describe uncertainty questions' (описание вопросов неопределенности) is incorrect here. Uncertainty is of a probabilistic character which can be expressed by a degree of this uncertainty). 521-523 - To give examples for the notions being used. 627-628 - To give a reference to an example of developing conceptual models while conducting risk assessment. 643 - To give a reference to an example of a quantitative expression of certainty. Ditto for 716, 770. 549 - 553 - To include information whether the target gene is expressed in a particular part of a plant (e.g. in the tubers) or in all parts of the plant, with the indication of the expression level. It is desirable to include illustrations showing the instrument readings, e.g. the expression level, the insertion sites, the number of copies, the results of the Polymerase Chain Reaction, confirming the stability and integrity of the insertion. It is desirable to provide references to similar scientific research and testing, if any, or indicate that these are the 'know-how'. 1100, 1101, 1102, 1103 - These sentences contain repetitions. The meaning of 'anti-nutritional factors' (антипитательные факторы) is unclear to Russian-speaking specialists. We suggest to use the term 'antimetabolites' (антиметаболиты), used in the Russian language literature. 2524, 2525 - We think that the notion of 'antagonism' (антагонизм) is ill-phrased. We suggest the following version of explanation of the term as relevant to LMOs and biodiversity: 'Antagonism is an interaction of two (and more) types of living organisms in which individuals of one species negatively influence the individuals of other species while competing for occupying the same ecological niche and partially or fully forcing the competitors out of it'. 2564-2565 - It is desirable to specify the phrasing of the term 'cumulative effect' (кумулятивный эффект). For example, the cumulative effect is an effect (action) enhanced as a result of combined or multiple action of LMOs or products thereof in the receiving environment; the accumulation and summation of this action.

Belgium: The testing was conducted as an individual exercise. Different levels of agreement/disagreement (i.e. 'Neutral' or 'Agree') with regards to the practicality of the Roadmap were expressed amongst people.

Bolivia (Plurinational State of): (translation) On line 203, under INTRODUCTION, the guidance should mention that in order to conduct a Risk Assessment there must be adequate economic resources available. On lines 286, 287, and 288, the type of information in question must be clearly specified. Sensationalist information or information drawn from press articles must be avoided. A note of clarification in this sense should be included.

Brazil: The practicality of the Guidance could be improved. There is a lack of clarity on how to relate the different steps of the assessment. It is also important to consider the nature of the potential damage and to compare it with common practices of agricultural production, as well as those associated to human and animal health. There is also a need to avoid considering the gene flow for every LMO as damage, when this is a common phenomenon in nature. The Guidance should avoid prescriptive approaches and provide the necessary flexibility for the development of biotechnology. More specifically: - Step 1 is not well explained: is that a identification of potential adverse effects step (where the question 'What could go wrong' could be applied) or a 'what, why and how' step (line 398)? This step should be better explained as critical to the success of RA. An inappropriate risk hypothesis may misdirect the whole risk analysis process and lead to the imposition of unnecessary controls to manage risk. Although all the necessary information is listed in the Step 1, there is a lacking of clarity about how to link this information in a logical way to define a causal pathway. - There are some sentences in the rationale of the text that creates complexities instead of explaining the purpose of each step. For example the sentence 'These includes any changes in the LMO, ranging from nucleic acid (including any deletions) to gene expression level to morphological changes' (lines 426 - 427) does not help to explain the step 1 of RA and the idea of this sentence is already covered in the 'points to consider' section. Also most of the examples in the text are not explained well enough to be used as practical examples (eg. lines 429-431; lines 513-514). - To be a 'scientifically plausible scenario' (line 407) is there is a need to have a concrete pathway linking the proposed dealings with potential adverse effect. Only these scenarios should be considered in detail in risk assessment. - The step 1 is based on the scope and context established in the planning phase so the ´points to consider´ section are the detailing of factors considered in this context to identify possible adverse effects. The way 'points to consider' are presented in the Guidance is therefore confusing due to: some information required are part of the previous step 'establishing the context' (line 458-450 and line 460-461); some information are being redundant (line 453-456 and 473-475) ; some information requirements are presented in the context of many factors (like type of irrigation, amount of herbicide applications, methods for harvesting and disposal etc. - line 493 and 494) and some information are required in a context using vague concepts (like cumulative effects – line 495). The entire 'points to consider' section could benefit from a simplification. - In the step 3 there are many ´points to consider´ that are part of the ´establishing the context ´section like line 600, line 601-603 and line 604. Those factors are out of place and can make more difficult to follow a logical pathway in the process.

Colombia: (translation) On lines 262 to 265, where it says: ‘Information, including data, may be considered relevant if they are linked to protection goals or assessment endpoints, contribute to the identification and evaluation of potential adverse effects of the LMO, or if they can affect the outcome of the risk assessment or the decision,’ the word ‘affect’ should be changed to ‘support’ (in the Spanish version it should be changed from ‘afectar’ to ‘soportar’). The guidance considers the need to establish protection goals and assessment endpoints (lines 328-370), which must be identified prior to analyzing the characteristics of the LMO in the receiving environment. However, it is important that the guidance clearly state that these should be of a general and basic nature and aimed at defining and targeting the processes in the risk assessment, and that they should never be so specific as to generate non-objective assessments with preconceived ideas that would entail losing sight of the general picture of the LMO and its characteristics. In this sense, it would be very helpful if the roadmap were more instructional and user-friendly, including more practical examples that really guide assessors in the ‘how to,’ so as to enable a consistent and solid approach to the risk assessment. Consequently, it would be important to indicate that protection goals and assessment endpoints are the initial questions used for formulating the problem, but that they may change after an objective analysis of the characteristics of the LMO. It would also be interesting to combine what is proposed on lines 328 to 370 with what is mentioned on lines 371 to 373. With respect to the ‘The choice of comparators’ section (lines 377-408), this section elaborates on paragraph 5 of Annex III of the Cartagena Protocol, but it is not clear what it is referring to when it says ‘risks posed by the non-modified recipients or parental organisms in the likely potential receiving environment’ (in Spanish, ‘riesgos planteados por los receptores no modificados o por los organismos parentales en el probable medio receptor’). The Spanish translation is imprecise, as it is not clear what is meant by ‘riesgos planteados,’ which could be interpreted either as: a) The risks that the non-modified recipients or parental organisms ‘may face’ (‘puedan tener’) in the potential receiving environments with respect to the LMO; or b) The ‘risks represented’ (‘riesgos que representan’) by the non-modified recipients or parental organisms in the potential receiving environment, even though they are not LMOs. Because of these two possible interpretations it is not clear if what needs to be assessed is the risk that the LMO (transgene-conferred trait) represents or poses for its non-modified receptors and parental organisms in the potential receiving environment, or if it is the level of risk that needs to be assessed by comparing whether the risks represented by the LMO are caused by the intrinsic traits of the species or by the transformation, through an exercise that compares the risks posed by the non-modified recipients and parental organisms in the potential receiving environment to the risks posed by the LMO. That is, thus distinguishing the risk that the species as such represents from the risks associated with the transformation (transgene-conferred trait). Similarly, we suggest avoiding the use of new terminology, and referring to the ‘comparator’ (‘comparador’) as ‘recipient organism or parental organism’ (‘organismo receptor y organismo parental’), as it could lead to confusion or generate ambiguities in the processes of interpretation. On lines 271 to 275, where it says: ‘knowledge and experience of, for example, farmers, growers, scientists, regulatory officials, and indigenous and local communities [may also be used]’ (‘considerar también el conocimiento y la experiencia de, por ejemplo, campesinos, agricultores, científicos, funcionarios de organismos normativos y las comunidades indígenas y locales…’), the following phrase must be added: ‘This knowledge will be used provided such information meets the standards established under paragraph 1, Article 15 of the Cartagena Protocol’ (‘Este conocimiento se considerará siempre y cuando dicha información cuente con los estándares establecidos en el artículo 15 numeral 1 del Protocolo de Cartagena’). The use of the terms ‘risks’ (‘riesgos’) and ‘hazards’ (‘peligros’) of LMOs in lines 418-419 must be revised. In that respect, it should be noted that the Cartagena Protocol refers always to ‘risks,’ without categorizing LMOs as inherently ‘hazardous’ (‘peligrosos’), and this must be maintained in the guidance, which must refer only to ‘risk’ conditions and not to ‘hazardous’ conditions. On lines 262 to 265, where it says: ‘Information, including data, may be considered relevant if they are linked to protection goals or assessment endpoints, contribute to the identification and evaluation of potential adverse effects of the LMO, or if they can affect the outcome of the risk assessment or the decision,’ the word ‘affect’ should be changed to ‘support’ (in the Spanish version it should be changed from ‘afectar’ to ‘soportar’). The guidance considers the need to establish protection goals and assessment endpoints (lines 328-370), which must be identified prior to analyzing the characteristics of the LMO in the receiving environment. However, it is important that the guidance clearly state that these should be of a general and basic nature and aimed at defining and targeting the processes in the risk assessment, and that they should never be so specific as to generate non-objective assessments with preconceived ideas that would entail losing sight of the general picture of the LMO and its characteristics. In this sense, it would be very helpful if the roadmap were more instructional and user-friendly, including more practical examples that really guide assessors in the ‘how to,’ so as to enable a consistent and solid approach to the risk assessment. Consequently, it would be important to indicate that protection goals and assessment endpoints are the initial questions used for formulating the problem, but that they may change after an objective analysis of the characteristics of the LMO. It would also be interesting to combine what is proposed on lines 328 to 370 with what is mentioned on lines 371 to 373. With respect to the ‘The choice of comparators’ section (lines 377-408), this section elaborates on paragraph 5 of Annex III of the Cartagena Protocol, but it is not clear what it is referring to when it says ‘risks posed by the non-modified recipients or parental organisms in the likely potential receiving environment’ (in Spanish, ‘riesgos planteados por los receptores no modificados o por los organismos parentales en el probable medio receptor’). The Spanish translation is imprecise, as it is not clear what is meant by ‘riesgos planteados,’ which could be interpreted either as: a) The risks that the non-modified recipients or parental organisms ‘may face’ (‘puedan tener’) in the potential receiving environments with respect to the LMO; or b) The ‘risks represented’ (‘riesgos que representan’) by the non-modified recipients or parental organisms in the potential receiving environment, even though they are not LMOs. Because of these two possible interpretations it is not clear if what needs to be assessed is the risk that the LMO (transgene-conferred trait) represents or poses for its non-modified receptors and parental organisms in the potential receiving environment, or if it is the level of risk that needs to be assessed by comparing whether the risks represented by the LMO are caused by the intrinsic traits of the species or by the transformation, through an exercise that compares the risks posed by the non-modified recipients and parental organisms in the potential receiving environment to the risks posed by the LMO. That is, thus distinguishing the risk that the species as such represents from the risks associated with the transformation (transgene-conferred trait). Similarly, we suggest avoiding the use of new terminology, and referring to the ‘comparator’ (‘comparador’) as ‘recipient organism or parental organism’ (‘organismo receptor y organismo parental’), as it could lead to confusion or generate ambiguities in the processes of interpretation. On lines 271 to 275, where it says: ‘knowledge and experience of, for example, farmers, growers, scientists, regulatory officials, and indigenous and local communities [may also be used]’ (‘considerar también el conocimiento y la experiencia de, por ejemplo, campesinos, agricultores, científicos, funcionarios de organismos normativos y las comunidades indígenas y locales…’), the following phrase must be added: ‘This knowledge will be used provided such information meets the standards established under paragraph 1, Article 15 of the Cartagena Protocol’ (‘Este conocimiento se considerará siempre y cuando dicha información cuente con los estándares establecidos en el artículo 15 numeral 1 del Protocolo de Cartagena’). The use of the terms ‘risks’ (‘riesgos’) and ‘hazards’ (‘peligros’) of LMOs in lines 418-419 must be revised. In that respect, it should be noted that the Cartagena Protocol refers always to ‘risks,’ without categorizing LMOs as inherently ‘hazardous’ (‘peligrosos’), and this must be maintained in the guidance, which must refer only to ‘risk’ conditions and not to ‘hazardous’ conditions.

Costa Rica: (translation) The tool is designed for people with experience on the subject of risk assessment. Countries and individuals with no experience will require training. The tool does not contribute a new approach, but it does help systematize information. As it is not a very rigid guide, it can be adapted for analysis.

Czech Republic: I propose to delete in line 407 the term 'scientifically plausible scenarios'. The sentence should be: 'In this step, risk assessors develop meaningful risk hypotheses…' I propose to change the text in line 423: 'considered in the context of any suitable comparator (e.g. the non-modified recipient or other reference line already adapted to receiving environment…' This is important in case of LMO with tolerance to abiotic stress, where the recipient species can be seriously handicapped in comparison to other naturally occurring species in receiving environment.

Denmark: No suggests for improvement

El Salvador: (to be translated) Lineas 324-328 A pesar de que la guía es útil, no es del todo amigable (fácil de usar) cuando se debe implementar paso a paso; en el caso de El Salvador cuya planta de reguladores y asesores de riesgos es limitada en comparación con otros países de la región (Costa Rica, Honduras) su contenido por momentos resulta un tanto confuso y difícil de aplicar, a menos que los reguladores tengan experiencia en temas similares. Líneas 351-370 Es poco práctica en la mayoría de circunstancias. En la mayoría de los casos, es la especie que se utiliza como comparador con un énfasis en las variedades cultivadas ya que estos proporcionan la mayoría de la información y los mejores indicadores pertinentes de los riesgos planteados por el organismo parental no modificado Líneas 428-431 Se debe distinguir claramente entre los ensayos de campo y las liberaciones comerciales ya que varios de estos tipos de efectos no parecen relevantes en ambos escenarios y considerando las decisiones que cada país tomará de acuerdo a sus propias condiciones, se debe revisar esto. Líneas 523-525, 580-582, 615-618 y 623-626 No establecen criterios del todo claros durante la fase de planificación (el contexto de la evaluación de riesgo), sobre todo en lo que se refiere a ‘riesgo indeterminado’ no resulta del todo práctico ya que siempre hay un conocimiento no finalizado (o incompleto) por lo que de mantenerse esta condición se estaría aplicando a todos los riesgos que se identifiquen en cada momento del proceso. Líneas 619-622 Sería muy conveniente que se utilizara ejemplos prácticos. Línea 634 No establece una clara diferencia entre las estrategias de gestión de riesgos que se han establecido como parte del contexto y las que eventualemente estarán surgiendo de la evaluación del nivel de riesgo global. Lineas 672-677 No se hace mención de distinguir estos usos de la vigilancia a largo plazo en la práctica.

European Union: The EU welcomes the endorsement of the risk assessment principles as demonstrated by the adoption of the case-by-case approach, comparative analysis with a selected comparator, 6-step approach to the risk assessment with an emphasis on the problem formulation, coverage of all areas of risk and the principles adopted in the monitoring post release. However the EU considers that the requirement (line 290) for various forms of uncertainty to be considered and described in each step of the risk assessment could be considered burdensome and disproportionate. It would be acceptable for the uncertainty for each indentified risk to be described, where relevant, under step 4 'An estimation of overall risk ….' The EU would also like to note that, in the EU, the risk assessment of an LMO for experimental purposes (i.e. a field trial) is the responsibility of the Member State on whose territory the release is to take place, therefore the European Commission is not in a position to comment on the practicality or any other aspect of the guidance in this respect and relies here on the comments made by its Member States.

Germany: - Line number 127: The Cartagena Protocol is mentioned as 'protocol' without further explanation. To the uninitiated reader it is not self-evident which Protocol is meant. Therefore, the Cartagena Protocol on Biosafety to the Convention on Biological Diversity should be referred to as such in this first sentence of the Preface of the Road map. - General: All mentioned articles should be linked to their respective texts for easier and quicker access to relevant text passages. - General: Especially in Part I, the structure of the document might be more easily navigated by lettering or numbering the headings. - General: The quick link option to the 'Use of Terms' section is very helpful and should be offered with more consistences. - Line number 263-265. Information is confusing. It is implicated that only independent experts can perform the risk assessment. Changing the wording to e.g. „Independent experts with the relevant background in the different scientific disciplines can serve in an advisory function during the risk assessment process or even perform the risk assessment themselves.' might be more appropriate and clear. - Footnote 11: This explanatory document could also be mentioned earlier and more clearly. - Footnotes 12 and 13 should link to the respective text passage. - Line number 343 ff: Sadly, an alternative to the comparative approach is not discussed but may become necessary considering future LMOs. - Line number 370: Link to references is missing (cp. Line 297-298). - Line number 371 ff: The mentioned Annexes and Paragraphs should be linked for easier access and understanding. - Line number 421: '(iv) transfer genes to other organisms/populations, and (v) become genotypically or phenotypically unstable' are per se no adverse effects and should therefore be omitted in the list. Still the mentioned processes may be part of a causal link or pathway to a secondary adverse effect and could be mentioned in this relation. - Line numbers 422-424: It is emphasized that the LMO should be considered in context of a comparator. It should be mentioned, that in special cases, were it is difficult to identify an appropriate comparator, the applicability of comparative approach may be limited.. An introductory guideline such as the road map might also propose solutions (maybe in a future revision of the road map?) - Line numbers 452/ 2173-2174: The given definition of ‘unintended gene products’ is not helpful. Maybe one or two examples would be advantageous (e.g. potential protein encoding sequences crossing the plant-construct insertion sites or elongation of the intended protein due to faulty or lacking terminator sequences etc.) - Line numbers 460-461: Should read for practicality reasons: 'Availability of data on the likely receiving environment which may serve as a basis for the risk assessment' since neither 'sufficient' nor 'meaningful baseline' is sufficiently well defined in this context to be useful for practical risk assessments. - Line number 491-492: To simplify reading and implementation point (p) should be reduced to '(p) Potential adverse effects of the incidental exposure of humans to (parts of) the LMO (e.g., exposure to modified gene products in pollen)'. The naming of 'the toxic or allergenic effects that may ensue' is redundant to 'Potential adverse effects' and can therefore be omitted. The second part of the point should be placed in a separate point, since agricultural practices that may be used are not directly connected to the issue of 'incidental exposure'. Moreover (p) might be better placed under Step 2 (exposure assessment) - Line numbers 493-494: new point (q) may read: 'Potential adverse effects of changes in agricultural practices such as type of irrigation, number and amount of pesticide applications, methods for harvesting and waste disposal, etc, induced by use of the LMO. Where use of other regulated products or practices are changed interplay with the respective risk assessments and regulations needs to be considered.' . Practicality of the risk assessment approach may be impaired, if no clear separation to risks from other regulated products is kept. - Line number 501: It is slightly confusing that the first sentence starts with a reference to step 4. The first sentence in explaining what should be done in step 2 should maybe start with: 'In this step the risk assessor evaluates the likelihood that each of the potential adverse effect identified in step 1 will occur.' - Line numbers 519-522: should read: 'If the likelihood of adverse effects being realized is difficult to assess it may be useful to reverse order of Steps 2 and 3 (s. Line 567 and Fig 1).' An assumption of 100% likelihood may lead to an erroneous indication of high risk although not supported by facts. - Line numbers 527 and elsewhere: the term 'likely potential receiving environment' is difficult to comprehend and to a certain degree redundant. - Line numbers 560 ff: Again, the evaluation of consequences should be done in context of adverse effects caused by a comparator. There are no hints given on what should be done if no comparator is available. - Line number 598: The terms 'combinatorial and cumulative effects' should be linked directly to 'Use of Terms' like all other terms before and not via footnote. - Line numbers 716-720: The respective articles should be linked. - Annex: Flowchart for the risk assessment: The instructions in the flowchart should be concise and brief and should inform the reader on what each step is about. Further information is given in the text and it is not necessary to repeat it here.

Honduras: (translation) The text of the guidance poses no concrete questions that are relevant to the risk assessment case studied and does not give the risk assessor the possibility of evaluating the context during the ‘problem formulation’ process. For example, it poses no questions regarding the type of pathogen in question and its relationship with the environment, how the pathogen is dealt with under conventional conditions (with toxic pesticides), what the susceptibility of other plant and insect species or microorganisms not targeted by conventionally-used agrochemicals is, among other questions. We found that the guidance was not appropriate for use by people who are not risk assessment specialists. Risk assessments must be conducted by interdisciplinary technical teams in order to cover the technical aspects of each specific case that is being assessed for risks. In this case, a risk assessment team must include specialists in agronomy, plant pathology, entomology, weed control, integrated pest management, molecular biology, genetics, ecology, and soils. In particular, risk assessment committees must be professional and objective. The guidance is too prescriptive in its conceptual principles. It is hard for the novice assessor to distinguish in each specific case what is relevant (the ‘need to know’) from what is interesting (the ‘nice to know’). It does not appear to follow the structure that is normally used for risk assessments in a real context, both those conducted by developers and those conducted by assessors. Another problem is that while the guidance is useful in the sense that it includes different steps to be followed, it does not provide adequate instructions for following the different steps when applied to a specific problem. Although it takes both risk and exposure into account, it is not clear on how to combine everything and follow the process of specific assessment, for example, of the non-target organisms. The text of the guidance poses no concrete questions that are relevant to the risk assessment case studied and does not give the risk assessor the possibility of evaluating the context during the ‘problem formulation’ process. For example, it poses no questions regarding the type of pathogen in question and its relationship with the environment, how the pathogen is dealt with under conventional conditions (with toxic pesticides), what the susceptibility of other plant and insect species or microorganisms not targeted by conventionally-used agrochemicals is, among other questions. We found that the guidance was not appropriate for use by people who are not risk assessment specialists. Risk assessments must be conducted by interdisciplinary technical teams in order to cover the technical aspects of each specific case that is being assessed for risks. In this case, a risk assessment team must include specialists in agronomy, plant pathology, entomology, weed control, integrated pest management, molecular biology, genetics, ecology, and soils. In particular, risk assessment committees must be professional and objective. The guidance is too prescriptive in its conceptual principles. It is hard for the novice assessor to distinguish in each specific case what is relevant (the ‘need to know’) from what is interesting (the ‘nice to know’). It does not appear to follow the structure that is normally used for risk assessments in a real context, both those conducted by developers and those conducted by assessors. Another problem is that while the guidance is useful in the sense that it includes different steps to be followed, it does not provide adequate instructions for following the different steps when applied to a specific problem. Although it takes both risk and exposure into account, it is not clear on how to combine everything and follow the process of specific assessment, for example, of the non-target organisms.

Hungary: It would be more helpful and more practical if the flowchart for the risk assessment process which is now found in the annex at the end of the repsective section were at the very beginning of the Roadmap section. With this slight change an overview would be given for the user which would assist in the understanding of the risk assessment procedure.

India: 1. The pre-face and introduction section to the Road Map for risk assessment of LMOs is well drafted. However the explanatory text in the subsequent sections needs to be greatly simplified. The practicality of the document can be significantly improved by removing the overtly complex explanations and terminologies, as these could lead to different interpretations by countries/risk assessors. 2. ‘Rationale’ section under each of the five steps is written in a complex language and needs to be rewritten in a simple language (2-3 lines) and supported by links to explanatory resource materials or examples. 3. Some of the sentences in rationale are complex and also repetitive and do not explain the purpose of each step. Some examples are as under: • Line 223 - the term ‘sufficient’ introduces subjectivity to a scientific risk assessment process • Line 263-265 regarding the availability of independent experts is part of a regulatory process and does not fit within the overarching Principles of Scientific risk assessment. • Line 278-283 deals with lack of information or knowledge due to experimental variability is an issue which relates to statistical validation of a study or faulty experimental design which is already covered in line 261-262 and does not fit under ‘uncertainty’. • The concept of ‘uncertainty is captured in lines 267 to 277. • Line 284-296 is a repetition of lines 267 -277 and may be deleted as it is superfluous. . • Line 302, the term ‘each’ may be deleted • Line 314 to 318; reference to protection goals and assessment end points is inclusive of national laws, guidelines, obligations under international agreements as indicated in line 200-203. The line 314 to 318 may be appropriately redrafted. • Line 353 is stating the obvious. • Line 319-320 lacks clarity and needs further explanation. • Line 360-363 'When the likelihood --------------------non-modified organisms' is superfluous and may be deleted. • Lines 396-397 are repeated again in line 428-429. • 394 – 431 – the language is too complex and needs to be greatly simplified to provide guidance on how to actually complete the process of Step 1. While dealing with Step 1, the utility of biology documents with respect to the non-modified or parental organisms needs to be introduced. • LINE -402 – 404 -identification of 'protection goals' and 'hazard identification' are both part of problem formulation based on which 'assessment ends points' are decided. Therefore this statement needs to be redrafted. • Line 432 - the word ‘ Parental Organism’ may be inserted after non-modified organism for the sake of uniformity. • Line 460 – the word ‘meaningful’ may be changes to scientific or relevant • Line 449 – 456 is prescriptive and not a guidance • Line 469-470 relates to risk management and not part of Step 1 • Line 495 - Sentence is not complete. • Line 502-504 needs more explanation on how evaluation of likelihood and consequences can be undertaken in the inverse order. It appears there is a mix up of risk hypothesis and risk assessment. • Line 527 -553 is crop specific but guidance document is for all LMOs therefore it needs to be generalised in a simplistic manner. • Line 554-557 is a management issue. Exposure due to gene flow and incidental exposure due to handling transport etc should be separated as impact due to gene flow is covered in other sections • Line 595-597 needs more explanation with examples and reference to resource material. • Line 601-603 - not part of ERA but food safety assessment. • Line 635-636 should form part of Step 2 and is already covered in that section. • Step 5 ‘Rationale’ is too broad and there are sections here that go into realm of decision making process. • Line 527-553, Line 684- -688 and Line 694 -696 - it is not clear if this is linked to confined field trials (CFT) or commercial releases. • Line 713-723 on ‘Related Issues’ prescribes how a decision making process is to be followed by a Party which is outside the mandate and scope of this document and may be deleted.

Japan: This guidance contains several 'Points to consider' unable to consider since there is NO internationally agreed guidance of how to address the issue and NO technical consultation has been made regarding the issue. These following 'Points to consider' should be deleted: 'baseline' in line 460 to 461, 'Potential adverse effects concerning target organisms such as pests developing resistance to the target trait and weeds developing resistance to the herbicide' in line 469 to 470, 'changes in farm management practices; dispersal of the LMO through mechanisms such as seed dispersal or outcrossing within or between species, or through transfer into habitats where the LMO may persist or proliferate; as well as effects on species distribution, food webs and changes in bio-geochemical characteristics' in line 478 to 481, 'Cumulative effects with any other LMO present in the environment' in line 495, 'long-term adverse effects related to the exposure to the LMO' in line 529 to 530, (iii) in line 595 to 597, (b) in line 598 to 599, 'Individual risks and any interaction among them, such as synergism or antagonism' in line 633, 'Broader ecosystem and landscape considerations, including cumulative effects due to the presence of various LMOs in the receiving environment' in line 635, and (g) in line 704 to 706. Also, 'non-target organisms' in line 489 and line 593 is NOT clearly defined and it is unable to consider. Also, 'agronomic practices' in line 594 has nothing to do with LMO. Also, since there are so many 'agricultural practices,' 'dissemination of the recipient organism,' and 'abundance of volunteers' in line 589 to 590, it is quite difficult to take all in.

Liberia: Step 1: Point to consider characrerization of LMO (i). Include agronomic traits of the non modifed reccipinent organism.

Malaysia: Please refer to response in #12

Mexico: (translation) 1. The working group considers that the guidance should be designed for regulators as its main target audience; however, it could be of use for all stakeholders or actors involved in the introduction of LMOs into the environment. (page 6, par. 1, of the Spanish version) 2. It should be explicitly stated under the objective that the document is a guide and not a method. 3. A brief description of the stages involved in the risk assessment process should be included at the beginning of the document (line 267, page 8, par. 1, Spanish ver.); placing hyperlinks for each stage at the beginning of the corresponding section would provide greater efficiency. Proposed text: ‘The Roadmap consists of 5 stages established on the basis of Annex III: Stage 1 - Identification of novel characteristics associated with potential adverse effects; Stage 2 - Assessment of the likelihood of the adverse effects being realized; Stage 3 - Assessment of the consequences should the adverse effects be realized; Stage 4 - Estimation of the overall risk; Stage 5 - Recommendation. 4. On line 761 (page19, Spanish ver.) the text should be modified so that the paragraph is more in line with the English version, as the Spanish translation does not fully reflect the sense of the original text. Where for the English ‘risk assessors,’ it says, in Spanish, ‘los asesores en riesgos’ (‘the assessors in risks’), it should say ‘los asesores de riesgo’ (‘the assessors of risks’). Where in English it says ‘identified individual risks’ it should say ‘estimated individual risks,’ and consequently, instead of the Spanish translation ‘riesgos individuales determinados,’ it should say ‘riesgos individuales estimados.’ 5. There was agreement in the working group that the Guidance document needs to be read and studied several times before it can be implemented. 6. The inclusion of a simplified version of the diagram at the beginning should be considered.

Netherlands: - Part I states that the information it presents is relevant to the risk assessment of all types of LMOs and uses (Roadmap lines 180 – 81). It also notes that the Roadmap has been developed largely based on LM crop plants (lines 181 – 2). This causes a strong bias in the rationale and approach of this Part, leaving Part I of low practicality for other LMOs (e.g. LM fish and micro-organisms). - In some cases, this is compensated for by the sections in Part II presenting information on specific types of LMOs or traits (LM mosquitoes). However, the Guidance leaves ambiguity how to mutually use Parts I and II for the specific types of LMOs and traits discussed in Part II (e.g. LM trees). - Part I does not provide instructions how to use the available information and presented points to consider to ask the relevant questions for the purpose of performing the consecutive steps of the risk assessment, in particular Step 1 (problem formulation).

New Zealand: These are general comments. The guidance is high level and very academic (i.e. it does not provide on-the-ground practical advice). It is very complex, uses lots of technical language and references to other Protocol sections and other documents etc. • It is unclear who the intended audience for this guidance is (i.e. how much previous experience with LMOs/ risk assessment practices do you need to be able to understand this guidance?). This document will not provide adequate guidance for non-experts to carry out a 'case-by-case' risk assessment 'on the ground' especially for less 'mainstream' GMOs or activities (e.g. vaccinations). An option to improve this is to provide real life case studies for a range of LMOs (from GM animals, plants and microorganisms, viruses) and uses (from field trials, commercial cultivation, vaccines) to show how different regulators actually carried out the risk assessment. For example how did Regulator X when assessing LMO Y; o Frame the risk assessment/define the scope (e.g. What was within the scope and what was out? What are the underlying assumptions/scenarios? What fell outside the Protocol mandate and how was this dealt with? i.e. if there were risks still to be addressed). o Decide what comparator to use (if needed at all?). o Decide the information was sufficient for the activity. o Identify and deal with uncertainty. o Decide what expertise was required for the risk assessment (e.g. toxicologists, ecologists, commercial growers etc.) o Deal with different activities (e.g. field test versus commercial releases). o Use pre-existing information drawn from previous risk assessments. o Use scientific consensus positions to inform the risk assessment e.g. current opinion on HGT. o Describe the likelihood, consequence and risk characterisation used (quantitatively or qualitatively, how are the terms defined).

Norway: The practicality of the Roadmap may be improved by articulating the inter-linkages of the risk assesment process with risk management and risk communication. Furthermore, describing the risk assessment process in the context of decision-making more fully, as illustrated in the flowchart, would improve the practicality of the Roadmap.

Peru: (translation) It is too general, with subjective terms. It lacks objective criteria that would preclude different interpretations, depending on the risk assessor (evaluator). The terms ‘monitoring’ and ‘surveillance’ are not defined in the English version of the document. This has led to confusion in the Spanish translation, where ‘vigilancia’ (‘surveillance’) is used instead of ‘monitoreo’ (‘monitoring’). Any definitions provided for both terms must highlight their differences. SUGGESTION: Other documents on risk analysis (e.g., ISPM No. 11, Pest Risk Analysis For Quarantine Pests, by the IPPC), containing specific criteria, should be consulted. The Roadmap must include more specific values or criteria. As it is now, the revised document consists merely of guidelines.

South Africa: It is important for the introduction (line 188) to already clearly make the distinction between the role of state party risk assessors (risk analysis undertaken by regulator) and the actual risk assessment conducted by the applicant. Some useful overarching elements are introduced that are important and provide greater clarityfor countries wanting to undertaken risk assessments as contemplated by the Cartagena Protocol on Biosfaty. Of particular interest is the introduction of elements of the concept of Problem Formulation and Options Assessment (PFOA) without any explicit reference or clear explaination. As a result, the elements introduced are not integrated into the overall risk assessment framework. Problem Formulation and Options Assessment can be used to simplify the risk assessment process and identify potential assessment endpoints. The concepts of protection goals and assessment endpoints are introduced but not fully explained. These tend to be sovereign in nature and it is important that these are not presecribed in any form. Line 421 (iv) transfer genes to other organisms/populations - not necessarily a harm, but potentially a pathway to a harm It may therefore be useful to distinguish between harm and pathways to harm in this section..

Turkey: All questions in the road map needs to be taken out and presented as a questionaire at the end or begining of the quidance. Wording shoud be used to give explanation for the questions. Questinaire Step I Points to consider regarding characterization of the LMO 1. Relevant characteristics of the non-modified recipient organism a. Biological characteristics aa. Life span bb distribution cc self polinated or cross polinated dd isolation distance etc. 2. its taxonomic relationships aa family cc genus dd species ee wild relatives ff distribution of wild relatives etc All related things sould be in the questinaire as stated above, across them related references needs to be given in paranthesis like (1, 2, 5). Therefore for risk assessment a standart application would be possible.

Viet Nam: It need to re-consider following items: - its taxonomic relationships; - its origin, centres of origin and centres of genetic diversity; - ecological function Because, the above categories don't reflect any adverse affect of the LMOs and make the assessor difficult to imagine what kind of adverse affect relating to these categories. Therefore, it should clarified specific adverse effects in each categories

Other Governments

Australia: Lines 324-328 does not provide any practical guidance on what differences might apply and why. Lines 351-370 are impractical in most circumstances. In most cases it is the species that is used as the comparator with an emphasis on cultivated varieties as these provide the most information and the best relevant indicators of risks posed by the non-modified parental organism. This is also the level at which most biology documents are prepared for use in risk assessments of LMOs. For most indicators of risk varietal differences are not significant. For those few indicators such as levels of endogenous toxins that may be relevant (more typically in food safety testing), the range of values in cultivated varieties gives a more realistic indication of acceptable levels than a single (near-)isogenic line, which may be aberrant in some way. In practice, many LMOs released are made from backcrossing into elite varieties from the original transformant that may be no longer cultivated, and therefore not relevant as the non-modified comparator. Lines 428-431 should carefully distinguish between field trials and commercial releases as several of these types of effect do not seem relevant to field trials (as tested on Australian case study DIR 102). Lines 460-461 do not provide practical guidance in the absence of what is meant by ‘sufficient data’ or ‘meaningful baseline’. Lines 519-522 do not work in practice. For example, a toxin expressed by a plant may have multi-trophic effects, such as the plant is consumed by an insect, which is eaten by another invertebrate, which is then eaten by a bird, which in turn might be eaten by an animal (e.g. a snake), which may then be affected by the toxin. The level of uncertainty increases the higher up the trophic scale we go. Therefore, from lines 519-522 we should assign 100% likelihood of an adverse effect to the snake if we have not specifically tested the toxins effects on the snake? This is not done in the two Australian case studies. Lines 523-525, 580-582, 615-618 and 623-626 are not practical if clear criteria are not established during the planning (context) phase. In particular, the use of ‘indeterminate’ risk (line 624) is impractical as there is always incomplete knowledge and so would be applied for all risks all of the time. This is out of step with the two Australian case studies and regulatory risk assessment methodology in general where uncertainty is already considered as part of the likelihood and consequence assessments. Lines 619-622 are particularly confusing and difficult to put into practice as no guidance is given on what is meant by ‘multiple lines of evidence’, ‘quantitatively or qualitatively weighted’, or ‘combined’. Practical examples would help. Lines 630-636 are missing the key point in practice, namely, how the estimates of likelihood and consequences are going to be combined to give a level of risk either individually or overall. Lines 633 and 635-636 are already catered for in practice in step 1 as seen in the two Australian case studies. Line 634 does not clearly distinguish risk management strategies that are established as part of the context and those that arise from evaluation of the overall level of risk. For the field trial case study (DIR 102) the controls proposed by the applicant for restricting spread and persistence form part of the risk context and then used in step 1 as consistent with Annex III, not at step 4 as suggested by this guidance. Lines 672-677 can be confusing in a regulatory setting where monitoring normally refers to monitoring for compliance with licence conditions. No mention is made to distinguish these uses of the term monitoring in practice. Nor is there mention of other monitoring related activities that form part of the licence conditions in the two Australian case studies, namely, adverse effects reporting obligations and contingency plans. Line 683. An additional point to consider as important in practice is whether or not a proposed risk management measure may introduce additional risks or increased level of identified risks.

Canada: The general guidance provided by the document is not conducive to the very specific recommendations that appear frequently. Specific recommendations for a general case are not useful or helpful. A good example is the extensive list of specific elements provided as points to consider for only the molecular characterization (line 440-452). There is no link to why any of these aspects may be important or a recognition or how many of these components, if any, would be required if the risk assessor has high familiarity with the trait and organism and/or the phenotype is well characterized. The cases considered in the testing treated the molecular characterization in terms of how they related to the expressed product and possible routes of exposure. Aspects such as the copy number, and site of insertion are generally irrelevant to that assessment. A consideration of phenotypic and genotypic changes is only relevant if it has a realistic linkage to a harm Many of the molecular characterization element identified may very well be irrelevant in all but a very few specialized cases but they are presented as 'point to consider' although they would be rarely be points to consider and function more to encourage unfocussed data collection. Concentrating on small changes in genotype (line 453) is not a useful exercise unless linked to a hypothesis. Product efficacy features such as in the points to consider in Line 451 are not part of the risk assessment. Genotypic or phenotypic instability would rarely be a hazard but more frequently a product failure and does not warrant consideration unless it can be linked to a specific harm. Similarly, herbicide tolerance is an aspect of the applied pesticide product not the LMO and in any case is not a hazard but an inevitable outcome of product use and would only be significant if it resulted in the failure of all possible control options, a high unlikely scenario.(line 469). In the cases tested, evolution of herbicide tolerance in weed populations is considered in the context of management advice to a user rather than a point to consider for the risk assessor. – Overall, the 'points to consider regarding characterization of the LMO' (lines 432 to 456) would be more practical if it was guiding the evaluators to determine the potential harms rather than to collect information for an extensive list of parameters. For example, instead of listing the various components of a thorough molecular characterization, the guidelines could trigger the evaluator to (1) determine if any toxic sequences have been inserted into the host organism, (2) determine if any endogenous toxic gene could have been upregulated resulting from the genetic modification, (3) determine if any antibiotic resistance gene sequence have been inserted into the host genome that have clinical significance, (4) determine if potential genotypic instability could result in a specific hazard, etc. Such an approach would allow for incorporation of concepts such as long history of safe use and familiarity and would also avoid collection of information that would not be useful for the risk assessment, such as information related to product efficacy. The purpose of the data collected for a risk assessment is not the same as data collection to satisfy scientific curiosity ('need to know vs. nice to know'). This fundamental concept has not been captured anywhere in the guidance despite its critical importance to the risk assessment. - In the first paragraph about the identification and consideration of uncertainty (lines 267-274), it is important to clearly state that the consideration of uncertainty and its importance to effective decision making is subject to a great deal of discussion and the importance will be variable, depending on where uncertainty occurs. - In the section about the identification and consideration of uncertainty (lines 267-297), it would be more accurate to say that 'Communicating uncertainty adds precision to the communication of outcomes of the risk assessment' rather than 'Considerations of uncertainty strengthen the scientific validity of a risk assessment' as the degree of uncertainty can be helpful to risk managers and decision makers when they weigh options. The section on uncertainty from 267-297 was unhelpful in practice and seemed more geared to confound a clear process then to enhance the end product. Although reviewers did not consider all of the literature referenced, it really did not provide much additional clarity on the practical application in risk assessment process. It is unclear how an uncertainty analysis, especially considering that this is generally a subjective judgment, strengthens the 'scientific validity of the risk assessment' (line 275). The science, if sound, will stand on its own merits but communicating the degree of uncertainty can be helpful to risk managers and decision makers when they weigh options. It would be more accurate to say that communicating uncertainty adds precision to the communication of the outcomes of the risk assessment. The acknowledgment in this section that more information can result in more uncertainty is useful but no examples are provided. This could be helpful for context and highlight the pitfalls in unfocussed data requests. Uncertainty is inherent and associated with risk assessment (not 'an inherent and integral element' as stated in line 267). This section is more likely to paralyze a novice risk assessor with indecision since complete information will never be available and both the importance and degree of uncertainty is highly subjective, despite the many attempts at quantification and it is not at all clear that extensive considerations of uncertainty really do enhance the final risk assessment as affirmed in this section.

United States of America: Overall, the Roadmap document (Part I of the Guidance) is not as practical as it could be for the novice risk assessors, for the following reasons: 1. The document does a very poor job of setting the context for comparative risk assessment by making clear our extensive experience in dealing with non-LMOs, not only non-LMOs modified by human intervention but also non-LMOs that are continually evolving without human intervention. As a consequence, our extensive experience with evaluating and dealing with potential environmental risks with non-LMOs is largely ignored in the document, even though this experience is key to using our prior knowledge to evaluate LMOs. The document gives the impression that the occurrence of outcrossing and instability of genotypes or phenotypes are something unique to LMOs (and that these phenomena are indications of environmental risk). 2. The document does not acknowledge the existing experience of over 40 years in evaluating potential environmental risks from LMOs. In contrast, the document gives the impression that there is a great deal of uncertainty and inexperience worldwide. 3. The document does a very poor job of providing practical guidance on risk assessment related to limited or confined environmental releases of LMOs (with LM plants, these are often referred to as field tests). This is relevant for the document discussion of topics beginning at line 218 and extending through the remainder of the document. The document says that some information may not be needed for confined environmental releases, but there is no tie-in with the concepts described elsewhere in the document (especially in the discussion on the likelihood of an adverse effect occurring). 4. The section on 'uncertainty' uses the term in a manner different from the way that the term is used in the Protocol. Paragraph 8(f) of Annex III states 'Where there is uncertainty regarding the level of risk, it may be addressed by requesting further information on the specific issues of concern or by implementing appropriate risk management strategies and/or monitoring the living modified organism in the receiving environment.' The underlined emphasis is added to highlight that according to Annex III the level of risk is determined at the end assessment, not with each step and consideration. 5. The document introduces additional terminology that is not consistent with the Protocol text, and the new terminology is not well explained. Perhaps the most striking examples of this occur in the section 'conducting the risk assessment', especially the use of 'hazard identification', exposure assessment, hazard identification, etc., rather than the text in Annex III of the Protocol (see lines 376-380 and onward through following pages). Additional terms are used, yet their meaning in relationship to the text in Annex III of the Protocol is not explained, including 'causal link and pathway', 'non-target organisms', 'target organisms', etc. No guidance is provided in the document to explain that in many cases there will be no target non-target organisms (e.g., LM-plant modified to tolerate drought conditions).

Organizations

ENCA EPA interest group On Risk Assessment and Monitoring of Genetically Modified Organisms (GMOs): The paras starting line 398 – to line 417 describe a quite narrow approach for step I. From our perspective there should be no restrictions to RA in advance. We propose to delete in line 407 'scientifically plausible scenarios'. The sentence could possibly read: 'In this step, risk assessors develop meaningful risk hypotheses….' Additionally in line 415 it is proposed to introduce the following changes: 'It is important to define a link or pathway between the LMO and a possible adverse effect, ….. (this proposal is meant to also cover indirect effects and long-term effects which are missing in the proposed approach)

GenOk - Centre for Biosafety: The treatment of dealing with uncertainty in the risk assessment would be better served by outlining a framework that elaborates on the three main components for describing of uncertainy, namely its nature, source and level for each identified uncertainty. The guidance could be improved by adopting some common, simplified approaches currently under use.

Global Industry Coalition (GIC): The current structure of the Guidance makes the testing difficult because it does not resemble the structure used in the BCH risk assessment reports posted and does not reflect the structure of most risk assessments that are conducted by developers to support regulatory applications. This makes the location of relevant information complex and time consuming. Further, there are elements in the Guidance that could not be found in any of the risk assessment reports. In the Guidance, each of the steps to follow for a risk assessment listed comprise a consideration of various issues: gene flow (including vertical and horizontal gene transfer), effects on target and non-target organisms (including toxicity, allergenicity and multi-trophic effects), changes in management practices, etc. By structuring the Guidance in this way, it is unclear how a novice risk assessor will understand the principles underpinning establishing a link between the different steps within the area of assessment or issue under consideration to complete a risk assessment. There is insufficient guidance on discerning 'need to know' versus 'nice to know' information necessary to conduct a risk assessment. For example in Step 1, the Guidance lists the points to consider, but when it moves to the other steps there is no clear link on how information from Step 1 (hazard identification) is used with information in Step 2 (exposure) and Step 3 (hazard) to complete Step 4 (risk characterisation). The Guidance resembles a list of potential hazards and exposure scenarios without context and with no clear guidance on how to integrate the various pieces of information in performing a risk assessment in practical terms. The Guidance sometimes wanders into areas of policy and fails to present scientific consensus, this is not useful to experienced risk assessors in countries that have functioning regulatory systems that follow a scientific approach.

Public Research and Regulation Initiative (PRRI): The Road map is not very practical, because of its very nature and because of the way it is written. The language is very dense and of a ‘legal negotiation’ type, with often sentences of several lines that are extremely difficult to follow for non native English speakers. Moreover, the way risk assessment is introduced gives the wrong impression that there must always be risks (e.g. language such as 'the risk posed by the LMO'). This is underlined by the flow chart which in all cases ends with 'consideration of risk management'. Further, the text gives the wrong impression that risk assessment is a process with which we have hardly any experience and that is riddled uncertainties. In fact, the detailed elaboration of uncertainty gives the wrong impression that this field is faced with more uncertainties than any other type of risk assessment in the field of biology. In addition, the text gives the wrong impression that natural phenomena such as out crossing and instability of genotypes or phenotypes is an unusual phenomenon that means risk. All this is pervasive throughout the document, and cannot be pinpointed to specific lines to be changed. A general clean up would be advisable. While we very much support the original idea to elaborate on the steps and points to consider in environmental risk assessment as outlined in Annex III and by pointing users to relevant background materials, the way it is done will give the Road Map little practical value, because novice risk assessors will either ignore the dense text or be deterred by the sheer notion of risk assessment.

Third World Network: We found the Guidance very practical - the language is easy to understand, and the document is well structured and logical in its progression. Practicality could be improved by prodiving more elaboration on the related issues that are part of the decision-making process, in order to assist Parties in making the linkages from risk assessment to these issues, inter alia, risk management, capacity-building, public awareness and participation, socio-economic considerations, and liability and redress. The Flowchart is a usefull illustration of the whole process, but could be accompanied by further explanatory detail.

Zelenyi Svit / Friends of the Earth Ukraine: Guidance is good instruction and practical tool for step-by-step procedure of risk assessment.

USEFULNESS AND UTILITY

Parties

Austria: Line 185: include for completeness of argument: …, including those of limited duration and scale as well as long-term and large-scale releases. Line 204: avoid 'tiered'. Suggestion for formulation: 'The Roadmap describes the risk assessment process as a sequence of five steps, in which the results of one step are relevant to the other steps. This step-wise structure is drawn from the outline presented in Annex III, Para 8 of the Protocol.' Line 229 - 231: The expression 'Data quality should be consistent with the accepted practices of scientific evidence-gathering and reporting and may include independent review of the methods and designs of studies' is not sufficiently clear. We suggest to use the following wording: 'An independent review of the design and methods of studies used for risk assessment, and the quality of reporting may be included to ensure appropriate data quality'. Line 260 - 265: The two bullet points listed under 'Additional considerations with regard to scientific information' target different issues, the latter one addressing availability of scientific expertise for conducting risk assessments. It is suggested to include both considerations as separate statements. Line 398 ff: We propose to use the term 'risk hypotheses' throughout for clarity –in substitution for 'risk scenarios' (Line 399), 'scientifically plausible scenarios' (Line 407). In our opinion the statement in Line 407 can be shortened to: 'In this step, risk assessors develop meaningful risk hypotheses….' Line 415: We suggest to change to: It is important to identify direct or indirect links or pathways between a characteristic of the LMO and possible adverse effects, to generate information during risk assessment that will be useful for decision-making' The chapter on 'The choice of comparators' (Lines 343 ff) is regarded very important and crucial for implementation of the guidance. However testing indicated that to appropriately assess effects in managed ecosystem comparisons need to include specific management conditions. This is considered relevant for most applications of LM crop plants and given the importance of these LMOs (cf. Line 181 – 183) should be explicitly indicated with reference to related chapters (e.g. Line 476 ff). Additionally we suggest to include in Line 424 ... taking into consideration the new trait(s) of the LMO, 'and associated changes in management'.

Belgium: - The testing was conducted as an individual exercise. Different levels of agreement/disagreement (i.e. 'Neutral' or 'Agree') with regards to the usefulness and utility of the Roadmap were expressed amongst people. - Line 190: The comparative approach is an important aspect of the risk assessment of LMOs. That could be highlighted already in the introduction. We propose to change the end of line 190 as follows: '…and on a case-by-case basis in relation to the risks posed by the non-modified recipients or parental organisms in the likely potential receiving environment.' - Line 585: We suggest adding under 'points to consider': 'Relevant knowledge and experience with non-modified organisms with similar phenotypic characteristics in the likely potential receiving environment.' This is particularly relevant for LM plants tolerant to abiotic stress. - Lines 654-659 ('In evaluating the acceptability of the overall risk of the LMO, it is important to consider whether risk management options can be identified that could address identified individual risks and the estimated overall risk as well as uncertainties. The need, feasibility and efficacy of the management options, including the capacity to enact them, should be considered on a case-by-case basis. If such measures are identified, the preceding steps of the risk assessment may need to be revisited in order to evaluate how the application of the proposed risk management measures would change the outcome of the steps'). More attention should be drawn on the importance of this paragraph, especially the re-conduction of the overall risk assessment, revisiting every steps of the risk assessment of the LMOs including the risk management options. This is of great importance as the final recommendation may be highly influenced by the presence/absence of risk management options. - The scale and duration of the environmental use is an important point to consider to determine the nature and level of detail of information that is needed for the risk assessment, and to identify and verify plausible risk hypothesis. Although this point is highlighted on page 10 and briefly addressed in step 2 (lines 533-535) and in step 3 (lines 569-570) of the Roadmap, it is not enough stressed and considered in the further description of the 5 steps of risk assessment (in particular step 1), including the points to consider. This leads to insufficient distinction between the environmental risk assessment of field trials (in which different types of trials could also be distinguished) and commercial releases.

Brazil: One useful measure adopted by several regulatory agencies is the establishment of communications mechanisms that risk assessors might use in order to ask for additional information. The Guidance could be improved by the inclusion of such mechanisms. - The 'Quality and relevance of information' (line 222) section does not allow an evaluation on how much and which type of data are needed in different cases of risk assessment (eg. field trial x commercial release). It also does not allow the establishment of the relative value of different types of information (eg. an opinion of an expert with the parent organism can be less reliable or relevant than a validated study conducted in accordance with international protocols). Some orientation in this sense would be useful. - The 'Identification and Consideration of Uncertainty' (line 266) section does not present different approaches to deal with different kinds of uncertainty (eg. expert opinion, clear definition of key words etc). The approaches presented are only additional data and risk management measures that can make the process of risk analysis more complex without giving the necessary confidence for the risk assessor to reach a conclusion. - The ´Establishing the context and scope´ section (line 300) should be more straight related with the context as the necessary information to sets the criteria against which risk will be evaluated (eg. genetic modification, parent organism, receiving environment, proposed activities with GMO and previous releases). In the Guidance there are some points listed that are vague (eg. line 329-332) and the introduction of broad parameters (eg. ecological function). - The iterative nature of risk assessment process is mentioned in line 373, but the concept of iterative should be better exploited due to its importance to the process. Iterative means, for example, the result of ongoing accumulation of information (data from applicant, expert advice, literature search) where any step during risk assessment can be reviewed. - The RoadMap is difficult to be applied in the RA process, for the information required are presented as a check list and is difficult to select which information are important or not for the case being evaluated and there are no suitable examples presented. - In the step 2 the way likelihood is expressed is not well addressed. Although quantitative and qualitatively expression is mentioned there is no explanation in the proposed Guidance why for biological systems qualitative terms are used and neither the assessment scale used for this terms, making more difficult to understand this step. - In the step 3 there is also an description of qualitative terms without any explanation on the importance of the definition of these terms in the context of a LMO risk assessment. 'Major', 'intermediate', 'minor' or 'marginal' consequences are not clear enough to help the evaluation. There is also a recommendation 'Parties may consider describing these terms and their uses in risk assessment guidelines published or adopted by them' but will be more useful if there is an example of how this terms may be used in the context of RA. - In the step 4 the terms 'high', 'medium', 'low', 'negligible' and 'indeterminate' are mentioned in the text without definitions or the presentation of a risk matrix that can be used to estimate the level of risk, making more difficult to understand this step. In the 'points to consider' section there is also a mention of 'broader ecosystem and landscape consideration...' (lines 635-636) that should be considered in the step 1 and not in the last step of RA.

Colombia: (translation) The guidance should be aimed at all Parties, but with a greater emphasis on Parties that still lack a solid regulatory and/or technical framework for risk analysis. Therefore, the guidance must provide step-by-step instructions for formulating the problem in the context of the scientific method when an LMO is going to be used. As noted in the previous item, the use of the term ‘hazard’ (‘peligro’) (lines 438-448) must be revised. However, for the ‘hazard identification,’ ‘exposure assessment,’ ‘hazard characterization,’ and ‘risk characterization’ processes it is necessary to discuss the methodology that could be used to address those elements (for example, matrices, forms of measurement, etc.), with the aim of proposing a standard method. In connection with lines 458 to 460, which refer to defining a causal link or pathway between a characteristic of the LMO and a possible adverse effect, it would be very useful if detailed examples of correct and incorrect ways of addressing them could be provided. The section comprised by lines 552 to 803 should go into detail regarding the methodology proposed for ‘characterizing the risks,’ ‘weighing the risks,’ ‘evaluating the consequences,’ ‘estimating the overall risk,’ and ‘recommendations.’ It would be very useful for the Parties if the proposed methodologies were explained in this section in a more practical and illustrative way (with graphs, diagrams, matrices, etc.). The current approach tends more towards a conceptual review and a rationale of the why than towards a how-to, which should be the ultimate purpose of a guide. It is important to highlight that there are international standards for assessing both human food and animal feed, which in the case of Colombia are the procedures used. Consequently, it is suggested that the guidance should try to combine all of these elements. In view of the above, the roadmap should include more appropriate guidelines that are applicable in general to different types of LMOs.

Czech Republic: In line 229: the statement concerning the data quality should be defined 'Used data should be of acceptable scientific quality.' Lines 263 to 265: are of great importance. Usually, two to three evaluators are asked to prepare RA, but usually all of them experienced in the same field (e.g. plant biology). Some special problems concerning chemistry, biochemistry, or population genetics are not so exactly evaluated although they could be important for some assessment endpoints.

Denmark: No supplementary text necessary

El Salvador: (to be translated) Líneas 290-296 Tal y como está planteado da la impresión de que todas las incertidumbres resultarán o pudieran ser importantes y deben ser abordadas. El enfoque debe estar restringido a las fuentes de incertidumbre que afectan al riesgo general de una forma que pueda afectar de manera significativa las decisiones. Líneas 410-413 y 417-421 Sólo se abordan los cambios sin especificar cuál podría convertirse en un efecto adverso Líneas 412 413 - El término "objetivo" debe sustituirse por "punto final de evaluación" ya que podría dar lugar a confusión con los "Organismos no objetivo" y "Organismos Objetivo" utilizado en otras partes del documento. Líneas 523-525, 580-582, 615-618 y 623-626 No clarifica del todo el ejercicio que se debe hacer durante la fase de planificación (Contexto de la Evaluación de Riesgo), sobre todo en lo referente a ‘Riesgo Indeterminado’ Líneas 598-599 No tienen claro si esto se refiere a los efectos o los efectos adversos? Sería conveniente reforzar esto con ejemplos. Líneas 619-622 sería muy conveniente que se utilizara ejemplos prácticos.

Egypt: - Issues of human health in the context of risk assessment of LMOs are still not well reflected in the roadmap. - There is need for more direct reference to socio-economic considerations which could be geographically and popularly relevant in decision making for specific LMOs.. - The preparation of concrete examples, including LMOs case studies, to be used in conjunction with the roadmap to assist in training and capacity building activities in developing countries would add to the usefulness of the guidance materials prepared.

European Union: Line 578 - It is not clear in the guidance if the reversibility of an effect is referring to an intentional or unintentional effect. For example, in the case of an LMO plant that has been modified to be resistant to a pest the overall effect may be a reduction in the pest poopulation in a region however there may also be an unintentional effect on a non target organism population. It should be described more clearly in this section exactly what is meant by reversibility of an effect and whether the risk assessor should consider the intentional or unintentional effects or both. In some cases scientific data and evidence of the reversibility of an effect may not be readily available. It would be more practical to consider the 'potential for recovery' rather than the reversibility of an adverse/unintentional effect. Line 660 refers to the recommendation that acceptability of risk should take into account potential benefits for the environment, biodiversity and human life. The EU does not support an approach to risk assessment that balances risk acceptability with benefits. If this text is to remain 'should' should be replaced with 'may' to reflect the view that not all parties agree with this approach.

Honduras: (translation) The difference between evaluating the ‘practicality’ (‘practicidad’) and the ‘utility’ (‘utilidad’) of the Guidance is not clear. Many on the comments made regarding section 9 apply to this section. The questions are aimed at an apparently negative point. The assessor is directed at all times to find a threat, without any effort to contextualize the threat, including a ‘problem formulation’ section. For example, to assess the risk of traveling by car or by plane (there is always the risk of a deadly accident) the advantages or benefits of these means of transportation are assessed against traveling by foot. In some sections of the guidance, what is described as a threat could in fact be highly desirable (e.g., the potential for the phytopathogenic fungi resistant potato to spread like weed, if what is sought is a higher potato yield. This case is hypothetical as potatoes cannot turn into ‘invasive weeds’ without human intervention). In order for the guidance to be useful, the questions must be more open and unbiased. There needs to be a receptiveness to consider other aspects, both favorable and unadvisable, in addition to those limited by Annex III. The process must not lead the assessor to directly identify the assumed problem. It must define both the context and the actual problem and those derived from it; the definition of the problem and the so-called endpoints need to be inferred. The guidance takes into account some of the main concepts of risk assessment, but in order to put this into context a problem formulation section needs to be added. In some parts, the guidance is too prescriptive and should allow for some flexibility.

Hungary: The content of footnote number 16 is a very important part of the guidance. Therefore, it should be included in the relevant part of the guidance, line 466.

India: 1. The Article 15, 16 and Annex III of the protocol are applicable to all types of environmental releases and the same has been reiterated in the roadmap (lines 184-185). However there is no distinction made between the risk assessment considerations for small scale experimental releases under confined conditions for the purpose of field trials or large scale/commercial releases under unconfined conditions in the roadmap. Therefore, it is difficult for risk assessor to select which information will be essential for either of the above two scenarios. It is important to recognize that confined field trials are needed to generate data for risk assessment and therefore the information requirements for the same are much different and limited as compared to large scale environmental releases. 2. Each of the step listed in the guidance requires consideration of various issues such as gene flow, effect on target/non-target organisms, changes in management practices etc. It is difficult to understand for risk assessors particularly who are doing it for the first time or having limited experience to establish links between area of assessment or issues under consideration. 3. Some of the text is also verbose, prescriptive and restrictive and can be modified / deleted to make the document more easy to understand. In step 3 several points in the points to consider are almost covered in the section on 'establishing the context and scope' e.g., line 600, 601-603 and 604. 4. The case by case approach as indicated Annex III has been indicated at the beginning of introduction (line 190). In line 206 -208, the word ‘iterative’ introduces ambiguity to a scientific risk assessment process. Information requirement for a risk assessment is captured in the screening and scoping of impacts exercise in an Environmental Impact Assessment which is akin to identification of Protection Goal and Assessment End Points. The process of identifying the information requirements is country specific. Risk Assessment is based on available scientific information at the time of conducting the risk assessment. 5. In several places, the key terms have been introduced suddenly in the text, without providing a background. For example even the term 'Protection goal' and its linkages to the concept of ‘Problem Formulation’ is not clearly understood in many countries and is not used presently in several regulatory systems. 6. The other terminologies which are not well explained and need further elaboration include • Line 198 - unintended effects, • Line 279 – lack of information and incomplete knowledge can be reconciled. • Line 303 and 317 - Risk threshold • Lines 396-397 - terms such as ‘direct’, ‘indirect’, and ‘immediate’, ‘delayed’ are not explained in the 'Use of terms' on page 57. Further usage of these terms in the context of the Nagoya Kuala Lumpur Supplementary Protocol (NKLSP) on liability and Redress is not appropriate as definition of damage under Article 2 (b) is with reference to ‘response measures’ and therefore outside the scope of this guidance document. NKSLP is yet to come into force. • Line 431 and 598 - ‘combinatorial effects’ and ‘cumulative impacts’ • Line 438 and 467 under the ‘points to consider’ regarding characters of LMOs, it has been mentioned to list characteristics such as ‘origin’ besides centres of origin and centres of genetic diversity. It is not clear in what context the word ‘origin’ has been used and introduces subjectivity to a scientific process which is avoidable. • Line 435 ‘gene products’ • Line 524- highly likely, likely, unlikely highly unlikrly • Line 541- ‘Anthropogenic mechanisms’ • Line 581 – ‘major’, ‘intermediate’, ‘minor’ or ‘marginal’ • Line 624 – ‘high’, ‘medium’, ‘low’, ‘negligible’ or ‘ intermediate’

Japan: As stated [above], 'Points to consider' unable to consider make this guidance less usefull; they need to be solved.

Malaysia: GENERAL COMMENTS 1. It is a difficult document to read. We suggest that a more user friendly document is developed. 2. Points mentioned should be separated/differentiated to what is essentially needed and what is good to have. 3. Language used is too complicated. It should be simplified. 4. Content is repetitious and too wordy. 5. Case examples should be extracted out from the main document. 6. For countries looking at assessment of LMOs for the purpose of Food, Feed and Processing only, this document is not useful at all. 7. Use bullet points in the formatting to enhance clarity. 8. Have sections and numberings so that it will be easier for any cross references. 9. A lot of information can be put in appendices/explanatory notes. Keep the main document simple…with headers, etc. Examples, options, alternatives should be mentioned in appendix. 10. 'Points to consider' are all useful points. May avoid listing this points using alphabets. Use numbering so that it is easy for reference and to break it up into smaller segments if necessary. SPECIFIC COMMENTS 1. Title of the Document TITLE - it may be changed to 'Guidelines' as it serves as an option only 2. Line 195 - Put the 3 items in bullets 3. Line 218 OVERARCHING ISSUES - Change title to – BASIC PRINCIPLES FOR RISK ASSESSMENT PROCESS 4. Line 219 and Line 225 - a number of 'issues' is stated. The term 'issues' gives a negative connotation. Use more neutral language. Positive language will get better co-operation from scientists. Terms like issues. Suggestion is to replace with 'points'. 5. Line 241 - Have a header for the paragraph – information linked to protection goal. It is too wordy. Suggest changing into a more clearly structured format. Shorter paragraphs. 6. Line 237 and line 244 - There must be consistency of words – 'data', 'information', 'relevant data' are all used in this document. Too many repetitive terms. Suggestion is to use 'information' and explain that information includes data, raw data and others. All unnecessary explanations can be put in glossary. 7. Line 266-274 Identification and Consideration of Uncertainty - Rather lengthy explanation of uncertainty. Suggest putting as appendix. 8. Line 275-277 - Suggest removing line 275 – 277. It provides unnecessary justification to do a risk assessment. It is already mentioned in the main CBD document that uncertainty is a concern. 9. Line 299-340 - PLANNING PHASE OF THE RISK ASSESSMENT. Instead of providing lengthy possibilities, have a short concise checklist. Have a summary document on how to do a risk assessment. 10. Line 343-370 - CHOICE OF COMPARATORS. Suggest shifting this part to Appendix. Just simply mention in the main document that the appropriate comparator must be chosen and provide a cross reference to the Appendix. 11. Line 365-370 - Suggest removing line 365-370. It’s confusing, and there is no need to refer to 'other risk assessment'. The paragraph seems ambiguous and does not add value or any useful information. It may cause confusion. 12. Line 371 - 387 - CONDUCTING THE RISK ASSESSMENT. Explanatory notes should not be in the main document. Just keep to main steps. 13. Line 272-387 - Suggest putting in explanatory notes as Appendix. 14. Line 393-431 Step 1 - Simplify the rationale; keep to one page, just the main points. Any additional information can be put in Appendix. 15. Line 500-525 Step 2 - Rationale can be simplified some more. 16. Line 519-522 - It is not logical to 'assign a likelihood of 100% that an adverse effect will occur…' There is an inconsistency in the terminology used as the assessment described is qualitative (Line 523-525). There should not be a percentage value that is used for uncertainty (quantitative value) but instead a qualitative value description should be used for consistency. 17. Line 526-557 - Points to consider – can reduce the explanation and examples. If there is already a Training Manual that complements this document, then there is no need for so much of explanation and examples. 18. Line 561-582 Step 3 - Rationale can be simplified some more. 19. Line 611-628 Step 4 - Rationale can be simplified some more. 20. Line 641-682 - Rationale can be simplified some more. 21. Line 707 - Is this supposed to be risk/benefit analysis or scientific benefit analysis?

Mexico: (translation) 1. Line 283 (page 8, Spanish ver.). In the section on overarching issues, the working group considers that it would be relevant to include a reference to the criteria published by the EFSA (Regulation (EC) No. 178/2002) regarding good practices and quality assurance for scientific findings through quality processes in accordance with ISO 9001 principles and the INEX process (the internal process of the EFSA), as well as their transparency and independence. 2. Further elaboration should be provided on the ‘context’ in which each actor conducts a risk assessment and how the ‘points to consider’ could be affected. a) In line 389 (page 10, Spanish ver.), the phrase ‘In establishing the context,’ (in Spanish, ‘para el establecimiento del contexto’) should be added at the beginning of the following sentence: ‘Several points may be taken into consideration, as appropriate, that are specific to the Party involved and to the particular risk assessment.’ So that the amended sentence reads: ‘In establishing the context, several points may be taken into consideration , as appropriate, that are specific to the Party involved and to the particular risk assessment.’ (In Spanish, ‘Existen varios puntos que pueden tomarse en consideración para el establecimiento del contexto, según corresponda, que son específicos de la Parte involucrada y de la evaluación del riesgo en particular.’). Moreover the English version says ‘Party involved’ not ‘Party committed’ (‘Parte comprometida’) as appears in the Spanish. The Spanish translation should be modified to ‘Parte involucrada’ to reflect the original. b) In Line 394 (page 10, Spanish ver.), where it says ‘(iii) Protection goals, assessment endpoints, risk thresholds and management strategies as laid down, for instance, in relevant legislation of the Party’ it should say ‘(iii) Protection goals, assessment endpoints, risk thresholds, and management strategies and capacities, as laid down, for instance, in relevant legislation of the Party.’ (in Spanish, ‘iii) metas de protección, puntos finales de la evaluación, umbrales del riesgo, estrategias y capacidades de gestión tal como están establecidos en la legislación de la Parte.’) 3. Stage 3 should be further developed, as, while the wording is clear, it poses some difficulties from the practical point of view due to the levels of uncertainty with respect to the information and how to interpret it. 4. A definition for ‘adverse effect’ should be included. 5. The Spanish translation of the following terms should be revised: a) Line 497 (page 13, Spanish ver.): Where it says ‘arriesgan hipótesis’ as the translation of ‘risk hypotheses,’ it should say ‘hipótesis de riesgo.’ b) Line 644 (page16, Spanish ver.): Instead of ‘ecological range’ (‘rango ecológico’) it should say ‘ecological niche’ (‘nicho ecológico’). c) Line 648 (page 16, Spanish ver.): Where it says ‘prácticas de crianza’ as the translation of ‘rearing practices,’ it should say ‘prácticas de crianza, de manejo de cultivo’ (‘rearing practices, cultivation management practices’). c) Line 666 (page 17, Spanish ver.): Where it says ‘…de maquinaria, productos mixtos u otros medios’ (‘...machinery, mixed produce or other means,’), it should say ‘…de maquinaria, prácticas agrícolas, sistemas de producción, productos mixtos u otros medios’ (‘...machinery, agricultural practices, systems of production, mixed produce or other means’). c) Line 691 (page 17, Spanish ver.): Where it says ‘...the reversibility (or lack thereof)’ (in Spanish, ‘…la reversibilidad (o falta de ella)’), it should say ‘...the reversibility (reversible without actions, or with minor actions, greater or irreversible actions’ (‘…la reversibilidad (reversibles sin acciones, o con acciones menores, acciones mayores o irreversibles)’). c) Line 692 (page 17, Spanish ver.): Where it says ‘...(es decir, organismos individuales, por ejemplo de especies protegidas, o poblaciones)’ (‘(i.e., individual organisms – for example of a protected species – or populations)’), it should say ‘…(por ejemplo organismos individuales, poblaciones, comunidades o ecosistemas protegidos)’ (‘...(for example, protected individual organisms, populations, communities or ecosystems)’). c) Line 707 (page 17, Spanish ver.): Where it says ‘...otros enfoques de ordenación’ (‘...other management approaches’), it should say ‘…otras alternativas de manejo’ (‘...other management alternatives’).

New Zealand: See above comments

Norway: Text Line 273 'There is no internationally agreed definition of 'scientific uncertainty', nor are there internationally agreed general rules or guidelines to determine its occurrence.' does not provide the user with anything useful, the same can be said of many terms defined in this guidance. A more useful phrasing in our view would be to replace the last two sentences, that starts on Line 271 with a hybrid sentence 'The issue of uncertainty and the determination of its occurance is dealt with – sometimes differently – in each international instrument incorporating precautionary measures.' In relation to the descriptions, particuarly in the rationale to step 1, are in some places unnecessarily complicated english, should be simplified here and throught the Roadmap to enhance the understanding of non-native language users.

Peru: (translation) The document is too general; it should be more specific.

Republic of Moldova: From our point of view it would be appropriate to include into Roadmap on risk assessment of LMOs specific considerations related to the adverse risks of LMOs to the human health.

South Africa: The introduction of specific elements in the document under the heading Point's to consider is useful for providing greater context and clarity. However, the language then utilized in these sections is prescriptive. The headings should therefore be rephrased to 'points that may be considered as appropriate' The roadmap was reviewed in the context of 2 applications. It was found that in some instances the information required under points to consider was not available in the dossiers.In particular elements relating to: - Horizontal gene flow - cumulative impacts on other LMOs - Multitrophic effects In addition, it was noted that the structure of the Guidance waas not aligned with the structure followed in the risk assessments analysed which used the Annex III structure

Uruguay: (translation) It would be very useful if the guidance could go into greater detail about the implications of the ‘technological package’ associated with the event (539-541) (447), (633-636), (654-655), with concrete references to the advisability of considering the technological package within the risk assessment or independently from it, thus considering only the event itself.

Other Governments

Australia: Lines 200-201 is confusing as determination of what is considered an adverse effect is used to derive assessment endpoints, not the other way round. Also, this provides little useful guidance if legislative protection goals are broad, such as, ‘to protect the environment’. Lines 290-296 give the misleading impression that all uncertainties are significant and should be dealt with. Instead the focus should be restricted to uncertainty sources that impact on the overall risk in a way that may significantly affect decisions. Line 275 is not useful as it is highly contentious and is even contradicted by the example on lines 519-522 as assigning a likelihood of 100% that an adverse effect will occur whenever there is a high level of uncertainty (e.g. experimental trials described in lines 254-259) can be expected to be scientifically invalid on most occasions. Uncertainty is central to the concept of risk. Therefore, the risk assessment provides a systematic framework to consider uncertainty. It would be better to say that explicit considerations of uncertainty provide transparency in the decision making process. Lines 343-370 would be assisted by acknowledgment that the non-GM comparator have associated risks (e.g. gluten in wheat and barley, glycoalkaloids in potatoes, gossypol in cotton, weediness in rice), which serves as a baseline to determine if the GM version has additional or increased risks. In addition, there should be recognition that in some circumstances, the appropriate comparator may be a GMO (e.g. with new forms of GM cotton in environments where GM cotton is already the standard cultivated form of cotton). Lines 377-378 ‘risk assessment is defined as a science-based process…..’ should delete ‘defined as’ as parts of the Roadmap that describe adverse effects and risk acceptability rely on policy (i.e. non-science). In the two Australian case studies the adverse effects identify the values that are affected (i.e. constitute harm) and expressed in the Risk Analysis Framework (OGTR 2013, see http://www.ogtr.gov.au/internet/ogtr/publishing.nsf/Content/42D3AAD51452D5ECCA2574550015E69F/$File/raffinal5_2.pdf). Only after explicit description of these values is evidence (including science-based evidence) used to test if these values are affected. Lines 401-406 are confusing and do not fit well with the simple, clear sentence at line 398. The two Australian case studies are more consistent with line 398. Lines 410-413 and 417-421 only discuss changes without specifying what an adverse effect might be. The two Australian case studies put emphasis on what the potential adverse effects might be. Lines 412,413 – the term ‘target’ should be replaced by ‘assessment endpoint’ as it could lead to confusion with ‘non target organisms’ and ‘target organisms’ used elsewhere, eg line 419. Lines 426-427 – should also include behavioral changes. Lines 428-431 are not useful without clear examples provided using existing LMOs. It also involves ambiguous use of effects. Are combinatorial and cumulative effects shorthand for combinatorial and cumulative adverse effects? Lines 469-470 are highly contentious. This is not meaningful without stating the protection goals. If they are protection of agricultural yield or food security then they may be relevant. However, in the Australian context, as described for these two case studies, the protection goal is the environment. These suggested adverse effects would be considered as economic/stewardship issues, not environmental adverse effects. Lines 476-481 are very confusing as they mix together different types of consideration (use, intended receiving environment/receiving environment due to potential for spread and persistence, nature of the receiving environment, pathways to adverse effects, and adverse effects) in an incoherent manner. Each point to consider should be simple and clear. Lines 509-514 seem to indicate that risk hypotheses/scenarios can be established in the absence of a causal link, which is very likely when there is no actual causal link. It also contradicts the clear statements at lines 415-416 and 505-506. Lines 547-552 can be misleading by confusing likelihood of adverse effects occurring with the likelihood of a single link in the causal pathway to an adverse effect. It could be made clearer by adding ‘as a step in a pathway to an adverse effect’, after ‘compatible species’, in line 547. Lines 564-565 – as for lines 509-513, this text describes factors which should have already been taken into consideration in development of the risk scenarios – this should be clarified in the text. Lines 586-597 are not clear that they indicate magnitude of the consequences as they do not explicitly describe an adverse effect, but seem more to do with step 1 or 2. Lines 598-599 are not clear if this is referring to effects or adverse effects? Without examples this does not seem useful. Line 604 is not clear what the adverse effect is as invasive plants (i.e. spread and persist) do not necessarily give rise to adverse effects (as is the case in the two Australian case studies). Suggest rewording to indicate that data from field trials could provide information about potential for invasiveness and impacts in the environment. Lines 612-636 give little useful guidance on the notion of level of individual risks as opposed to overall level of risk and how these are actually achieved. As uncertainty is considered in both likelihood and consequences, it will already be reflected in the level of risk estimated from the combination of these two elements. As discussed currently, it sounds like a new consideration. Lines 620-621 have little use and provide less insight than the parent statement at paragraph 8(d) of Annex III. Multiple lines of evidence may well be used to obtain the likelihood and consequences ratings, but what has this got to do with combining these to obtain an estimated level of individual risks, which are then combined in some unexplained way to obtain an overall level of risk. In the two Australian case studies the likelihood and consequences are combined through a risk matrix that can be more clearly linked to Annex III than to these lines. Lines 664-668 are confusing as only relevant if it affects the estimates of the overall level of risk.

Canada: In practice, risk assessors in Canada found the guidance confusing and unworkable for most cases considered. The LMO risk assessments considered for testing did not follow the format of the Roadmap in their reporting or the hazards considered. The language is confusing and inconsistent with the terminology generally used. Speculative hazards such as those discussed in line 431 where gene products combine for an unexpected hazard has never been identified in any of the cases reported on the BCH. The addition of these terms on several points in the guidance is not useful. The points to consider format suggests that normal phenomena such as gene flow to wild relatives is a hazard. Line 180 refers to the broad intended applicability of the Roadmap, including to limited scale releases and field trials. In testing the Roadmap was only marginally applicable for risk assessment of field trial of an LMO plant. Much of the information identified in the Roadmap would be unknown or irrelevant in the context of a confined field trial release. For example, potential for vertical gene transfer (line 482) is a relevant consideration for field trials, but other considerations listed in the documents, as potential changes to existing agronomic or pest management practices (line 478), are not. Other important considerations for the receiving environment (line 471) for a field trial of an LM plant, as control of land and proximity to the related cultivated species are not included at all in the document. In general, considerations of larger landscape effects are irrelevant in the context of a small scale field trial with limited duration. The objective of field trials is generally to learn about the LMO and generate the information that is under consideration in the Roadmap in contrast to a request for larger scale release where most information about the LMO should be known. In this context the Roadmap has little applicability to a confined field trial. Small scale research field trials have different considerations that are mostly related to inspection, monitoring and risk management. Since this Roadmap has limited utility for risk assessment related to field trials, we suggest removing the paragraph beginning at line 184. Directive Dir 2000-07: Conducting Confined research Field Trials of Plant with Novel Traits in Canada: http://www.inspection.gc.ca/plants/plants-with-novel-traits/applicants/directive-dir2000-07/eng/1304474667559/1304474738697 provides guidance to risk assessors on how to determine necessary and sufficient information and to applicants who wish to conduct field experiments. Many other competent authorities have similar guidance. Gene transfer is not a hazard, it is a characteristic of living organisms, but the outcome of gene transfer may be. The concept of combinatorial and cumulative effects has not been captured in guidance or risk assessment of LMOs elsewhere. These speculative and unresolved concepts in terms of a LMO risk assessment add little value to the process of hazard identification and in fact these concepts add confusion and hamper utility. Lines 299-340 on the planning phase of the risk assessment describe processes which differ from country to country and are rooted in law and derived regulations. The division in roles between risk assessment, risk management and decision making is rooted in policy and institutional structure. Although some competent authorities use consultative mechanisms, this is a function of policy and rooted in regulation, often as a requirement for public transparency and is not a condition for conducting a risk assessment of an LMO. With the wide divergence of models for implementation, the guidance offered is of limited utility and provoked more questions than practical guidance. Risk assessors are required to identify risks and their acceptability is decided by risk manager. The rest of the suggested steps would either be primary steps in developing regulations or guidelines or a special case where an LMO was proposed for release and these primary steps had not already occurred. This section should be eliminated as it is not generally applicable and adds extensive confusion. - Lines 335 to 337 should be more specific and include categorization of the likelihood, magnitude of consequences, and the matrix to be used to estimate risk. Defining the quality of scientific information, (line 229-239) is a policy decision by competent authorities and the guidance here is prescriptive in terms of policy guidance but vague in scientific terms. For example, there is guidance to use appropriate statistical methods but no guidance as to what those may be. The Roadmap acknowledges that there is no international guidance document but fails to acknowledge that risk assessors will bring professional expertise to bear and will be capable of making those determinations more effectively on their own using their own standards. The recommendation that methods be transparent and sufficiently detailed for independent verification (line 235) is unrelated to the quality of data submitted but rather addresses whether the competent authority wishes to verify submitted information. In Canada, Developers are responsible for the data they submit and misrepresentation would constitute a breach of the regulations. Independent verification does not take place. This policy recommendation is well outside of the scope of the Roadmap.

United States of America: Perhaps one of the most useful and practical suggestions for novice risk assessors is not addressed in the guidance document at all, and it would require extensive additional text to elaborate on the need for risk assessors to use relevant experts who know about the non-LMO versions of the organism and can set the context and extent of worldwide experience with confined releases and unconfined releases of the species of the organism in question. There is an over-emphasis throughout the document on the predictive utility to risk assessors of molecular genetic characterization of the LMO. In actual practice, the phenotype of the LMO is of far greater predictive utility in assessing potential environmental risks, and this approach has the added benefit of drawing upon our experiences with evaluating and using non-LMO organisms that have similar or identical phenotypes. In order to increase the usefulness of this document, it should describe how confinement approaches for such releases serve to minimize the likelihood of adverse environmental impacts from the LMO release, even when detailed information on the specific LMO is not available. This is a well understood principle with testing LMOs and non-LMOs, and its value should be more extensively developed in the document.

Organizations

Academy of Sciences of Moldova: It is necessary to include into Roadmap on risk assessment of LMOs specific considerations related to the adverse risks of LMOs to the human health.

Eco-TIRAS NGO: From our point of view it would be appropriate to include into Roadmap on risk assessment of LMOs specific considerations related to the adverse risks of LMOs to the human health.

ENCA EPA interest group On Risk Assessment and Monitoring of Genetically Modified Organisms (GMOs): Line 229: term 'accepted practice'. This term is not clear and should be better explained. Line 235 : add: 'Reporting of data, source of data and methods….'The whole bullet point starting 235 ff is felt to be very important. Line 260 ff. 'Additional considerations'. These bullet points are difficult to understand and would need further explanations. **Altogether the chapter on Overarching issues is very useful and important and is strongly endorsed.** Last para of the chapter on comparators: Line 369: we propose to quote more precisely: '…please refer to Part II, Section B, of this Guidance.'

GenOk - Centre for Biosafety: The issue of defining the problems and issues to address up front is an essential component of the risk assessment process, as discussed in the 'context and scoping' and 'step 1' in the 'conduction the risk assessment' sections. Therefore, term 'problem formulation process' on line 403 is perhaps in need of defining

Global Industry Coalition (GIC): The Guidance does describe the main concepts within the methodology used in conducting a risk assessment, and makes clear some key distinctions that risk assessment considers, e.g., hazard and exposure as unique components of risk. However, development of a succinct section on problem formulation is recommended, as well as further explanation on how to determine what information is relevant to characterise exposure and hazard for a particular area of assessment and how to characterise risk. This section should outline and clearly define key elements of the risk assessment such as: protection goals, assessment endpoints and measurement endpoints. The Guidance should provide more insights as to how risk assessments discern 'need to know' versus 'nice to know' information necessary to conduct a risk assessment. Listing potential hazards and exposure scenarios without context (real-world examples) and with no clear guidance on how to integrate the various pieces of information in performing a risk assessment in practical terms should be avoided. The 'prescriptive' tone and policy-based statements in the Guidance should be revised, as should the inappropriate notion that risk assessment occurs in discrete 'steps'. Overall, the Guidance should be reduced in length to represent a true scientific consensus based on real-world experience for it to become useful.

Public Research and Regulation Initiative (PRRI): Although the practical value of the Roadmap for specific cases is very limited, it can – after revision – be quite useful, to further explain the basis of risk assessment, in the same way as in the past an OECD 'Preamble' document laid the basis for subsequent specific documents to build on.

State University of Moldova: We consider important to include human health considerations as part of Roadmap on risk assessment of LMOs.

Third World Network: The Guidance is extremely useful. Improvements in terms of utility would include the development of further guidance on specific topics of risk assessment, selected on the basis of the priorities and needs indicated by the Parties, and with the view of moving towards achieving operational objectives 1.3. and 1.4 of the Strategic Plan and its outcomes. We believe that it is not that useful to assert that 'there is no internationally agreed definition of 'scientific uncertainty', nor are there internationally agreed general rules or guidelines to determine its occurrence.' What would be more useful would be to outline a framework on how to deal with uncertainty, including focusing on the three main components for describing uncertainy, i.e., the nature, source and level of uncertainty.

CONSISTENCY WITH THE CARTAGENA PROTOCOL

Parties

Austria: The case study used for testing underlined the importance of the concept included in the Roadmap - consistent with the Protocol - that 'special attention (should be paid) to protected areas and centres of origin and centres of genetic diversity' (cf. Line 564, footnote 16). The testing case exemplifies that a conclusive assessment of relevant risk hypotheses, e.g. potential impacts on non-target-organisms, including effects on 'rare, endangered, protected species and/or species of cultural value' (footnote 16), needs to be conducted to be able to devise appropriate risk management strategies. Line 267 - 270: To increase consistency with the Protocol reference should also be made to Article 10 para 6 when describing appropriate ways to deal with identified uncertainties. Lines 290 – 293: The case-study used for testing identified that identification of uncertainties associated with the potential occurrence of specific adverse effects can result in determining that the respective data basis is not allowing completion of a specific assessment. In such a case no specific conclusions regarding that risk issue can be drawn. Such a possibility should be identified in the text of the indicated paragraph.

Belarus: Moreover it should be noted that Article 15 of the Cartagena Protocol on Biosafety provides that risk assessment is conducted for determining and evaluating potential adverse effects of LMOs not only on conservation and sustainable use of biological diversity but in view of risk to human health. We would like to emphasize the importance of LMO risk assessment to human health and the necessity for the maximum coverage of the issues which should be taken into consideration by experts when conducting such risk assessment. We propose to develop a separate Guidance on the issues of LMO risk assessment to human health. [translated from original in Russian] The guidance is consistent with the Cartagena Protocol on Biosafety. At the same time it should be noted that Article 15 of the Cartagena Protocol stipulates that Risk assessment is undertaken not only to identify and evaluate the possible adverse effects of LMOs on the conservation and sustainable use of biological diversity but taking into account risks to human health. We would like to note the importance of assessing the risks that LMOs pose to human health and the need to maximally highlight the issues which should be taken into consideration by the experts when undertaking the said risk assessment. We suggest developing a separate 'Guidance on assessing risks posed by LMOs to human health'. It is worthwhile to note the fact that people (the community) are greatly worried by the problem of LMOs' impact exactly on their health.

Belgium: - The testing was conducted as an individual exercise. Different levels of agreement/disagreement (i.e. 'Neutral' or 'Agree') with regards to the consistency of the Roadmap were expressed amongst people.

Brazil: The Guidance must be focused specifically on risk assessments. Decision BS-VI/12 instructed Parties to test the document in 'actual cases of risk assessment and in the context of the Cartagena Protocol on Biosafety'. Therefore, it is clear that the Guidance must not go beyond the scope of the Protocol. Nonetheless, the section 'Related Issues' is dedicated not to risk assessment in itself, but to 'other issues that may be part of the decision-making process' (lines 714-715). Furthermore, the section also addresses 'a number of other issues, which are not mentioned in the Protocol (e.g., co-existence, ethical issues)' (line 721) In this topic, the work of the AHTEG goes beyond what Parties asked it and, for this reason, section the section 'Related Issues' must be removed from the Guidance.

Colombia: (translation) The Guidance has been developed in accordance with the provisions of the Protocol, but it does not go beyond what the Protocol says, that is, it does not develop methodologies, or step-by-step instructions, among other elements, that respond to the actual needs involved in carrying out a risk assessment. In monitoring, however, it does provide some new conceptual elements and elaborates a bit further on the subject beyond what is established in the Cartagena Protocol.

Costa Rica: (translation) Agree, because it considers transboundary movements from an environmental perspective. However, as it has an environmental approach it might not adequately address human health issues. These could be addressed more thoroughly and applicably with this tool.

Czech Republic: No special text modifications are needed. The evaluation of an LMO takes into account also the interactions with other species present in receiving environment.

Denmark: This represents a step forward relative to the Cartagena protocol, because of its more precise descriptions and definitions of what is needed for the risk assessment

El Salvador: (to be translated) Líneas 178-179 Se pueden interpretar de manera equivocada en el sentido que se podría entender que las Autoridades Nacionales Competentes puedan ir (durante el proceso de Evaluación de Riesgo) más allá de los alcances que el Anexo III plantea. Líneas 219-296. A nuestro juicio esta sección no es compatible con los apartados 3 a 6 en "Principios Rectores" del anexo III, lo que parece para llenar una función similar. Líneas 466-468. No está claro que el Protocolo establece una acertada distinción entre la diversidad biológica en los centros de origen y los centros de diversidad genética en contraste con la diversidad biológica en otros lugares. Líneas 605-606 Deberían referirse como consecuencias de efectos adversos y no como consecuencias de los efectos.

Honduras: (translation) The guidance is in many parts inconsistent with the Protocol. Both exceed in several instances the aim of providing an objective guide for evaluating the risks of an LMO in a specific context. The guidance at times goes beyond the mandate of the Protocol. The problem is also that the guidance in its current wording could lead to erroneous assumptions, such as, for example, that LMOs are in fact always hazardous and that what is being assessed is their level of hazardousness, when the exercise is actually a ‘risk assessment.’ Much of the text of the risk assessment guidance may confuse non-specialized regulators who have time constraints due to their other duties. People with no experience in a technical and complex area cannot critically assess a technical document and that is exactly what is being asked in this ‘guidance testing’ exercise: that a technical issue be evaluated by non-specialists. This could lead novice regulators to make hurried decisions without much critical analysis.

India: The scope, mandate and understanding in developing this document were specifically to elaborate on Annex –III to the Cartagena Protocol on Biosafety. While the broad structure and the 5 steps outlined in the document are consistent with the steps in Anne III to the Protocol, the text of the document is not in conformity. The basic reason for the same is that Annex III is based on scientific considerations, whereas the guidance document extends into policy considerations and administrative issues. For example under the section on 'Overarching issues in the risk assessment' sub section ‘quality and relevance of information’, several points/terms being used are administrative in nature. The terms such as 'independent review', sufficient quality of information etc are administrative/operational procedures and subject to different interpretations. In addition there have been use of new terminologies as mentioned in the previous sections which is not consistent with the CPB . As mentioned earlier, inclusion of the Section on ‘Related Issues’ extends beyond the realm of this exercise into decision making by a Party with respect to a LMO. Guidance for decision making is a process after completing the risk assessment process and therefore outside the scope. Relevance of including issues such as Socio-economic consideration which are still being discussed at the Cop-MoP level and the Nagoya Kuala Lumpur Supplementary Protocol on Liability and Redress in the context of Annex III is not Issues such as ethical issues, co-existence etc is outside the scope of the Protocol itself and introduces a high level of subjectivity to a scientific risk assessment process.

Japan: 1) RA under the CPB is 'to identify and evaluate the potential adverse effects of LMOs on the conservation and sustainable use of biological diversity'. (Annex III 1) On the other hand, RA explained in the Roadmap seems to identify and evaluate broader adverse effects on the whole environment caused not only by LMOs but also by agricultural practices. In order to be consistent with the CPB, at least the following parts beyond the CPB should be revised. - In line 172, line 177, line 182 : 'environmental risks' - In line 635 : 'Broader ecosystem and landscape considerations' - In line 573-575 : 'the adverse effects associated with the exsisteing practices or with practices will be introduced along with the LMO (such as various agronomic practices, for example, for pest or weed management)' --- Those are not the adverse effects caused by LMOs. 2) Monitoring under the Annex III of the CPB is the monitoring of the assessed LMO in the receiving environment where there is uncertainty regarding the level of risk. (Annex III 8(f)) Such monitoring does not include 'general monitoring'. In order to be consistent with the CPB, at least the following parts related to 'general monitoring' should be deleted. - In line 676-677 : 'Monitoring can also be applied as a tool to detect effects that were not anticipated in the risk assessment and long-term adverse effects.' --- Such monitoring is not the 'monitoring' stated in the Annex III. - In line 692 : 'general monitoring' 3) The words which have no scientific consensus among the Parties, such as 'cumulative effects', should not be used in the Roadmap. At least the following parts should be revised. - In line 429, line 495, line 598, line 635 : 'cumulative'

Malaysia: The training manual developed was not consistent with Annex III or the requirements as listed in this Guidance document.

Mexico: (translation) 1. The text should be adjusted to reflect the objectives of Annex III: a) Line 219 (page 6, Spanish ver.). Where it says: ‘This Guidance consists of three parts. In Part I, the Roadmap for Risk Assessment of LMOs is presented,’ it should read instead: ‘This Guidance consists of three parts. Part I presents a Roadmap for Risk Assessment of LMOs for conservation and sustainable use of biological diversity in the likely potential receiving environment.’

Netherlands: - The presented information is largely focused on unconfined commercial releases. Field trials are addressed insufficiently to yield the Guidance practical for this particular use. The environmental risk assessment of LMOs in case of spillage during handling and transport is left unmentioned.

Norway: The issue of baselines is relevant here in the context of the Protocol's focus on the receiving environment. Thus it is not only a comparison of a LMO and a non-LMO in the receiving environment that is important, but changes to the receiving environment itself resulting from the release of the LMO that must be additionally considered. We suggest adding in line 424, to read, '…in the likely potential receiving environment, or baseline enivornmental conditions prior to the release of the LMO'. This is important to reinforce the subsequent points to consider in (g) and (i).

Peru: (translation) No improvements are suggested.

Republic of Moldova: According to the Annex III of the Protocol it is required to provide taxonomic status and common name, source and relevant biological characteristics of the donor organism. In this regard we suggest to reflect this provision in the Roadmap on risk assessment of LMOs, under the lines 441-443, to introduce specific requirements for characterization of donor organisms, to be sufficient for risk evaluators.

South Africa: It was recognized that the guideline is consistent with the limited focus of the CPB and is therefore not encompassing issues that may not be biodiversity related.

Other Governments

Australia: Lines 178-179 are open to misinterpretation as going beyond the scope of Annex III. Paragraph 2 of Annex III states that 'Risk assessment is, inter alia, used by competent authorities to make informed decisions regarding living modified organisms.' Therefore, the primary target group should be competent (national) authorities. Any use beyond this group should be seen as an unintended benefit, not its primary objective. Risk assessment of LMOs in the two case studies used here (DIR 102 and 108) were done by the Australian competent national authority. Lines 204-205 states that risk assessment methodology is a ‘tiered process’, which implies a hierarchy rather a flat, albeit connected, process as described in Annex III. Both Australian case studies are consistent with Annex III rather than a ‘tiered process’. Would be better described as a ‘stepwise process’. Line 269 is missing and/or to be a strictly correct quote from Annex III. Lines 219-296. This section is not consistent with paragraphs 3-6 under 'Guiding principles' of Annex III, which would seem to fill an equivalent role. The section ‘Quality and relevance of information’, lines 222-265, seems to misinterpret 'Risk assessment should be carried out in a scientifically sound and transparent manner….' in paragraph 3 of Annex III. Instead the Roadmap restricts attention regarding scientifically soundness and transparency to information used in a risk assessment rather that to the overall process of risk assessment itself. Indeed, nothing is provided in the Roadmap to indicate scientific validation of its own guidance. Similarly, the discussion in the section ‘Identification and consideration of uncertainty’, 266-296, seems to be restricted to matters related to uncertainty regarding the level of risk as addressed in the methodology section of Annex III, paragraph 8(f), rather than uncertainty in the overarching sense as discussed in paragraph 4 of Annex III. Both sections of the Roadmap, ‘Quality and relevance of information’ and ‘Identification and consideration of uncertainty’ are important considerations in risk assessment. However, as described here, they would be better suited in the section ‘conducting the risk assessment’, line 371, and with a more appropriate title such as ‘risk assessment methodology under the microscope’. However, this would highlight the absence of guidance on the section, 'Guiding principles' in Annex III. Part of this gap could be filled in part by substituting lines 219-296 with lines 343-370 of the Roadmap. However, lines 346-364 are problematic (see below) and new text would still be needed to provide guidance on paragraphs 3, 4 and 6 of Annex III. The Australian case studies are consistent with Annex III, but not the Roadmap in these matters. Lines 346-364 are inconsistent with lines 344-345, which is a direct quote from paragraph 5 of Annex III. The context (comparator) is risks posed by the non-modified recipients or parental (missing in line 333) organism. This implies a comparative risk assessment in which risks posed by the LMO are considered against the risks posed by the non-modified recipients or parental organism. The Roadmap limits discussion to changes of genotype and phenotype in the LMO, rather changes in the nature and level of risks. These risks should be considered in the context relevant and appropriate to the organism under consideration, and does not imply that comparisons require ‘the same time and location, and under the same environmental conditions’, line 352. For example, risks regarding potential adverse effects to beneficial organisms may be compared between the LMO (e.g. a Bt crop) and the non-modified recipient under different environmental conditions (e.g. different pesticide types/application regimes) if these are appropriate to differences in standard management practices that are expected to apply under normal production conditions. Lines 466-468. It is not clear that the Protocol makes a clear distinction between biological diversity in centres of origin and centres of genetic diversity (emphasised in the Roadmap) as opposed to biological diversity elsewhere (e.g. nature reserves or World Heritage areas). Lines 586-597 and line 604 are inconsistent with 8(c) as they are not about the consequences of adverse effects but about causal pathways to potential adverse effects. Lines 605-606 should be consequences of adverse effects not consequences of effects. The Australian case studies are consistent with Annex III, but not the Roadmap in these matters. Lines 620-621 are misleading. Estimating the level of overall risk (as well as individual risks) is 'based on the evaluation of the likelihood and consequences of the identified adverse effects being realized', paragraph 8(d) of Annex III, not a ‘best estimate of risk from multiple lines of evidence’. It uses non-evidential methods to combine likelihood and consequence assessments. The Australian case studies are consistent with Annex III, but not the Roadmap in this matter.

Canada: The Roadmap is structured in close alignment with the Annex 3 and captures concepts such as case by case, comparative risk assessments. Where it strays is the in inclusion of factors outside of risk assessment and risk management such as product efficacy (described above.) and the extensive discussion of policy elements such as data quality, consultation with stakeholders and selection of experts. (line 263) The section on risk management blurs the line between decision making and risk management and as in many other areas of the document, recommends policy. This is not in line with Annex 3. The last section on related issues has no place in this document. It is not complete enough to be useful, is subjective in the topics chosen and is far outside of the scope of the Roadmap which is intended to enhance the guidance in Annex 3.

United States of America: In the recapitulation of the reason for the guidance, this document focuses primarily on the potential problems with LMOs, whereas the Protocol and the parent convention emphasize the role of biotechnology in providing environmental benefits (and the need for a Protocol to enable sharing of these benefits, even before countries have their own regulatory mechanisms in place to facilitate the necessary transboundary movements of LMOs). This is primarily a case of missed opportunities to set the tone of the document in the Preface and Background sections.

Organizations

Academy of Sciences of Moldova: According to the Annex III of the Protocol it is required to provide taxonomic status and common name, source and relevant biological characteristics of the donor organism. We suggest to introduce this provision in the Roadmap on risk assessment of LMOs the specific requirements for characterisation of donor organisms, to be sufficient for risk evaluators.

Eco-TIRAS NGO: According to the Annex III of the Protocol it is required to provide taxonomic status and common name, source and relevant biological characteristics of the donor organism. In this regard we suggest to refelect this provision in the Roadmap on risk assessment of LMOs, under the lines 441-443, to introduce specific requirements for characterisation of donor organisms, to be sufficient for risk evaluators.

ENCA EPA interest group On Risk Assessment and Monitoring of Genetically Modified Organisms (GMOs): There is only weak reference to the precautionary approach. In order to ensure better consistency with the Protocol, we would like to propose additional wording for line 293 complementing the last sentence of the para: '…in order to help applying the precautionary approach in an adequate manner' The whole para would then read: 'The various forms of uncertainty are considered and described in each step of the risk assessment. In addition, when communicating the results of a risk assessment, it is important to describe, quantitatively or qualitatively, what impact uncertainty may have on the estimated level of risk and on the conclusions and recommendations of the risk assessment in order to help applying the precautionary approach in an adequate manner'

GenOk - Centre for Biosafety: Protection of human health is mentioned in as a core protection goal under the Protocol, yet the consistency between this important goal and its consideration in the Guidance is disproportional. Greater weight and consideration should be given to appropriate and robust strategices for appraising the risks to human health in the document.

Global Industry Coalition (GIC): The Guidance goes beyond the recommendations in the Cartagena Protocol. The 'prescriptive' tone and policy-based statements in the Guidance should be revised. The Guidance should be reduced in length and represent a true scientific consensus so it is in line with risk assessments procedures followed by most parties.

Public Research and Regulation Initiative (PRRI): While the verbatim quotes of provisions of the Protocol are obviously consistent with the text Protocol, the overall flavour of the documents (e.g. the notion that there must be risks, the disproportionate emphasis on uncertainties and the absence of the notion that this technology is anticipated to have great benefits goes against the background and legal basis of the Protocol, such as: • Access to and transfer of biotechnology are essential elements to attain the objectives of the CBD (Article 16 CBD) • Parties agree to promote and advance priority access to the results and benefits arising from biotechnologies (articles 19.1 and 19.2 of the CBD) • Modern biotechnology has great potential for human well-being (Preamble CPB) These references should be included and repeated at least frequently as the references to risks and uncertainties.

State University of Moldova: We suggest to reflect special provision and characteristics of donor organisms under lines 441-443 of the Roadmap, as it is requested in the Annex III of the Protocol.

Third World Network: We find the Guidance consistent with the Cartagena Protocol on Biosafety. It is a consistent and coherent elaboration of Annex III. There are however some missing elements, the most obvious being the issue of risks to human health. The Protocol clearly states that its objective is to contribute to ensuring an adequate level of protection in the field of the safe transfer, handling and use of living modified organisms resulting from modern biotechnology that may have adverse effects on the conservation and sustainable use of biological diversity, taking also into account risks to human health, and specifically focusing on transboundary movements. In addition, the scope of the Protocol shall apply to the transboundary movement, transit, handling and use of all living modified organisms that may have adverse effects on the conservation and sustainable use of biological diversity, taking also into account risks to human health. Article 15 of the Protocol obliges Parties to undertake risk assessment, based, at a minimum, on information provided in accordance with Article 8 and other available scientific evidence in order to identify and evaluate the possible adverse effects of living modified organisms on the conservation and sustainable use of biological diversity, taking also into account risks to human health. Several other provisions of the Protocol, inter alia, Articles 2, 10, 11, 12, 16, 17, 18 and 21, as well as Annex III, also refer to the need to take into account risks to human health. There is therefore strong justification for the Guidance to also include risks to human health in order to enable and assist Parties to meet their obligations under the Protocol.

Zelenyi Svit / Friends of the Earth Ukraine: Probably should be also provided referance to Alma-Aty amendment to Aarhus Convention as potential instrument of public involvemnt in decision making

PAST AND PRESENT EXPERIENCE WITH LMOs

Parties

Bolivia (Plurinational State of): (translation) Line 333: Where it says ‘risk thresholds,’ add ‘according to each specific case.’ Line 345: Add ‘production development policies.’ After line 349: Add another item that includes ‘Establishing a baseline for the wild type, ‘native’/conventional organism.’ Line 380: Where it says ‘…by the non-modified recipients,’ include ‘wild type/native.’ Line 548: Where it says ‘… toxic or allergenic effects,’ add also ‘balanced feed used for different animal species and which will also be consumed as meat by human beings.’ Line 604: Take into account climate change factors that are modifying these aspects. Line 655: …including sexually compatible species and an ecosystem approach. Line 656-660: Emphasize that seeds of non-modified organisms must be certified. Line 672: Add ‘carcinogens.’ Line 727: Add at the end ‘for which operating capacity must be developed in the country.’

Brazil: The institution responsible for risk assessments under the Brazilian law (the National Technical Commission for Biosafety – CTNBio) considers that the national legislation is more advanced than the Guidance. Therefore, testing the Guidance was not particularly beneficial for knowledge development. According to CTNBio the Brazilian legislation allows risk assessments in independent sections that enables a particular approach to each of risk assessments area, according to scientific progress, considering the objectives of protection (invasiveness, non-target organisms). Each of these sections comprise a risk rating, which is characterized as a hazard exposure, where the problem in question is clearly defined. Another relevant aspect is that the national legislation allows the consideration of the available scientific information. In addition, the Brazilian law facilitates the communication between the risk assessor (CTNBio) and the risk managers (CTNBio and bodies of supervision and registration) which unfortunately is still a gap in the Guidance. The Ministry of Agriculture, Livestock and Supply presented the following considerations: 'In the section ´Establishing the context and scope´ (line 300) the previous risk assessments (in different levels of appropriateness - with related surrogate systems, modified traits in other organisms, GMO or parent organisms overseas or the GMO/parent organism in the receiving environment) should be included as it can also be important to set up the context of the actual risk assessment. Risk assessment should be carried out on case-by-case basis and the required information may vary in the nature and level of detail (Annex III of the Protocol). However the fact that the information required can be simplified in some cases in not mentioned along any document. For instance, for ´RA of LMO plants with stacked genes or traits´ guidance there is no mention along the text of the possibility to reduce the information requirement in case the RA of stacked genes if individual events were already approved and if the evidences are showing that there is no interaction between the genes/proteins expressed (lines 759 - 952)'.

Costa Rica: (translation) The group that validated this tool is formed for the most part by people with experience in risk analysis who found it useful, notwithstanding the observations highlighted in previous items.

Czech Republic: All important questions which emerged in previous risk assessments were incorporated in Part I.

Denmark: The text is sufficient

El Salvador: (to be translated) En general esta parte de la guía necesita reestructurarse ya que la aplicabilidad de la misma está sujeta a las condiciones locales de cada país y aunque el proceso sea el mismo las experiencias suelen ser distintas en virtud de la capacidad instalada de cada país, un ejemplo claro se encuentra en las líneas 227-234 ya que los estudios que tienen por objeto los efectos adversos de los organismos vivos modificados han evidenciado deficiencias en estos aspectos, así que el que se agreguen algunos ejemplos resultaría muy útil.

European Union: The EU welcomes the clarity provided (line 645) with regard to the need, during the problem formulation, to identify protection goals, assessment endpoints and risk thresholds.

Honduras: (translation) The guidance is in many parts inconsistent with the Protocol. Both exceed in several instances the aim of providing an objective guide for assessing the risks of an LMO in a specific context. The guidance at times seems to exceed the mandate of the Protocol. The problem is also that, in its current wording, the guidance could lead to erroneous assumptions, such as, for example, that LMOs are in fact always hazardous and that what is being assessed is their level of hazardousness, when the exercise is actually a ‘risk assessment.’ Much of the text of the risk assessment guidance may confuse non-specialized regulators who have time constraints due to their other duties. People with no experience in a technical and complex area cannot critically assess a technical document and that is exactly what is being asked in this ‘guidance testing’ exercise: that a technical issue be evaluated by non-specialists. This could lead novice regulators to make hurried decisions without much critical analysis.

Hungary: However this part of the document provides an overall risk assessment guidance, it mainly focuses on GM plants. We are aware that Part II refers to the assessment of the specific types of LMOs and traits, but Part I should reflect more to the different types of LMOs, including GM animals, microorganisms etc.

India: The guidance document is envisaged to enhance the understanding of scientific risk assessment process for all LMOS but has been developed with focus on LM plants in view of the available experience (refer line 181-183) with LM plants. However this linkage has not been established in Part I and part II and needs to be reviewed. In fact the pattern followed in part II for Guidance on Stacked Events is pre-emptive and restrictive. It does not highlight / demonstrate (I) how a risk assessment is actually conducted (i) how the risk assessment process can be simplified using the available experience and (ii) risk assessment may not be applicable to all Protection Goals or end points . It is case /trait specific, nature of receiving environment and intended use. The complexity further arises from the fact that experience in conduction risk assessment for chemical hazards has been extrapolated for assessing risks from biological material / LMOs.

Japan: As stated [above], this guidance is filled with several 'Points to consider' unable to consider and we see no practicality in this document. We are not sure whether those who were involved in preparing this guidance had been involved in risk assessment in reality. If no, it would be required to have such person in this process.

Malaysia: 1. Some items in the document are impossible for developing countries to adopt. There is financial constraint to do some of the requirements 2. It is not a practical working document to conduct a risk assessment. However, it can be used as an additional reference. 3. A lot of information that is needed is not essential to make a decision. The document may be misinterpreted that everything that is listed must be done. 4. How much data would be considered enough from the list? The items for consideration cannot be generalized that it is applicable to all. 5. It is not a practical document to conduct a risk assessment for an experimental field trial. For countries that are trying to develop their modern biotechnology capacity, these requirements are overwhelming. 6. The document is not a print-friendly document as there are a lot of links attached. It will not be easy for a third world country with limited internet access to fully access the document with all the links.

Mexico: (translation) 1. The introduction should acknowledge that there are other approaches to risk assessment (worst case scenario, event tree analysis, etc.), besides the problem formulation approach. 2. Line 517 (page 13, Spanish ver.). Where it says ‘genotypic and phenotypic terms,’ it should say ‘genetic terms, phenotypic terms, and changes due to technology use and management.’ 3. Line 599 (page 15, Spanish ver.): The guidance should provide guidelines for establishing criteria to define types and levels of exposure.

Netherlands: - The concept of uncertainty is discussed in general terms only. It omits to address uncertainty in the context of experience in risk assessment and history of use that might have been obtained already with certain LMOs. Such existing experiences directly impact on the level of uncertainty. It also does not address the principal differences how the concept of uncertainty is handled in field trials and unconfined releases.

New Zealand: See above comments

Norway: The section 'choice of comparators' could better elaborate on the impact of different choice of comparators may have on the analytical robustness of the comparative assessment. Therefore we suggest the addition of the following sentence at the end of the paragraph on line 371; 'The impact of using additional comparators that are not (near)isogenic lines on the level of analytical rigour of the comparative assessment may be taken into consideration when deciding on appropriate comparators'.

Peru: (translation) Include examples on the subject.

Philippines: OVERALL: experience over almost 20 years not considered (vector sequences, horizontal gene flow, chronic effects, bioaccumulation, persistence of gene product in environment). Please see attached

Republic of Moldova: The structure of the Roadmap foresees that at the end of each step it is provided a link to the reference materials (web-sites). These references facilitate the evaluators to take into account the past and present experiences during the risk evaluation and decision making processes. These references materials might be updated and completed any time, so the newly scientific evidence is available to use during the risk assessment. The Roadmap is an open and living document that is guiding the evaluators with the minimum required questions, at the same time it might be updated and improved according the new scientific publications. >

South Africa: 485 Horizontal gene transfer in plants remains on the list, although it has not been demonstrated> 460 Baseline data not always available for new or novel applications. Unintended effects All the elements listed in the points to consider under 471 to 495 were not covered in the applications that were assessed as part of this exercise.

Other Governments

Australia: Lines 227-234 are important as studies purporting adverse effects from LMOs have typically shown shortcomings in these aspects. So the addition of some of these examples would be useful. Lines 275-289 are an idiosyncratic treatment of uncertainty that is not applied in the regulatory setting, including the two Australian case studies. In fact, the implied nature of ‘incomplete knowledge’ in the sentence at lines 286-288 contradicts the majority position in risk analysis, which treats incomplete knowledge as a form of epistemic uncertainty and reducible (to some degree in principle) by obtaining additional information. Lines 237-238. Sample material has never been required by the Australian competent national authority. Instead, relevant information is requested about the sample material from the applicant. Lines 222-265 ignores the reality that competent national authorities such as Australia provide the necessary independence for assessing the information either through regulatory staff, enlisted expert advice and consultation. Also, relevant information is determined by legislation. Lines 300-340 ignore the importance of other key parameters for establishing the context and seen in the two Australian case studies, namely the risk criteria for the likelihood and consequence assessments, and for combining the two; information from previous risk assessments of the same or similar LMOs; and proposed limits and controls to restrict spread and persistence of the LMO (particularly relevant for field trials). From experience in the Australian context and also shown in the two case studies, molecular information (lines 443-456) have not provided clear examples for identifying substantive risks; this is particularly marked for field trials.

Canada: - Include key and long established concepts for LMO risk assessment such as the use of familiarity to increase potential utility. Risk assessments are not linear but rather horizontal processes where hazards are considered in context, familiarity factors into the information required and the degree of hazard identification undertaken This is apparent in the LMO risk assessments posted on the BCH. - Line 367 suggests that the use of a non-modified counterpart may not be sufficient to assess the risk of a stress tolerant plant. For countries that have conducted risk assessments on these types of crops, there has been no evidence to date that this is the case. It is difficult to follow the scientific logic behind this statement and as practice to date contradicts this statement, it is counter-productive to include this in guidance. Since Annex 3 states unequivocally that LMO risk assessments are comparative it also departs from the guidance in Annex 3. Even if there is significant divergence from the usual domestic phenotype, the process would still be comparative as described in Annex 3, therefore this paragraph should be deleted. The inclusion of extensive text on indirect effects, synergistic and combinatorial effects ignores the long history of experience with LM crops where these risks have not been realized and provides no context for when they might be. The recurring emphasis on these speculative risks without actual context is counterproductive and confusing.

United States of America: As mentioned in comments above, the current content of the document gives the erroneous impression throughout that LMOs are likely to pose risks to biodiversity that differ from those posed by non-LMOs, yet this is not supported by global experience to date in both confined environmental releases of LMOs or in unconfined releases of LMOs. There are numerous places in the document where the facts could be made clearer, but one specific example would be to note the actual experience with LM plants on line 182. The text could cite the experiences worldwide that there have been no substantiated adverse effects on biodiversity arising from either confined or unconfined LM plants. Such environmental releases have occurred over the past 30 years at tens of thousands of sites worldwide. This type of information would be very useful in the early parts of the document to set the context for the overall discussion of risk assessment that follows. It would also pave the way for a more well-developed explanation that it is not the technique of genetic modification that is of primary interest in risk assessment, but rather the phenotype of the organism resulting from the genetic modification. The discussion on centers of origin, gene flow, etc., found throughout the document need to correct the impression left for readers of this version of the guidance that gene flow poses risks in and of itself (and that LMOs are unique in posing potential adverse effects on biodiversity). This is a pervasive short-coming of the document that cannot be pinpointed to a few lines, but they appear prominently in the section on Conducting the Risk Assessment (lines371-723) and the Flowchart (Figure1, lines 726-731).

Organizations

Academy of Sciences of Moldova: The structure of the Roadmap foresees that at the end of each step it is provided a link to the reference materials (web-sites). These references facilitate the evaluators to take into account the past and present experiences during the risk evaluation and decision making processes. These references materials might be updated and completed any time, so the newly scientific evidence is available to use during the risk assessment.

Eco-TIRAS NGO: The structure of the Roadmap foresees that at the end of each step it is provided a link to the reference materials (web-sites). These references facililtate the evaluators to take into account the past and present experiences during the risk evaluation and decision making processes. These references materials might be updated and completed any time, so the newly scientific evidence is available to use during the risk assessment. The roadmap is an open and living document that is ghuiding the evaluators with the minimum required questions, at the same time it might be updated and improved according the new scientific publications. >

Global Industry Coalition (GIC): The Guidance needs re-structuring and simplification. A problem formulation section should be included and clear explanation on how to link hazard, exposure and risk.

Public Research and Regulation Initiative (PRRI): As said, the way risk assessment is introduced gives the impression that there must always be risks (e.g. language such as 'the risk posed by the LMO'), that all the details will be needed in every case, and that risk assessment is a process with which we have hardly any experience and that is riddled uncertainties. To give the Roadmap better perspective and usefulness, it is advisable to include in the Background after the sentence 'However, it has been developed based largely on living modified (LM) crop plants because the experience to date with environmental risk assessments of LMOs has been mainly gained from these organisms.' (line 182) some further detail about the experience to date, with data from the last 4 decades about the number of risk assessments conducted, the number of field trials conducted, the area over which certain GM crops have been grown, with reference to an Annex with contact points of countries and organisations that have conducted these risks assessments, field trials and commercial planting, as well as the notion that the experience with these decades of risks assessments, field trials and commercial planting does not indicate any verifiable report of adverse affects to human health of teh environment.

State University of Moldova: These references materials might be updated and completed any time, so the newly scientific evidence is available to use during the risk assessment.

FURTHER COMMENTS

Parties

Belarus: The general conclusion of the guidance testing: Road map is an integral and comprehensive guidance and a reference material to concentrate experts' attention on main important issues which should be taken into consideration when conducting risk assessment and which may be missed. For example, it is absolutely right that in lines 453-456 emphasis is put on both intentional and unintentional genetic products, emergence and effect of which are frequently not taken into consideration.

Belgium: - The roadmap is straightforward, well-structured (cfr. 3 sections: overarching issues; planning phase; conducting phase) and comprehensive (cfr. listing of potential points to consider in the risk assessment). It represents a practical and useful tool to learn how to make a risk assessment of LMOs for inexperienced risk assessors.

Bolivia (Plurinational State of): (translation) As not everyone who participated in the exercise conducted in Bolivia was fully familiarized with the Cartagena Protocol, it was suggested that the guidance should mention that Risk Assessors must have a basic knowledge of the Protocol. The flowchart on line 806 is quite useful and in specific cases it could be used by assessors as a checklist.

Costa Rica: (translation) The ‘Problem Formulation’ must be addressed in order to understand how the guidance works. A graphical version of the guidance should be considered, to make it easier to understand. The instrument is a first attempt at providing directions for risk analysis. This guidance requires technical and financial assistance and cooperation for its implementation and continuity . The discussion of how to use it should be accompanied by expert criteria with case studies. It is important for the tool to be revised and, if necessary, improved periodically. It is important to take into consideration other guides that may complement this guidance, including, for example, D.Tree, ILSI, etc.

Czech Republic: Lines 707 to 708: in most risk assessments the evaluators focus in particular on the possible threat given by an introduction of LMO. To make the correct decision the benefit analysis should be included as mentioned in this Protocol.

Cuba: (to be translated) Definir en términos y definiciones Efecto adverso y Peligro. Dejar claro que la probabilidad y las consecuencias así como el riesgo se evaluan y se estiman para cada efecto adverso.

Egypt: The roadmap shows how conducting risk assessment relates to other stages in the overall appraisal process leading to decision making. It clearly emphasizes the importance of the pre-risk assessment scoping process and the post- risk assessment evaluation of its outcomes. The roadmap encourages serious brainstorming of those involved in risk assessment-decision making as to what information they may legitimately ask for and how to evaluate such information and integrate it into a package would help decision making.

European Union: The guidance was not tested using actual applications. The EC, as risk manager, has reviewed the guidance in collaboration with EFSA and considered if followed, if it would provide sufficent information and data to enable risk assessors to make informed decisions. Therefore in the following sections only the usefulness and the utility and comments under this section have been completed.

Honduras: (translation) A more instructional and user-friendly guide should be developed, so that it could be used by non-specialized technicians and could prompt them, if necessary, to consult specialists in the relevant technical fields (genetics, microbiology, plant pathology, biochemistry, etc.), based on the information provided by applicants. Questions should not be posed as if the hazard were a given. Questions should also underscore the benefits. The technical assessment should be conducted separately from social and economic considerations, as these correspond to a different group of specialists.

Hungary: In our opinion the issue of non-target organisms should be highlighted more in this Part, because a wide-range of adverse effects may appear as regards the various types and levels of non-target organisms. Regarding the handling of uncertainity the outcome of the overall risk assessment may result in two possibilities: a) the uncertainity may be dealt with risk management and/or monitoring; b) according to the precautionary approach which is the main principle of the Protocol the release of the certain LMO may be refused. This approach is only mentioned in lines 273-274. In our view more focus should be given for the above mentioned option b).

India: As the Guidance document is a negotiated text and not a consensus document, there are areas of policy, administration procedures appearing in the text leading to confusion in understanding the scientific considerations in the risk assessment process, which is the primary objective of the document. So much so that experienced risk assessors found it difficult to apply the document to real case scenarios. 2. Though the guidance does describe the main concepts within the methodology used in conducting a risk assessment, it use complex terminologies, which does not resemble the structure/terminology used in risk assessment in India as well as other regulatory authorities. It also does not resemble the procedures and protocols used by technology developers in generating safety data and in regulatory submissions. 3. Therefore placing relevant information in the context of the guidance document is complex and time consuming. In the present form the practical understanding of the risk assessment document shall be extremely limited, particularly to novice risk assessors in Parties which have never been involved in this process and may be applying it for the first time. 4. The questionnaire is not consistent to the purpose of titles generated for the purpose of purported comments on each section. Routine and repetitive queries may not elicit relevant response. In the current format, it is difficult to elucidate any original response.

Italy: The Guidance has been tested on a genetically modified plant with a single trait, the NK603 maize event. Taking into account the European regulatory framework about risk assessment, we think that the provided references in this section of the guidance are not sufficiently detailed. In our experience of risk assessors, we use the European guidelines such as those proposed by the European Food Safety Authority (EFSA, Guidance on the environmental risk assessment of genetically modified plants, EFSA Journal 2010; 8(11): 1879. [111.]. doi: 10.2903/j.efsa.2010.1879 available online: www.efsa.europa.eu/efsajournal.htm)

Mexico: (translation) 1. The guidance is a clear and straightforward document that is easy to understand for anyone with experience in risk analysis and assessment. Those who have no such experience will find it harder to understand. 2. The working group considered that, overall, the guidance’s objective of providing ‘a reference that may assist Parties and other Governments in implementing the provisions of the Protocol with regards to risk assessment, in particular its Annex III’ was met, and underlined that the guidance must be aimed at regulators and could be of use for other actors involved in the introduction of LMOs into the environment. 3. The working group identified a number of contributions that it considered relevant: (a) The main idea presented in the introduction regarding potential adverse effects and when such effects can or cannot be considered acceptable risks, depending on the ‘assessment endpoints’ that are in turn derived from the protection goals considered by each Party (protection goals are determined in turn by national policies, legislation, and Annex I of the CBD) (line 188, page 5, Spanish ver.). (b) The concept of uncertainty and the description of the nature of uncertainty as: (i) lack of information (in the Spanish version it says ‘pérdida,’ which is ‘loss,’ and it should be changed to ‘falta’) ; (ii) incomplete knowledge; and (iii) biological or experimental variability, as well as the need to incorporate uncertainty in the risk assessment (line 252, page 9, Spanish ver.). (c) The suggestion that the opinion and/or experience of various actors involved be taken into account, in particular that of experts, who may be able to contribute information and opinions based on knowledge and experience, thus strengthening the findings and elements that may be taken up by decision-makers (line 275, page 8, Spanish ver.). (d) The guidance succeeds in framing and providing context for the risk assessment in connection with the release of LMOs from the perspective of the obligations, responsibilities, and commitments assumed by the Parties under the CBD and the Cartagena Protocol itself (CPB). (e) The explicit inclusion of ‘centers of origin and centers of genetic diversity’ as one of the relevant characteristics of non-modified recipient organisms is very significant for megadiverse countries and centers of origin such as Mexico (line 525, page 13, Spanish ver.). 4. The document does not draw on existing experience with respect to living modified mosquitoes.

Netherlands: a) To promote a uniform interpretation of the questionnaire during the testing, a further specification has been defined of the terminology used: 'Practicality' is understood as: Does the Guidance Document allow you to perform every step of the Risk Assessment in a consistent manner: does it present concrete information to raise and answer the necessary Risk Assessment questions? 'Usefulness/Utility' is understood as: Does the Guidance Document present relevant information to help you understand and enable you to conduct a Risk Assessment? 'Consistency' is understood as: Does the Guidance Document contain elements that are very Practical or Useful, but are not related to the scope of the Risk Assessment in the Protocol? 'Experience' is understood as: Does the Guidance Document improve our understanding of the Risk Assessment methodology and does the Guidance Document make use of existing information? b) The Guidance Document was developed in response to a need for further guidance on Risk Assessment of LMOs. As such, the Guidance Document does find its application in the conducting of the Risk Assessment, not the verification of the outcomes. The testing has therefore been performed with the technical and scientific information that was available with the application of the different actual test cases.

New Zealand: See above comments

Peru: (translation) These are viewed as guidelines and not a roadmap, and the adjustments that need to be made are mainly in the translation of some of the terms. The following is suggested: In general: The term ‘ambiente’ should be used as the translation of ‘environment,’ instead of ‘medio ambiente.’ Line 239: The term ‘identificar’ should be used as the translation of ‘identify,’ instead of ‘descubrir’ (‘discover’). Line 253-254: The phrase ‘datos que provienen de diversos campos científicos’ should be used as the translation of ‘data generated from multiple scientific fields,’ instead of ‘datos que provienen de campos científicos múltiples’. Line 509: The phrase ‘establecer una línea de base de referencia significativa’ should be used as the translation of ‘establish a meaningful baseline,’ instead of ‘establecer una base de referencia significativa’. Line 533-535: The word ‘transgénesis’ is spelled ‘transgenesis’ (no accent). Line 544: Where it says ‘afectos’ as the translation of ‘effects,’ it should say ‘efectos’. Line 549: Where it says ‘cumulativos’ as the translation of ‘cumulative,’ it should say ‘acumulativos’. Line 591: Where it says ‘alérgenos’ as the translation of ‘allergens,’ it should say ‘alergenos’ (no accent). Line 704: Where it says ‘como ser ‘ as the translation of ‘such as,’ it should say ‘como son’.

Republic of Moldova: In our view the Roadmap on risk assessment is an extremely practical, useful and efficient document that can be very helpful for my country to develop institutional and personnel capacities for risk assessment and decision making on LMOs. The testing of the Roadmap was carried out during the Eastern European Regional Course on Integrative Impact Assessment of LMOs under the Cartagena Protocol 'Advancing LMO assessment: Principles, Practice and Progress' , that was held during 3-8 February 2014 in Chisinau, Republic of Moldova, and many positive feedback were received from all participants of the course. To be mentioned that Republic of Moldova was represented at the course by members of the National Biosafety Commission, national competent authorities in the field (Ministry of Environment, Ministry of Agriculture, State Environmental Inspectorate, etc. ), scientific researches in various relevant fields for biosafety representing Academy of Science, State University of Moldova, Agrarian University of Moldova, State Institute for Phytotechnical Research, National Institute for Horticulture and Food Industry. During the testing of the Roadmap session the participants made risk evaluation exercise based on the real dossier of risk assessment on GM potato, provided by the Netherlands authorities via BCH, and which was submitted by BASF Plant Science Company GmbH. The results of this report reflect general view of participants related to the quality and usefulness of the Roadmap Guidance. The overall appreciation of moldovan participants that the Roadmap is extremely useful, practical and efficient to guide national evaluators in the risk assessment process. The Roadmap is fully consistent with the provisions of the Cartagena protocol and its Annex III.It is to be mentioned that the Roadmap is very well structured, it contains information that reflect fundamental knowledge of various fields/disciplines of science. It is written in scientific language but at the same time it is clear and understandable for decision makers and risk evaluators.

South Africa: Line 622 Risk matrices is a usefule, simple tool that is useful. It may be a good idea to include an example of a completed risk matrix as a reference This document focuses on conservation and sustainable use of biodiversity as the scope of the CPB requires. It should be recognised, however, that the introduction of LMOs can have impacts beyond biodiversity effects which can be positive ror negative. For Example (i) risk benefit scenarios where there is clear benefit for society (ii) the development of resistance tand its impact on the sustainability of LMO crops . There are differnce roles for the roleplayers (competent authority, regulatory authority, notifier etc) in implementing this guidance. There is a distinction to be made between the risk assessment conducted by the notifier and the risk analysis conducted by the competent authority or regulator. It is important to have greater clarity that different roles could be taken on by different roleplayers and this may differ from country to country. A section that provides this clarity may be useful. Different roleplayers will play a key role/assume responsibility for different stages of the risk assessment. At times, more than one role player may be responsible for generating data or contributing to the risk assessment process. It is important to use the terms consistently. The terms competent authority and regulator are used interchangeably.

Spain: Part I, the ‘Roadmap,’ is the most well-developed section of the Guidance. It specifies all the steps necessary to conduct a risk assessment, including general issues such as quality and relevance of the information. Therefore, it enables assessors to conduct the assessment from the planning stage on, focusing on aspects that may not be as important for the more experienced assessors, but that are necessary for those who undertake an assessment for the first time. This section describes in great detail the points to consider, but leaves some margin so that each assessor can adapt the level of demand of the information required to the characteristics of the specific release (scale, time), which is something we consider positive. However, because of this margin, inexperienced assessors might find that some aspects need to be more thoroughly defined. For that reason, the level of agreement we have assigned to the practicality and utility criteria is ‘agree.’ As for the criterion of consistency with the Cartagena Protocol, we have assigned a level of agreement of ‘strongly agree,’ because this part of the Guidance corresponds fully to the requirements set out in Annex III.

Tajikistan: This section of the Guidance is user friendly due to its logic nature and structure, usability and informativeness. It is focused on such basic points, which must be taken into account when conducting risk assessment, as follows: 1) quality, actuality and completeness of submitted information; 2) scientific approach, methods of information submission; 3) specific character of each case. In addition, the Guidance is in consistency with accumulated experiences and reflects the current state of scientific knowledge in genetics, biology, molecular biology, etc.

Uruguay: (translation) It would be advisable to go into further depth on the subject of how to factor in uncertainty in risk assessments, how to describe it in quantitative terms when there is no agreement on a definition of scientific uncertainty (314).

Other Governments

Canada: The cases considered in the testing are those were conducted by experienced risk assessors but at least one risk assessor had no familiarity with the cases chosen. None reported that the guidance was helpful. All noted that steps suggested in the planning phase, and overarching issues sections did not seem to be reported in the LMO risk assessments’ at least for LM plants on the BCH. There seemed to be both prescriptive and vague guidance combined and the depth of detail was not consistent. It was felt that the document needed to be reconfigured to better reflect the information contained in reports on the BCH and thus the accepted practice of competent authorities.

United States of America: The Roadmap should be revised to present a more balanced view of existing experience in doing environmental risk assessments of LMOs. It should also make clear that global experience with non-LMOs can provide a useful context for considering whether the phenotype of an LMO is likely to cause adverse effects to the environment. It would be best to revise the Roadmap before attempting any further topic-driven documents, such as those in Parts II and III of the guidance. Preparation of guidance on specific risk assessment topics should be done when there is a body of experience upon which to base the specific guidance. The topics in Parts II and III were chosen because of interest in the topics, rather than whether there was sufficient experience from numerous risk assessments upon which to base a document providing sound guidance. Until such a body of experience is developed, it is more prudent to continue with the case-by-case approach. It is unclear whether the statement in lines 152-154 state that the guidance is a 'living document' is the aspiration of the authors or the Parties to the Cartagena Protocol on Biosafety. After more than 5 years’ work by the AHTEGs and online experts to arrive at this rather weak draft of the guidance document, it seems worth considering whether this is the best mechanism for novice risk assessors to gain a better appreciation of how to conduct environmental risk assessments in a manner consistent with Annex III of the Protocol.

Organizations

Academy of Sciences of Moldova: The testing of the Roadmap was carried out during the Eastern European Regional Course on Integrative Impact Assessment of LMOs under the Cartagena Protocol 'Advancing LMO assessment: Principles, Practice and Progress' , that was held during 3-8 February 2014 in Chisinau, Republic of Moldova, and many positive feedback were received from all participants of the course. To be mentioned that Republic of Moldova was represented at the course by members of the National Biosafety Commission, national competent authorities in the field (Ministry of Environment, Ministry of Agriculture, State Environmental Inspectorate, etc. ), scientific researches in various relevant fields for biosafety representing Academy of Science, State University of Moldova, Agrarian University of Moldova, State Institute for Phytotechnical Research, National Institute for Horticulture and Food Industry. The Roadmap is fully consistent with the provisions of the Cartagena protocol and its Annex III.It is to be mentioned that the Roadmap is very well structured, it contains information that reflect fundamental knowledge of various fields/disciplines of science. It is written in scientific language but at the same time it is clear and understandable for decision makers and risk evaluators.

Eco-TIRAS NGO: In our view the Roadmap on risk assessment is an extremely practical, useful and efficient document that can be very helpful for my country to develop institutional and personnel capacities for risk assessment and decision making on LMOs. The testing of the Roadmap was carried out during the Eastern European Regional Course on Integrative Impact Assessment of LMos under the Cartagena Protocol 'Advancing LMO assessment: Principles, Practice and Progress' , that was held during 3-8 February 2014 in Chisinau, Republic of Moldova, and many positive feedback were received from all participants of the course. To be mentioned that Republic of Moldova was represented at the course by members of the National Biosafety Commission, national competent authorities in the field (Ministry of Environment, Ministry of Agriculture, State Environmental Inspectorate, NGOs etc. ), scientific researches in various relevant fields for biosafety representing Academy of Science, State University of Moldova, Agrarian University of Moldova, State Institute for Phytotechnical Research, National Institute for Horticulture and Food Industry. During the testing of the Roadmap session the participants made risk evaluation exercise based on the real dossier of risk assessment on GM potato, provided by the Netherland authorities via BCH, and which was submitted by BASF Plant Science Company GmbH. The results of this report reflect general view of participants related to the quality and usefulness of the Roadmap Guidance. The overall appreciation of Moldovan participants that the Roadmap is extremely useful, practical and efficient to guide national evaluators in the risk assessment process. The Roadmap is fully consistent with the provisions of the Cartagena protocol and its Annex III.It is to be mentioned that the Roadmap is very well structured, it contains information that reflect fundamental knowledge of various fields/disciplines of science. It is written in scientific language but at the same time it is clear and understandable for decision makers and risk evaluators.

GenOk - Centre for Biosafety: The guidance makes does not reflect existing experience with environmental risk assessments which strongly considers what is catagorized as 'related issues' in the guidance, such as how it is done in practice in Norway. Line 715 should be changed to reflect this experience, to say 'Other issues that may be part of the risk assessment or decision-making process, as appropriate, and that are mentioned in other articles of the Protocol, include

Global Industry Coalition (GIC): The GIC sponsored an analysis to 'test' the risk assessment Guidance following the concept note made available by the Secretariat. The goal was to develop a methodology for testing the Guidance and report on the results of using this method. The search engine on the BCH website was used to identify records for risk assessments completed related to commercial production. Based on the BCH information, for this project, risk assessments conducted in different countries for one particular product were used. A detailed Excel spreadsheet was developed to present the results of the testing in a visual format. The 'recommendations' and 'points to consider' outlined in the Guidance were entered and compared with information extracted from each of the risk assessment reports selected. Examination of the spreadsheet reveals which of those elements were discussed in the reports and what elements described in the Guidance did not appear in the reports. For more details see Q.67

State University of Moldova: The State University of Moldova, Faculty of Biology and Pedology consider that the Roadmap on risk assessment is an extremily practical, useful and efficient document that can be very helpful for my country to develop institutional and personnel capacities for risk assessment and decision making on LMOs. The testing of the Roadmap was carried out during the Eastern European Regional Course on Integrative Impact Assessment of LMos under the Cartagena Protocol 'Advancing LMO assessment: Principles, Practice and Progress' , that was held during 3-8 February 2014 in Chisinau, Republic of Moldova.The overall understanding of participants from the State University of Moldova is that the Roadmap is extremely useful, practical and efficient to guide national evaluators in the risk assessment process. The Roadmap is fully consistent with the provisions of the Cartagena protocol and its Annex III.It is to be mentioned that the Roadmap is very well structured, it contains information that reflect fundamental knowledge of various fields/disciplines of science. It is written in scientific language but at the same time it is clear and understandable for decision makers and risk evaluators.

Zelenyi Svit / Friends of the Earth Ukraine: I propose include as recommendation to conduct also consultations with experienced NGOs and communnity of tergetting area for better understanding social and some other possible accompanying effects from GMO releasing.

LMOS WITH STACKED GENES OR TRAITS

PRACTICALITY

Parties

Bolivia (Plurinational State of): (translation) Some of the sections could provide other examples of cases conducted in Latin America and even, if possible, include publications in Spanish.

Brazil: The present study was based in a test of a bean genetically modified for disease resistance. This trait is produced with a different technology from the one considered by the guide. Thus, the Guidance was not helpful in this point, since it lacks the necessary flexibility to cover all the new techniques of genetic modification and new traits. The Guidance should be improved in order to be more useful in different situations, regardless of the methodology of modification or inserted phenotype.

Colombia: (translation) In the cases to consider, concrete examples should be provided in detail, as it is not clear how to implement what is proposed. In lines 905 to 910, some elements are not clear due to a lack of methodologies or diagrams explaining what the text is trying to convey.

Czech Republic: The uncertainty (how many combinations of newly stacked genes could arise, see statement in line 909) is great and this section surely does not include all potential risks. In our case (the field trials with Phytophthora infestans resistant potato plants) this section does not cover it precisely (see definition in lines 771, 780, 790). Nevertheless, LMO contains three transgenes two of them conferring the resistance to the pathogen and one the resistance to herbicide.

Denmark: Appear to cover relevant aspects

El Salvador: (to be translated) La orientación no resulta del todo práctica debido a que el marco de referencia de cada evaluador es distinto.

Mexico: (translation) 1. The Spanish translation must be revised as in several instances what has been translated into Spanish is not consistent with the ideas conveyed in the English version. 2. On page 24 of the Spanish version, the second sentence in the first paragraph of the Introduction, which reads: ‘As a result, the number of stacked genes in a single LM plant and the number of LM plants with two or more transgenic traits is growing.’ (in Spanish, ‘Como resultado, está creciendo la cantidad de genes apilados en una única planta viva modificada y la cantidad de plantas vivas modificadas con dos o más rasgos transgénicos.’) should be eliminated, as it is not useful and instead only leads to confusion. 3. The working group found the document to be practical, as it takes into account issues such as the accumulation of risks, among other things.

Netherlands: - The scope of this section takes the wrong assumption that a risk assessment is already available for LM plants with the single genes or traits. This leaves many LM plants out of the scope of this section. Also, the availability of a risk assessment for the individual lines is not a necessary prerequisite to allow for a risk assessment of the stacked line. - For stacked genes or traits the focus in the problem formulation should be on possible interactions that may take place between the individual genes or traits. This is left undiscussed in the section and some of the points to consider that are mentioned lack scientific rationale. - The presented information does not follow the structure of the respective steps in the risk assessment. It leaves ambiguity how to mutually use Parts I and II for the specific types of LMOs and traits discussed in Part II. Provide instructions how to use available information and points to consider to ask the right questions for the purpose of performing the respective steps in the risk assessment, in particular Step 1.

Peru: (translation) With respect to Planning: Line 880: Add an introductory text explaining that the planning phase consists only in the choice of comparators. Explain why there are no other items in the planning phase. Line 899: Add a text pointing out the need to develop protocols for megadiverse countries with multiple isogenic lines, supported by baseline studies. Line 909: Add a text pointing out the need to have specific protocols indicating the minimum number of crossings for different species. Line 1005: Add a subparagraph (e) ‘Effects on native/local biodiversity.’ Line 1029: Add the following phrase at the end of subparagraph (c): ‘... and its non-modified wild relatives’ (in Spanish ‘...y sus contrapartes silvestres no modificados’). Line 1062: Specify the precise methods that can be used depending on sensibility, specificity, reliability, and availability.

Other Governments

United States of America: This section on stacked genes or traits is focussed solely on LM plants, yet the rationale for developing a guidance section on this topic is lacking. In fact, experience from doing safety assessments on non-LM plants would not suggest any need to evaluate the environmental safety of the offspring of two plants when each of the parents are considered to be environmentally safe. This is the first faulty premise of this section, that such guidance is logical or needed. It is not logical to advocate such a guidance section on stacked traits in LM plants, nor does it have any basis in the evidence of centuries of plant breeding and selection done by humans. Therefore, this does not seem to be a topic suitable for development of a guidance section. We understand that the AHTEG had only a very brief deliberation when choosing this and the other special topics for developing guidance sections, and that there was disagreement on these decisions. It is unfortunate that so much effort has been expended on this topic, and it is advisable to retract this section. In addition to these shortcomings, the section provides no practical advice to risk assessors evaluating requests for confined environmental releases.

Organizations

Public Research and Regulation Initiative (PRRI): One of the key paradigms of LMO risk assessment is that it is comparative, i.e. that indentified risks should be considered in the context of the risks posed by the non-modified recipients or parental organisms. To be of practical value for risk assessment, any guidance document should therefore: 1) provide a clear explanation of what happens in nature and with conventional breeding, 2) identify cases that go beyond what happens in nature and with conventional breeding that could give rise to safety questions, and 3) give practical how to address such questions. This document does nothing of the sort 1) This document does not provide any introduction that places the topic in the context of the fact that organisms are in fact sets of thousands - and often tens of thousands - of 'stacked genes'. Neither does the document explain that it is in fact the aim of every farmer, and therefore of every plant breeder, to have as many useful genes ‘stacked’ in one plant, e.g. genes for pest and disease resistance, combined with genes for better taste and higher yield, etc etc. In short, the document should start with a ‘setting the scene’ that stacked genes are a natural phenomenon and a very common result of breeding, and all with interactions between genes and pathways. However, rather than explaining this, the introduction and the entire document give the impression that stacked genes are somehow a rarity and above all that somehow stacked genes are prone to risk. To give just one of many examples that can be found in the text 'During cross-breeding, changes may occur to the molecular characteristics of the inserted genes/genetic elements at the insertion site(s) as a result of recombination, mutation and rearrangements.'. What it should say is ' as with any genes in an organism .......'. 2) The document does not give any guidance how to identify cases that go beyond what happens in nature and with conventional breeding that could give rise to safety concerns and that could give rise to safety questions. The document does not make reference to the growing list of literature that indicates that there is in general no scientific justification to look separately at stacked genes if the individual events have already been assessed (e.g. Kok et at: Plants with stacked genetically modified events: to assess or not to assess?, TibTech, 2013.12.001). The document should make this clear and then offer practical guidance how to identify the cases where a separate assessment would be warranted. However, rather than doing this, the document suggests that in all cases stacked genes should be subject to renewed assessment. 3) The document does not give practical guidance how to address particular safety questions that would have arisen from the previous steps in which cases have been identified where a separate assessment would be warranted. However, rather than doing this, the document comes with an endless list of ‘points to consider, without explaining in which cases those points would be relevant, how those questions could be answered in a way that is relevant for risk assessment. Examples of such points are: Level of heterozygosity among the non-modified recipient organisms used to produce the parental LM plants; phenotypic variability among non-modified hybrids produced through crosses between the non-modified recipient organisms; Number of crossings and the use of intermediate stacked LM plants as additional comparators; Phenotypic changes that may indicate underlying changes to any of the transgenes and genetic elements present in the stacked LM plant. Etc etc On various points, the document does not only provide little to no practical guidance when and how to answer questions, but instead sends the reader in a direction that shows that the authors have very little understanding of the topic itself. For example, the suggestion that the stacking of various insecticidal proteins in an LM plant could result in a faster development of resistance in target organisms has no basis. If anything, the stacking of various insecticidal proteins will delay resistance development. In short, this section is best retracted.

USEFULNESS/UTILITY

Parties

Brazil: The Guidance does not cover the type of technology applied to the test.

Czech Republic: The section mentions the problems connected with the higher variability among LMOs with stacked genes (see line 800). The unrelated non-LMOs were used even in these field trials with transgenic potato to obtain some basis of resistance data. The variability increased also due to different pathovars which emerged each year and due to climatic changes favorable or less favorable to the spread of oomycete. Two other lines refer correctly to the occurrence of LMOs with stacked genes in environment (line 909 and line 919). The line 864: '…of interactions between the novel and endogenous genes…' (see also line 850) refers rather to common situation which concerns any single transgene to be introduced. Such information could be in this special section omitted.

European Union: Line 788 It should be stated more clearly that re-transformation and co-transformation is not considered in this part of the document but that such applications should be covered on a case-by-case basis and that these LMOs may be considered to be and subsequently assessed as a single event.

Japan: As stated [above], 'Points to consider' unable to consider make this guidance less usefull. They should be deleted.

Mexico: (translation) 1. Line 911 (page 24, Spanish ver.); line 1160 (page 31, Spanish ver.); line 2357 (page 62, Spanish ver.): Where it says ‘reproducción convencional’ (‘conventional reproduction’) as the translation of ‘conventional breeding’ it should say ‘mejoramiento convencional’ (‘conventional improvement’). 2. Line 954 (page 25, Spanish ver.): Where it says ‘Variabilidad fenotípica entre híbridos no modificados’ (‘Phenotypic variability among non-modified hybrids’) it should say ‘Variabilidad fenotípica entre híbridos o variedades no modificadas’ (‘Phenotypic variability among non-modified hybrids and varieties’).

Peru: (translation) Line 941: Add at the end of the paragraph: ‘...and to human health’ (in Spanish, ‘...y a la salud humana’). Line 969: Add a subparagraph (e) that reads: ‘In assessing health risks, the risk assessments must be preclinical in animals and long-term clinical in human beings …’ (in Spanish, ‘Las evaluaciones de riesgo sobre la salud deben ser preclínicas en animales y clínicas en seres humanos a largo plazo…’) Line 1052: Mention that there are more modern methods for detecting stacked events, such as, for example, qPCR.

Other Governments

United States of America: As stated in the comment above, this section is likely to mislead novice risk assessors that there is some inherent reason why environmental risk assessments need to be done for the offspring of two parents who have already been deemed unlikely to pose adverse effects on biodiversity. Such a notion is contrary to worldwide experience with plant breeding over thousands of years.

Organizations

Public Research and Regulation Initiative (PRRI): See explanation under point 17.

CONSISTENCY WITH THE PROTOCOL

Parties

Brazil: Annex III of the Protocol covers the information that needs to be considered in a risk assessment in a flexible way. The Guidance, being a non-binding document, must reflect the same non-prescriptive approach.

Czech Republic: The importance of this section will increase with increased number of different LMOs introduced in the recipient environment.

European Union: Line 901 consideration of the segregation of transgenes warrants a more detailed explanation and discussion in the document.

Mexico: (translation) Line 920 (page 25, Spanish ver.). Under the Objective and Scope section, the text that reads: ‘This guidance also includes considerations for unintentional stacked events as the result of natural crossings between stacked LM plants and other LM plants or sexually-compatible relatives in the receiving environment.’ should be moved down to line 1053 (page 28, Spanish version) under the ‘Crossing and segregation of transgenes’ section.

Netherlands: - The section on LM plants with stacked genes is not fit for field trials.

Peru: (translation) Line 941: As there is no mention of the effects on health, the guidance is not consistent with Annex III of the Cartagena Protocol.

Other Governments

United States of America: If 'consistency with the Cartagena Protocol on Biosafety' means supporting its intentions as well as its words, this section of the guidance falls far short. The Cartagena Protocol does not advocate using poor science or accumulated knowledge in risk assessment, yet this section does not use logic or accumulated knowledge. As stated in the comment above, experience from conducting safety assessments on non-LM plants would not suggest any need to evaluate the environmental safety of the offspring of two plants when each of the parents are considered to be environmentally safe. There is nothing inherent in the techniques of modern biotechnology that would contradict observations from non-LM plants. It would be best to discontinue work on this section, since its fundamental premise is scientifically flawed and unsupported by empirical and experimental knowledge. Repeatedly in this section, the authors say that certain evaluations 'may be considered' or 'may be relevant', yet there are no concrete examples from experience in which these actually indicated that the plants with stacked traits were likely to pose an increased risk to biodiversity. Such guidance is not constructive, in that it sends risk assessors on a path to request and evaluate information that does not bring them closer to a valid assessment of actual likely risks. As such, these faulty guidance recommendations prevent countries from being able to conduct a timely evaluation and reach a decision on the transboundary movement of LM plants. Such delays inhibit the ability of countries to share the benefits of modern biotechnology, a chief goal of the Convention on Biological Diversity and the Cartagena Protocol on Biosafety. These vague statements regarding what 'may be considered' likewise give no practical advice on how to conduct a risk assessment when someone wants to conduct a confined, short-term field test with LM plants with stacked traits. This failure hinders the goal of the Protocol to provide a way for countries to have transboundary movements of LMOs for research and development, and thereby accrue some of the benefits that others have been able to realize.

Organizations

Public Research and Regulation Initiative (PRRI): The Protocol requires that risk assessment is scientifically sound and transparent, and conducted in line with the comparative paradigm. As explained above, this document does not do any of this. There is nothing scientifically sound about suggesting that stacked genes are somehow a rarity and above all that somehow stacked genes are more prone to risk than with crossings happening in nature or with conventional breeding. There is nothing transparent about listing a blur of points to consider, without indicating in which type of cases those points would be relevant, how those points should be addressed and how the results will be relevant to risk assessment. Specific line numbers cannot be given, because this problem is pervasive in the entire document.

PAST AND PRESENT EXPERIENCE WITH LMOS

Parties

Brazil: The Guidance does not cover the type of technology applied to the test.

Czech Republic: The present experience is not very frequent. Some crops with stacked genes are already prepared (especially the combination of different cry genes and their expression together with herbicide resistance), but the transgenes are usually physically linked the cases not covered by this section.

Denmark: No improvements necessary

Japan: Same as what is stated [above]

Mexico: (translation) Scientific experience and information indicate that it is important to assess stacked events, in accordance with the considerations set forth in the document (Part II).

Netherlands: Introduce the experiences with LM plants with stacked genes in risk assessment and history of safe.

Peru: (translation) Line 1069: The guidance should incorporate bibliographic references and links to web pages with more current LMO experiences.

Other Governments

United States of America: In the instances in which countries have done assessments of LM plants with stacked traits, it has not been for the purpose posited in this section. Lines 880-882 make the erroneous statement: 'Likewise, the evolution of resistance in target organisms (e.g., insect pests) to such stacked LM plants could happen faster than the development of resistance to the parental LM plants.' Stacking of insect resistance traits is done to slow the development of resistance in inset populations. In any event, the development of resistance in pest and pathogen populations occurs with non-LM plants, too. In fact, most of the phenomena that this section ascribes or implies as unique to LM plants are common to non-LM plants, also. The section on methods to distinguish stacked trait plants from mixtures of non-stacked parental lines is disingenuous when it states in lines 943-944: 'Based on the considerations above, the detection of each and all individual transgenes in a stacked event, if needed or required, may become a challenge and may need special consideration.' In fact, there are no such reliable, practical methods, to use in the case of LM plants in agriculture.

Organizations

GenOk - Centre for Biosafety: Text line 1041 discusses the use of '-omics' approaches in risk assessment. As these techniques are currently being applied, particularly in relation to iRNA LMOs, its characterization as 'as it becomes available' is no longer applicable. The phrasing 'as it becomes available' should be removed to reflect current experience

Public Research and Regulation Initiative (PRRI): See point 20

FURTHER COMMENTS

Parties

Bosnia and Herzegovina: I think there should be more examples of risk assasments for this section, but also in other sections. Generaly, text is a little bit 'dry', and more examples would make more accessible to interested parties.

Brazil: The Guidance does not cover the type of technology applied to the test.

Czech Republic: The risk assessment of genetically modified potatoes with improved resistance to Phytophthora infestans took into account the negligible possibility of out-crossing of the transgenes to other cultivated potato plants. Fortunately, such transfers are not dangerous as potatoes are propagated vegetatively. As for the herbicide resistance imidazolin herbicides are not allowed for application to this crop in the Czech Republic.

El Salvador: (to be translated) Tratando de encontrar algunas coincidencias encontramos que con respecto a la realización de la evaluación de riesgos, la metodología es consistente con el Anexo III, pero es un poco debil en cuanto a su vínculo con el proceso de la orientación.

Mexico: (translation) 1. Line 889 (page 24, Spanish ver.). Include the retransformation cases in Part II also, under section A, with reference to stacked genes. 2. The working group considered the following text as relevant input: ‘The larger the number of different sexually-compatible LM plants, stacked or not, being cultivated in the same environment, the more variations and complexity of new stacked LM plants may occur. The presence of sexually-compatible LM plants being cultivated in the likely potential receiving environment of the stacked LM plant under consideration is to be taken into account when establishing risk scenarios or hypotheses during step 1 of the risk assessment.’

Netherlands: See 17. Under Part I.

Slovenia: Regarding the section Methods for distinguishing the combined transgenes in a stacked event: clarification is needed in order to give more information on possibilities of a stack detection on a single test individual (e.g. one plan, one kernel) what could be important in monitoring plan development.

Spain: This section focuses on the assessment of plants that express stacked events, which are understood as events generated by the accumulation of single events through breeding of two living modified plants. For this assessment to be conducted, the single events must have been previously assessed, and in this sense we have given a rating of ‘neutral’ to the practicality of this section, as it must necessarily refer to part 1, the ‘roadmap.’ As for the ‘experience’ criterion, in our opinion the Guidance treats stacked events as a generation of GMOs, when in fact they are the most widely used events. For that reason, we have rated it ‘neutral.’

Uruguay: (translation) Same reference to the previous section regarding the associated technological package.

Other Governments

Australia: This part of the Roadmap was tested on DIR 108, a commercial release of a stacked LM canola. The same issues that apply to the Roadmap also apply here and if corrected would allow application to stacked traits without the need for additional guidance. More specific comments include the following. Lines 794-817 are not relevant as the comparator is the species Brassica napus. Therefore the issues of choice of non-modified comparator are straight forward and the innate heterozygosity part of the normal considerations and are not particular to a stacked event. With regards to conducting the risk assessment, the methodology was consistent with Annex III but followed little of the guidance material. Lines 819-869 were considered of little relevance and did not give rise to any realistic risk scenario. In lines 870-900, only lines 877-878 were considered relevant, but did not give rise to an identified risk that warranted detailed consideration. In the Australian context lines 901-949 are part of all risk assessments and therefore subject to the normal considerations that form part of the Roadmap.

United States of America: Please see the previous comments and suggestion to retract this section. The scientific rationale is consistently weak or lacking.

LM CROPS WITH TOLERANCE TO ABIOTIC STRESS

PRACTICALITY

Parties

Belgium: - The testing was conducted as an individual exercise. Different levels of agreement/disagreement (i.e. 'Neutral' or 'Agree') with regards to the practicality of this section of the Guidance were expressed amongst people.

Bolivia (Plurinational State of): (translation) Line 908: Include or mention, if possible, parameters for the considerations; make them more specific. Line 996 on other chemical products: Specify that hormones, fertilizers, growth promoters, flowering promoters, and others are also considered.

Czech Republic: Line 1007: 'taking into account risks to human health.' These seem to be connected especially with the products of the genes introduced. Such genes usually originate from non-crop plants which can withstand harsh environmental conditions. Nobody has experienced such species as food. The cold-shock protein from B.subtillis (expressed in MON87460) has been experienced as food and it is safe.

Peru: (translation) In general: The term ‘al estrés’ should be used as the translation of ‘stress’ (as in ‘tolerance to abiotic stress’) instead of ‘a la tensión’ (‘tension’). Line 1077: The following text should be added at the end of the paragraph: ‘…refer to only one event and add a text with a reference to the development of the LMOs, including a link to consult’ (in Spanish, ‘…referirse a un solo evento y agregar un texto que haga referencia al desarrollo de los OVM, incluyendo un link de consulta’). Line 880: Add an introductory text explaining that the planning phase consists only in the choice of comparators. Explain why there are no other items in the planning phase.

Other Governments

Canada: The comments on the Roadmap apply to this document as well. Fixing the flaws in the Roadmap would render this document unnecessary.

United States of America: In the Introduction to this section, lines 957-959 state: 'While the same general principles used in the risk assessments of other types of LMOs also apply to LM plants with increased tolerance to abiotic stress, there are a number of specific issues that may be of particular importance when assessing the risks of LM plants tolerant to abiotic stresses'. In fact, none of the specific issues cited subsequent to this are unique to LM plants modified to tolerate abiotic stress. The issues around pleiotropic effects and potential changes in invasiveness are not unique to LM plants modified for increased tolerance to abiotic stresses, nor are they unique to non-LM plants crossed with other non-LM plants. The authors provide no context of how such issues of pleiotropic effects or potential invasiveness are evaluated in non-LM plants, and that would have been useful information to place the evaluation LM-plants in context. This section does not provide any practical distinctions to aid risk assessors when evaluating requests for confined environmental releases of LM plants modified to tolerate abiotic stresses.

Organizations

Public Research and Regulation Initiative (PRRI): The Introduction to this sections starts with 'there are a number of specific issues that may be of particular importance when assessing the risks of LM plants tolerant to abiotic stresses', but what then follows throughout the entire section are issues that are not specific to risk assessment of abiotic stress traits. The resulting document is therefore nothing more than a repetition of the general statements in the RoadMap, e.g: 'Some of the issues that could arise from the introduction of LM plants tolerant to abiotic stress into the environment and which may lead to adverse effects include, for example: a) increased selective advantage(s), other than the intended tolerance trait, which may lead to potential adverse effects (e.g., resulting from the introduction of a transcription factor affecting more than one trait); b) increased persistence in agricultural areas and increased invasiveness in natural habitats; c) adverse effects on organisms exposed to the LM plant; and d) adverse consequences of potential gene flow to wild or non-modified relatives'. None of this is specific to abiotic stress and therefore the document does not provide any useful or practical guidance specific to that topic. This is comes back repeatedly from the points to consider, e.g.: 'Does the tolerance trait have the potential to cause an increase of the invasiveness, persistence or weediness of the LM plant that could cause adverse effects to other organisms, food webs or habitats? '. Or: ' Any intended or unintended change that may lead to selective advantage or disadvantage acquired by the LM plant under other abiotic or biotic stress conditions that could cause adverse effects;' Or:' it is of particular importance that the assessment of potential adverse effects of LM plants with tolerance to abiotic stress be conducted in relation to the ‘likely potential receiving environment’ of the LM plant under consideration' Or: ' The likely potential receiving environment where exposure to the LM plant may occur and its characteristics such as information on geographical, climatic and ecological characteristics, including relevant information on biological diversity, centres of origin and centres of genetic diversity'; None of this is specific to abiotic stress, and none of this provides any practical guidance. It is striking that the organisers of this testing have asked to apply this guidance to 'real cases'. Irrespective of the case taken, none of this guidance is specific enough to help identify in which cases certain points to consider would be relevant, and how those points could be answered. All this is pervasive throughout the document, and cannot be pinpointed to specific lines to be changed. A general, drastic revision would be appropriate, whereby a clear distinction should be made for guidance in case of releases for confined field trials and guidance for placing on the market.

USEFULNESS/UTILITY

Parties

Belgium: - The testing was conducted as an individual exercise. Different levels of agreement/disagreement (i.e. 'Neutral' or 'Agree') with regards to the usefulness and utility of this section of the Guidance were expressed amongst people. - Lines 1009-1050: As outlined in the document, the choice of comparators and the experimental design (stress vs. non-stress conditions) may present specific challenges for this type of LM plants. Although the Guidance provides some explanation on how to deal with this issue, it is a bit confusing and unclear with regards which comparator(s) and which comparative endpoint(s) should be used in which case(s) and under which condition(s). Providing some concrete (even theoretical) examples would certainly be very useful. - Lines 1040-1043: The possible use of 'Omics' in the comparative assessment is not specific to LM plants with tolerance to abiotic stress. We suggest moving this sentence to the relevant section of the Roadmap.

Bolivia (Plurinational State of): (translation) Line 850: Is there any source that can be cited for this statement or a link where examples of these events are given? Line 881: Include bibliographic reference.

Czech Republic: No improvements are needed. The Protocol mentions even aspects that are not frequently evaluated (e.g. the cross-talk between abiotic resistance and biotic resistance of plant LMO, see line 974).

European Union: The EC is supportive of the approach adopted which resonates with that proposed in the EFSA guidance, for example, the need to assess the unintended effects, to test GM plants in representative receiving environments under representative stress conditions and the availability of appropriate non GM comparators.

Other Governments

Canada: Same comment as above.

United States of America: Consistent with the comment above, this section of the guidance does not introduce any new topics that are not already addressed in the overall Roadmap (Part I of the guidance). Most of the Roadmap was written with a strong focus on LM plants, and the discussion in this section adds little to the information presented previously in the Roadmap. Like the Roadmap and other sections on LM plants, this section lacks information that is useful or helpful to the novice risk assessors trying to decide what information is relevant for confined environmental releases as compared to unconfined environmental releases. It would have been useful if this section made clear global experience with non-LM plants, as well as existing varieties bred and selected for their ability to tolerate abiotic stresses. This is a common occurrence with many non-LM crop plants, forages, ornamentals, and forest trees, but the text neglects to make this as clear as it could be.

Organizations

Public Research and Regulation Initiative (PRRI): See above comments

CONSISTENCY WITH THE PROTOCOL

Parties

Czech Republic: This part also generates materials useful for the application of the article 19 - Handling of Biotechnology and Distribution of its Benefits of the Convention on Biological Diversity.

Other Governments

United States of America: If 'consistency with the Cartagena Protocol on Biosafety' means supporting its intentions as well as its words, this section of the guidance falls far short. The text follows the same recipe for restating the elements found in Annex III of the Protocol, but it does little to clarify when and why it may be relevant to consider certain attributes of the LM plants modified to tolerate abiotic stresses. The text of this section, like the rest of the guidance, is written in a manner that gives the reader the impression that these traits and their potential effects on biodiversity are unique to LM plants, thereby inhibiting researchers, developers, and governments from pursuing modern biotechnology techniques to achieve plants which can tolerate abiotic stresses, such as drought, flooding, and increased salinity.

Organizations

Public Research and Regulation Initiative (PRRI): See comments under point 30

PAST AND PRESENT EXPERIENCES WITH LMOS

Parties

Belgium: Since the current experience with the risk assessment of LM plants with tolerance to abiotic stress is very scarce, an update of this section of the Guidance will probably be needed after more experience has been gained.

Bolivia (Plurinational State of): (translation) Lines 1052 to 1058: Eliminate that paragraph and in the previous paragraph indicate that there are other techniques, such as omics technologies, that could be more accurate for these cases. Line 1065 is more appropriate than the paragraph that is suggested should be eliminated.

Czech Republic: No improvements are suggested.

Peru: (translation) Line 1316: The guidance should incorporate bibliographic references and links to web pages with more current LMO experiences.

Other Governments

Australia: This part of the Roadmap was tested on DIR 102, a field trial of abiotic stress tolerant LM wheat/barley. The same issues that apply to the Roadmap also apply here and if corrected would allow application to abiotic stress tolerant traits without the need for additional guidance. More specific comments include the following. This guidance material does not acknowledge the extensive experience with introducing abiotic stress tolerance through conventional breeding and other (indiscriminate) forms of mutagenesis. In practice these are considered desirable traits that potentially allow more uniform yields to be produced in existing agricultural areas, not to extend growing into even more marginal zones (lines 1083 and 1151-1156), where there is competition from better adapted species. Lines 1009-1050 are not relevant as the standard methods using the parental species as the comparator and comparing risks. Lines 1052-1162 were largely irrelevant as this case study is a field trial, which identified risk scenarios that are typical of LM plants and did not need special considerations. Nevertheless, the introduced characteristics were considered in terms that might affect proposed controls to restrict spread and persistence of the LMOs (e.g. lines 1135-1136), which apply to all LM plant applications. No additional controls were deemed necessary other than those determined of the basis of the biology of the non-modified parental species. Furthermore, data is expected from these field trials that might answer some of the issues raised in lines 1052-1162. In addition, Australia applies a weed risk assessment methodology that covers unintended effects, which would encompass pleiotropic effects, cross-talk and unintended effects from genes introduced and intended to provide abiotic stress tolerance. Therefore, there would be no need for the 'omics' technologies suggested at lines 1040-1043.

Canada: The document suggests that the use of a non-modified counterpart may not be sufficient to assess the risk of a stress tolerant plant. For countries that have conducted risk assessments on these types of crops, there has been no evidence to date that this is the case. It is difficult to follow the scientific logic behind this statement and as practice to date contradicts this statement, it is counter-productive to include this in guidance. Those familiar with profiling technologies acknowledge that this is a powerful tool for research but not to generate relevant risk assessment data. The statement about the use of 'omics' technologies is attached to speculative risks and unless a clear explanation of how those risks could arise, why that is different from any other LMO plant and how transcrptomics can address those risks, this section should be eliminated. It is scientifically unsound and devalues the document. The discussion on cross talk in stress tolerance mechanisms is interesting but is not contextualized to any potential resulting harms. Countries have long experience with a wide variety of stress tolerant plants that are derived through conventional breeding. The document makes no mention of existing experience and once again ignores the concept of familiarity.

United States of America: The text does a poor job of depicting the knowledge gained from non-LM and LM plants modified for increased tolerance to abiotic stresses. There is virtually no information in the text nor its references cited to inform novice risk assessors of the vast array of stress tolerant non-LM plants used, bred, and selected around the world. The case of LM-plants is not much better in the text, yet there have been numerous plant species evaluated and tested in confined field tests around the world in response to pressing needs in agriculture, horticulture and forestry. Some LM plants for abiotic stress have completed the risk assessments at the commercialization phase, as well, but the text makes no mention of them.

Organizations

Public Research and Regulation Initiative (PRRI): See comments under point 30

FURTHER COMMENTS

Parties

Belgium: This Guidance is practical and useful by pointing in a few pages to the specific issues related to LM plants with tolerance to abiotic stress. The problematic of potential 'pleiotropic effects' in these LM crops is well explained and helpful for the risk assessment.

Czech Republic: The Introduction of this Guidance (lines 956-1007) refers to all specific ways leading to the LMOs resistant to abiotic stress. Some instructions have more general character. I have no experience with the risk assessment concerning LMO resistant to abiotic stress but I used the data concerning maize MON87460 where no adverse effects were observed during cultivation under either well-watered or water-limited conditions. As the additional source of RA the text 'Novel aspects of the environmental risk assessment of drought-tolerant genetically modified maize and omega-3 fatty acid genetically modified soybean' prepared by NIPHEN in 2007 was used.

El Salvador: (to be translated) Líneas 1009-1050 no son consistentes como los métodos estándar utilizando las especies parentales como comparador y comparación de los riesgos.

Other Governments

Canada: The document has little grounding in the current experience with stress tolerant LM plants or the extensive experience with stress tolerant plants derived through conventional breeding technologies. There is no reference to either current or past experience. As a consequence, it provided little useful guidance to risk assessors.

United States of America: As with other sections of the guidance, any section on LM plants for abiotic stress should be emphasizing for the novice risk assessor the relevant characteristics that might be of concern. Other than the vague discussion on pleiotropic effects, the other points to consider focussed on issues relevant to weed risk assessment, a topic that is well-developed for non-LM plants. This section should have highlighted the extant weed risk assessment models, how they are used, which data are most relevant, and how recommendations are reached for non-LM plants. This would help to set a constructive context for risk assessors of the LM plants modified for abiotic stress tolerances.

RISK ASSESSMENT OF LM MOSQUITOES

PRACTICALITY

Parties

Bolivia (Plurinational State of): (translation) Line 1258: The word ‘invasividad’ as the translation of ‘invasiveness’ should be replaced by ‘invasión’ (the word ‘invasividad’ does not exist in Spanish).

Colombia: (translation) While there are conceptual elements developed which are new, it is necessary, as mentioned throughout this evaluation, to generate elements to show how to address the issues in a practical way, more than in a theoretical sense. This is particularly necessary in this case, which poses new challenges when it comes to establishing methodologies for risk assessment. Moreover, with respect to the contents of lines 1783 to 1786, the wording should be improved to clarify whether it is the mosquitoes that are generating changes in land management or vice versa, as this significant aspect is not clear.

El Salvador: (to be translated) Luego de haber realizado varias comparaciones hemos encontrado que la estructura actual de la Guía es muy diferente a la estructura usada en evaluaciones de riesgo publicadas en el sitio web del BCH y no refleja la estructura de evaluaciones de riesgo que llevan a cabo los desarrolladores. Esto significa que es difícil encontrar la información relevante para la prueba, lo que lleva una inversión de tiempo mayor a la prevista; aunado a ello se corro el riesgo de que cuando los evaluadores sean un poco inexpertos lejos de servir como guía podría en algún momento confundir.

Malaysia: Please refer to General Comments response in #12

Netherlands: - This section does not provide instructions how to use the available information and presented points to consider to ask the relevant questions for the purpose of performing the consecutive steps of the risk assessment, in particular Step 1 (problem formulation). - The presented information does not follow the structure of the respective steps in the risk assessment.

Peru: (translation) Not all the participants found that the document was practical. It needs to be improved in terms of accuracy, and examples need to be added.

Other Governments

Canada: The title should be changed to reflect what is covered in the text; in other words, that the scope of the guidance is limited to LMO mosquitoes that are important as vectors of human and animal pathogens and parasites. To be practical, the section (not ‘document’ as used under the heading ‘Objective and Scope’ lines 1439, 1443 and 1446) on ‘Conducting the Risk Assessment’ should include a sub-heading on ‘Containment’ based on the rationale that given the uncertainties with the possible effects of LMO mosquitoes, a limited release (trial) may be desirable and therefore information should be sought on a description of any means of containing or controlling the release. Here is a suggestion: 'Containment (biological, physical, chemical, temporal) of the living modified mosquito Rational: Given the uncertainties with the possible effects of widespread release of LM mosquitoes into the environment, limited release in a particular geographic zone may be desirable. Description of any means of containing or controlling the release of the living modified mosquito inside of the intended target zone. Points to consider: (a) Description of physical containment and its effectiveness (b) Description of chemical containment and its effectiveness (c) Description of biological containment and its effectiveness including success rate of separating sexes or induction of sterility. (d) Description of temporal or other means of containment and their effectiveness' There are a number of rather speculative possibilities under the heading ‘unintentional effects’ that are not practical in risk assessment; if these are of concern, then what would be useful here is some suggestions for a research program that would reduce the uncertainty. Certainly there is closely related research that could be used as a surrogate. Experienced risk assessors are always dealing with some uncertainty. Unacceptable uncertainty provides further rationale for the need for containment and therefore, argues for its addition to the guidance. Under ‘points to consider’, is this an exhaustive ‘wish-list’ of concerns? Only the most plausible should be kept; item (n) relates to a human-driven consequence that is not related to the LMO mosquito and, in most countries, would not be among the considerations for a risk assessment such as this.

USEFULNESS

Parties

Bolivia (Plurinational State of): (translation) Line 1313: Take up some other aspects that the Australia example considers in its Risk Analysis.

European Union: The guidance should include the following (1) Advice on the use of non-GM surrogates (i.e. sterile mosquitoes through radiation) to inform on interactions with biotic and abiotic environment(s), and; (2) Further guidelines on selection of comparators including the possible need for alternative comparators (to non GM parental line).

Malaysia: Please refer to General Comments response in #12

Netherlands: The specific nature of field trials is not sufficiently addressed in this section. It should be made clear whether the paragraph on risk management strategies applies to field trials and /or commercial unconfined release into the environment. 1667: The heading ‘unintentional transboundary movements’ is inappropriate legal wording and should be replaced by ‘potential for dispersal and methods to prevent this’.

Peru: (translation) Provide precise examples connected with actual events; the guidance needs to be more concise.

Other Governments

Canada: See above. Usefulness is related to practicality so the comments on practicality overlap with those on usefulness. Points to consider (i) and (j) are covered by point (k). Point (n) is not relevant to the RA of the mosquito. The Guidance would be clearer if the heading were ‘Points that could be considered’. Lines 1714 to 1718 is a ‘point to consider’ when talking about containment strategies and should not be placed under RM (see text above regarding containment).

CONSISTENCY WITH THE PROTOCOL

Parties

El Salvador: (to be translated) La Guía no provee suficiente orientación en como discernir lo que se necesita saber para la evaluación de riesgo (‘need to know’) y lo que pueda parecer interesante desde un punto de vista científico (‘nice to know’) pero que no aporta datos que dan información útil para la evaluación.

PAST AND PRESENT EXPERIENCE WITH LMOS

Parties

Colombia: (translation) While the guidance does not develop a specific methodology, it does contribute information.

El Salvador: (to be translated) Algunos de los puntos recomendados en la Guía, no se encuentran en las evaluaciones de riesgo examinadas. Las evaluaciones de riesgo publicadas en el sitio web de la BCH (escritas por las autoridades del país que evalúan el riesgo) contienen un resumen de la evaluación de riesgo que las autoridades competentes llevaron a cabo para el producto OVM. Estos resúmenes no siempre reflejan toda la información que se considera durante la evaluación o toda la información que proporcionó el desarrollador.

Malaysia: The points to consider are quite comprehensive and based on the case study that was used to test the guidance document, all relevant points have been taken into consideration.

Netherlands: Include reference to past experiences with the SID technique (non LMO self-limiting techniques). 1730 – 5: It is important to include a more explicit mentioning of the relationship between environmental risk considerations of the LM mosquitoes and the human health benefits.

Peru: (translation) Incorporate information from countries that have already conducted risk analyses on the subject. For example: Brazil.

Other Governments

Canada: Line 1612 should say: 'The likelihood and consequences of this hazard can be gauged by assessing the fitness of the LM mosquito with the transgene should the self-limiting mechanism fail to prevent spread of the transgene.' 1703 should begin with 'Where a risk has been identified that warrants a response through risk management, risk assessors should consider risk management strategies such as monitoring the LM mosquitoes to ensure that the technology is functioning as intended and to identify any unintended adverse effects.' Paragraph starting at line 1746 should read 'There are other issues that may be taken into consideration in the decision for environmental releases of LM mosquitoes used for control of wild-type mosquitoes that are vectors of human and animal pathogens and parasites which are not covered by Annex III of the Protocol. They encompass, inter alia, social, economic, cultural and health issues associated with the use of LM mosquitoes to control wild-type mosquitoes that are vectors of human and animal pathogens and parasites or, alternatively, the use of chemical pesticides or other means to achieve the same result. The use of LM mosquitoes will require broader considerations of how target-disease risk affects human behavior, veterinary medicine, public health practices and national health priorities in order to address the catastrophic human tragedy caused by exposure to wild-type mosquitoes that are vectors of pathogens and parasites of human health and veterinary importance.'

FURTHER COMMENTS

Parties

Brazil: Although the Guidance does not cover the type of technology applied to the test, the National Health Surveillance Agency considers that, 'in addition to environmental concerns, risk assessments on LM mosquitoes should also taking into account risks to human health', in accordance with articles 1 and 4 of the Protocol.

Colombia: (translation) In the case of these types of LMOs whose main purpose is to reduce the incidence of vector-borne diseases (emerging, reemerging, tropical), it is important that the guide describe the aspects that are specific to the risk assessment from the perspective of epidemiology (changes in vector dynamics, host dynamics, etc.).

Mexico: (translation) Reference needs to be made to past outcomes in the management of living modified mosquitoes, as everything is being based on assumptions. Given that some actual situations already exist, it would be very useful to have such information. For example, the comments and reactions of the population upon seeing that mass releases are carried out.

Netherlands: See 17. Under Part I.

Peru: (translation) In general: More examples must be added, as the assessment was focused only on mosquitoes. Line 1587: The term ‘gene-drive’ should be left in English and not translated into Spanish. Line 1602: Where it says ‘para transmitir principal, pero no exclusivamente, patógenos…’ as the translation of ‘to transmit pathogens mainly, but not exclusively,’ it should say ‘para principalmente, pero no exclusivamente, transmitir patógenos…’. Lines 1606-1607: Where it says ‘microrganismo’ and ‘microrganismos’ as the translation of ‘microorganism’ and ‘microorganisms,’ it should say ‘microorganismo’ and ‘microorganismos’, respectively.

Other Governments

Canada: In the context of LM mosquitoes for control of wild-type mosquito vectors, one should bear in mind the enormous potential public health good that could be achieved. The LM mosquito guidance would likely benefit from linking with the OECD Mosquito Biology Consensus document now (2014) under development by the Working Group on Harmonization of Regulatory Oversight in Biotechnology.

RISK ASSESSMENT OF LM TREES

PRACTICALITY

Parties

Colombia: (translation) The guidance is based more on repeatedly-stated conceptual elaborations, without using specific cases to clearly illustrate how to follow the roadmap. This is evident in lines 1318-1374, where the context is not clearly identified, which means that the risk assessment presented is not completely consistent with the roadmap established. Similarly, in the planning phases of the risk assessment there is still an evident lack of methodological developments that would guide in the ways of addressing these processes. On the contrary, there are repetitive references to rationale, points to consider, etc. The problems set out in lines 1411 to1423 are not explained in a way that illustrates how to address them. In this sense, we could say that for this type of LMO in which the temporal scale is too large to determine the risks, it would be relevant to propose biological scenario modeling strategies and systems, in order to determine ecological dynamics that will allow for the identification of potential risks. Towards improving this aspect of the guidance and enhancing its applicability, it would be very useful to discuss methodological and conceptual aspects in the roadmap and, in the specific cases, propose practical elements drawn from actual cases, to guide the assessor in how to address a risk assessment’s case-by-case. Since non-experienced assessors are familiar with the basic elements, providing examples of practical applications would enable them to gain experience through the guidance.

El Salvador: (to be translated) Según la guía, el proceso de evaluación de riesgos consiste en una revisión crítica de los datos disponibles con el fin de identificar y posiblemente la cuantificación de los riesgos derivados de, por ejemplo, los fenómenos naturales, la tecnología, las prácticas agrícolas, procesos, productos, agentes por lo que en las condiciones en que nuestro marco regulatorio y operativo (específicamente en El Salvador) se encuentra, los lineamientos acá vertidos resultan un poco difíciles de aplicar por lo que sería bueno que se avanzara en la elaboración y presentación de ejemplos prácticos en esta sección para que aquellos países que no hemos entrado en esta experiencia tengamos la oportunidad de ilustrar mejor los posibles escenarios que se vaya presentando..

Germany: - Part I 'Conducting the risk assessment' introduces steps of a risk assessment. The same heading is also used in the Part II documents, but only specific issues of risk assessment are covered in those. Thus, it might be more accurate to change the Part II headings to „Conducting the risk assessment – special points to consider'. - Line numbers 1234-1236: It is pointed out that a comparative approach may be challenging for LM tree species for which there is little or no information with regard to their ecological functions or interactions in the likely receiving environment. The need for alternative approaches should be addressed.

New Zealand: These are general comments. The guidance is high level and very academic (i.e. it does not provide on-the-ground practical advice). It is very complex, uses lots of technical language and references to other Protocol sections and other documents etc. • It is unclear who the intended audience for this guidance is (i.e. how much previous experience with LMOs/ risk assessment practices do you need to be able to understand this guidance?). This document will not provide adequate guidance for non-experts to carry out a 'case-by-case' risk assessment 'on the ground' especially for less 'mainstream' GMOs or activities (e.g. totally contained field tests). An option to improve this is to provide real life case studies for a range of LMOs (from GM animals, plants, trees and microorganisms, viruses) and uses (from field trials, commercial cultivation, vaccines) to show how different regulators actually carried out the risk assessment. For example how did Regulator X when assessing LMO Y; o Frame the risk assessment/define the scope (e.g. What was within the scope and what was out? What are the underlying assumptions/scenarios? What fell outside the Protocol mandate and how was this dealt with? i.e. if there were risks still to be addressed). o Decide what comparator to use (if needed at all?). o Decide the information was sufficient for the activity. o Identify and deal with uncertainty. o Decide what expertise was required for the risk assessment (e.g. toxicologists, ecologists, commercial growers etc.) o Deal with different activities (e.g. field test versus commercial releases). o Use pre-existing information drawn from previous risk assessments. o Use scientific consensus positions to inform the risk assessment e.g. current opinion on HGT. o Describe the likelihood, consequence and risk characterisation used (quantitatively or qualitatively, how are the terms defined).

Other Governments

Canada: This below information pertains to forest trees. Concerning the section on The likely potential receiving environment specifically the text section, I would recommend including information pertaining to the potential longer life span of LM trees on the landscape, in particular to consider with more detail the potential impact on ecosystem processes (secondary ecological impact, or unintended impact). Ecosystem processes drive the composition and function of forest and include such processes as geomorphological (erosion and sedimentation rates), hydrological cycling (water holding capacity and surface-flow patterns) and biogeochemical cycling (nutrient mineralization and immobilization rates). Impacts on ecosystem processes can have cascading effects (e.g. alterations in atmospheric exchanges (trace gas fluxes and carbon balance) and soil processes (nutrient cycling and microbial diversity)) that can result in an alteration in the integrity of the ecosystem. In the section addressing the likely potential receiving environment(s) point (c) slightly addresses this point, but there should be a more detailed description (as given above). Additionally it is important to consider potential impacts at a landscape level (secondary ecological impact, or unintended impact). A landscape can be defined as a heterogeneous land area (including heterogeneous habitats) composed of a cluster of interacting ecosystems (Forman and Godron (1986). LMO-related alterations in ecosystems processes over time may impact processes of interacting ecosystems. This level of interaction would be highly complex. This may be an impact that is more relevant when there is an ecologically sensitive ecosystem (e.g. aquatic, wetland, and riparian ecosystem) adjacent to the LMO site. Additionally under Risk management strategies ‘Points to consider:’ for this section. An important point to consider should be the potential impact on biodiversity associated with cultural and management practices associated deployment of the LMO (secondary ecological impact, or unintended impact). This could occur due to changes in cultural practices such as application of fertilizer associated with the introduction of the LMO. This could have potential impacts on the environment, including effects on the diversity and ecosystem complexity of non-targeted species. Foreman RTT and Godron M. 1986. Landscape ecology. John Wiley and Sons, New York. 619 pp. This dispersal section is very vague and this can be misleading for forest tree species. I recommend specifically addressing the means of dispersal. For examples, gene flow associated with sexual propagation: (i) Potential for gene flow to a wild relative. The LMO may hybridize with sexually comparative species (non-targeted species within the same genus and even other genera) and have an impact on the environment through the production of hybrids and their progeny. This form of gene flow is due to the production of transgenic pollen or wild-type pollen may fertilize a transgenic ovary. There are four basic elements associated with the likelihood and consequences for this form of gene flow: a) distance of pollen movement from the transgenic tree; b) synchrony of flowering between the LMO and wild-type species; c) sexual compatibility between LMO and the wild-type species; d) ecology of the wild-type species. The LMO transgene can become permanently established in the wild-type populations(s) if it becomes introgressed into the genome. The likelihood of this is depended on environmental selection pressures. A transgene which increases fitness will more likely persist in the wild-type population. (ii) Potential for gene flow of the LMO. Sexual propagation may occur through the formation and dissemination of transgenic seed (transgenic pollen fertilizing a transgenic ovary). This does not involve hybridization with non-targeted species. 3b) Gene flow associated with vegetative propagation: (i) The potential for gene flow can also involve the dispersal of LMO’s vegetative parts and their subsequent establishment. (e.g. stem or root segments). Note, for 1-3 there may be addition potential impacts related to the length and scale of LMO release, and the LMO population size relative to the wild-type population(s). New unintended impacts may arise due to the release of LMOs on a large scale over long durations, which are related to stochasticity in climatic and biological conditions.

United States of America: Consistent with the comments above for other sections of the guidance, this section of the guidance on LM trees does not introduce any new topics that are not already addressed in the overall Roadmap (Part I of the guidance). A number of statements are made in this section that imply that the characteristics of some trees warrants a unique section of the guidance, but a rationale for this suggestion is not given.

Organizations

Public Research and Regulation Initiative (PRRI): The same applies here as for the section on abiotic stress tolerance, and here too the resulting document is therefore nothing more than a repetition of the general statements in the Roadmap. For all of the lengthy discussion in the guidance document there is little real guidance. There is little mention of utilizing our centuries or millennia of experience with non-LM crop plants/trees. We know that some of our current agricultural practices are environmentally unsound and many of our current crop/tree varieties require these practices to sustain their cultivation. To take these practices and non-LM varieties as gold standards seems to preclude the consideration of benefits derived from LMOs. That is why we must draw on the base of knowledge with non-LM trees to evaluate both the potential risks and benefits. We suggest to consider OECD biology documents in this context one.

USEFULNESS

Parties

Belgium: - The testing was conducted as an individual exercise. Different levels of agreement/disagreement (i.e. 'Neutral' or 'Agree') with regards to the usefulness and utility of this section of the Guidance were expressed amongst people. - With respect to the field trial with LM poplar used as case-study, one could notice that important principles, such as the duration and the scale of the intended use have been mentioned in the Guidance (e.g. lines 1343-1347 ('In determining the likelihood of an adverse effect of an LM tree, an assessment of the exposure to the LM tree should take into account the expected duration of the trees’ presence in the receiving environment, the nature of the transgenic traits, the intended use of the LM tree (e.g., processing, trade routes), as well as dispersal mechanisms. Given the late onset of reproductive maturity of a number of tree species, pollen and seed production may not occur during field trials'). This is done in a very concise, ‘poor’ elaborated way, which is understandable if the Guidance is considered as a framework through which links refer to background document with more detailed and specific information. However, it remains questionable whether these points will sufficiently capture the attention of inexperienced risk assessors or non-specialized users so as to allow them to address these aspects with appropriate consideration in their risk assessment. - The capacity of vegetative propagation is mentioned in line 1298 but could already be mentioned in lines 1187-1188. This characteristic is not specific to trees but it could have a big impact in the overall risk if one take into account specific characteristics of trees (perennial, height, etc.).

Colombia: (translation) A guidance instrument is expected to give step-by-step instructions of how to go from point A to point B. In this sense, the guidance lacks utility due to insufficient practical elements for applying it.

El Salvador: (to be translated) Aplica el comentario vertido en la pregunta 45.

Germany: - Footnote 25: It is slightly irritating that it is mentioned that some experts are of the opinion that fruit trees should not be included. As they are included, it does not matter to the uninitiated reader that there was dispute concerning them. Also, as the dispute is not further explained (do fruit trees have other, special concerns?) this information is unneeded. - The text of footnote 27 should be included in the paragraph.

Netherlands: Include and complete several essential considerations (presence of genetic elements, propagation methods).

Other Governments

Canada: There is a significant risk to have all trees (forestry and fruit) addressed in section C. Modern commercial apple production and other orchard fruits, for example, have more in common with field crop production than plantation forestry. Given the consideration of all trees under this section, there are many sweeping generalizations and this can be interpreted as implying that all GE trees may pose the same risks and must be evaluated by the same standards.

United States of America: The usefulness of this section of the guidance is hindered by confusion in the scope that echoes at various places through the document. The problem of scope begins with the apparent contradiction of line 1169 which states: 'Forest biodiversity is one of the seven thematic programmes of work under the Convention on Biological Diversity (CBD).' Yet in the following paragraph, lines 1177-1180, the authors state that the scope of this section is not limited to forest trees, but rather encompasses all true trees. The contradictory echoes reappear throughout this section, especially with the generalizations that seem to describe primarily forest trees (lines 1182-1212), yet the text compares trees to annual crop plants rather than non-trees.

Organizations

Public Research and Regulation Initiative (PRRI): This section can be useful, but not in the form it is in now.

CONSISTENCY WITH THE PROTOCOL

Parties

El Salvador: (to be translated) Aplica el comentario vertido en la pregunta 45.

Germany: - Line number 1357: The consequences of the cultivation of whole LM trees on ecosystem functions and biodiversity should be considered and not only the consequences of the modified trait. We suggest rephrasing this sentence accordingly .

Other Governments

United States of America: As with other sections of the guidance, the section on LM trees follows the organization of Annex III with 'points to consider'. However, the rationale for certain statements and assumptions to support the 'points to consider' is lacking or faulty throughout this section.

PAST AND PRESENT EXPERIENCE WITH LMOS

Parties

Bolivia (Plurinational State of): (translation) Line 1602: Consider the case of Brazil, and, if possible, provide some publications of the studies that are being conducted with living modified mosquitoes.

El Salvador: (to be translated) Aplica el comentario vertido en la pregunta 45.

European Union: The lifespan of trees and the likelihood that such LMOs may be released in unmanaged ecosystems presents a challenge to risk assessors and risk managers when considering the monitoring requirements for these LMOs. These factors, among others related to LM trees, are the subject of ongoing discusions in the EU.

Netherlands: Introduce the experiences with LM trees in risk assessment and history of safe.

New Zealand: See above comments

Other Governments

United States of America: The text does a poor job of depicting the knowledge gained from non-LM and LM trees. As mentioned above, there are a number of sweeping generalizations in the text that imply that these characteristics are unique to tree species (especially long life, symbiotic relationships, complex ecological relationships, etc., mentioned lines 1182-1212). There is virtually no information in the text nor its references cited to inform novice risk assessors of the vast array of non-LM trees used, bred, and selected around the world in horticulture, fruit and nut production, wind and soil erosion control, and pulp and timber production. The case of LM-trees is a bit better in the text, describing some of the tree species evaluated and tested in confined field tests around the world in response to pressing needs in agriculture, horticulture and forestry. Some notable LM trees have completed the risk assessments at the commercialization phase, as well. It is disappointing that this section gives little attention to the benefits that plant breeders see in using modern biotechnology techniques to introduce traits that otherwise take decades to incorporate into tree species. In some cases, such as the resistance of papaya to papaya ringspot virus, there were no known sources of resistance which breeders can turn to for resistance to this pathogen that attacks papayas wherever they are grown.

Organizations

Public Research and Regulation Initiative (PRRI): See above

FURTHER COMMENTS

Parties

Brazil: Although the Guidance does not cover the type of technology applied to the test, the Ministry of Agriculture, Livestock and Supply presented the following considerations on the risk assessment of LM trees: 'There is no practicality or utility of having a separate guidance for LM trees: for field trials or commercial release the RA of LM trees will take into consideration the information about the genetic modification, the biology of parental organism, the receiving environment, the proposed activity with the LM tree and previous risk assessment as much as is considered for LM crop. The fact that 'Because of their perennial growth and, in many cases, long lifespan and large size, trees may develop complex and multi-level ecological interactions with other organisms' (lines 1189 e 1190) does not creates any additional risks that can be not evaluated using the five steps approach described in the Road Map as noticed in the present test. The Guidance does not adequately address questions that could be inherent for LM trees and neither presents new specific 'points to consider', that could lead to different outcomes than those listed in the Road Map (part 1) : For instance, in the case of a confinement release the main point should be how the introduced trait might alter the biology of the tree regarding its ability to keep confined and the adequate measures for this confinement. Information from laboratory, green house or experience with similar genes introduced into crop plants or traits developed by traditional breeding could be used as source of information. If there is still a lack of information about the phenotype of a LM tree in the environment, measures to reproductively isolate the confinement could be adopted. In the case of a field trial, not all information is available yet as this is still a research step and the field data is necessary to assess the risks of a commercial release, however these factors are not clear in the Guidance leading to an endless looping. In the 'points to consider' questions related with presence of genetic elements and propagation methods, long life span, phenotypic characterization and stability of the modified genetic elements, dispersal mechanisms, exposure of the ecosystem to LM trees and potential consequences are listed but they are nothing else than further elaboration of steps 1, 2 and 3 applied to LM trees, including a lot of redundancy among the listed topics'.

Netherlands: See 17. Under Part I.

New Zealand: See above comments

Peru: (translation) Line 1739: Where it says ‘Diptera’ as the translation of ‘Diptera,’ it should say ‘Díptera’. Line 1783: Where it says ‘...cambios en la ordenación de la tierra...’ as the translation of ‘...changes in land management,’ it should say ‘...cambios en el manejo de la tierra...’. Line 1787: Consider irrigation practices only. Line 1801: Where it says ‘de especies A. gambiae’ as the translation of ‘A. gambiae species,’ it should say ‘de especies de A. gambiae’. Line 1909: Where it says ‘...como la vigilancia de...’ as the translation of ‘such as monitoring of,’ it should say ‘...como el monitoreo de...’. Line 1917: Where it says ‘...puede considerarse la vigilancia durante ...’ as the translation of ‘Monitoring during...may also be considered,’ it should say ‘...puede considerarse el monitoreo durante...’.

Spain: In this section, as in the ‘roadmap,’ the points to consider in the assessment are set out in detail, so we have assigned high values of agreement to the practicality and utility criteria. As for the experience criteria, in our opinion the knowledge acquired thus far in the assessment of living trees has been taken into account, but it would be important to highlight that such knowledge is more limited than in the case of herbaceous plants, which last less in the environment.

Other Governments

United States of America: Overall, there is little unique information presented in this section that is not already in the Roadmap (Part I) of the guidance. The general statements (lines 1181-1212) introducing the reader to trees are not enough to inform readers who are unfamiliar with trees, and overly generalized for those who know about trees. The existing information, including OECD biology documents on specific tree species provides more practical information for risk assessors, and there are more informative resources on various tree species that could be cited in addition to the OECD consensus biology documents.

MONITORING OF LMOS RELEASED INTO THE ENVIRONMENT

PRACTICALITY

Parties

Austria: Practicality was considered in regard to the targeted purpose. Part III (Monitoring of LMOs released into the environment) was regarded as an instrument to introduce important elements of a monitoring framework, and thus highly relevant for the development of appropriate monitoring approaches for specific LMOs. Therefore it is considered to be of high overall practicality. The points to consider included in chapters 1-4 are providing reference to aspects which need to be elaborated when drafting monitoring plans for implementation. In case of our testing we considered that complementing additional guidance providing additional detail is available at the EU level (EFSA (2011): Scientific Opinion of the Panel on Genetically Modified Organisms on the annual Post-Market Environmental Monitoring (PMEM) report from Monsanto Europe S.A. on the cultivation of genetically modified maize MON810 in 2009. EFSA Journal 9(10), 2376. doi:10.2903/j.efsa.2011.2376). Line 1822 ff: General Monitoring Against the background of EU requirements for monitoring few information is provided concerning general monitoring. Taking into account that the general focus is laid on issues, which are highly connected with other parts of the guidance, e.g. the Roadmap on risk assessment (cf. Lines 672 – 677) this is considered acceptable.

Bolivia (Plurinational State of): (translation) Line 2119 The usefulness of indicators (add) and comparators ....

Brazil: Brazil follows its national legislation on monitoring both for the experimental stage and for commercial use. The legislation considers the national protection goals.

Colombia: (translation) The guidance emphasizes surveillance and frames surveillance and monitoring in a single context, which is somewhat vague in Colombia’s case, where the terms surveillance and monitoring have been incorporated as two separate things in regulatory frameworks.

El Salvador: (to be translated) La Guía debería restringirse a cuestiones científicas que explican la metodología para llevar a cabo una evaluación de riesgo de forma práctica.

Czech Republic: All steps involved in the procedure are mentioned in lines 1851 to 1855. Line 1886: 'The selection of monitoring methods should also take into account the level of sensitivity and specificity…' The estimation of both characters can be rather difficult. Methods with higher sensitivity can reveal the higher variability of parameter which in turn can pose a problem to find the base line.

Netherlands: Make clear why and when specific or general monitoring applies to what types of LMOs. Introduce practical guidance as to how monitoring should be carried out. Distinguish between monitoring of field trials and commercial releases. Adjust this section to make it applicable to all types of LMOs. Address the relationship between the outcomes of the risk assessment and monitoring.

Norway: Although it is clearly stated that monitoring for human health is part of the guidence, its description contains little information that would allow practical human health monitoring. Therefore the sentence starting on line 1794 should be amended to conclude '…'yet may require additional methods or approaches'.

Peru: (translation) In general: The term ‘gene-drive’ should be left in English in the Spanish text. Line 2147: The difference between the terms ‘monitoring’ and ‘surveillance’ should be clarified. A definition for both terms should be included. Lines 2066-2067: Where it says ‘...no específicos de los organismos vivos modificados...’ as the translation of ‘ not specific to LMOs,’ it should say ‘...no específicos a los organismos vivos modificados...’. Lines 2164-2168: Where it says ‘Por consiguiente, la metodología de la que se deriva la base de referencia se describirá en el plan de vigilancia de modo de verificar que proporcionará información útil en relación con el medio ambiente en el que puede ser liberado el organismo vivo modificado. Cuando se analizan los datos de la vigilancia habrá que considerar la variación natural y antropógena que puede darse en los datos de la base de referencia.’ as the translation of ‘Therefore, the methodology by which the baseline is derived should be described in the monitoring plan in order to verify that it will provide useful information in relation to the environment where the LMO may be released. Natural and human induced variation that may occur in baseline data should be taken into account when analyzing monitoring data’, it should say ‘Por consiguiente, la metodología que determina la línea de base debería ser descrita en el plan de monitoreo a fin de verificar si proporcionará información útil en relación al ambiente donde el OVM podrá ser liberado. Las variaciones naturales o inducidas por humanos en la línea de base deberían tomarse en cuenta cuando se analizan los datos de monitoreo.’ Line 2174: Where it says ‘antropógena ‘ as the translation of ‘human induced,’ it should say ‘antropogénica’. Line 2210: Where it says ‘ciclos vitales’ as the translation of ‘life cycles,’ it should say ‘ciclos de vida’.

South Africa: A greater emhasis on the distinction between General monitoring and Case Specific monitoring to be a applied consistently within the section.. Each time the concept of monitoring is mentioned it must be clear whether its case specific or General monitoring being refered to. 1839 should refer to Case Specific monitoring plan This is the only place in the document that the roleplayers are named - notifier, competent authority, regulators There are significant costs associated with a comprehensive monitoring plan if all of these elements are incorporated.

Viet Nam: Post-monitoring is a important tool to recognize any adverse effects, if any, occurs when releasing LMOs into environment. However, monitoring also cost a lot of money. So it should have some specific recommendations for developing countries who have limited resources, to carry out post-monitoring.

Other Governments

United States of America: This section of the guidance does not seem to be very practical for use in real situations in which countries are trying to achieve the protection goals of the Cartagena Protocol on Biosafety. This section of the guidance seems to confuse detection of LMOs (lines 1775-1778) and monitoring adverse effects of LMOs released into the environment (lines 1786-1788). The document chooses to categorize monitoring as either case-specific monitoring or general monitoring (lines 1798-1799) and explains in the accompanying footnotes that some of the experts in the online forum and AHTEG thought that general monitoring should not be part of the guidance. In terms of practicality, the general monitor approaches are not well-designed to yield information that would reliably indicate a causal relationship between the environmental release of an LMO and some purported adverse effect. This conceptual shortcoming, and lack of expert agreement on general monitoring, should be more clearly highlighted in the text (i.e., more than a footnote should warn of the weak basis for advocating general monitoring). It is good that the text of line 1800 directs the reader to the purpose of monitoring as described in paragraph 8(f) of Annex III of the Protocol, but it is not very practical if the subsequent text of this section doesn’t provide some real-life examples of environmental monitoring that was useful in detecting the levels of risk identified in the risk assessment, i.e., the levels of adverse effects to biodiversity. The text in lines1822-1831 states that general monitoring is used in some approaches, but does not resolve the inability of such general monitoring to link an adverse effect to the release of the LMO.

Organizations

Public Research and Regulation Initiative (PRRI): The text starts with 'Monitoring of LMOs released into the environment may allow the detection, in a timely manner and as early as possible, of changes that may lead to adverse effects'. This is indeed done of the uses of monitoring, but what seems to happen throughout the document is that the texts moves away from 'detecting changes' to 'detecting LMOs', suggesting that the mere presence of LMOs are adverse effects. It is advised to start this section with a listing of the various uses of monitoring, and to stick to monitoring changes. Another issue that hinders the practical use of this section is that the distinction between 'case specific' monitoring and 'general monitoring' is introduced, but then keeps hanging in the air. It should be made clear that general monitoring has nothing to do with LMOs, is protection goal driven and has very little – if any – potential to establish causality. To take away this confusion, the text should make clear that as of 'Development Of A Monitoring Plan' is referring to case specific monitoring. It would probably be best to move the general monitoring to a footnote, to keep the flow of the text.

USEFULNESS

Parties

Austria: Line 1865: Monitoring of the exposure to LMOs is considered an important aspect to address potential uncertainties regarding estimates for exposure used for risk assessment (cf. Roadmap e.g. Lines 323, 505ff & 556ff). Therefore we suggest to add: 'Monitoring of the exposure to LMOs may be a highly relevant element of an overall monitoring approach.'

Bolivia (Plurinational State of): (translation) Line 2137: Consider the social and economic aspects of each country or region. Line 2227: Include an additional point: ‘(j) Variations due to climate change.’

Brazil: Each country should consider its methodologies for monitoring and protection.

Czech Republic: In para '3. Choice of monitoring sites' the shift of the field trial is not mentioned. Such attitude is obligatory for long lasting trials with aim to find the next season some volunteers. It introduces some additional environmental variability. Line 1980: '….to inform competent authorities about the results yearly.' In para 4. should be mentioned that the management practices (see lines 1974 to 1975) and especially substantial changes (irrigation, application of different insecticides or fungicides etc.) should be reported.

El Salvador: (to be translated) La Guía debería restringirse a cuestiones científicas que explican la metodología para llevar a cabo una evaluación de riesgo de forma práctica.

European Union: The approach, case specific monitoring to confirm assumptions made during the risk assessment process and general monitoring to monitor for unanticipated adverse effects, is supported by the EU.

Japan: As stated [above], 'Points to consider' unable to consider make this guidance less usefull. They should be deleted.

Norway: A more detailed description of how existing monitoring networks could be utilized in monitoring would improve the usefulness of the guidance. Suggest replacing the term 'programmes already established' from line 1831 with 'existing monitoring networks'.

South Africa: n the context of monitoring, it must be clearly stated that the parameters outlined only refer to commercial/large scale environmental releases. It would be useful to include a section that highlights how Problem Formulation and option assessment could be used to reduce the monitoring requirements and allow for a more focussed monitoring plan.

Viet Nam: As recommended in the question No. 25.

Other Governments

United States of America: Throughout this section of the guidance, the authors have incorrectly confused monitoring for the presence of LMOs with monitoring for the adverse effects arising from the environmental release of LMOs. The Protocol states the purpose of environmental monitoring when there is uncertainty in the level of risk identified in the risk assessment. The text needs to clarify that (1) not all environmental releases of LMOs are likely to cause adverse effects on biodiversity (then give specific cases illustrating this point both from examples of confined and unconfined environmental releases of LMOs), (2) monitoring is done to address uncertainty in the level of risk from a risk assessment and therefore needs a clear scientifically plausible hypothesis to test, and (3) any parameter or indicators need to have a strong basis for indicating likely adverse effects, not merely measuring some change in the environment.

Organizations

Public Research and Regulation Initiative (PRRI): See above

CONSISTENCY WITH THE PROTOCOL

Parties

Austria: To highlight consistency and interconnections between different parts of the guidance reference should be made to text sections contained in the Roadmap which address monitoring requirements (e.g. Roadmap Lines 267ff, 672-677, 689-693).

Brazil: It is important to bear in mind that article 33 of the Protocol provides the necessary flexibility for Parties to adopt its own methodologies for monitoring and protection. Therefore, the Guidance must adopt the same approach.

Colombia: (translation) It is acknowledged that an effort has been made, in the sense that it is able to go somewhat beyond what is contained in the Cartagena Protocol.

Czech Republic: The monitoring mentioned in CPB stresses the identification of stabilizing components in environment. The LMOs could have such function but the monitoring is aimed to detect their destabilizing function.

El Salvador: (to be translated) La Guía debería restringirse a cuestiones científicas que explican la metodología para llevar a cabo una evaluación de riesgo de forma práctica.

Japan: 1) Monitoring under the Annex III of the CPB is the monitoring of the assessed LMO in the receiving environment where there is uncertainty regarding the level of risk. (Annex III 8(f)) Such monitoring does not include 'general monitoring'. In order to be consistent with the CPB, at least the following parts related to 'general monitoring' should be deleted. - In line 1783-1784 : 'The Convention on Biological Diversity (CBD) covers in its article 7, 'Identification and Monitoring' --- Such monitoring is not the 'monitoring' stated in the Annex III. - In line 1798-1799, line 1822-1831, line 1936-1937 : 'general monitoring' 2) The words which have no scientific consensus among the Parties, such as 'cumulative effects', should not be used in the Roadmap. At least the following parts should be revised. - In line 1816, line 1902 : 'cumulative'

Netherlands: - Monitoring for the use of LMOs field trials is not discussed properly. Most of the suggested elements for a monitoring plan do not apply to field trials or the requested information will not be available. However, especially for field trials monitoring plays an important role in data gathering about uncertainties.

Norway: As human health is an important protection goal under the Protocol, and for Norway, the consistency of the monitoring guidance with the Protocol could be improved by the inclusion of points related guidance to human health monitoring where possible.

South Africa: The link between monitoring and risk management needs to be more clearly articulated. To this end, it would be helpful if the roadmap could incorporate monitoring as part of riak management and include it in the last phase.

Other Governments

United States of America: The text is inconsistent with the Protocol when it states in lines 1833-1835 that: 'A monitoring plan is developed when the recommendation of a risk assessment and/or the national biosafety policy calls for monitoring activities to be carried out in conjunction with the environmental release of the LMO.' This differs from what is stated as the purpose of environmental monitoring (lines 1779-1782 correctly direct the reader to paragraph 8(f) of Annex III for the purpose). The text in lines 1833 brings national policies into the discussion, and this is clearly beyond the scope of the Protocol text. It is strongly suggested that the authors revise the text so that the reader can clearly understand that change in some parameter does not mean that there an adverse impact on biodiversity has occurred (e.g., the section on Choice of indicators and parameters, lines 1857-1879). Among other things in this section, line 1859 should make it clear that in order to be consistent with the Protocol, the monitoring is for adverse effects, not just effects. Further clarification is needed throughout this section to address whether the proposed monitoring can actually indicate a causal relationship between the environmental release of the LMO and the adverse effect observed. Monitoring in the sense of the Protocol is not verifying compliance with laws and regulations governing LMOs, but there are places in which novice risk assessors and others are likely to get this impression from many of the supporting documents cited for this section. Such background documents are not appropriate for this section. In addition, monitoring in the sense of the Protocol is not detection of LMOs, and these discussions should be clarified or removed from the text (lines 924-944, 1775, 1847-1849) and supporting documents cited for this section of the guidance (too numerous to cite here).

Organizations

Public Research and Regulation Initiative (PRRI): The introduction of general monitoring is not something that appears in nor follows form the Protocol itself.

PAST AND PRESENT EXPERIENCE WITH LMOS

Parties

Bolivia (Plurinational State of): (translation) Line 2737: Add the following text at the end: ‘and revision of contingency plans.’

Bosnia and Herzegovina: This part of document was extremely usefull, detailed and applicable.

Brazil: Each country should consider its methodologies for monitoring and protection.

Czech Republic: I have no such knowledge.

Egypt: The section on monitoring of the guidance provides a useful approach to develop a monitoring plan focusing on 'what, how long, how and where to monitor' and how to communicate the results of both general and case specific monitoring. It could however benefit from further focusing and articulation.

El Salvador: (to be translated) La inclusión de una sección describiendo la formulación del problema sería muy útil para enfocar la evaluación de riesgo y decidir qué información es necesaria para caracterizar el peligro y la exposición en cada tema de la evaluación.

Japan: Same as what is stated [above]

Netherlands: No experiences gained with LMOs in risk assessment and history of use have been included in this Part of the Guidance.

Peru: (translation) Actual experiences should be taken as a model.

South Africa: This component was not incorporated into the actual cases of risk assessment considered. More thoroughly integrating the concept of Problem formulation and option assessment could be used to reduce the monitoring requirements and allow for a more focussed monitoring plan.

Other Governments

United States of America: As mentioned in the comment above, this section of the guidance does not provide a strong rationale for why the described 'general monitoring' approach is likely to yield reliable information as to whether an environmental release of an LMO has actually caused an adverse impact on biodiversity. There are no examples cited in the text of when such general monitoring has yielded such information, yet the authors seem to advocate such an approach. The section on establishing baselines (lines 1916-1929) is rather vague about how such baseline information has actually been used to indicate that the release of an LMO has caused adverse impacts on biodiversity. Here, as elsewhere in this section, the authors have incorrectly implied that change equals likely adverse effects to biodiversity. It needs to be clarified consistently in this and other sections of the guidance that mere change does not mean that there are adverse effects to biodiversity.

FURTHER COMMENTS

Parties

Colombia: (translation) The guidance does not distinguish between ‘seguimiento’ (‘follow up’), ‘vigilancia’ (‘surveillance’) and ‘monitoreo’ (‘monitoring’), and one would expect that, in the case of monitoring especially, it would provide input that would guide the parties in how to further the validation of medium and long-term risk analyses, in a context of potential cumulative-additive effects and in territorial spatial scenarios.

Czech Republic: The monitoring is more problematic than RA. It is affected by forced changes in agrotechnique (the occurrence of climatic changes or the occurrence of pests etc.). The notifier usually performs only general monitoring as the field trials are placed in distance from any protected area. My personal experience concerns only such monitoring which is usually short-term. The greatest problem in monitoring of LMOs released into the environment represents the fact that a long-term monitoring is missing.

Denmark: We suggest that a minimum required power is considered for evaluation of hypotheses

Italy: In our opinion it is correct to include General Monitoring in the guidance.

Netherlands: See 17. Under Part I.

Norway: It may be useful to the reader to highlight that exsting experience on LMO monitoring is very limited, and as evidenced from prior environemental monitoring efforts in other fields has shown that the monitoring plan must be carefully considered and sufficiently detailed to make sure the outcomes are meaningful to the stated objective, or will likely not yield useful results.

Peru: (translation) The terms ‘monitoring’ and ‘surveillance’ should be defined and included in the glossary.

Slovenia: In the section Monitoring of MLOs released into the environment a development of Monitoring plans in the cases of stacked events should also be considered.

South Africa: The section would be greatly improved through a reorganization of the text, Sections 1829 to 1831 would provide greater clarity if they were incorporated after line 1792. There is need to distinguish the various levels at which monitoring happens. There is monitoring in the context product development and that which is undertaken as part of validating a risk assessment and constitutes risk management. This input could provide greater clarity to the section on monitoring. It should be clear, that the the monitoring infered in this text is for commercial large scale releases. There reference to best available science in line 1894 is unnecssary in the context of the prior principles articulated.

Spain: This is the section that we find less specific when it comes to applying it to an actual situation. As it fails to refer to concrete situations or to give examples, it has a more limited practicality and utility than the previous sections, in which the requirements of different types of GMOs are specified. The objective and scope are in our opinion properly described, but in setting out the monitoring plan the guidelines given are so general that they do not facilitate its application. In this sense, the Protocol is much less specific in terms of the requirements of this type of monitoring programs, so we have assigned a value of ‘strongly agree’ to the consistency criterion.

Other Governments

United States of America: This section of the guidance does not appear to be well supported by knowledge gained through actual use. Most of the background documents cited for this section illustrate the problems with using reports that are not subsequently substantiated. This is the case for almost all of the background documents cited for the section heading. For the background documents cited for the subheadings of this section on monitoring there are two additional cited documents, namely one abstract (by two members of the AHTEG on RA, 'Observational science in the environmental risk assessment and management of GMOs [2012]') and a food safety study on pigs (Carman, et al, A long-term toxicology study on pigs fed a combined genetically modified (GM) soy and GM maize diet [2013] ). In general, the background documents here and throughout the guidance need to be re-examined for relevance to the actual final text.

GENERAL COMMENTS

Parties

Belarus: We would like to emphasize that testing was prepared and conducted very thoroughly with experts’ involvement of the Parties from all regions of the planet. Taking into account the fact that at present many countries have no sufficient experience in assessing LMO risks, we propose to conduct repeated testing of the Guidance when a rather large number of comments and suggestions will be accumulated after practical application of the given Guidance.

Belgium: - Broadly speaking, the Guidance is a useful tool to learn about the fundamental principles and criteria of the risk assessment of LMOs, and the specific points to be considered when assessing specific types of LMOs and traits. As compared for example with the EFSA Guidance on the environmental risk assessment of genetically modified plants (EFSA Journal 2010;8(11):1879), it is more straightforward and concise and therefore easier to read for the risk assessor. It might be less useful for applicants for which a more detailed Guidance is needed. - The Guidance is not self-sufficient to conduct a risk assessment. Other relevant sources of information should be consulted. - The Guidance lists many potential points to be considered in a risk assessment. This led to different feedbacks from individuals who were involved in the testing: On the one hand, the messages are useful to understand the principles and criteria of the risk assessment of LMOs. This gives inexperienced risk assessors the opportunity to avoid missing important points to be considered in a risk assessment. As potential point to be considered in a risk assessment cover a wide range of study areas, it is very useful to start screening them on basis of a list which is as comprehensive as possible. On the other hand the information remains of poor help when practically conducting a risk assessment for a specific LMO. It makes difficult, in particular for an inexperienced risk assessor or user, to formulate appropriate testable hypothesis really supporting the risk characterization. To verify that all data have been provided in a dossier and all risk hypotheses adequately tested, it is important to be able to verify these hypotheses. Risk assessors could developed many scientific hypothesis that, although broadening scientific knowledge, would not really inform the risk assessment ('nice to know' vs. 'need to know'). In that respect, the Guidance could be more efficient by proposing specific examples of adequately formulated risk hypotheses, including selection of assessment endpoints and ways of collecting relevant data supporting the risk assessment. In addition, examples illustrating the implementation of the guidance and the risk assessment methodology for specific cases could be a way forward to improve the utility of the guidance. - One of the main added-value of the Guidance is that it provides a structured access through a single link to many references that can give more details about specific aspects of the risk assessment. A potential difficulty with the current system is that the list of background documents is in some cases very long and the scientific quality and relevance of the documents very variable. In consequence the use of these background documents might not be very efficient, in particular for non-specialized users.

Bolivia (Plurinational State of): (translation) In general, the participants in the work meetings suggested that more case studies be included in the region’s BCH page, and, if possible, in Spanish.

Brazil: The Guidance on Risk Assessment of LMOs might be helpful for countries that do not have a legal framework for biosafety and biotechnology or do not have experience in the area. For countries that already have a legal framework in place, as it is the case of Brazil, the Guidance is not very practical and could be improved. In this sense, some measures could be useful, such as: the establishment of communications mechanisms that risk assessors might use in order to ask for additional information; further explanation on how to relate the different steps of the risk assessment; provide flexible orientation on how to consider new technologies and new generations of LMOs. As biotechnology advances quickly, it is important that any regulatory strategy for this area of knowledge be flexible and scientifically grounded.

Colombia: (translation) This guidance provides a good conceptual review, but a guide for a subject as technical as risk analysis needs to address more practical elements through processes or procedures that would clarify the elements contained in Annex III of the Protocol. In this sense, the guidance appears to have been based more on drawing on the Protocol than on elaborating on it, and thus it seems that there are few elements that are novel as compared to what is established in the Protocol. As for the discussion on the subject contained in lines 525 through 529, this would be more appropriately addressed in the working group on social and economical aspects. Moreover, some of the reference documents are not adequately reflected in the guidance, as many of the concepts and the methodologies proposed are explained more clearly in the reference documents than in the guidance. Lastly, Colombia recognizes and values the effort made by the Secretariat to standardize risk assessment criteria and methodologies, and the observations made here are meant to be constructive. Also, it is important to continue with this initiative by preparing a document that gathers the elements addressed here and those presented by the other Parties.

Czech Republic: The testing of the Guidance on Risk Assessment of Living Modified Organisms was conducted by the risk assessor RNDr. Oldřich Navrátil, CSc., who is working for the Institute of Experimental Botany of the Academy of Sciences of the Czech Republic and is a member of The Czech Commission for the Use of Genetically Modified Organisms and Genetic Products.

Cuba: (to be translated) Ordenar alfabeticamente la sección de términos y definiciones

Denmark: Our general impression is quite positive – the Guidance document directs the reader to consider aspects of LMOs in relevant contexts. The document could be improved by supplementing the text with overview tables and/ or figures showing the elements to consider in the risk assessment. Specific guidance on how to investigate and quantify the effect of transgenes on biotic interactions is needed. A possible method has previously been suggested (Damgaard and Kjær, 2009). Rosemary Hails recommended in an earlier guidance document the adoption of the method of Damgaard & Kjær (2009) Damgaard & Kjær 2009. Competitive interactions and the effect of herbivory on Bt-Brassica napus, Brassica rapa and Lolium perenne. Journal of Applied Ecology, 46, 1073–1079

European Union: As a risk manager the European Commission (EC) is not responsible for the environmental risk assesment (ERA) of individual LMOs, this is the remit of the European Food Safety Authority (EFSA).This exercise has thus been completed in collaboration with EFSA. Whilst no specific application was tested and no direct comparison has been made with the EFSA guidance, where there are similarities that we support, or gaps that we consider should be addressed, we have considered the usefulness and the utility of the guidance but have not commented on the practicality. Some Member States have conducted a more detailed analysis using specific assessments.

India: No comments can be made on Part II on specific types of LMOs and traits as the roadmap needs significant revisions and therefore its applicability to specific LMOs and traits cannot be tested or commented in the present form. As per decision of COPMOP6, the guidance is to be tested using actual cases of risk assessment conducted in accordance with Annex III of the Cartagena Protocol. There are no risk assessment summaries/documents available for GM trees or GM mosquitoes as the risk assessment of these product groups has not yet been completed. No such risk assessment summaries are available for testing. Annex III provides a well-structured approach to risk assessment, that needs to be applied on a case by case basis. Coupled with various resource documents available, it is sufficient to conduct risk assessment of specific product groups/traits and can be elaborated by Parties as per their regulatory requirements. It is strongly opined that development of specific guidance be left to Parties and not pursued.

Italy: In general, we retain the guidance consistent with the requirements of the Cartagena Protocol. Nevertheless, we would like to suggest the following improvements to the guidance: - The Part I should be more detailed - Although Part II has not been tested by our working group, we welcome the deepening of risk assessment on specific traits of organisms, and we consider useful to provide additional specific cases. - About Part III, our evaluation is very positive and we considered important to include also the General Monitoring in the guidance. - We suggest to include the 'background materials' at the end of each part of the guidance. In order to improve the 'practicality' of this online bibliography we suggest to add a search bar in order to filter documents for keywords such as LMOs, Genes or Organisms. Furthermore, the references can be further implemented and updated.

Mexico: (translation) The process through which the guidance document was tested can be consulted at: http://www.conacyt.gob.mx/cibiogem/index.php/protocolo-de-cartagena/acciones-de-mexico The guidance was tested in Spanish using the Spanish version. Additionally, in some sections, the Spanish version was compared to the English version and some inconsistencies were found, which are noted in some of the comments. Overall, greater care with translations is suggested, as in some cases the idea that is being conveyed is lost or distorted. The evaluation of Part II of the document, concerning living modified mosquitoes, was conducted by a different working group, and the details of that evaluation can be consulted at: http://www.conacyt.mx/cibiogem/index.php/grupos-de-trabajo/aedes-aegypti This evaluation process was limited by the fact that no example of an application for release into the environment was available and what was considered was the experience of some experts that have participated in projects involving confined use of living modified mosquitoes.

Netherlands: This questionnaire presents the results of The Netherlands of the testing of the Guidance on Risk Assessment of LMOs in accordance with Decision-VI/12 of COP-MOP. The main findings are reported with the respective questions in the questionnaire. Below, a recommendation for further improvement of the Guidance is presented. Explanatory notes on the testing procedure followed by The Netherlands The testing has been performed by risk assessors and risk assessment experts from governmental authorities in their personal capacity. The testing was done in two consecutive workshops: in workshop 1 the context, objectives and execution of the testing were framed and the actual cases were selected. Workshop 2 presented the results from the individual testing by the workshop participants and concluded on recommendations to further improve the Guidance Document.  To promote a uniform interpretation of the questionnaire during the testing, a further specification has been defined of the terminology used: 'Practicality' is understood as: the Guidance Document allows you to perform every step of the Risk Assessment in a consistent manner: it presents concrete information to raise and answer the necessary questions to perform each step of the Risk Assessment. 'Usefulness/Utility' is understood as: how informative is the Guidance Document. Does it present relevant information. 'Consistency' is understood as: the Guidance Document contains elements that may be very Practical or Useful, but are not related to the scope of the Risk Assessment in the Cartagena Protocol. 'Experience' is understood as: 1. the Guidance Document improves our understanding of the Risk Assessment methodology, and 2. the Guidance Document makes use of existing information.  The Guidance Document was developed in response to a need for further guidance on Risk Assessment of LMOs. As such, the Guidance Document does find its application in the conducting of the risk assessment, not the verification of the outcomes. The testing has therefore been performed with the technical and scientific information that was available with the application of the different actual test cases (rather than the information of the risk assessment evaluation reports). Recommendations by The Netherlands on improvements of the ‘Guidance on Risk Assessment of LMOs’. Concentrate the further improvement of the Guidance on the elaboration of easy-to-use standalone sections for specific types of LMOs and traits: 1. Give priority to the development of case-specific sections in Part II of the Guidance for specific types of LMOs and traits, which gives recognition of the need to raise special points to consider, uncertainties and monitoring requirements that come with their unique nature and characteristics. 2. Ensure a standalone nature of the case-specific sections that gives proper account of the different uses (field trial and commercial releases). 3. Adhere to a concise and consistent format for the case-specific sections to preserve the readability and improve the effectiveness of the Guidance: o In the setup of the sections follow the structure of the respective steps in the risk assessment; o Support the problem formulation with the use of appealing examples and clear instructions how to use available information and points to consider to ask the relevant questions for the purpose of performing the consecutive steps of the risk assessment, in particular Step 1. o Use a clear text structure, maximum size (6 pages) and make reference to additional literature where relevant (including Part I Roadmap, Part III Monitoring and BCH background documents). o Provide a scheme in which the aspects of risk assessment are presented in a visual manner. Add numbers (e.g. paragraph numbers) to the scheme to allow for easy navigation to necessary information in the Guidance. 4. Identify and rank specific types of LMOs and traits that demand the elaboration of case-specific sections. 5. Focus Part I (Roadmap) on the step-wise approach of the risk assessment methodology with reference, where relevant, to the Training Manual on Risk Assessment of Living Modified Organisms. 6. Keep Part III (Monitoring) as a separate document with individual chapters for specific types of LMOs and traits. Address explicitly the nature, needs and conditions of monitoring for different uses (unconfined commercial release and field trials) and explain the relationship between monitoring and risk assessment and risk management, in particular in relation to the concept of uncertainty. Rationale The Guidance aims to assist in implementing the risk assessment of LMOs. To maximize the practicality and usefulness/utility the information presented in the Guidance has to be easy to use. The testing revealed that these conditions are not met in the current Guidance. Parts I and III present an all-inclusive reference on the conducting of a risk assessment and monitoring for all LMO types and traits, possible uses and receiving environment. This has led to an already bulky, yet incomplete, document. The testing revealed that Parts I and III still lack necessary information, both for specific types of LMOs, traits and possible uses (in particular field trials). Adding this information will further expand the document and challenge the easy to use condition. Part II of the Guidance recognizes that specific LMO types and traits can raise unique or special points to consider, uncertainties and monitoring requirements that merit a separate section ('emphasis to issues that may be particularly relevant when assessing the risks of the respective types of LMOs and traits'; lines 757 – 8). The testing revealed that, in some cases, the individual sections in Part II improve the practicality and usefulness/utility of the Guidance. This case-specific approach links up to the case-by-case principle of the risk assessment methodology. More information is available at : http://www.cogem.net/showdownload.cfm?objectId=A1B0E481-0607-F1E0-B96C7C00F9E6635A&objectType=mark.hive.contentobjects.download.pdf

Peru: (translation) In general: The translation for the term ‘environment’ throughout the document must be revised and translated as ‘ambiente’ instead of ‘medio ambiente’; also, the term ‘monitoring’ should be translated as ‘monitoreo’ instead of ‘vigilancia.’ Moreover, the acronym ‘OVM’ (‘LMO’) should be used for ‘organismo vivo modificado’ (‘living modified organism’). Line 138: The term ‘tomar’ should be used as the translation of ‘making,’ instead of ‘obtener’ (‘obtaining’). Line 138: The term ‘informadas’ should be used as the translation of ‘informed,’ instead of ‘fundadas’ (‘grounded’). Line 2285: Where it says ‘Base de referencia’ as the translation of ‘Baseline’, it should say ‘Línea de base’. Line 2299: Where it says ‘epistásicas’ as the translation of ‘epistatic,’ it should say ‘epistáticas’. Line 2307: It should say ‘daño’ (‘damage’) (consequence of the adverse effect). Line 2312: Where it says ‘Técnicas de la moderna biotecnología ‘ as the translation of ‘Techniques of modern biotechnology,’ it should say ‘Técnicas de la biotecnología moderna’. Line 2325: Where it says ‘saprofitos’ as the translation of ‘decomposer,’ it should say ‘saprófitos’. Line 2334: Where it says ‘Sistema de regulación genética’ as the translation of ‘Gene-drive system’, it should say ‘sistema de propagación gene-drive’.

Portugal: We consider that the guidance is a good working document to be used by the Parties in order to provide reference that may assist in implementing the provisions of the Protocol with regards to risk assessment.

Republic of Moldova: The Republic of Moldova considers the testing of the Roadmap on Risk assessment of LMOs was a very useful exercise. It has exceeded our expectations in our opinion the Roadmap is a high quality document that met our requirements for practicability, usefulness, utility and efficiency and we recommend to submit the document to the next COP- MOP of the Cartagena Protocol on Biosafety for approval.

South Africa: A significant focus on risk monitoring is not expected in light of the title. The title should change to reflect the contents of the document, namely risk assessment and risk monitoring. The inclusion of points to consider throughout the text may be misconstrued as prescriptive requirements for each section. The sections should be renamed 'Points that MAY be considered'

Tajikistan: The Roadmap section of the Guidance is user friendly due to its logic nature and structure, usability and informativeness. It is focused on such basic points, which must be taken into account when conducting risk assessment, as follows: 1) quality, actuality and completeness of submitted information; 2) scientific approach, methods of information submission; 3) specific character of each case. In addition, the Guidance is in consistency with accumulated experiences and reflects the current state of scientific knowledge in genetics, biology, molecular biology, etc. As for the general perception of the Guidance, the minor improvements are suggested in attached to this questionnaire Russian version of the Guidance and address some moments in translation of the document mostly.

Viet Nam: Some of the information in the part I of the Roadmap are specific for plants only such as 'Agricultural practices ' or 'Pest management ' practice while the scope of this part is for LMOs in general. Also, some of the information is quite general as recommended in previous part of this survey. Therefore, it's highly recommended to have a more useful guidance on environment risk assessment

Other Governments

Canada: The lack of practicality of the Roadmap was well illustrated by the testing process in Canada by experienced risk assessors and some novices. The organization is the key problem as it is difficult to reconcile the organization of the Roadmap with the actual process of conducting a risk assessment. The Roadmap describes an academic exercise that is not an accurate representation of real practice considers hazards without adequate context and requires that risk assessors become policy experts. The process described for the risk assessment is linear, and compartmentalized where the hazards are presented outside of the working context that risk assessors employ where hazards are considered and discarded when there are no realistic pathways from the hazard to any type of harm. This concept appears in the document but is never linked in any coherent way, even in the section on conducting the risk assessment. The document is presented in a way that does not acknowledge the professional judgment that is truly key to effective risk assessment and instead is replaced by prescriptive guidance on data quality. The hazard identification sections are not contextualized and as a consequence look like shopping lists. Additional points identified: - The scope of risks to human health in the context of an environmental risk assessment should be specified, e.g., topical exposure, etc. ). - Include list of references at the end of the document as is common practice. Including references as hyperlinks will only be useful when an internet connection is available and when the hypertext links are valid and active - Part I should be broadly applicable, hence the focus on plants should be removed. This includes references to plants (lines 543-546) and crops (lines 589-592).

Organizations

Public Research and Regulation Initiative (PRRI): PRRI warmly welcomes the MOP’s request that the current guidance be tested for parameters as usefulness, practicality and consistency, and PRRI welcomes the Secretariat’s approach of an online questionnaire in combination with the possibility of producing a paper version of the questionnaire. Two things appeared challenging in filling in this questionnaire: 1) using real life dossiers: irrespective of the case taken, none of this guidance is specific enough to help identify in which cases certain points to consider would be relevant, and how those points could be answered. 2) Providing text proposals for specific lines in the document: given that the concerns expressed above often referred to matters that were pervasive throughout the document, it is not possible to provide specific comments at specific lines.

Global Industry Coalition (GIC):1. INTRODUCTION The Guidance for the risk assessment of living modified organisms (LMOs) developed by the Parties to the Cartagena Protocol on Biosafety is currently under discussion. Following the decision at the sixth Meeting of the Parties (MOP-6), the Parties were encouraged to test the Guidance using actual cases of risk assessment and share their findings through the Biosafety Clearing-House (BCH) and the open-ended online forum. In the context of this testing, the MOP-6 requested to the Executive Secretariat to: a) Develop appropriate tools to structure and focus the testing of the Guidance; b) Gather and analyse, in a transparent manner, feedback provided as a result of testing on the practicality, usefulness and utility of the Guidance, (i) with respect to consistency with the Cartagena Protocol on Biosafety; and (ii) taking into account past and present experiences with living modified organisms; and c) Provide a report on possible improvements to the Guidance for consideration by the Conference of the Parties serving as the meeting of the Parties to the Protocol at its seventh meeting. The Secretariat developed a concept note and a questionnaire to structure and focus the testing of the Guidance. The concept note describes the testing process to follow: a) The objective of the testing is to evaluate the practicality, usefulness and utility of the Guidance on Risk Assessment of Living Modified Organisms with respect to consistency with the Cartagena Protocol on Biosafety, in particular Article 15 and Annex III, and taking into account past and present experiences with LMOs; b) The testing may be conducted by Parties, other Governments and relevant organizations through their risk assessors and other experts who are actively involved in risk assessment; c) The testing may be conducted by individuals or as a group initiative (e.g. workshops); d) The Guidance is to be tested using actual cases of risk assessment conducted in accordance with Annex III of the Cartagena Protocol, noting that the actual case of risk assessment itself is not the subject of the testing: a. The technical and scientific data of actual cases of risk assessment used in the testing may originate from various sources. These sources may include application dossiers, and previous or on-going risk assessment processes. Alternatively, the summaries of notifications may also be used; b. Irrespective of the source of the technical and scientific data described in subsection ‘d’ above, the actual cases of risk assessment used in the testing must be clearly identified either through references to Risk Assessment Records in the Biosafety Clearing House (BCH), or hyperlinks to their original source; c. The BCH Risk Assessment Records referring to the actual cases of risk assessment used in the testing may be generated either through the regulatory process of a country or through an independent or non-regulatory process. Reporting the results of the testing: a) The results of the testing are to be submitted through the BCH using the questionnaire common format that is made available for this purpose; b) The BCH Risk Assessment Records or hyperlinks to webpages containing information on the actual cases of risk assessment used in the testing are to be linked to the questionnaire; c) The results of the testing conducted by Parties and other Governments are to be submitted by their respective BCH National Focal Points and those by relevant organizations through their head offices; each Party, other Government or relevant organization may test the Guidance with as many actual cases of risk assessment available but may only complete and submit one questionnaire reporting their results. Taking these requests into consideration, Estel Consult Ltd. has developed an approach for testing the Guidance. This document describes the methods used and a summary of the initial findings. 2. DESCRIPTION OF THE TESTING METHODOLOGY USED 2.1. SELECTION OF RISK ASSESSMENT RECORDS FOR THE TESTING Following the Secretariat recommendation in the concept note, the first step was to map the risk assessment information available in the BCH website. The objective of the search was to identify and compare risk assessment records from different countries on the same product with the same scope of application. Using the search engine on the BCH website, the following criteria were used: ‘any country’, risk assessments for ‘Regulatory processes’ and ‘LMOs for introduction into the environment- commercial production’. This search returned 218 records, although close examination revealed that despite selecting ‘commercial production’, most of the records returned were confined field trial evaluations. However, the search was useful for determining which countries have posted records of risk assessment reports conducted with a scope that includes food, feed and cultivation. These countries are: Argentina, Brazil, Canada, EU, India and Japan. All of the risk assessments found were for genetically modified plants, no records for trees, mosquitoes, or abiotic stress traits were available. Since one of the objectives was to compare risk assessment reports from different countries on the same product with the same scope, the next step was to map the risk assessments posted by each of the countries for specific products (‘organism id’) and select those products for which records were available from more than one country. The search focussed on three crops: maize, cotton and soybean. This search showed that: • The only maize product for which risk assessment records are available for most of the countries listed above is the genetically modified maize MIR162 from Syngenta. Risk assessments including cultivation in the scope for this maize have been posted by Argentina, Brazil, Canada and Japan. • For cotton there are risk assessment records including cultivation in the scope from Argentina and Brazil for the Monsanto product MON531xMON1445. • For soybean the only product for which more than one country have posted risk assessment records including cultivation in the scope is the Bayer soybean A2704-12 (from Brazil and Argentina). 2.2. TESTING METHOD Risk assessment records including cultivation in the scope for MIR162 maize were downloaded from the BCH website. The records retrieved were: 102518 and 102523 from Argentina; 100880 from Brazil; 103211 from Canada and 101240 from Japan. A number of issues were encountered when retrieving the risk assessments: • The first issue was that the links for the records from Argentina do not work. • Another issue is that the records from Argentina are written in Spanish. For MIR162 the risk assessment report can be found at: http://64.76.123.202/site/agregado_de_valor/biotecnologia/50-EVALUACIONES/___favorable/_archivos/DOC_DEC_MIR162.pdf • Record 100880 from Brazil is in Portuguese, although links to the English and Portuguese documents are included, both links lead to the document written in Portuguese. The English document can be found at: http://www.ctnbio.gov.br/index.php/content/view/15160.html As mentioned earlier, all risk assessment records available on the BCH website are for genetically modified plants only. Risk assessments have been posted for stacked events, but they have not yet been tested. The initial approach in testing the Guidance was to align the information between the risk assessment reports and then align them with the Guidance. However, once the reports were examined, it became apparent that the format used, the information included and the level of information provided by each country were very different. Therefore, a decision was made to list in an Excel spreadsheet the steps described in the Guidance and information to be considered in each step. Each report was then reviewed and compared against the points outlined in the Guidance. The comparison focused on whether a country had addressed or considered a particular step and information. Following Secretariat’s directions, the risk assessment itself was not the subject of the evaluation. Once all the risk assessments were compared against the Guidance, a further evaluation was made to establish: • What elements in the Guidance appear in all the risk assessments? • What elements in the Guidance do not appear in any of the risk assessments? • What elements in the Guidance are considered in some risk assessments and not in others? During the evaluation, the practicality, usefulness and utility of the Guidance were also examined. Given the request from the Secretariat to use ‘actual cases of risk assessments’ for which links to webpages can be provided, the testing focused only on risk assessments posted on the BCH. It is important to note that these risk assessments, written by regulatory authorities, often provide a summary of the risk assessment and may not contain all the information provided by the developer or all the information considered by the evaluator. However, the approach used in testing the Guidance allowed the identification of elements addressed in all risk the assessment reports contained within each BCH record. It also allowed the identification of elements not included in these risk assessments. These elements may be examined further to determine if they are an essential part of the risk assessment. 3. FINDINGS 3.1. GENERAL COMMENTS Testing of the Guidance led to the following findings: • The format and prescription of the Guidance is confusing. The structure of the Guidance does not reflect the structure followed in any of the risk assessments analysed. The presentation of the methodology is not sufficiently similar to published methodologies that are conducted to support regulatory applications, and made it difficult to conduct the test. • The Guidance describes a strict step-wise approach, presumably due to interpreting Annex III.8 without taking into account the term ‘as appropriate’. The step-wise process to follow in the risk assessment described in the Guidance is: - Step 1: Hazard identification - Step 2: Exposure assessment - Step 3: Hazard characterization - Step 4: Risk characterization - Step 5: Report, summary and recommendations This is in contrast to the records reviewed in the BCH. • In the Guidance, each step comprises a consideration of various issues: gene flow (including vertical and horizontal gene transfer); effects on target and non-target organisms (including toxicity, allergenicity and multi-trophic effects); changes in management practices; etc. By structuring the Guidance in this way, it is unclear how a novice risk assessor would understand the principles underpinning establishing a link between the area of assessment or issue under consideration. There is insufficient guidance on discerning ‘need to know’ information necessary to conduct a risk assessment versus ‘nice to know’. For example in Step 1, the Guidance lists the points to consider, but when it moves to the other steps, there is no clear link on how information from Step 1 (hazard identification) is used with information in Step 2 (exposure) and Step 3 (hazard) to complete Step 4 (risk characterisation). So the Guidance almost becomes a list of potential hazards and exposure scenarios without context and with no clear guidance on how to integrate the various pieces of information for performing a risk assessment in practical terms. This approach does not allow a necessary connection between each phase of the risk assessment, and may generate confusion and lack of harmonization between risk assessments. The main problem is the seemingly prescriptive approach taken in the Guidance that is inconsistent with the more flexible, evidence based approach that occurs in practice as witnessed in the risk assessment records examined. • The risk assessment reports reviewed follow national guidelines. Each report is structured in a way that allows a separate risk assessment (where a hazard and exposure characterization are conducted) of each issue, such as weediness potential and effects on non-target organisms, considered by the country important to their environmental protection goals. This allows for risk assessments that are easier to understand, where the problem under assessment is clearly defined (problem formulation), information already available is considered and an analysis is conducted to determine what additional information may be needed to complete the risk assessment. • In addition to a prescriptive step-wise approach, the Guidance also has two generic sections, ‘overarching issues’ and ‘planning phase’. - The ‘overarching issues’ section contains discussions on data quality and relevance, and on uncertainty. However, none of the risk assessment records reviewed contained such a section or discussed these elements. Sections like ‘criteria for the quality of scientific information’, drift beyond guidance into policy statements that may run counter to national practices of risk assessment that are compliant with the Protocol. - The ‘planning phase’ section discusses how to establish the context, mentions the need to take into account protection goals, assessment endpoints and national regulations, and discusses the choice of comparators. While it is good that the Guidance brings attention to the need to take into account these elements, countries usually do this within their national system. Besides, problem formulation needs to be understood as a composite of protection goals, assessment endpoint and measurement endpoints; the latter are not even mentioned in the Guidance, yet they are necessary in enabling the translation of protection goals into concrete measureable indicators and parameters. The risk assessment reports reviewed quote the directives under which they were conducted and the scope of the application. No specific mention of protection goals, assessment endpoints or comparators was found in any of the reports. These two sections of the Guidance are more likely to bring confusion than clarity — they contain speculation and opinion on complex topics, which could potentially lead to conflict with national policies. One of the critical elements of a risk assessment is to facilitate the communication between the risk assessor and the risk manager, so the risk assessor can outline in a clear way what problems were addressed, what information was used to address them, what were the conclusions and what information supports these conclusions. The Guidance in its current form fails to provide this. The Guidance is useful in that it describes the critical distinctions between hazard and exposure and the need to consider both to characterize risk. However the Guidance does not offer much practical support to risk assessors as it is not clear how to match relevant hazard data with relevant exposure data to assess the risk for a particular area of assessment. The Guidance is supposed to cover all LMOs and all scopes of application, including risk assessments for confined field trials. However, the points to consider outlined in each of the steps are very broad and it is not immediately obvious what points would be applicable to a specific product or to a certain scope. One of the important elements that could be improved is the introduction of a problem formulation step (mentioned in the Guidance but not developed or explained in detail). This step would help risk assessors in focussing the risk assessment according to the product type, the receiving environment and the scope. This would also help in clarifying the key areas to assess, given national protection goals, what information is already available and relevant for the assessment and what information is missing to complete the assessment. Another potential improvement would be to re-structure the Guidance or provide examples to clarify how the Steps described fit together for the assessment of a particular risk or issue under consideration. 3.2. COMMENTS ON SECTIONS OF THE GUIDANCE This section provides general comments and observations that were made in the course of conducting the test. They are provided in a manner relevant to each of the sections of the Guidance tested. 3.2.1. Overarching issues The Guidance gives only brief mention to protection goals and assessment endpoints. These are key elements in the risk assessment, but they are not sufficiently explained in the Guidance to help risk assessors. For example, assessment endpoints are brought up in several sections, but the term ‘measurement endpoint’ is never mentioned. Understanding the difference between the two terms is key to understanding risk assessment. As such, this is a gap that should be addressed. Another topic raised in this section is the quality and relevance of data. The testing revealed that these two topics are not mentioned in any of the BCH risk assessment reports examined. Sections like ‘criteria for the quality of scientific information’, drift beyond guidance into policy statements that may run counter to national practices of risk assessment that are compliant with the Protocol. This is also the case for the section on relevance of information. Here it could be made clear that developers and evaluators could easily establish the relevance of data by using the problem formulation approach. This section on ‘criteria for the quality of scientific information’ also contains a discussion about uncertainty and a recommendation to identify sources of uncertainty and establish their nature. Again, none of the risk assessments reviewed during this project had an explicit mention of uncertainty or types of uncertainty. Usually evaluators deal with uncertainty in a practical way. Often the perception of ‘uncertainty’ stems not just from the data included in the risk assessment, but from lack of familiarity with risk assessment methodologies or a problem of communication between the developer and the evaluator. Since the environmental risk assessments (ERAs) are conducted on a case-by-case basis, different approaches may be taken for different risk assessments. Unless the approach is clearly outlined and explained, evaluators may have difficulties following or understanding the approach and may have different interpretations of the data or they may reach different conclusions. Establishing a good line of communication between developers and evaluators could be a good way to deal with uncertainty, rather than prescribe elaborated requirements to address uncertainty. Pre-consultation meetings are encouraged in some countries to allow discussions regarding the best approach to take in the ERA before the assessment starts. Discussions may also take place during the review to facilitate a common understanding. Failing this, evaluators often request clarification or ask for more data to complete the assessment. It is surprising that the Guidance does not strongly recommend this very helpful dialogue that occurs in most countries around the world. Uncertainty also often arises from the lack of familiarity with risk assessment methodology, either from the developer or the evaluator. Risk assessments are conducted to aid decision making and do not provide definitive answers regarding safety, there is always some degree of uncertainty. The more knowledgeable and experienced the risk assessors and evaluators are the more at ease they are with these concepts and the more able they are to deal with uncertainty. It is likely that the less experienced risk assessor will be greatly confused by this discussion on uncertainty and struggle to translate the Guidance into practicable terms. This section of the guidance also describes the importance of establishing the scope and context of the risk assessment, taking into consideration country policies and regulations and protection goals. The section also includes a reference to the comparative assessment and choice of comparator. In essence the section introduces some of the aspects of problem formulation, but does not explain properly how problem formulation can be used to make the risk assessment fit for purpose by guiding the compilation of information and the risk assessment approach. All of the risk assessments reviewed use the comparative approach and take into consideration background information on the conventional crop and the history of safe use. None of the reports make specific mention of protection goals, assessment endpoints or risk thresholds. None use a formal approach to problem formulation, although some do provide an introduction on the purpose of the assessment. 3.2.2. Conducting the risk assessment- STEP 1: Hazard identification ‘An identification of any novel genotypic and phenotypic characteristics associated with the living modified organism that may have adverse effects on biological diversity in the likely potential receiving environment, taking also into account risks to human health.’ In this section the guidance recommends to ‘identify changes in the LMO, resulting from the use of modern biotechnology, that could cause adverse effects on the conservation and sustainable use of biological diversity, taking also into account risks to human health. The potential adverse effects may be direct or indirect, immediate or delayed’. Some of the examples given of potential adverse effects are: (i) affect non-target organisms, (ii) cause unintended effects on target organisms, (iii) become persistent or invasive or develop a fitness advantage in ecosystems with limited or no management, (iv) transfer genes to other organisms/populations, and (v) become genotypically or phenotypically unstable. All these examples, with the possible exception of (i) and (iii) prejudge a possible outcome as potentially adverse without providing guidance on the nature of the ‘harm’. Effects on non-target organisms and persistence and invasiveness are areas of assessment that were considered in all risk assessments reviewed. However, there was no specific mention of ‘unintended effects on target organisms’ in any of the risk assessments. ‘Unintended effects on target organisms’ is a vague term and not clear what it means (usually effects on non-target organisms are evaluated) or what would be the mechanism of harm to the environment or its predictability (magnitude, duration, nature, etc.). Similarly, gene flow is a natural biological phenomenon, but it is presented to the novice as a potential adverse effect without proper explanation that the objective of the assessment is the consequences of gene flow, not its possible occurrence. All the risk assessment reports reviewed did consider whether sexually compatible wild relatives are present in the countries, but none of the reports contained an assessment of gene flow to other organisms (horizontal gene transfer). The terms presented as potential adverse effects in the section on Hazard Identification raise concerns about the objectivity of the Guidance and cast doubt on its broad usefulness for those not familiar with environmental risk assessment methods. Following the review of the risk assessment reports analysed, the following observations were made: • All of the risk assessment reports:  Use the comparative approach (history of safe use of the conventional crop) taking into account relevant characteristics of the conventional crop;  Consider the scope of the application;  Consider the molecular characterization data (insertion site, copy number, stability, integrity, insert sequence, etc);  Consider compositional and agronomic comparison data;  Consider expression data;  Assess potential for outcrossing and vertical gene transfer;  Assess persistence and invasiveness or weediness potential;  Assess effects on non-target organisms (NTOs). • None of the reports make specific mention to:  Direct, indirect, immediate or delayed adverse effects;  Cumulative effects;  Unintended effects on target organisms;  Horizontal gene transfer (only one country refers to this, but the assessment made is based on gene transfer from the GM plant to sexually compatible wild relatives, which in most countries is referred to as ‘vertical gene transfer’). Insect resistance management is one of the points recommended for consideration in this step as a potential environmental hazard. However, of the risk assessment reports reviewed only one country discussed this topic. This assessment shows the elements of the Guidance considered in all risk assessment reports and the elements outlined that that do not appear in any of the risk assessment reports (for example: ‘direct, indirect, immediate and delayed effects’ and ‘cumulative effects’). As discussed above, the Guidance prejudges hazards, which may be confusing when trying to implement this Guidance by a person without previous experience in risk assessment. 3.2.3. Conducting the risk assessment- STEP 2: Exposure characterization ‘An evaluation of the likelihood of adverse effects being realized, taking into account the level and kind of exposure of the likely potential receiving environment to the living modified organism.’ In this section the Guidance provides recommendations on the points to consider for assessing the likelihood of an adverse effect occurring and characterizing exposure. The Guidance acknowledges that the characterization of exposure can be quantitative or qualitative, but fails to explain how to do this. Qualitative expressions are suggested but with no clear criteria as to how to harmonise their use in the risk assessment. Following the review of the risk assessment reports, the following observations were made: • All of the risk assessment reports:  Use the comparative approach (history of safe use of the conventional crop) taking into account relevant characteristics of the conventional crop;  Consider factors that may affect the spread of the LMO;  Consider factors that may affect the persistence and establishment of the LMO;  Consider the likelihood of outcrossing;  Consider the expected type or level of exposure (level of expression is taken into account). • None of the reports:  Specify ‘plausible pathways of a hazard’;  Contain a specific quantification of exposure other than expression data in different tissues. This evaluation shows that all risk assessment reports consider exposure and use the comparative approach. In this case most of the points to consider recommended by the Guidance appear to be addressed. However, there is no useful information to characterise exposure in practical terms. 3.2.4. Conducting the risk assessment- STEP 3: Hazard characterization ‘An evaluation of the consequences should these adverse effects be realized.’ In this section the Guidance provides recommendations on the points to consider when conducting a hazard characterization. Following the review of the risk assessment reports, the following observations were made: • All of the risk assessment reports:  Use the comparative approach (history of safe use of the conventional crop) taking into account relevant characteristics of the conventional crop;  Consider results from field trials;  Consider if transgene introgression may occur. • Some risk assessment reports consider:  Relevant knowledge and experience with the LMO in similar environments;  Results from laboratory tests (most do, but there was no specific mention of this in the reports from two of the countries). • None of the risk assessment reports:  Discuss the duration of the potential adverse effect (short/long-term);  Discuss the mechanisms of the effect (direct/indirect);  Discuss cumulative effects;  Discuss the reversibility of the effect or ecological scale;  Specify a quantitative or qualitative measure. Here the testing revealed a greater variability in the risk assessments regarding hazard characterization. This could be due to differences in the level of detail provided by the reports. However, there are some elements proposed by the Guidance that do not seem to appear in the risk assessments, for example cumulative effects, direct, indirect immediate and delayed effects, reversibility of effects. No quantitative measures of hazard appear in the reports. 3.2.5. Conducting the risk assessment- STEP 4: Risk characterization ‘An estimation of the overall risk posed by the living modified organism based on the evaluation of the likelihood and consequences of the identified adverse effects being realized.’ This section recommends conducting a risk characterization taking into consideration both hazard and exposure. This could leave a novice thinking that risk assessment occurs in distinct, successive steps rather than a gathering of evidence based on a plan articulated in problem formulation. It also recommends considering ‘identification and consideration of uncertainty’. As discussed above, some inexperienced risk assessors may interpret this as a need to formally address uncertainty. The points to consider also include reference to the need to consider ‘Individual risks and any interaction among them, such as synergism or antagonism’. Presumably this last point refers to an assessment of stacks. Unless further explained, this point can lead to confusion as it is unclear when or why this is recommended. There is also a point referring to ‘Broader ecosystem and landscape considerations, including cumulative effects due to the presence of various LMOs in the receiving environment’. No further explanation is given on how to address this. Importantly, this is an example of the Guidance extending beyond science and experience into policy. The review of the risk assessment reports confirmed that all reports assess the risk considering exposure and hazard. However none of the reports makes specific mention of uncertainty, cumulative effects, broader landscape considerations or interaction between individual risks. 3.2.6. Conducting the risk assessment- STEP 5: Report, summary and recommendations ‘A recommendation as to whether or not the risks are acceptable or manageable, including, where necessary, identification of strategies to manage these risks’ This step recommends that risk assessors prepare a report summarizing the risk assessment process, identified individual risks and the estimated overall risk, and provide recommendation(s) as to whether or not the risks are acceptable or manageable and, if needed, recommendation(s) for risk management options that could be implemented to manage the risks associated with the LMO. This section discusses the use of ‘acceptability criteria’ and thresholds stressing that these are set by each country. The section also recommends considering a ‘scientific benefit analysis’. Benefit analyses are not usually an integral part of the risk assessment, although some countries may consider them during the decision-making process. However, this appears to be outside the scope of the risk assessment Guidance and more a matter of policy. The review of the risk assessment reports shows that all the reports make overall risk conclusions, but there is no mention of acceptability criteria or thresholds. No formal benefit analysis was found in any of the reports. Some of the reports recommended the preparation of an Insect Resistance Management (IRM) plan. 4. SUMMARY This document describes the approach followed to ‘testing’ the risk assessment Guidance following the concept note provided by the Secretariat and the main findings from this test. For this project, risk assessments conducted in different countries for one particular product were used. The goal was to determine whether similar approaches were followed by all countries and how these approaches matched with the approach recommended by the Guidance. The criteria used to select the risk assessments to test were that (1) they should be available on the BCH website; (2) there should be risk assessments from different countries for the same product. After mapping the information available on the BCH website the first product selected was MIR162 maize. Risk assessment reports from Argentina, Canada, Brazil and Japan were used for the testing and comparison. For the testing, the ‘recommendations’ and ‘points to consider’ outlined in the Guidance were entered in an Excel spreadsheet. Each of the risk assessment reports was then examined to determine which of the elements described in the Guidance were considered and discussed in the reports and which were not. Once all the records were compared against the Guidance, a further evaluation of the Guidance took place. This exercise showed that finding information in the BCH website is not easy as there are many links broken and the risk assessments that are retrieved can be in different languages. Comparing the reports was also challenging as the reports reviewed followed different formats, and contained different types of information and different levels of detail. Trying to align each report with the Guidance was also challenging. The current structure of the Guidance is very different from the structure of these reports, making the location of relevant information very time consuming. Experience in risk assessment was essential to complete this task. The test allowed the identification of elements described in the Guidance that were taken into consideration in all the risk assessments examined. There were elements in the Guidance that could not be found in any of the risk assessment reports. Given the request from the Secretariat to use ‘actual cases of risk assessments’ for which links to webpages can be provided, the testing conducted focused only on risk assessments published in the BCH. It is important to note that these risk assessments, written by regulatory authorities, often provide a summary of the risk assessment and they may not contain all the information provided by the developer or all the information considered by the evaluator. Therefore the elements that do not appear in any of the reports may need further examination to determine if they are an essential part of the risk assessment or they could be removed from the considerations altogether. 5. MAIN CONCLUSIONS Finding a practical methodology for testing the Guidance taking into account the concept note provided by the Secretariat is challenging, time consuming and requires a good knowledge of the risk assessment methods currently used to assess the environmental safety of LMOs. However, the method described in this document provided important insights into the practicality and usefulness of the Guidance, and allowed the identification of key issues that could be considered for improvement. The current structure of the Guidance makes the testing difficult because it does not reflect the structure of most risk assessments that are conducted to support regulatory applications. Concluding the latter point requires that the tester have experience in conducting and submitting risk assessments. It would probably be easier to test the Guidance within the context of particular areas of assessment, for example testing the points to consider recommended in each of the five steps for the assessment of persistence and invasiveness or the assessment of effects on NTOs, etc. This would facilitate the grouping of points to consider for each area of assessment and check whether they are considered in the actual case of risk assessment tested. The testing described in this document suggests that the Guidance is useful in that it describes the main concepts within the methodology used in conducting the risk assessment, and makes clear some key distinctions that risk assessment considers, e.g., hazard and exposure as unique components of risk. However, the Guidance would benefit from revision; in particular removing the prescriptive tone, the inappropriate notion that risk assessment occurs in discrete ‘steps’ and deleting all statements that are policy based. Overall, to become useful, the Guidance should be reduced in length to represent a true scientific consensus based on real-world experience. The development of a succinct section on problem formulation is recommended as well as further explanation on how to determine what information is relevant to characterise exposure and hazard for a particular area of assessment and how to characterise risk. Some of the ‘points to consider’ outlined in each of the steps do not appear in actual cases of risk assessment and their mention in the Guidance may not be helpful to guide inexperienced assessors, but bring confusion instead. Therefore, some of these points should be re-considered and if necessary removed.

Results of the testing of the Guidance on Risk Assessment of LMOs

Contents of this page

Guidance on Risk Assessment of LMOs

Testing questionnaire

Individual submissions

Notifications

Analysis of the results

Overall analysis

Section-by-section analysis

Suggestions for improvements