APPENDIX M — PUBLIC COMMENT RESPONSES

General Comments

Comment Number: GC1

Submitted by: Bill Galyean — Idaho National Laboratory

Comment: [Note: The footnote indications below do not appear in the submitted comment; they were added as reference points for the response.] Aside from the fact that I was a contributing panel member in the elicitation process, I want to express my compliments on the effort made by the NRC management and staff to produce realistic and useful results. I believe with the significant research performed in recent years coupled with the accumulated operating experience, reasonable estimates of LOCA frequencies can be made. NRC has recognized these facts and acted accordingly and appropriately. That said, I also wish to express my opinion on the some of the details of the elicitations process and portions of the subsequent analyses about which I disagree, but with the acknowledgement that had they been done differently, the results would not change significantly (i. e., the reported results would probably be reduced by less than an order of magnitude). The first issue relates to the interpretation of the LOCA frequencies and associated uncertainties. The instructions given to the panel members stated that we were to make a best-estimate1 of the “single ‘true’ value”2 for the industry-wide (or more accurately BWR-wide and PWR-wide) LOCA frequency. This is an issue because I do not believe a “single true value” exists for the LOCA frequency. Specifically, I believe each plant has differences in design, construction, operations, age, and maintenance such that in reality the plant-to-plant variation in LOCA frequencies will be quite large. These two interpretations can be made consistent however, if the “single true value” is viewed as an average or mean value of the population of frequencies. While on the surface this might seem to be a question of semantics and appears to have little significance, the implication of the interpretation on the uncertainty characterization is significant. If the “single true value” interpretation is employed, the question becomes, what does the uncertainty associated with this value represent. Since the implicit assumption is that plant-to-plant variability does not exist (otherwise how can there be a single frequency appropriate for the entire industry?), then there is no stochastic or aleatory uncertainty associated with the estimate. That is, the presence of outliers (or event plant-to-plant variability) in the population of nuclear power plants (NPP) is ignored. The uncertainty therefore represents the level of confidence of each panel member’s estimate of this single true frequency.3 Given that the uncertainty represents an individual’s confidence in their own estimate, what is the basis for automatically assuming the probability distribution associated with this confidence uncertainty is not symmetrical?4 Or to ask more specifically, with this interpretation of the point estimate value and associated interpretation of the uncertainty, why do the authors assume that the uncertainty surrounding each panel member’s estimate should be a lognormal distribution that is weighted toward higher (conservative) values? While I agree that the uncertainty associated with LOCA frequencies should be asymmetrically weighted toward higher values; this is based on the observation that not all plants are identical, and that if a LOCA occurs at a plant, that plant will likely be shown to be a poor performer, not representative of the fleet as a whole.5 This is not the same as assuming the confidence uncertainty surrounding the estimates provided by the panel members, of a “single true value” should be represented with a lognormal (which was done in the elicitation).6 [Note: the use of the lognormal distribution became entrenched in PRA with its use in WASH-1400 (1975). However, the model employed then was motivated by the sparse data available from the commercial nuclear power industry at that time. Data used in that study were collected from many different industries and sources. These data were used to develop a probability distribution of the population of possible values, effectively capturing the random component of the uncertainty. Specifically, the authors of WASH-1400 did not know which value in the population of values collected, was the most appropriate value to use. Therefore the entire population of values was characterized in the probability distribution. The data were not combined, or averaged to estimate a single value, and the uncertainty modeled with the lognormal distribution was not meant to describe a statistical confidence on a single true value, but used to describe the variability in possible values. Quoting directly from WASH-1400: “Because of the large spread, the failure rate data were treated as random variables, incorporating both the physical variability and the uncertainty associated with the rates. Moreover, since the study’s results were to apply to a population of approximately 100 nuclear plants, it was important to show the possible variability and uncertainty in this population.” The consequences of employing the lognormal distribution to characterize each individual panel member’s uncertainty, manifests itself in how the estimates were interpreted and processed (by the authors) and in the effects of the overconfidence adjustment made to the base case results. Specifically, the best-estimates solicited by the authors of this “single true value” were interpreted as median values of the lognormal distribution, allowing for the derivation (by the authors) of a higher mean value. [Note that for any probability distribution skewed toward higher values, the lognormal being one example, the mean will always be greater than the median.] This calculated mean value was then used to represent the panel member’s input to the aggregation process used for generating the LOCA frequency results. Additionally, this assumption of a lognormal for the uncertainty on each panel member’s estimate, has the additional impact of calculating an even higher (compared to the non-adjusted calculated mean) mean value after the widening of the uncertainty in the overconfidence adjustment. My concerns are twofold. First, the opinions of the panel members were solicited, and then modified (increased) by the authors.7 Irrespective of the instructions and discussions during the elicitation process, this point was viewed with dismay by more than one panel member. Second, the processes and analyses employed have introduced a conservative bias into the final base case results, with additional conservative bias8 inserted in the various sensitivity studies.9 This “creeping conservatism” is not necessarily undesirable given the significant uncertainties and the uses to which the results will be employed; however, it should be explicitly acknowledged and clearly stated rather than obscured in the details of the analyses.10

Response: The authors have identified ten separate issues in this comment, as indicated by the inserted footnotes. The responses to these issues are provided below.

Issue 1: The instructions given to the panel members stated that we were to make a best-estimate1 of the “single ‘true’ value”2 for the industry-wide (or more accurately BWR-wide and PWR-wide) LOCA frequency.

1. Issue 1 Response: The panel members were not asked to provide a ”best-estimate“ of any quantity. Rather, they were asked to provide a MV and LB and UB values for each question. The MV was defined such that, in the panel member’s opinion, the unknown true value for that particular question has a 50% chance of falling above or below the MV, with similar definitions for the LB and UB values (Section 3.8.5).

Issue 2: The instructions given to the panel members stated that we were to make a best-estimate1 of the “single ‘true’ value”2 for the industry-wide (or more accurately BWR-wide and PWR-wide) LOCA frequency.

2. Issue 2 Response: Additionally, the panel members were not asked to estimate a ’’single ’true‘ value“ of any quantity. Rather, the elicitation focused on estimating generic, or average, LOCA frequencies for the commercial fleet by combining the contributions from individual component failures. As stated in Section 2 of NUREG-1829, the generic BWR and PWR estimates were determined by first estimating the separate LOCA frequency contributions associated with specific BWR piping, BWR non-piping, PWR piping, and PWR non-piping failures for each panelist. These individual piping and non-piping component failure frequencies were then combined to estimate parameters of the total passive system LOCA frequency distributions for BWR and PWR plants at each distinct LOCA category and time period. Panelists were specifically instructed to consider broad plant differences in estimating these component failure frequencies and their uncertainties (Section 3). More information related to the instructions given to the panel can be found in Sections 2 and 3 of NUREG-1829.

Issue 3: If the “single true value” interpretation is employed, the question becomes, what does the uncertainty associated with this value represent. Since the implicit assumption is that plant-to-plant variability does not exist (otherwise how can there be a single frequency appropriate for the entire industry?), then there is no stochastic or aleatory uncertainty associated with the estimate. That is, the presence of outliers (or event plant-to-plant variability) in the population of nuclear power plants (NPP) is ignored. The uncertainty therefore represents the level of confidence of each panel member’s estimate of this single true frequency.3

3. Issue 3 Response: More precisely, as stated in Section 3.8.5, the elicited quantities for each question (MV, LB and UB) are percentiles of each panel member’s subjective distribution. As stated above, this subjective distribution should consider broad plant difference related to plant design that can affect the LOCA frequency. Also, except for the base case frequencies used to anchor the results, panel members were never asked about absolute frequencies, only about relative frequencies of specific components, subsystems or systems. Their responses were then combined to form estimates of LOCA frequencies.

Issue 4: Given that the uncertainty represents an individual’s confidence in their own estimate, what is the basis for automatically assuming the probability distribution associated with this confidence uncertainty is not symmetrical?4

4. Issue 4 Response: The distribution associated with the response to any question was assumed to be asymmetrical only if the stated LBs and UBs were not symmetrical about the MV.

Issue 5: While I agree that the uncertainty associated with LOCA frequencies should be asymmetrically weighted toward higher values; this is based on the observation that not all plants are identical, and that if a LOCA occurs at a plant, that plant will likely be shown to be a poor performer, not representative of the fleet as a whole.5

5. Issue 5 Response: As noted above and in NUREG-1829, the estimated LOCA frequencies are industry-wide or “generic, or average, estimates for the commercial fleet” (Section 2). Consequently, the uncertainties associated with the estimates pertain to these generic estimates and not to any individual plants whose LOCA frequencies may differ from the generic estimates. As stated in Section 2 of the NUREG, the panelists were instructed to account for broad plant-specific factors which influence the generic LOCA frequencies in providing uncertainty bounds, but not consider factors specific to any individual plants. Thus, the uncertainty bounds should include both contributions related to the uncertainty of the generic estimates as well as uncertainty due to broad plant-specific fleet differences.

Issue 6: This is not the same as assuming the confidence uncertainty surrounding the estimates provided by the panel members, of a “single true value” should be represented with a lognormal (which was done in the elicitation).6

6. Issue 6 Response: The lognormal, or split lognormal when the responses are asymmetrical, is a reasonable distribution for representing the responses to the various questions so that the responses can be combined to estimate LOCA frequencies. Based on the sensitivity analyses conducted in Section 7, the authors expect that the log-normal distribution assumption has an inconsequential impact on the bottom line parameter estimates, especially in light of the large uncertainties observed in the final results. Note that the report does not estimate the distributions of LOCA frequencies, but only the four bottom-line parameters (mean, median, 5th and 95th percentiles) of these distributions.

Issue 7: First, the opinions of the panel members were solicited, and then modified (increased) by the authors.7

7. Issue 7 Response: It is misleading to state that “.. .the opinions of the panel members [were] modified (increased) by the authors”. Rather, an overconfidence adjustment was applied to increase only the uncertainties for those panelists whose responses indicated more confidence than the group average. The panelists’ median estimates were not modified. The justification for the overconfidence adjustment is provided in Section 5.6.2. Responses were not modified in any other manner.

Issue 8: Second, the processes and analyses employed have introduced a conservative bias into the final base case results, with additional conservative bias8 inserted in the various sensitivity studies.9

8. Issue 8 Response: The only “conservative bias” that may have been introduced in the summary estimates was through the use of the overconfidence adjustment. However, as discussed in Section 5.6.2 and by reviewing the wide range of uncertainty estimates provided by the panelists, there is good reason to believe that at least some of the panelists may have been overconfident. Only those panelists that were more confident than the group median were adjusted. Furthermore, sensitivity studies indicated that the error factor adjustment used was a reasonable adjustment scheme. This method was not the most conservative adjustment scheme which could have been used.

Issue 9: Second, the processes and analyses employed have introduced a conservative bias into the final base case results, with additional conservative bias8 inserted in the various sensitivity studies.9

9. Issue 9 Response: Sensitivity analyses were conducted to examine the impact of the assumptions used to process the results. The assumptions and methods used to calculate the “baseline” LOCA estimates are clearly stated in Sections 5 and 7. Additionally, the methods used to conduct each sensitivity analysis are clearly explained in Sections 5 and 7 and comparisons are made to the baseline results so that the effects on the LOCA frequency estimates are readily apparent. Those sensitivity analyses resulting in the largest differences from the baseline estimates are clearly discussed in the report.

Issue 10: This “creeping conservatism” is not necessarily undesirable given the significant uncertainties and the uses to which the results will be employed; however, it should be explicitly acknowledged and clearly stated rather than obscured in the details of the analyses.10

10. Issue 10 Response: The NUREG report systematically documents the assumptions and analysis used to calculate the LOCA frequency estimates. The NUREG also discusses the effects of alternative assumptions and analysis methods using sensitivity studies. The analysis procedures utilized in the elicitation process were fully discussed with the panelists at several times throughout the process: during the base case review, prior to conducting the individual elicitations, during the presentation of the preliminary results, and during the preparation of the draft NUREG. Each panelist was also provided numerous opportunities to modify their results, including changes if they believed the initial instructions were unclear, or if the processing of their results did not actually reflect their solicited opinion. None of the panelists elected to provide wholesale changes to their estimates based on these issues. General feedback from the panelists about the elicitation process in general, and the processing of the results in particular, was favorable.

Comment Number: GC2

Submitted by Joseph Conen of the BWR Owners Group

Comment: Another issue is the inclusion of thermal fatigue as a degradation mechanism for the BWR feedwater line. We are not aware of any thermal fatigue issue other than the feedwater nozzles; that issue was resolved in the early 1980s through several mitigation measures including the installation of GE — designed triple thermal sleeve. A rigorous inspection program per NUREG-0619 is currently in place. Not a single one of hundreds of these inspections have turned up any evidence of cracking. Thus, thermal fatigue is not an issue in the NSSS portion of the BWR feedwater line. In addition, please refer to the letter from T. Essig (U. S.NRC) to T. J. Rausch (BWROG), dated June 5, 1998, subject: BWROG-Safety Evaluation of Proposed Alternative to BWR Feedwater Nozzle Inspections (TAC M94090). This letter provides evidence that the thermal fatigue of the FW nozzles is effectively managed at BWRs and provides the basis for revising the examination frequencies.

Response: The NUREG report acknowledges the improvements that have been implemented in BWRs as a result of the NUREG-0619 inspection requirements. As stated in Section 6.3.2, "There was a rash of feedwater nozzle cracks reported in the 1970 to early 1980 time period in BWRs. Plant and system modifications were implemented after a detailed study of the problem and augmented inspections are being conducted based on NUREG-0619 requirements. These mitigation measures have proven effective as no new thermal fatigue cracks have been discovered in these BWR feedwater nozzles over the last 20 years." However, as stated in Section L.2 when discussing the relative contributions for various BWR piping systems, "There was wide variability expressed for the feedwater system. Several participants thought that its susceptibility was similar to that of the recirculation system while others thought that it would make an inconsequential contribution.” This latter group generally thought that the mitigation programs in place for the feedwater system were generally effective and additional thermal fatigue locations within the feedwater system were not significant LOCA contributors.

However, rationale provided by some of the panelists who believe that thermal fatigue is a significant contributor to the LOCA frequency estimates is summarized in Section L.2 and Section 6.3.2. As stated in Section 6.3.2, “The BWR plants are expected to be more prone to thermal fatigue problems compared with the primary side of PWR plants because they experience greater temperature fluctuations during the normal operating cycle. In BWR plants, thermal fatigue remains an important contributor for the feedwater lines and the RHR system.” Additionally, “…thermal fatigue is an aging mechanism that could lead to a large LOCA because it does not manifest itself as a single crack, but as a family of cracks over a wide area. Thermal fatigue cracks also tend to propagate rapidly, and since it is not material sensitive (i. e., it can attack a number of materials), it is difficult to prioritize critical areas for inspections.” These reasons explain why thermal fatigue is still regarded as an important LOCA contributor by many panelists.

The general variability in opinion expressed by the elicitation panelists for feedwater, main steam, and RHR systems is summarized in the box and whisker plots in Figures L.6 through L.8. Some of the differences associated with the illustrated variability in these figures results from the diversity in opinion about the significance for thermal fatigue in BWR plants.