Package Integrity Testing in the Product Life Cycle—Test Method Selection and Validation

This article is compiled based on the United States Pharmacopeia (USP) – 2025 Edition

Issued and maintained by the United States Pharmacopeial Convention (USP)

DOWNLOAD PDF HERE

1 INTRODUCTION

Package Integrity Evaluation—Sterile Products 〈1207〉 provides guidance in the integrity assurance of product packages. This chapter describes package integrity verification during three product life cycle phases: 1) package development, and package processing and assembly validation; 2) product manufacturing; and 3) commercial product shelf-life stability assessments. Further, this chapter provides information on how to select, develop, and validate leak test methods.

2 PACKAGE INTEGRITY TESTING IN THE PRODUCT LIFE CYCLE

Appropriate packaging for a sterile product can be determined on the basis of a knowledge-gaining effort conducted during the product's life cycle.

2.1 Development and Validation

Package development

Package development begins with the preparation of a product–package profile (e.g., user requirements specification), which considers the product end use, stability requirements, and method of manufacture, as well as the anticipated storage, shipment, and distribution environments. This profile also defines the proper product–package quality requirements and the package's maximum allowable leakage limit (see Package Integrity Evaluation—Sterile Products 〈1207〉, Product–Package Quality Requirements and the Maximum Allowable Leakage Limit).

With this prospectively developed body of information, one can select each package component's materials of construction, choose suitable component sources, and establish critical physical attributes and component dimensional tolerances. Each component material, with its critical dimensional tolerances, directly affects the integrity of the final packaged product. Assurance of package integrity originates from the use of appropriate materials, accurate and optimum closing properties, matching dimensional fit, appropriate multi-component stack heights and tolerances, and consistent control of processes used to assemble the closed package.

The manner in which the package is processed, formed, or assembled is an important consideration in package integrity assurance. A preliminary assessment of package integrity at the end of the development phase under conditions representative of the marketed product manufacturing system is prudent.

These conditions include processes such as sealing operations and component sterilization. Where possible, processes are performed according to established and approved user requirement specifications. Establishing appropriate physical characteristic specifications for container materials, considering lot-to-lot variations, can ensure that the most extreme processing conditions anticipated (e.g., multiple sterilization cycles) do not physically damage materials in a manner that would adversely affect package integrity.

Finally, the robustness of the manufactured product–package system may be evaluated during the development phase by exposing a representative number of product samples to specified storage, shipment, distribution, and final product-use environments. These efforts may include studies that evaluate package integrity at the extremes of the finished product–package profile, not simply at optimal conditions. Given the complexity of some sterile packages, it may be useful for package integrity development studies to incorporate multi-point analyses of test packages manufactured within specified process parameters. Utilizing a common container–closure for which the producer has significant experience and process knowledge can supplement the overall development process and may help reduce the efforts needed.

A battery of integrity and seal quality test methods may be employed during product–package development, starting with techniques able to measure the product's inherent package integrity. Inherent package integrity of a viable package system conforms to the product's maximum allowable leakage limit.

Test package quantities for inherent package integrity verification may vary on the basis of: the complexity of the product–package, the specifics of the user specification requirements, and the prior experience of the producer.

Test quantity choice is also influenced by the confidence that can be placed on the package integrity test results as well as the level of integrity assurance desired. In some cases inherent package integrity verification may be more readily and economically determined by using empty or placebo-filled container–closure systems, thereby enabling larger sample quantities to be tested by the most sensitive and quantitative leak test methods without the risk of product formulation interference with the test method.

The outputs of the packaging development phase include the final user requirement specifications, which form the basis of production purchasing specifications for package components. Also during the development phase, the final equipment user requirement specifications are developed for package material cleaning, sterilization, and forming; sealing or assembly equipment; and allied materials supply and component feed systems. These user requirement specifications provide purchase specifications for the acquisition of equipment or for the vetting of potential contract manufacturers.

Package processing and assembly validation

Final confirmation of acceptable inherent package integrity is generally part of a larger process validation activity for the overall production process. The scope depends upon the product type and whether the organization has previous experience with the container–closure system.

All processes germane to the sterilization and formation of a package having integrity are to be evaluated against the user requirement specifications established in the package development phase, including likely process extremes. For example, inherent package integrity verification may consider extremes of package assembly variables such as line speed, heat-sealing temperature, screw-cap application torque, and vial-capping forces, as well as resterilization processes, labeling, and secondary and tertiary packaging processes. Validation test requirements and scope should fit the statistical requirements and capabilities of each process, taking into account both package and package-line complexity, as well as prior experience with similar product–packages.

Testing done during technical transfer from the product development site to the manufacturing sites will assist in determining whether the user requirement specification targets and the control ranges established in development require any modification when packages are made on a full manufacturing scale.

Integrity test methods for package processing and assembly validation studies are meant to verify the packages' continued conformance to the product's maximum allowable leakage limit. For some product–packages, the most fitting integrity test methods for this life cycle phase may have a detection limit greater than the maximum allowable leakage limit. Methods able to reject largely leaking packages such as those caused by defective or out-of-specification components, package damage, and/or package misassembly are appropriate. Seal quality tests suitable for use include those able to monitor package processing and/or assembly consistency.

Successful validation will result in a package that meets its user requirement specifications. The primary objective in package development and subsequent validation is to arrive at a quality product–package prepared using processes that reliably and consistently run within specified operating parameters as defined in the user requirement specifications, yielding critical package defects at a satisfactorily low rate. When performed, in-process and end-product package integrity testing should complement, not replace, thorough package development efforts.

2.2 Product Manufacturing

To ensure the quality of the manufactured product–package, it is critical to specify components of sufficient quality and to select vendors carefully. The factors to consider include:

• Acceptable results of the initial vendor or supplier evaluation
• Appropriate vendor acceptance quality limits and statistical sampling plan(s), or relevant certification
• Incoming component quality verification, including statistical assessment of quality against purchase specifications
• Appropriate procedure(s) for establishing corrective and preventive action when a vendor falls short of quality expectations.

Package integrity re-evaluation is considered when changes occur in the product, package design, package materials, or manufacturing/processing conditions. The extent of change control efforts are evaluated on a case-by-case basis, while the level of testing required to support the change is determined based on impact assessment.

The product's package integrity profile is compiled over the course of commercial manufacture. This database of ongoing leak and seal quality test results serves to flag potentially harmful package integrity excursions that may be linked to operative variations in package component design/material and package assembly/processing. Not all packaged-product damage can be readily detected post-assembly; therefore, processing procedure controls and monitors may be relied upon to safeguard against such problems. The proper combination of package leak tests, complementary package seal quality tests, and visual inspection is largely based on package failure results observed in earlier production validation and package development studies.

Testing production lot samples can provide a measure of package integrity confirmation, while entire testing by nondestructive means is able to yield an ongoing assessment of integrity assurance. In some cases regulatory requirement dictates the level of testing performed. For other product-packages, justification for the level of testing is based on statistical process control results generated during the validation phase, and later, on the basis of routine manufacturing product-quality trending analyses.

For example, glass or plastic ampules closed by heat fusion are customarily subjected to 100% nondestructive leak testing. Products sealed under vacuum require appropriate package assembly validation supplemented by testing over time to ensure that the vacuum is retained. Similarly, integrity assurance of packages that require a specific, non-reactive, inert gas headspace is based on appropriate package assembly validation along with testing for rise in reactive gas or water vapor content over time.

2.3 Commercial Product Stability

Container-closure integrity tests have been recommended as alternatives to sterility testing as part of commercial product stability programs. The goal is to ensure package integrity as a function of long-term product storage (1). [NOTE-Package integrity tests do not replace product release sterility tests.] Testing product for sterility is a poor measure of product-package integrity and also will not ensure product-package integrity over the shelf life of the product when performed as part of the stability program. In addition, a package may be in no danger of microbial ingress and yet be unable to maintain the gas headspace content necessary for product quality. Validated package integrity test methods using technologies such as those described in this chapter are more sensitive and reliable than product lot sterility testing for detecting a breach in package integrity that could lead to sterility loss or failure in relevant product physicochemical quality attributes.

Ideally, package integrity tests selected to support marketed-product stability studies are able to verify the absence of the smallest leaks of concern for a given product-package system. In other words, the product's maximum allowable leakage limit falls within the chosen test method leak detection range. However, the test methodology most appropriate for a particular product-package system may be unable to detect the very smallest leaks of concern. In some cases the package contents interfere with the ability of leak test methodologies to detect the smallest leaks. For example, proteinaceous ingredients or even salts may clog leak pathways, inhibiting leak detection by gas flow methods such as vacuum decay or mass extraction. It is prudent to understand the product's potential to interfere with the selected leak test method, both initially after package assembly and over time.

In situations such as those described above, the package integrity test chosen to support stability studies should have a detection limit as close as feasible to the product's maximum allowable leakage limit. An understanding of what the method is capable of evaluating and how this is applicable to microbial integrity assurance is warranted.

For products that demand package headspace content preservation, it is appropriate that the integrity test for stability studies verify the continued presence of specific headspace gases or subatmospheric pressure over time. Satisfactory results verify the absence of leaks that could jeopardize product sterility as well as relevant physicochemical quality attributes. The required duration for monitoring container vacuum or headspace gas content maintenance that equates to microbial barrier assurance and liquid product loss prevention can be predicted on the basis of gas flow kinetics, should a leak pathway exist that is large enough to allow microbial entry and product formulation loss.

Stability test samples intended for package integrity evaluation are kept at labeled storage conditions for the marketed product. Test sample quantities chosen for each testing time point are to be relevant and sufficiently representative of the purpose of stability testing, which is to indicate whether integrity is affected by the stability conditions. Sample quantity selection takes into account all prior development and validation testing.

As noted in reference (1), if an integrity test is nondestructive to the product or package, samples that pass package integrity testing may be further used in the stability testing for that specific test period or interval. However, samples should not be tested for package integrity at one time point (e.g., 12 months), then stored for further stability testing at a later time point (e.g., 24 months). While not specifically noted in reference (1), it is logical to assume that test samples earmarked for integrity verification over the course of the stability study could be checked for integrity before placement on stability if a nondestructive leak test method is used. This would be akin to the common practice of visually inspecting test samples prior to placing them on stability. In this way, subsequent integrity failures can be attributed to stability storage, rather than to other causes.

Change to read:

3 TEST METHOD SELECTION CRITERIA

No single package leak test or package seal quality test method is applicable to all product–package systems. Test method selection is made on a product–package on a case-by-case basis. Often more than one test method is employed during a given product's life cycle.

Package test method selection as a function of product life-cycle phase, along with important integrity considerations, are discussed in Package Integrity Testing in the Product Life Cycle. A broad discussion of additional selection criteria for leak test methods follows. Specific leak test method examples are provided for informational purposes but are not intended as recommendations. The attributes and general capabilities of tests can be found in Package Integrity Leak Test Technologies 〈1207.2〉 and are helpful in the method selection process.

3.1 Package Contents

When selecting a leak test method, the first determining factor is the nature of the package contents. Whether the package contains a liquid or solid formulation, and whether it contains a headspace of inert gas, air, vacuum, or even no headspace at all, will influence leak test method choice.

For example, if testing liquid-filled packages by vacuum decay or mass extraction, test vacuum conditions may trigger some formulations to solidify inside leak paths, thereby blocking gas leakage flow and making the test ineffective. Alternatively, electrical conductivity and capacitance tests can be used, but only if the liquid product is more electrically conductive than the package materials.

3.2 Package Design, Materials of Construction, and Mechanics

Packages vary widely in their design, their materials of construction, and the mechanisms whereby package closure is affected. Each of these variables influences the leak test method choice, as the following examples illustrate.

Rigid or flexible packages

Rigid (nonporous) packages can tolerate the pressure or vacuum-challenge conditions required in several leak tests, including tracer liquid ingress, vacuum decay, mass extraction, and some tracer gas tests. In contrast, some flexible packages can tolerate differential-pressure test conditions only if special tooling is used to restrict package expansion and prevent subsequent seal damage. Package restriction is necessary to ensure consistent differential pressure conditions across the package barrier.

Moveable or fixed components

Moveable package components, such as syringe stoppers (plungers or pistons), may require restraint to prevent their dislocation during the differential pressure test conditions required for a majority of leak test procedures (e.g., tracer liquid test, pressure or vacuum decay test, mass extraction test, bubble emission test, and microbial challenge by immersion test).

Polymeric materials

When exposed to test vacuum conditions, package materials such as plastics and some elastomers may outgas volatiles that raise vacuum-decay leak test results as well as mass extraction results, falsely implicating package leakage. Plastic packages that are highly permeable to tracer gases may not be compatible with helium tracer gas leak detection, as helium permeating through the package could be mistaken for package leakage or may mask small leaks. Special fixtures to limit tracer gas permeation effects and isolate tracer gas exposure to the seal area under test have been used to mitigate such difficulties.

Metallic materials

Packages made of foil laminate materials may prove incompatible with electrical conductivity and capacitance leak detection, which works best with relatively nonconductive package materials. However, aluminum caps used to secure vials closed with elastomeric closures pose no hindrance to electrical conductivity and capacitance tests, even for finding leaks located between the closure and the vial finish.

Transparent or opaque materials

Packages made of transparent or translucent materials allow for visual inspection and electromagnetic wave passage. Therefore, transparent or translucent materials can be tested by laser-based gas headspace analysis techniques, as well as tracer-liquid ingress or microbial ingress. Opaque packages are incompatible with testing approaches that require visual inspection of the package contents.

3.3 Closure Type and Mechanics

Refer to Package Integrity Evaluation—Sterile Products 〈1207〉, Closure Type and Mechanics for a discussion of the closure type categories represented by the various product–package systems. The design of the closure system and its leakage restriction function (i.e., the maximum allowable leakage), plus the types of defects anticipated, influence the integrity test method choice.

3.4 Maximum Allowable Leakage Limit

The maximum allowable leakage limit as a function of the product–package quality requirement is discussed in Package Integrity Evaluation—Sterile Products 〈1207〉, Product–Package Quality Requirements and the Maximum Allowable Leakage Limit. The following integrity test method selection considerations are offered in light of that discussion.

Sterility and product formulation content must be preserved; gas headspace preservation is not required

Integrity tests for this product quality category include those able to verify that the maximum allowable leakage limit that prevents liquid and microbial ingress is not exceeded. Tracer gas tests performed using the vacuum mode and laser-based gas headspace analysis test methods are two examples. Both have been shown to be sensitive enough to quantitatively analyze leakage through the smallest leak paths found to pose the smallest chance of liquid leakage or microbial ingress in rigid packaging. Such tests have also proven useful for defining relationships among package design, component fit, package assembly parameters, and leakage rate, even in the absence of package defects.

Leak testing of product-filled packages during later product-life-cycle phases often requires other tests. Leak test methods available for this phase of the product life cycle are able to be validated to reliably detect defects a few micrometers and larger. Examples of such physicochemical leak tests include vacuum or pressure-decay tests, mass extraction methods, electrical conductivity and capacitance tests, and liquid tracer tests.

Sterility, product formulation content, and gas headspace content must be preserved

For this product quality category, leak test options include those that directly check for package headspace pressure and/or content, such as laser-based gas headspace analysis techniques. The detection limit for such methods is a function of the method's ability to accurately measure package headspace content or absolute pressure at the product acceptance limit, given the package headspace volume and the time lapse after package assembly. Such methods have broad application throughout the product life cycle.

Sterility must be preserved; product access is required

The product quality subcategory relates to the additional integrity requirement for those products that are contained in multiple-dose packages once the product is accessed by the end user. As discussed in Package Integrity Evaluation—Sterile Products 〈1207〉, Product–Package Quality Requirements and the Maximum Allowable Leakage Limit, Sterility Must Be Preserved; Product Access is Required, physicochemical as well as microbiological test methods designed to characterize and verify the unique barrier properties specific for the given container–closure system and its end-use application may be required (2).

Elastomeric closures of multiple-dose parenteral products are meant to reseal, limiting microbial ingress and product leakage during and between product access via needle puncture. The test in Elastomeric Closure Functionality in Pharmaceutical Packaging/Delivery Systems 〈382〉, Functionality Tests, Self-Sealing Capacity (CN 1-Dec-2020) is a blue dye liquid tracer test intended to screen closures for their ability to prevent gross liquid leakage post puncture. Caution is advised when solely relying on this test to prove a given closure's ability to reseal. Other test method(s) may need to be designed and implemented that more fully characterize the closure's leak prevention capabilities for the given product–package system and product end-use application.

3.5 Deterministic or Probabilistic Methods

A "deterministic leak test method" is one in which the leakage event is based on phenomena that follow a predictable chain of events, and leakage is measured using physicochemical technologies that are readily controlled and monitored, yielding objective quantitative data. Most deterministic leak test methods rely on the predictable movement of gas that inevitably occurs through an open leak path, given specific differential pressure and/or partial pressure test conditions (e.g., tracer gas, laser-based gas headspace analysis, pressure decay, vacuum decay, and mass extraction). The electrical conductivity and capacitance test is a deterministic leak test approach that relies on the more predictable presence of liquid near a leak path, rather than the less predictable movement of liquid through a leak. Deterministic methods are characterized as being capable of reproducibly detecting leaks at clearly defined and predictable detection limits. Because the majority of deterministic leak test methods described in this chapter require no special test sample preparation, sample preparation error is eliminated.

A "probabilistic leak test method" is stochastic in nature in that it relies on a series of sequential and/or simultaneous events each associated with uncertainties, yielding random outcomes described by probability distributions. Thus, the findings are associated with uncertainties that necessitate larger sample sizes and rigorous test condition controls to obtain meaningful results. Typically, sample size and test condition rigor are inversely related to leak size. Therefore probabilistic leak test methods are more challenging to design, develop, validate, and implement, especially when used to find leaks near the upper and lower limits of the test method's detection range. Probabilistic methods include microbial challenge tests, as well as some physicochemical tests. These methods include bubble emission tests, tracer liquid tests (employing either qualitative or quantitative measurement methods), and tracer gas tests by the sniffer probe approach.

A deterministic leak test method having the ability to detect leaks at the product's maximum allowable leakage limit is preferred when establishing the inherent integrity of a container-closure system. Deterministic methods may also be chosen if test sample quantities are limited, when checking for rarely occurring leaks of concern and/or when the potential risk for failing to find leaks of a given size or type is too great. Probabilistic methods are best chosen when the method outcome requirements demand a specific probabilistic testing approach. Probabilistic tests are further discussed in Method Outcome and Quantitative or Qualitative Methods of Analysis below.

3.6 Physicochemical or Microbiological Methods

"Physicochemical leak test methods" are those that use physical and/or chemical analysis techniques to test for package integrity. Physicochemical methods include both deterministic and probabilistic leak test methods. "Microbiological leak test methods" are probabilistic methods of analysis that use viable microorganisms to evaluate test sample integrity. This chapter includes the microbial challenge test by immersion exposure.

The following section includes additional information on the test method outcome criterion that is related to the microbiological challenge test method.

3.7 Method Outcome

The outcome(s) sought from leak test results will often drive the test method choice. These desired outcomes may include:

• Leak path presence of detection
• Leak path location determination
• Leakage rate measurement
• Liquid egress/ingress potential
• Microbial ingress potential

All leak test methods are meant to identify the leak path presence. Often, leak tests are able to provide additional outcome information. Methods that also provide evidence of leak path location include electrical conductivity and capacitance tests, bubble emission tests, tracer gas sniffer probe tests, and some tracer liquid tests. Methods that provide a measure of whole-package leakage rate include laser-based gas headspace analysis, mass extraction, pressure and vacuum decay methods, and tracer gas tests by the vacuum mode.

Microbial challenge tests provide information on the degree of protection afforded by the product-package against microbial ingress that occurs via active growth or motility through leak pathways and/or by liquid carrier passive transport through leak pathways. Microbiological challenge tests help to clarify the risks to product sterility posed by specific package materials, package designs, or potential package barrier breaches. Sterility risks linked to particular environmental exposure or product use conditions may also demand a microbiological challenge methodology. In summary, all leak test methods at minimum detect leaks; some methods may provide more information, but no single method alone can yield all four outcomes listed above.

3.8 Quantitative or Qualitative Methods of Analysis

The measure of analysis can play a part in leak test method selection. In other words, does the method yield quantitative data that allow for objective analysis, or are the data strictly qualitative and require more subjective interpretation? Leak test methods that use a "quantitative measure of analysis" include electrical conductivity and capacitance tests, laser-based gas headspace analysis, mass extraction, pressure and vacuum decay, tracer gas tests (especially when testing via the vacuum mode), and tracer liquid tests that use quantitative analysis (e.g., spectrophotometric analysis).

Conversely, a "qualitative measure of analysis" is based on subjective observation of a specific quality, attribute, or characteristic of the test sample, e.g., a visual check for turbidity when evaluating microbial challenge test samples. Bubble emission tests that report visible evidence of continuous bubbling, and tracer liquid tests that rely on visible evidence of dye migration are other examples of subjective and qualitative analysis. Because qualitative measurement results are subject to interpretation, they may be prone to human error. When method considerations permit, leak test methods that yield quantitative measurements are preferred.

3.9 Leak Test Detection Limit

The detection limit of a leak test is the smallest leakage rate or leak size that the method can reliably detect, given the product-package of interest. A large variety of measurement units are used to describe leakage rates and leak sizes when specifying the detection limits (and detection ranges) of leak test methods. This often leads to confusion when comparing the performance claims of various instrument manufacturers or examining test results generated by multiple methods.

To address this, Table 1 presents the relationship between orifice size (assuming a perfect hole of negligible length) and the rate at which dry air would pass through such a hole when exposed to 1 atmosphere (atm) differential pressure at a specified temperature.

These leakage rates and leak sizes are theoretical approximations and are not definitive.

Table 1 services two purposes. First, it is meant to help the reader better grasp the relationship between theoretical hole diameter and the gas leakage rate. Second, it provides a common measurement scale that can be referred to later in this chapter to more simply state leak detection limits for the various technologies described. For example, test technology X found in published studies to detect leaks as small as about 8 µm would be referred to in (1207.2), as having an approximate leak detection limit of row 4 in Table 1.

Leak detection limit should not be the only or perhaps even the primary basis for choosing a test method. Often, the best method for a given application is dictated by other factors. For example, a tracer gas leak test method having an extremely small leakage rate detection limit may be the proper choice for establishing the inherent package integrity of a stoppered glass vial as a function of capping machine parameters during package development. Yet this method would be an inappropriate choice for rapid on-line testing in routine manufacturing. Instead, an electrical conductivity and capacitance test with a larger leak detection limit, able to test product-filled packages at on-line speeds, may be the best option. Refer to Detection Limit for a discussion on how to determine the limit of detection.

Table 1. Gaseous Leakage Rate versus Orifice Leak Size^a

^a This table is not intended for ranking test methods but helps express leak detection capability.

^b Dry air leakage rate measured at 1 atm differential pressure across an orifice leak at 25°.

^c Nominal diameter orifice sizes assume negligible leak path length.

3.10 Leak Test Method Range

Leak test method range is the interval between the smallest and largest leak size (or leakage rate) that can be detected by a given leak test method with a suitable level of accuracy and precision. All leak test methods have an optimum detection range. Therefore, it is possible that additional tests may be needed to catch those leaks of concern that fall outside the chosen leak test method's detection range. For example, a test method able to find the smallest leaks may miss gross leakage such as a missing package component. Vision systems that check for major package defects such as a missing package component or a package crack may be required. Finally, before using a test method to check for large leaks, the impact of gross defects on leak test instrumentation should be considered; some instruments may malfunction or become damaged upon exposure to a leaking product or damaged packages. Refer to Range for additional detection range information.

3.11 Nondestructive or Destructive Methods

The need to preserve the test product-packages may influence the decision to select a nondestructive leak test method, rather than a destructive one. Destructive test methods damage the test sample and/or expose it to potential contaminants; hence, the product is not recoverable. Only nondestructive test methods are appropriate for leak testing product-package units earmarked for commercial or clinical study distribution.

Examples of destructive leak test methods include tracer liquid ingress tests, bubble emission tests, and microbiological challenge tests.

Tracer gas and pressure decay leak tests are considered destructive if they require that the package barrier be compromised in order to introduce tracer or pressurizing gas into the assembled package. Examples of nondestructive leak test methods include mass extraction and vacuum decay leak tests, as well as noninvasive gas headspace analysis tests. Electrical conductivity and capacitance leak tests are deemed nondestructive if it can be shown that electrical current exposure causes no harm to the product; in rare instances, exposure has triggered headspace ozone formation, causing product oxidation (3).

3.12 Off-Line or On-Line Methods

Off-line leak testing is generally performed on a stratified random sampling of the product lot, away from the manufacturing line. Off-line package evaluation allows for the use of any validated nondestructive or destructive leak test and seal quality test option compatible with the product-package. Off-line testing can accommodate slower test cycle times; for methods in which test time is a performance factor, a slower off-line test is often more sensitive than its on-line counterpart. Off-line tests typically are less costly to perform as they utilize bench-top or smaller scale equipment without the sophisticated product-handling machinery required to support higher speed on-line processes.

On-line leak testing is commonly integrated into a continuous fill and seal product-package manufacturing process. A prerequisite for an on-line leak test method for entire lot testing is that it be nondestructive to the package and its contents. On-line testing can potentially provide greater assurance that all packages have integrity and can yield instant feedback in the event of package misassembly or breakage, enabling real-time line corrections. For some test systems, incorporating large-scale leak detection equipment into a complex high-speed product-package filling and assembly line can be prohibitive. Higher line speeds leading to shorter leak detection test cycle times can limit test method detection capability. In addition, the impact of instrument downtime on the production run as a result of leaking packages or possible equipment malfunction is an important consideration. A separate leak testing line may be set up outside the sterile manufacturing suite to allow for full lot testing without the complications of leak test integration with package filling/sealing operations. A few examples of on-line leak test technologies include electrical conductivity and capacitance, vacuum decay leak, and noninvasive laser-based gas headspace analysis tests.

4 TEST INSTRUMENT QUALIFICATION, METHOD DEVELOPMENT, AND METHOD VALIDATION

Leak test methods are validated in order to demonstrate method effectiveness. Method validation is preceded by instrument/equipment qualification, followed by test method development. The following discussion specific to leak test methods is intended to supplement the guidance for analytical instrument qualification presented in Analytical Instrument Qualification (1058), plus the guidance for method validation provided in Validation of Compendial Procedures (1225).

4.1 Instrumentation and Equipment Qualification

The qualification of instruments or equipment to be used for leak testing includes:

evaluation of instrument/equipment functionality, and

determination of test system detection capabilities using appropriate calibration tools or reference standards to simulate with-leak test conditions.

4.2 Method Development and Validation

After successful instrument/equipment qualification, leak test method parameters are developed and optimized to ensure a leak test method is able to meet all relevant leak detection performance criteria specific for the test product–package system. The following properties of a valid test method are defined as they relate to package integrity tests.

Accuracy

“Accuracy” is a measure of the method's ability to correctly differentiate packages that leak above the claimed detection limit from those that leak below this limit (i.e., do not leak). Accuracy provides a measure of false positive and false negative occurrence.

Alternatively, for those methods that deliver an outcome that is a direct quantitative measure of gas leakage rate (or of gas concentration or gas pressure), accuracy is a measure of the method's ability to produce an outcome comparable to a true standard.

For example, helium mass spectrometry provides a direct measure of helium leakage rate. Accuracy is the closeness of the instrument reading to the certified leakage rate of a nationally recognized traceable standard.

Precision

“Precision” is the ability of the method to yield reliable, repeatable data. Precision includes: repeatability (e.g., repeat testing of a homogeneous test sample population), ruggedness (within laboratory tests performed, for example, by multiple operators on multiple days, using multiple instruments; also known as intermediate precision), and reproducibility (among laboratories tests)

The level of precision to which a leak test method is validated is often a function of resource availability (e.g., one instrument versus multiple instruments) and intended test method application (e.g., use of the method at one test site only versus across multiple test sites).

Specificity

“Specificity” is the ability of the method to accurately differentiate between leaking and nonleaking packages, despite interfering factors that may cause false detection. For example, when employing tracer gas leak detection using helium mass spectrometry (vacuum mode), excessive helium permeation through the package wall may mask small package leaks or may be falsely interpreted as leakage in no-defect packages.

Detection limit

Refer to Leak Test Detection Limit for an introduction to this topic. The detection limit of a leak test is specific for a given testing approach when performed using a specific instrument make/model in evaluating a given product–package system. Utilizing the principles described in 〈1225〉, detection limit is demonstrated by challenging packages with and without known defects by the leak test method for multiple test days by multiple operators. The intended application of the method will dictate the level of precision required (i.e., whether to incorporate multiple operators/instruments/laboratories, etc.).

A common challenge with small leak detection is the potential for interfering factors to be misconstrued as leakage presence. For example, vacuum decay leak tests measure the rise in pressure inside an evacuated chamber containing the test package. Package leakage causes chamber pressure to rise, but package material volatiles, test system moisture, and package expansion may also do the same. Gas permeating through a package wall detected by a tracer gas test may be mistaken for leakage. Bubbles emitted during a bubble test may actually be the result of package surface outgassing, volatilization of dissolved gases in the immersion fluid, or the release of trapped air between package components.

The absence of a leak detection signal may also be misinterpreted as the absence of leaks. For example, tracer liquid tests may fail to reliably detect small leaks due to any one of a number of factors including air locks, product, or debris in the leak path; liquid surface tension; leak path geometry; or insufficient differential pressure test conditions. The same is true for microbial ingress tests that are further subject to the inherent variability of living microorganisms.

In short, false negative results that miss leaks and false positive results that incorrectly suggest leak presence are possible with any leak test method. Therefore, test method detection limit is determined by comparing readings of intentionally defective packages to nondefective ones. Leak detection limit determination test units consist of a randomly ordered population mix of negative and positive control units (refer to Negative and Positive Controls). Nondestructive test methods can employ the same set of units for multiple test exposures, while destructive methods will require a new set for each test series. Control subset unit quantities are chosen based on several factors: 1) the deterministic or probabilistic nature of the outcome, 2) the inherent package-to-package variability that may influence test results, and 3) the statistical confidence level required by the test acceptance criterion. The positive control subset includes units with defects sized to the anticipated detection limit, in addition to units with leaks bracketing this size limit. If the detection range is to be established, controls having large defects are included. Positive controls representing a wide defect size range are especially important for probabilistic methods in order to clearly understand leak detection likelihood as a function of leak size.

Because of the many product-packages and leak-testing options available, the resultant leak detection limit is more meaningfully stated when the negative and positive control subsets used, the test precision level, and the test results are summarized. The following is an example of expressing a test method's limit of detection:

“The detection limit for method X was determined to be 5 ± 2 µm. Validation studies found defects of this nominal size were detected 95% of the time; all larger defects were detected 100% of the time. Studies included three replicate test series performed on multiple days by multiple operators in a single laboratory using one instrument. Detection limit was determined using product-filled packages. Test units in each series included a negative control subset of 300 units (each without defect) and a positive control subset of 90 units (each having a laser-drilled defect ranging in nominal size from 1.5 ± 0.6 µm to 15 ± 3 µm). Each defect was independently size-certified by comparing the dry air leakage rate at 1 atm differential pressure (leak inlet pressure of 1 atm versus outlet pressure of approximately 1 Torr) at 25° to that of standard orifice leaks.”

[Note—This is one example of how test method detection limit could be expressed and is not to be considered compulsory or restrictive either in content or level of detail.]

Leak test technologies exist that are able to detect leaks even smaller than can be artificially created in a positive control test set. Two examples follow. In both cases, a limited number of positive controls can serve to verify that the instrument set-up is able to detect leaks of specific type and in specific package locations but not to determine leak size detection limit capability.

• Tracer gas tests by helium mass spectrometry performed in the vacuum mode can detect leaks as small as about 10⁻¹¹ mbar·L/s, which is about 5 logarithmic leakage units smaller than leakage through a hole 0.2 µm in size. For such methods, leakage rate detection ability can only be verified by using nationally recognized gas leakage rate standards to introduce leaks into the test system. Because gas permeation can be mistaken for leaks, it is important to experimentally establish the relationship between tracer gas leakage rate and permeation rate as a function of test cycle time for a given package system.
• Laser-based gas headspace analysis is another approach that may be able to identify the presence of leaks smaller than can be artificially created. For such methods, the limit of detection can be mathematically predicted on the basis of gas flow kinetics and is a function of the time lapse between analyses, and the smallest gas content or pressure change that can be reliably detected by the instrument for the given package system.

Suantitation limit

“Quantitation limit” is that lowest leakage rate or leak size that can be determined with acceptable accuracy and precision under the stated experimental conditions. Laser-based gas headspace analysis is a method that may be evaluated for quantitation limit. For most leak test methods, detection limit is more meaningful.

Linearity

“Linearity” is the ability of the method to elicit test results that are mathematically proportional to leak path size or leakage rate. Deterministic leak test methods that exhibit linearity include laser-based gas headspace analysis and tracer gas analysis (vacuum mode). Other methods such as vacuum decay, pressure decay, and mass extraction also produce results that correlate to leak size or leakage rate. However, test findings are generally intended to identify leak presence and perhaps to understand relative leak size; they are not typically relied upon for leak size or leakage rate quantitation. Electrical conductivity and capacitance tests and all referenced probabilistic methods are not validated for linearity.

Range

Leak detection range is defined in Leak Test Method Range. Leak test method range is explored in test method development to better understand leak test method detection limitations. For methods being relied upon to detect leaks within a specified size range, detection at the upper range limit may be confirmed in validation. Range is evaluated by using sets of negative controls and appropriately sized larger-defect positive controls. The large-defect positive control subset may include defects of various types likely to occur for the given product–package system (refer to Type Defects).

Robustness

“Robustness” is the method's ability to accurately identify leaking versus nonleaking packages despite small but deliberate variations in procedural parameters, providing an indication of the method's suitability during normal usage. One way to evaluate robustness is to perform the test using test parameters bracketing optimal or normal test specifications. Parameters to be varied are those having the greatest impact on test results; variation should reflect instrument performance accuracy. For example, the robustness of a vacuum decay leak test with a test cycle time of 30 s (accurate to within 0.5 s) might be demonstrated during method development by verifying method performance at set test cycle times of 29.5 s and 30.5 s.

4.3 System Suitability

One outcome of test method development and validation is the establishment of a system suitability test(s), also called a performance verification test(s). “System suitability” is a manner of ensuring that the leak test method, including all factors that may be subject to variability that may impact test results (e.g., instrumentation, analysts, test sample preparation, and the test environment), is adequately controlled and maintained in such fashion that the method is rugged and robust. System suitability is important for all leak test methods. Extra care may be required for probabilistic methods more prone to variability.

For example, a highly instrumental method such as a mass extraction test may be checked for system suitability by leak testing a master package unit both with and without the added challenge of external air introduced to the test system via a calibrated leak adjusted to the method's limit of detection. If the method requires preliminary test package preparation (e.g., a drying step), then it would be appropriate to demonstrate that an intact package prepared in the prescribed manner elicits the anticipated mass flow rate response. System suitability can be performed at the beginning and end of each testing sequence for added method assurance.

System suitability for a test more probabilistic in nature takes into account multiple test method variables and may require greater numbers of challenge samples (i.e., positive and negative controls) for adequate method assurance. For example, tracer gas tests via a sniffer probe may require a routine demonstration that the operator is able to successfully differentiate packages without defect from those with leaks (ranging in size from smallest to largest, located at various package positions) in a blinded challenge study. Alternatively, the operator could be challenged by randomly introducing defective samples (unknown to the inspector) among the test sample population
during routine test procedures.

4.4 Microbial Ingress Risk Comparison

Appropriate leak test method validation practices mirror compendial analytical test validation guidelines. In the past, validation of physicochemical leak test methods used for pharmaceutical sterile product packages also routinely included a direct or indirect comparison of physicochemical leak test results to microbiological challenge data. While an understanding of the relationship between physicochemical leak test method capabilities and microbial ingress risk may be important, an experimental comparison may not be needed or useful in all cases. [NOTE-The reader may refer to Package Integrity Evaluation-Sterile Products (1207), Product-Package Quality Requirements and the Maximum Allowable Leakage Limit for a discussion of maximum allowable leakage limit and inherent package integrity.]

The following situations illustrate when a comparison study of the microbial ingress or liquid leakage risk to physicochemical leak test method capability relationship is likely not needed. [NOTE-This is not intended to be a complete or exhaustive listing of all situations for which comparison studies would not be required or useful but is provided for illustration only.]

• If the validated physicochemical leak test method has a proven detection limit at or below the product-package maximum allowable leakage limit.
• If the validated physicochemical leak test method is not being used to verify the absence of all leaks of concern. Instead, the method is being used to find leaks notably larger than the maximum allowable leakage limit. For example, a rapid on-line test shown to reliably detect leaks of 25-150 µm in nominal diameter is being used to reject damaged or misassembled product-filled packages within the method's leak detection range.

The following situations illustrate when an experimental indirect or direct comparison of the microbial ingress risk (or liquid leakage risk) to physicochemical leak test method capability relationship may be useful. [NOTE-This is not intended to be a complete or exhaustive listing of all situations for which comparison studies could prove beneficial but is provided for illustration only. These examples are not compulsory but are provided for instruction only.]

• If the validated physicochemical leak test method is being relied upon to measure or confirm the inherent package integrity of a product-package system, but the method's limit of detection is notably greater than the maximum allowable leakage limit. For example, a package has a maximum allowable leakage limit of less than 6 * 10 ^ - 6 mbar L/s as measured by helium mass spectrometry (equivalent to holes less than approximately 0.2 ± 0.1 µm in nominal diameter). This limit was chosen based on published literature references. But the leak test method of choice for testing product-filled packages placed on stability is able to detect leaks equivalent to holes 3 µm in nominal diameter and larger. A study correlating microbial ingress or liquid leakage risk to defect type/size can provide a measure of the likelihood of microbial ingress (and/or liquid leakage) at the leak test method detection limit, and thus can provide an understanding of ability of the stability leak test to identify leaks of concern.
• If the validated physicochemical leak test method is being used to measure or verify the inherent package integrity of a product-package system, but the maximum allowable leakage limit is either lacking or not well defined. For example, a unique package is being used for which the maximum allowable leakage limit that will ensure absence of product loss or microbial ingress has not been determined. A study comparing the risk of microbial ingress or liquid leakage to leak type/size, and in turn to the likelihood of detection by the physicochemical leak test method may be useful.

When performing microbial ingress risk assessment studies it is important to keep in mind the probabilistic nature of microbial ingress. To achieve the most meaningful data, large population sets of negative and positive controls should be used. Test protocols should be thorough and well designed, taking into consideration the multiple factors and variables that can influence results. As suggested above, liquid leakage risk assessment studies may substitute for microbial ingress risk studies assuming the risk of liquid leakage is equivalent to or greater than that of microbial ingress, with appropriate justification.

4.5 Negative and Positive Controls

"Negative controls" are packages with no known leak, and "positive controls" are packages with intentional or known leaks. Negative and positive controls are designed and assembled for use in method development and validation with consideration given to container-closure design, materials of construction, characteristics of anticipated package leaks, and impact of product contents on test results. Negative and positive controls should represent packages assembled in a typical manner as the product being tested using normally processed components, the exception being the intentionally created leak in each unit of the positive control subset. Some leak test methods may necessitate positive controls that simulate test product headspace and formulation contents as well. System suitability checks for some test methods employ negative and positive controls (refer to System Suitability).

Test blanks should not be confused with negative controls. For example, liquid tracer leak detection by spectrophotometric analysis may require a blank solution without the liquid tracer element to confirm instrument baseline performance.

A "master" is a type of negative control test unit. It is a package prototype, model, or facsimile made to simulate the test package in shape and design. Masters may be made of solid material such as plastic or metal, or they may be simply a designated container-closure unit.

Masters are no-leaking mock packages often used in system suitability verification tests for leak tests to verify instrument performance, such as for vacuum decay or mass extraction testing.

Defect creation methods

The positive control set typically represents a range of package defect sizes and types. Numerous approaches have been used to create package defects. When creating positive controls, a fundamental awareness of leakage dynamics as a function of different defect types and materials of construction is important.

Placing a so-called "hole" or break in the package wall is one positive control creation approach. In this case the defect materials of construction are identical to the package itself, thus potential test method interference due to product exposure to the package material can be readily identified. An example of such interference is the clogging of small leak paths by product formulation observed during vacuum decay, mass extraction, or tracer gas (vacuum mode) tests.

Laser drilling is often used to create package defects commonly referred to as holes. However, these so-called holes are not pristine orifices but are non-cylindrical and asymmetric and may consist of a tortuous matrix of micro-cracks. While laser-drilled defects are not dimensionally ideal, they offer the advantages of closely simulating actual package defects (e.g., cracks) and do not require the introduction of foreign materials, such as wires, tubes, or epoxies, that may influence leakage dynamics. Laser-drilled defects are often sized for nominal diameter by comparing dry air leakage rate through the defect at specified conditions of pressure and temperature to flow rates through standard orifice leaks in thin metal plates. Currently, laser-drilled defects in rigid glass or plastic components can be made as small as about 2-3 µm in nominal diameter and to about 5-10 µm in nominal diameter in flexible thicker wall package materials. Smaller diameter defects tend to clog from handling, environmental debris, or flexible package wall deflection.

Glass micropipettes can be used to simulate single-orifice defects as small as about 0.1 µm in diameter. The tip diameter can be nominally sized using air flow measurements. When creating positive controls, micropipettes are inserted through a break in the package wall, then an appropriate sealant is applied to the insertion site. Challenges to micropipette use include ensuring a complete seal between the micropipette perimeter and the package wall and avoiding micropipette tip damage.

In addition, air trapped in the pipette tube can interfere with leak test methods that depend on fluid flow through the leak path, such as tracer liquid tests and microbial challenge ingress tests.

Microtubes (also called microcapillaries) inserted through the package wall and fixed in place with sealant are another means of creating positive control defects. Microtubes can be made of a variety of materials, can be cut to any length and can be as narrow as 2 µm in cross-sectional diameter. Microtubes are often employed as a substitute for a smaller-bore, shorter-length leak path when performing leak tests that rely on gas flow measurements. However, caution is advised before choosing lengthy, larger-bore microtubes to simulate an orifice leak of smaller diameter for leak tests that rely on the passage of liquids or microorganisms. Fluid dynamic theory correlating fluid flow through capillary tubes to pass through smaller bore holes is based on the unimpeded passage of ideal liquids through capillaries at equilibrium pressure conditions. Liquid product formulation and aqueous media flow into and through a microtube is complicated by numerous factors including liquid surface tension, liquid viscosity, surface contact angle, airlocks, particulate blockage, and tube-wall and tube-end finishes. Microbial ingress through microtube defects relies more on the presence of liquid in the tube than on the physical barrier to passage or grow-through afforded by the tube diameter (4). Microtubes are a logical choice when creating defects representing channel defects. In this case microtube length should mimic as closely as possible the actual package barrier thickness (package wall or seal width). Microtube use challenges include effecting smooth, perpendicular cuts of microtube ends and adequately sealing microtubes into the test sample wall.

Other commonly used defect creation methods include inserting a needle through the package wall; placing a wire, microfilament, or film between sealing surfaces; and adhering a holed, thin metal plate onto package surfaces. It is important to note that defects made by using an object foreign to the package (e.g., needle, film, wire plate) may display gas, liquid, or microbial leakage dynamics markedly different from that of actual defects (5). Positive controls made by such means are easy and inexpensive approaches for creating larger size defects useful for test method feasibility studies and for exploring a test method's detection range upper limit.

Type defects

"Type defects" are positive controls representing realistic package flaws. Type defects are important to include in method validation studies to explore a leak test method's practical application in detecting realistic package failures and flaws. A few types of defect examples are listed below:

• Heat sealed bag: 1) weak seal, 2) wrinkled seal, 3) seal gap, 4) seal channel, 5) product entrapment in seal
• Stoppered vial package: 1) vial finish channel defect, 2) loosely capped stopper, 3) product trapped between stopper and vial
• Prefilled syringe: 1) needle shield punctured by staked needle, 2) defective plunger
• Ophthalmic dropper bottle: 1) loose cap, 2) missing or poorly inserted dropper tip, 3) defective tip or cap

Assigning precise sizes to type defects is generally not meaningful or feasible because of their inherent irregularity and complexity. Type defects are defined in more qualitative, descriptive terms such as those listed above. Because no leak test method can find all possible defects in a given container-closure system, information collected from type defect tests can be used to identify alternative approaches to detecting or limiting the occurrence of critical imperfections not readily found by the chosen leak test method.

5 REFERENCES

1. Department of Health and Human Services. Container and closure system integrity testing in lieu of sterility testing as a component of the stability protocol for sterile products. Guidance for industry. Rockville, MD: U.S. Food and Drug Administration, Center for Biologics Evaluation and Research (CBER), Center for Drug Evaluation and Research (CDER), Center for Devices and Radiological Health (CDRH), Center for Veterinary Medicine (CVM); 2008.
2. Bagel S, Wiedemann B. Extension of in-use stability of preservative-free nasalia. Eur J Pharm Biopharm. 2004;57(2):353-358.
3. Bigwarfe P Jr, Domin-Turza A, Hwang E, Leidner S, McGinley C, Olson K, Nachtigall M. Presentation. Annual Meeting of the American Association of Pharmaceutical Scientists 2008. Elucidation of an unusual mechanism of drug product degradation caused by a leak detection instrument. Lake Forest, IL: Hospira; 2008.
4. Keller S, Marcy J, Blakistone B, Hackney C, Carter H, Lacy G. Application of fluid and statistical modeling to establish the leak size critical to package sterility. J Appl Pack Res. 2006;1:11-21.
5. Morrical BD, Goverde M, Grausse J, Gerwig T, Vorgrimler L, Morgen R, Büttiker J-P. Leak testing in parenteral packaging: establishment of direct correlation between helium leak rate measurements and microbial ingress for two different leak types. PDA J Pharm Sci Technol. 2007;61(4):226-236.