Evaluation of Screening Technologies for Assessing Medicine Quality

This article is compiled based on the United States Pharmacopeia (USP) – 2025 Edition

Issued and maintained by the United States Pharmacopeial Convention (USP)

DOWNLOAD PDF HERE

1 INTRODUCTION 

The proliferation and spread of substandard and falsified (SF) medical products has been and continues to be a growing global concern. Examples have been reported on a range of SF medicinal products, including but not limited to essential medicines. These products threaten global public health by jeopardizing patient safety, adding to the cost of care, diminishing confidence in health workers and systems, increasing the risk of treatment failure, wasting valuable resources, and contributing to the development of drug resistance. 

Fortunately, many analytical technologies and tools exist to evaluate product quality within the laboratory. In general, these technologies are well understood and characterized. Over the last decade, many of these tools have been miniaturized for portability to help combat the growing proliferation of SF medical products. One of the benefits of the field-deployable platform of these technologies is that it facilitates the screening of samples as part of a risk-based testing approach. 

The capabilities of the portable screening technologies complement, but do not obviate the need for, laboratory-based technologies and their confirmatory power. Portable tools may help conserve the limited resources of these laboratories and drive a sustainable “work smart” approach to ensuring medicine quality by utilizing complementary methodologies of progressively increasing complexity to rapidly and reliably analyze large numbers of samples. The following examples describe where and how these technologies are currently used or may be used in the future: 

• Manufacturing controls 
• Supply chain screening 
• Border control 
• Customs inspection 
• Post-market quality surveillance or regulatory monitoring 
• Point-of-care screening 

However, the potential and actual capabilities and limitations of many of these portable screening tools, and especially their performance in field settings, have not been established. To address this need, this chapter provides the structure and suggested requirements for performing an appropriate and pragmatic review of a given technology. Such a review would typically comprise general information, a performance evaluation, and a field evaluation. The information garnered from the review can then be used to inform selection, procurement, and deployment of the technology of interest. Within this chapter, the term technology refers to screening instruments, solutions, technologies, or tools that provide basic information about the authenticity or quality of a drug substance or product. 

2 APPLICATION OF THE TECHNOLOGY 

The first and most important aspect of an evaluation is to identify the target application of the technology. A clear understanding of the intended application of any technology will help identify the parameters that need to be evaluated and inform the decision as to whether multiple screening technologies might be needed in a particular setting. For example, some screening technologies evaluate non-chemical features of a product, such as appearance, packaging, labeling, and origin. The technology will need to correctly identify a falsified medicine, but it will not need to determine the level of chemical impurities within the drug product. 

Furthermore, the intended application of the technology comprises that of both the original equipment manufacturer (OEM) and the user. If these applications align, then the initial part of an evaluation should involve reviewing the technology against the manufacturer's claims. However, if these applications do not align, further steps should be taken to evaluate the technology against the user's requirements and intended application. Intended applications may vary based on factors such as: 

• Medicine types 
• Anticipated users 
• Geographic locations 
• Points within the supply chain 
• The public health consequences related to poor-quality medicines within supply chains 

3 GENERAL INFORMATION ABOUT THE TECHNOLOGY 

The first step of a review, prior to implementing the performance and field evaluations, is to acquire general information about the technology of interest. This should include, but is not limited to, answering the questions listed below. 

Specifications, Relative Cost, and Data 

SPECIFICATIONS

• Questions regarding specifications include: 
• Which model or version of the technology is being reviewed? 
• What technique (e.g., Raman spectroscopy, infrared spectroscopy, chromatography) does the technology employ?
• What are the dimensions (i.e., size, weight) of the technology? 
• What type of power source (i.e., electricity, battery, solar) is needed to run the technology? 
• What additional power requirements (i.e., converters, power conditioners, uninterrupted power supply) are involved? Are there any safety precautions that should be taken into consideration before using the technology (i.e., lasers, heat generating)? Is waste generated when using the technology? If so, what kind of waste, and how is it safely handled? 
• How easily can the technology's equipment be cleaned? 
• What additional resources (i.e., chemicals, water, electricity) are required to clean the technology? 

RELATIVE COST

• Questions regarding relative cost include: 
• What is the up-front cost of the technology? 
• What are the recurring costs and how much are they (e.g., consumables, maintenance costs)? 
• What is the cost of performing a test? 
• Will the technology and intended methods be utilized with enough frequency to offset the cost? 
• What warranties are available when purchasing the technology, and how comprehensive are they? 

DATA

Questions regarding data include: 

• What alternative languages are available for the hardware/software? 
• Does the technology have a readily accessible user manual? 
• Does the technology need or have internet connectivity capabilities? 
• What format are data less in? 
• Can data be transferred between devices? 
• Can data be transferred to external sources? 
• How can the data be displayed? 
• Can the data be easily summarized into a report that supports decision-making? 
• Can the software capabilities maintain data integrity? 
• What are the levels of software permissions (i.e., administrator, user, guest), and what level of access do they provide?
• Does the instrument have a usable bar or Quick Response (QR) code reader? 

It is important to note that some of these questions complement and supplement those listed in 5. Field Evaluation. Generally speaking, the questions listed in that section can be answered through the review of publicly available information on the technology of interest or discussion with the OEM. 

4 PERFORMANCE EVALUATION 

The analytical performance portion of an evaluation may include parameters discussed in various guidances and standards on method development and validation. These include Validation of Compendial Procedures 〈1225〉 (1); Analytical Instrument Qualication 〈1058〉 (2); the Food and Drug Administration (FDA) Guidance for Industry, Analytical procedures and methods validation for drugs and biologics (3); and the International Council for Harmonisation (ICH) (harmonised tripartite guideline), Validation of analytical procedures: text and methodology Q2(R1) (4). However, except for 〈1058〉, these references are geared toward the validation of a method; their application in the context of this chapter should therefore focus on the performance qualification of a specific instrument rather than a procedure. 

Technology Applications and Analytical Performance Characteristics to Evaluate 

This section provides potential reviewers with the performance characteristics to evaluate, depending on the intended applications of a given technology. Ideally, this analytical performance evaluation will take place in a laboratory where variables can be controlled and a confident assessment can be made regarding the technology‘s analytical capabilities and whether they align with the reviewer's needs. With the exception of Application I, below, the performance evaluation should also encompass a comparison of the analytical results of the screening technology with those of a well-accepted confirmatory or compendial procedure (e.g., high-performance liquid chromatography) using reference material or standards. This will help inform the application and implementation of the technology within a broader quality assurance and quality control system. 

Table 1, Table 2, and the Applications below, have been adapted from 〈1225〉 and tailored to the context of this chapter. However, they provide a broad guide and should be adapted to the needs of the user and the capabilities of the technology of interest. Definitions of the listed characteristics are included in the Glossary. 

QUANLITATIVE APPLICATIONS

Application I: Verification of packaging, labeling, origin, and appearance of the sample or drug product 

Application II: Identification of bulk drug substances or active pharmaceutical ingredients (APIs) in finished pharmaceutical products Application III: Identification of contaminants or impurities in bulk drug substances or finished pharmaceutical products 

Table 1. Qualitative Analytical Characteristics to Evaluate 

QUANLITATIVE APPLICATIONS

Application IV: Quantification of major components of bulk drug substances or APIs in finished pharmaceutical products Application V: Quantification of contaminants, impurities, or adulterants in bulk drug substances or finished pharmaceutical products Application VI: Determination of product performance characteristics (e.g., disintegration, dissolution, drug release) in finished pharmaceutical products. 

Table 2. Quantitative Analytical Characteristics to Evaluate 

5 FIELD EVALUATION 

A field fitness evaluation of a screening technology should be conducted as a follow-up to a performance evaluation. This part of the review assesses the ability of a technology to operate effectively and efficiently in field settings, which may include, but is not limited to, resource-limited areas. Certain parameters should be assessed and certain questions answered. Although some of these parameters can be evaluated through online searches and discussion with the instrument manufacturers, other parameters will need to be evaluated in the field and this should be taken into consideration when planning a review. The parameters and questions listed in the section below are not exhaustive and some may not be applicable, depending on the technology. However, they provide baseline information necessary for making an informed decision about the field suitability of a given technology within a given country. 

Determine whether the results provided by the technology when analyzing a medical product in the field correspond to results obtained from the performance evaluation in the laboratory. This does not mean that the exhaustive analytical testing done in the laboratory needs to be replicated in the eld, but a smaller, pragmatic subset of samples and tests should be run to give the reviewer confidence that the technology does, indeed, function in a field setting. 

5.1 Access, Handling, Maintenance, and Repair 

Some questions to address include: 

• Is the technology commercially available in the country? 
• In countries where the technology is commercially available but needs to be imported, what are the marketing authorization requirements for importing the technology? 
• What are the control requirements for exporting this technology out of its country of origin or distribution? 
• Are there local service providers or authorized distributors in the country? 
• Can the technology be maintained and repaired by trained users or is a local service provider required? 
• If there is a local service provider, then where is it located and how can it be reached (i.e., in country office, via email, or by phone)? Are there other shipping requirements if there is a need to send the technology for calibration or repair? 
• Are there additional resources required to calibrate, recalibrate, or repair the technology (i.e., internet access, reagents, standards, spare parts)? 
• How accessible are these additional resources in terms of availability and affordability? 
• How often are recalibration and performance maintenance of the technology required? 

5.2 Durability and Use

DURABILITY

Some considerations regarding durability of the technology include: 

• What is the operating temperature range of the technology? 
• How is the technology affected by humidity? 
• How are results affected by changes in temperature or humidity? 
• How tolerant is the technology of various changes in the operational environment? 

Temperature 

Humidity 

Dust 

Vibration 

Electromagnetic interference 

Light 

Water 

Electrical variability (e.g., voltage, surge, frequency) 

• How tolerant is the technology of rough handling by users? 
• Is the technology intrinsically safe and suitable for the intended environment (e.g., temperature, humidity)? 
• Can the technology withstand a drop test [i.e., according to U.S. Military Standard [MIL-STD]-810G (5), International Electrotechnical Commission [IEC] 60068 (6), or ASTM D5276-98 (7)]? 

USE 

Some considerations regarding use of the technology include: 

• Is the technology suitable to use for the intended application? 
• What level of training (e.g., remote vs. in-person, time, complexity) is required to use the tool? 
• What are the sample preparation requirements for analysis and is the testing destructive or non-destructive? 
• What additional accessories, consumables, reagents, and standards are required for sample analysis? 
• How long does it take to analyze a sample and how easily can the results be interpreted? 
• How many samples can be run simultaneously? 
• What offline data analysis is required to interpret the results? 
• What offline data analysis is available for experienced users? 
• What requirements are there for fleet management of multiple instruments? 
• What types of products (e.g., tablets, injections) can the technology analyze? 
• Can models and/or methods developed on another instrument be remotely transferred to field units as needed? 

5.3 Protocol and Statistics 

PROTOCOL

When evaluating a screening technology for suitability, the applicable sections of this chapter should be used to develop a robust, practical, and ideally, standardized protocol that specifically outlines the work to be done. After selection of the applicable sections, additional variables need to be taken into account to ensure that the evaluation generates evidence-based data that can inform decision making. These variables include but are not limited to the following: 

• Number of units of the screening technology needed to determine variability between instruments (any evaluation should include at least two units of the technology in question) 
• Types of dosage forms (e.g., solid oral dosage forms, injectables) to use for the evaluation 
• Types of APIs to use for the evaluation 
• Number of batches per product to use for the evaluation 
• Number of units per sample to use for the evaluation 

SATISTITICS

The prudent use of statistics as part of an evaluation is a critical component for ensuring the integrity and reliability of any data generated. When possible, the specific statistical approaches and methodologies to be used should be identified, and a professional statistician should be consulted to devise an appropriate protocol. Concomitantly, it is valuable to collect available information from the literature and specific OEMs about the statistics associated with various applications of the technology. This approach provides a strong baseline to expand upon. 

The following statistical guidances, references, and standards can be used to help with planning statistical approaches and methodologies: 

• Analytical Data—Interpretation and Treatment 〈1010〉 (8) 
• Statistical Tools for Procedure Validation 〈1210〉 (9) 
• Annals of Internal Medicine Research and Reporting Methods—QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies (10) 
• ASTM E2586-16 Standard practice for calculating and using basic statistics (11) 
• BMJ Research Methods & Reporting—STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies (12)
• Guidance for Industry and FDA Staff—Statistical guidance on reporting results from studies evaluating diagnostic tests (13)
• International Organization for Standardization (ISO) 3534-2, Statistics—vocabulary and symbols—part 2: applied statistics (14)
• ISO 3534-3 Statistics—vocabulary and symbols—part 3: design of experiments (15) 
• ISO 5725-1 Accuracy (trueness and precision) of measurement methods and results—Part 1: general principles and definitions (16) 

6 GLOSSARY 

For reference purposes, some definitions are based on 〈1225〉. 

Accuracy: The accuracy of an analytical procedure is the closeness of the test results obtained by that procedure to the true value. The accuracy of an analytical procedure should be established across its range. [Note—The definitions of accuracy in 〈1225〉 and the ICH harmonized tripartite guideline, Validation of analytical procedures: text and methodology Q2 (R1) (4) correspond to unbiasedness only. In the International Vocabulary of Metrology and documents of ISO, “accuracy” has a different meaning. In ISO 5725, accuracy combines the concept of unbiasedness (termed “trueness”) and precision, and is defined as the closeness of agreement between a test result and the accepted reference value.] 

Adulterant: Any substance that has been (1) mixed or packed with a drug so as to reduce its quality or strength or (2) substituted wholly or in part for the drug. 

Detection limit: The detection limit is a characteristic of limit tests. It is the lowest amount of analyte in a sample that can be detected, but not necessarily quantitated, under the stated experimental conditions. Thus, limit tests merely substantiate that the amount of analyte is above or below a certain level. The detection limit is usually expressed as the concentration of analyte (e.g., percentage, parts per billion) in the sample. Linearity: ICH documents define linearity as the ability to obtain test results that are proportional to the concentration of analyte in the sample across a given range. 

Precision: The precision of an analytical procedure is the degree of agreement among individual test results when the procedure is applied repeatedly to multiple samplings of a homogeneous sample. The precision of an analytical procedure is usually expressed as the standard deviation or relative standard deviation (coefficient of variation) of a series of measurements. Precision may be a measure of either the degree of reproducibility or of repeatability of the analytical procedure under normal operating conditions. In this context, reproducibility refers to the use of the analytical procedure in different laboratories, as in a collaborative study. Intermediate precision (also known as ruggedness) expresses within-laboratory variation, as on different days or with different analysts or equipment within the same laboratory. Repeatability refers to the use of the analytical procedure within a laboratory over a short period of time using the same analyst with the same equipment. Quantitation limit: The quantitation limit is a characteristic of quantitative assays for low levels of compounds in sample matrices, such as impurities in bulk drug substances and degradation products in finished pharmaceuticals. It is the lowest amount of analyte in a sample that can be determined with acceptable Precision and Accuracy under the stated experimental conditions. The quantitation limit is expressed as the concentration of analyte (e.g., percentage, parts per billion) in the sample. 

Range: The range of an analytical procedure is the interval between the upper and lower levels of analyte (including these levels) that have been demonstrated to be determined with a suitable level of Precision, Accuracy, and Linearity using the procedure as written. The range is normally expressed in the same units as the test results (e.g., percent, parts per million) obtained by the analytical procedure. Robustness: The robustness of an analytical procedure is a measure of its capacity to remain unaffected by small but deliberate variations in procedural parameters listed in the procedure documentation. Robustness provides an indication of the procedure's suitability during normal usage. Robustness may be determined during development of the analytical procedure. 

Screening: The process by which a sample undergoes an initial assessment of quality to determine acceptability or the need for additional testing. 

specificity: ICH documents define specificity as the ability to identify the analyte in the presence of components that may be expected to be present, such as impurities, degradation products, and matrix components (4). [Note—Other reputable international authorities (e.g., International Union of Pure and Applied Chemistry, AOAC-International) have preferred the term “selectivity,” reserving “specificity” for those procedures that are completely selective.]The definition of specificity, above, has the following implication for the tests discussed below:

1. Identification tests ensure the identity of the analyte. 
2. Purity tests ensure that all the analytical procedures performed allow an accurate statement of the content of impurities of an analyte (e.g., heavy metals, organic volatile impurities). 
3. Assays provide an exact result, which allows an accurate statement of the content or potency of the analyte in a sample. Supply chain: The system of manufacture, distribution, and dispensing of pharmaceutical products. 

7 REFERENCES 

1. US Pharmacopeial Convention. Validation of Compendial Procedures 〈1225〉. In: First Supplement to USP 41. Rockville, MD: USP;2018.
2. US Pharmacopeial Convention. Analytical Instrument Qualication 〈1058〉. In: USP 41. Rockville, MD: United States Pharmacopeial Convention. 2018:7005–7011. 
3. Food and Drug Administration. Analytical Procedures and Methods Validation for Drugs and Biologics. Rockville, MD: FDA; 2015. www.fda.gov/regulatory-information/search-fda-guidance-documents/analytical-procedures-and-methods-validation-drugs-and-biologics.
4. International Council on Harmonisation. Validation of Analytical Procedures: Text and Methodology: Q2(R1). Geneva Switzerland: ICH; 2005. https://www.ich.org/leadmin/Public_Web_Site/ICH_Products/Guidelines/Quality/Q2_R1/Step4/Q2_R1__Guideline.pdf.
5. Department of Defense. Environmental Engineering Considerations and Laboratory Tests (MIL-STD-810G). Washington, DC: DoD;2008. everyspec.com/MIL-STD/MIL-STD-0800-0899/MIL-STD-810G_12306. 
6. International Electrotechnical Commission. Environmental Testing—Part 1: General and Guidance (IEC 60068-1). Geneva, Switzerland: IEC; 2013. https://webstore.iec.ch/publication/501. 
7. American Society for Testing Materials. Standard Test Method for Drop Test of Loaded Containers by Free Fall (ASTM D5276–98). West Conshohocken, PA: ASTM International;2017. www.astm.org/Standards/D5276.htm. 
8. US Pharmacopeial Convention. Analytical Data—Interpretation and Treatment 〈1010〉. In: USP 41. Rockville, MD: USP;2018:6706–6721.
9. US Pharmacopeial Convention. Statistical Tools for Procedure Validation 〈1210〉. In: USP 41. Rockville, MD: USP;2018:7622–7633.
10. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, Leeang MM, Sterne JA, Bossuyt PM. QUADAS-2: A revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. October 2011;155(8):529–536. 
11. American Society for Testing Materials. Standard Practice for Calculating and Using Basic Statistics (ASTM E2586–16). West Conshohocken, PA: ASTM International;2016. www.astm.org/Standards/E2586.htm.. 
12. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: A updated list of essential items for reporting diagnostic accuracy studies. BMJ. October 2015. 351(1):h5527. 
13. Food and Drug Administration. Statistical Guidance on Reporting Results from Studies Evaluating Diagnostic Tests. Rockville, MD: FDA;2007. www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm071287.pdf.
14. International Organization for Standardization. Statistics—Vocabulary and Symbols—Part 2: Applied Statistics (ISO 3534–2:2006). Geneva, Switzerland: ISO; 2006. www.iso.org/standard/40147.html. 
15. International Organization for Standardization. Statistics—Vocabulary and Symbols—Part 3: Design of Experiments (ISO 3534–3:2013). Geneva, Switzerland: ISO;2013. www.iso.org/standard/44245.html. 
16. International Organization for Standardization. Accuracy (Trueness and Precision) of Measurement Methods and Results—Part 1: General Principles and definitions (ISO 5725-1:1994). Geneva, Switzerland: ISO;1994. www.iso.org/standard/11833.html.