Real World Test Plan

2024 REAL WORLD TESTING PLAN

GENERAL INFORMATION

Developer Name: DrFirst, Inc.
Product Name(s): Rcopia4 Web
Version Number(s): 4
Certified Health IT
Product List (CHPL) ID(s): 15.04.04.1375.Rcop.04.00.0.171227
Developer Real World Testing Page URL: www.drfirst.com/realworldtestplan

JUSTIFICATION FOR REAL WORLD TESTING APPROACH

Our approach to Real World Testing (RWT) verifies that production users can send and receive electronic prescribing messages without error through system-level transaction monitoring.

STANDARDS UPDATES (INCLUDING STANDARDS VERSION ADVANCEMENT PROCESS-SVAP AND USCDI)

• We are using NCPDP SCRIPT version 2017071 for all electronic prescribing and medication history messages
• This update applies to certification criterion 170.315 (b)(3) Electronic Prescribing
• The module CHPL ID is 15.04.04.1375.Rcop.04.00.0.171227 (Rcopia 4)
• Measures used in overall approach
  

DESCRIPTION OF MEASUREMENT/METRIC

Describe the measure(s) that will be used to support the overall approach to Real World Testing.

• Artifact is a document that captures the values for the measures listed below
• All checks measure electronic messages sent or received only using the NCPDP SCRIPT 20170715
• Metric is the percentages of:
  • New prescriptions sent electronically and verified vs. all sent 
  • Pharmacy renewal responses sent and verified vs. all sent 
  • Pharmacy change responses sent and verified vs. all sent 
  • Pharmacy fill messages received  vs. all new prescriptions electronically verified  
  • Medication history requests sent and acknowledged vs. all sent 
  • New prescriptions sent electronically, with patient diagnosis vs. without diagnosis. 
  • Controlled substance prescriptions sent electronically vs. printed

ASSOCIATED CERTIFICATION CRITERIA

JUSTIFICATION FOR SELECTED MEASUREMENT/METRIC

The ability of a practitioner to communicate with a pharmacy using a standard electronic messaging format is a core feature of Rcopia and is essential for compliance with criterion 170.315 (b)(3).  The chosen metric measures the degree to which our system conforms to the NCPDP SCRIPT 20170715 messaging format by displaying the success rate of each type of supported message. 

CARE SETTING(S)

EXPECTED OUTCOMES

• We expect electronic transaction success rates to consistently be over 95%.
• We expect the rates of inclusion of patient diagnoses to increase year over year.
  

SCHEDULE OF KEY MILESTONES

ATTESTATION

Authorized Representative Name: Peter Dahlberg
Authorized Representative Email: pdahlberg@drfirst.com 
Authorized Representative Phone: 301-231-9510 ext. 3227 
Authorized Representative Signature: Peter Dahlberg 
Date: 10/18/2023

2023 REAL WORLD TESTING RESULTS

GENERAL INFORMATION

Developer Name: DrFirst, Inc.
Product Name(s): Rcopia4 Web
Version Number(s): 4
Certified Health IT Product List (CHPL) ID:  15.04.04.1375.Rcop.04.00.0.171227
Criterion: §170.315(b)(3) Electronic Prescribing 
Real World Testing Page URL:  www.drfirst.com/realworldtestplan
Testing Period: 3/26/2023 - 7/1/2023

CHANGES TO ORIGINAL PLAN

SUMMARY OF TESTING METHODS AND KEY FINDINGS

CARE SETTING: AMBULATORY INTERNAL MEDICINE

CARE SETTING: AMBULATORY BEHAVIORAL HEALTH

TESTING OUTCOMES

• The three most prevalent transactions succeeded at rates above the expected rate of 95%. These transactions are for new prescriptions (including controlled substances), renewal responses, and medication history requests.
• The success rate for change request responses fell below 95% at 76.3% (Ambulatory Internal Medicine) and 86.0% (Ambulatory Behavioral Health).
• The rate at which providers included diagnosis codes on prescriptions became more consistent between specialties this year. The rate for Ambulatory Internal Medicine increased (4.9% in 2022 and 14.1% in 2023) whereas the rate for Ambulatory Behavioral Health decreased (44.6% in 2022 and 13.4% in 2023).

TESTING CHALLENGES

2023 REAL WORLD TESTING PLAN

GENERAL INFORMATION

Developer Name: DrFirst, Inc.
Product Name(s): Rcopia4 Web
Version Number(s): 4
Certified Health IT Product List (CHPL) ID:  15.04.04.1375.Rcop.04.00.0.171227
Real World Testing Page URL:  www.drfirst.com/realworldtestplan

JUSTIFICATION FOR REAL WORLD TESTING APPROACH

Our approach to Real World Testing (RWT) verifies that production users can send and receive electronic prescribing messages without error through system-level transaction monitoring.

STANDARDS UPDATES (INCLUDING STANDARDS VERSION ADVANCEMENT PROCESS-SVAP AND USCDI)

• We are using NCPDP SCRIPT version 20170715 for all electronic prescribing and medication history messages. 
• This update applies to certification criterion 170.315 (b)(3) Electronic Prescribing. 
• The module CHPL ID is 15.04.04.1375.Rcop.04.00.0.171227 (Rcopia 4). 
• Measures used in overall approach

2022 REAL WORLD TESTING RESULTS

GENERAL INFORMATION

Developer Name: DrFirst, Inc.
Product Name(s): Rcopia4 Web
Version Number(s): 4
Certified Health IT Product List (CHPL) ID:  15.04.04.1375.Rcop.04.00.0.171227
Real World Testing Page URL:  www.drfirst.com/realworldtestplan
Testing Period: 4/1/2022 - 8/31/2022

CHANGES TO ORIGINAL PLAN

SUMMARY OF TESTING METHODS AND KEY FINDINGS

Testing was conducted by monitoring e-prescribing transmissions to detect rates of success for the NCPDP SCRIPT transactions that are required by ONC. The set of transactions was limited to those made by DrFirst retail clients to exclude traffic generated through affiliated EHR clients that hold their own ONC certifications.  

2022 REAL WORLD TESTING PLAN

GENERAL INFORMATION

Plan Report ID Number: [For ONC-Authorized Certification Body use only]
Developer Name: DrFirst, Inc.
Product Name(s): Rcopia4 Web
Version Number(s): 4
Certified Health IT Product List (CHPL) ID:  15.04.04.1375.Rcop.04.00.0.171227
Real World Testing Page URL:  www.drfirst.com/realworldtestplan

JUSTIFICATION FOR REAL WORLD TESTING APPROACH

Provide an explanation for the overall approach to Real World Testing, including an outline of the approach and how data will be used to demonstrate successful Real World Testing¹.

All measures should reasonably align with the elements within a Real World Testing plan, the scope of the certification, the types of settings in which the Certified Health IT is marketed, and other factors relevant to the implementation of the Certified Health IT Module(s). The justification should reflect how each element within the plan is relevant to the developer’s overall strategy for meeting the Real World Testing Condition and Maintenance of Certification requirements.

Note: A single Real World Testing Plan may address multiple products and certification criteria for multiple care settings.

Our approach to Real World Testing (RWT) verifies that production users can send and receive electronic prescribing messages without error through system-level transaction monitoring.

STANDARDS UPDATES (INCLUDING STANDARDS VERSION ADVANCEMENT PROCESS-SVAP AND USCDI)

Both required and voluntary standards updates must be addressed in the Real World Testing plan. Real World Testing plans must include all Certified Health IT updated to newer versions of standards prior to August 31 of the year in which the updates were made.

Describe approach(es) for demonstrating conformance to all certification requirements using each standard to which the Health IT is certified. List each version of a given standard separately. For each version of a standard submit the following:

• Identify standard versions
• Indicate what certification criteria in which product(s) has been updated
• If reporting for multiple products, identify the certification criteria that were affected by the updatefor each of the associated products
• CHPL ID for each Health IT Module
• Method used for standard update (e.g., SVAP) Date notification sent to ONC-ACB
• If SVAP, date notification sent to customers
• Measure used to demonstrate conformance with updated standard{s)
• Which certification criteria were updated to USCDI and/or to which version of USCDI was the certification criteria updated?
• We are using NCPDP SCRIPT version 20170715 for all electronic prescribing and medication history
• This update applies to certification criterion 315 (b)(3) Electronic Prescribing.
• The module CHPL ID is 15.04.1375.Rcop.04.00.0.171227 (Rcopia 4).
• The update is covered under SVAP.

MEASURES USED IN OVERALL APPROACH

Each plan must include at least one measurement/metric that addresses each applicable certification criterion in the Health IT Module’s scope of certification. Describe the method for measuring how the approach(es) chosen meet the intent and purpose of Real World Testing.

For each measurement/metric, describe the elements below:

• Description of the measurement/metric
• Associated certification criteria
• Justification for selected measurement/metric
• Care setting(s) that are addressed
• Expected Outcomes

DESCRIPTION OF MEASUREMENT/METRIC

Describe the measure(s) that will be used to support the overall approach to Real World Testing.

• Artifact is a document that captures the values for the measures listed
• All checks measure electronic messages sent or received only using the NCPDP SCRIPT 20170715
• Metric is the percentages of:
  • New prescriptions sent electronically and verified all sent
  • Pharmacy renewal requests received and verified all received
  • Pharmacy renewal responses sent and verified vs. all sent
  • Pharmacy change requests received and verified vs. all received
  • Pharmacy change responses sent and verified all sent
  • Pharmacy fill messages received and verified vs. all received
  • Medication history requests sent and acknowledged all sent
  • Medication history responses received and acknowledged vs. all received
  • New prescriptions sent electronically for juvenile patients, with patient weight without weight.
  • New prescriptions sent electronically, with patient diagnosis vs. without
  • Controlled substance prescriptions sent electronically printed

ASSOCIATED CERTIFICATION CRITERIA

List certification criteria associated with the measure and if updated to 2015 Edition Cures Update criteria.

315 (b)(3} Electronic Prescribing

JUSTIFICATION FOR SELECTED MEASUREMENT/METRIC

Provide an explanation for the measurement/metric selected to conduct Real World Testing.

The ability of a practitioner to communicate with a pharmacy using a standard electronic messaging format is a core feature of Rcopia and is essential for compliance with criterion 170.315 (b)(3). The chosen metric measures the degree to which our system conforms to the NCPDP SCRIPT 20170715 messaging format by displaying the success rate of each type of supported message.

CARE SETTING(S)

The expectation is that a developer’s Real World Testing plan will address each type of clinical setting in which their Certified Health IT is marketed. Health IT Developers are not required to test their Certified Health IT in every setting in which it is marketed for use. Developers should address their choice of care and/or practice settings to test and provide a justification for the chosen approach.

Note: Health IT Developers may bundle products by care setting, criteria, etc. and design one plan to address each, or they may submit any combination of multiple plans that collectively address their products and the care settings in which they are marketed

List each care setting which is covered by the measure and an explanation for why it is included.

EXPECTED OUTCOMES

Health IT Developers should detail how the approaches chosen will successfully demonstrate that the Certified Health IT:

1. is compliant with the certification criteria, including the required technical standards and vocabulary codes sets;
2. is exchanging EHi in the care and practice settings for which it is marketed for use; and/or,
3. EHi is received by and used in the Certified Health

(from 85 FR 25766)

Not all of the expected outcomes listed above will be applicable to every Certified Health IT Module, and Health IT Developers may add additional description of how their measurement approach best addresses the ongoing interoperability functionality of their product(s). Health IT Developers could also detail outcomes that should not result from their measurement approach if that better describes their efforts.

Within this section, Health IT Developers should also describe how the specific data collected from their Real World Testing measures demonstrate expected results. Expected outcomes and specific measures do not necessarily have to include performance targets or benchmarks, but Health IT Developers should provide context for why specific measures were selected and how the metrics demonstrate individual criterion functionality, EHi exchange, and/or use of EHi within Certified Health IT, as appropriate.

• We expect electronic transaction success rates to consistently be over 95%.
• We expect the rates of inclusion of juvenile patient weight and of patient diagnoses to increase year over

SCHEDULE OF KEY MILESTONES

Include steps within the Real World Testing plan that establish milestones within the process. Include details on how and when the developer will implement measures and collect data. Key milestones should be relevant and directly related to expected outcomes discussed in the next section.

For each key milestone, describe when Real World Testing will begin in specific care settings and the date/timeframe during which data will be collected.

ATTESTATION

The Real World Testing plan must include the following attestation signed by the Health IT Developer Authorized representative.

Note: The plan must be approved by a Health IT Developer authorized representative capable of binding the Health IT Developer for execution of the plan and include the representative’s contact information.²

This Real World Testing plan is complete with all required elements, including measures that address all certification criteria and care settings. All information in this plan is up to date and fully addresses the Health IT Developer’s Real World Testing requirements.

¹ Certified Health IT continues to be compliant with the certification criteria, including the required technical standards and vocabulary codes sets; Certified Health IT is exchanging EHi in the care and practice settings for which it is marketed for use; and EHi is received by and used in the Certified Health (85 FR 25766)

² https://www.federalregister.gov/documents/2020/05/01/2020-07419/21st-century-cures-act-interoperability-information-blocking-and-the-onc-health-it-certification#p-3582