A clinical data warehouse solution can effectively serve end users only if it is effectively adopted in practice by the target users. At Recombinant By Deloitte we have learned that a system that is functional and delivers large data volumes may have strong potential value, but if it is not used effectively or not useful then it will eventually cause the project to fail. Because user adoption is a major risk in a data warehouse or front-end application investment, we provide containing services to help determine that both new and existing solutions are effectively adopted among user populations.
Historically, common reasons for poor adoption rates include:
- Use of data from applications does not fit into existing or planned work flows
- Poor communication of changes and updates
- Lack of awareness of the system's tools and capabilities
- Lack of training/expertise to effectively utilize the data and tool
- Users do not see sufficient value in the resulting applications of the data
- Data loaded does not include enough relevant or complete content to answer pertinent questions
- Low quality data creates distrust of results and overall dissatisfaction of the warehouse applications
A number of support services are available for deployment to address these risks, enabling adoption to remain a center of focus throughout the lifecycle of the build-out. The services and service levels are constructed for our clients based on the scope and needs of each project.
Engaging Active Users
End-User Meetings
An agile software development approach requires frequent end-user meetings to get user feedback. This helps developers set direction and priorities during the iterative steps (a.k.a. 'sprints'). During the prototype and design phases, a small set of users are included in meetings to help direct each iteration through in-person feedback sessions. These meetings model the close user engagement which will be continued through the course of the project. This approach engages active users through:
- Increased commitment through design ownership by the end users
- Focuses developers on features that provide user value
- Promptly corrects low value efforts
- Specialist users who can help recruit and support new users
At the end of each iteration, when a set of data and tested code is available, face-to-face meetings are scheduled with identified users to preview the next software/data release candidate and solicit feedback on new features/improvements before the updates are promoted to the production systems. This feedback is used to prioritize the list of work items for the next iteration and to identify any bugs requiring specific domain experience to identify.
Once the user base grows beyond the initial size of identified stakeholders, scheduling face-to-face meetings may not be practical. Therefore, not all users are involved in the dialog for each iteration. User review meetings include identified stakeholders and are scheduled at the discretion of the project management team. The stakeholder groups evolve throughout the course of an initial build as new users are included in the dialog and divided into an advisory board and analysis specific focus groups.
Power User Development
A small set of power users is developed through their involvement in the acceptance testing and the roll-out of the solution. They perform the role of supporting end users who need peers to guide them through the application.
Release Notes
When updates are promoted into production systems, release notes are posted on the project wiki and sent to an opt-in distribution list. The contents of release notes include changes to the application and changes to available data.
New User Ramp-Up Tools
A number of packaged tools and services can be created to accelerate adoption, as follows:
- Training seminars
A team member/primary contact is designated for delivery of scheduled seminars either in person, or through web-based demonstrations. These include a basic training script and slides for the training seminars. The seminars are scheduled on an as-needed basis to teams or individuals looking to get started.
- End-user documentation
A web-based end-user help guide accompanies end-user applications regarding the functionality in each application. It is made available through a "help" link clearly labeled on the page. Where possible the help is directly embedded into the applications through links.
- Recorded tutorials
Walkthroughs of the applications are created through screen capture videos with a narrator providing details of workflows within the application. The tutorials are similar to training seminar content, but archived and available for download.
- Use cases/Test cases
The test cases used for the development are re-written as tutorials with step-by-step walk-throughs.
- Project wiki and feature tracker
Recombinant By Deloitte provides an internet enabled enterprise wiki and feature request tracker to clients, that can be extended to client sites or can use wiki tools hosted within client networks. The project wiki, available to all members within the client site can be updated by Recombinant By Deloitte staff with content on a regular basis by various members of the team and end-user community. The feature tracker is used to allow for triage and response to stakeholder requests throughout the deployment process.
- Consult offers
Since some users may be reluctant to engage in a dialog about specific questions regarding available data, the Data Request Manager (DRM) can allow them to work through a consult process. The team will offer to provide data consults through the reporting analyst(s) to answer specific questions for new and experienced users to support specific data analysis activities. This can both drive usage of the warehouse as well as frame new use cases for greater automation within front-end tools.
Measuring and Monitoring Effectiveness
In order to determine activity, a number of monitoring programs will be put into place, as follows:
- Quarterly adoption review
Creation of presentation content and statistics for adoption reviews on a quarterly basis by steering committees in a review meeting in order to set goals and review the current status. Facilitation of review activities.
- Usage reporting from application logs
Usage of the end-user applications can be logged at a transaction level. The usage logs feed a set of reports for determining the volume of use by production users, top users and teams, and areas in use by production users. These reports can be reviewed monthly by a business analyst to identify and communicate areas of improvement for adoption.
- Compiling positive or negative feedback
Both positive and negative feedback stories can be compiled in a project wiki and associated with the person providing the feedback and with the associated date. The feedback will at times be solicited and then compiled into a quarterly review document.
