Technology today is personalized. Fueled by massive increases in computing power and data, industries from film to insurance to healthcare are investing in products and services that seek to cater to individuals’ preferences, skills, and background. This is a guide to the emerging, personalized, technology-enhanced future of medical education, precision medical education (or PME).
One45 was founded in 2001 — 19 years ago. Consider some of the changes in technology in that time:
- In 2001, you looked up driving directions on MapQuest, printed them out, and hoped that if you stayed the course those directions would steer you correctly to your destination.
- Today, Google Maps predicts your commute time to work for you and gives you an optimized route with voice-propelled turn by turn directions. It will even offer up modifications to that route as you go based on shifting traffic patterns.
- In 2001, you drove to Blockbuster and spent hours, or at least I did, browsing shelves for an interesting-looking movie, only to be disappointed with the movie and the late fees when you finally returned it.
- Today, Netflix uses your viewing history to predict shows you’ll like. It offers an almost unlimited set of options to you based on those predictions. It funds movies and TV series based on those predictions so that it can make more exciting viewing content, and the cycle repeats.
- In 2001, Google was one of many search engines. Search results were littered with spam, and Internet directories like Yahoo! played an important role in making sense of the web.
- Today, Google has so much data on your searching and browsing patterns (from Chrome and Google Search) that it often seems to know what you want before you do.
Even the social sciences are embracing these trends. This Nature article is a good overview.
The element that unites the technology experiences of today is personalization. Advances in computing power, massive increases in data and the rise of deep learning and machine learning now permit the mass customization of experiences for everyone within reach of a mobile device.
In healthcare, a direct concern to medical education, the personalization transformation is well underway. Academic medical centers like Stanford Medicine and Cumming School of Medicine have announced “precision health” initiatives, among many others. So has the NIH. Precision health is the idea that you can predict and prevent disease, and personalize care, by tapping into ever increasing and available volumes of health data. A growing list of companies, such as Medial EarlySign, have launched businesses designed to train machine learning algorithms on health data sets, and to tap into this opportunity.
Precision health is an idea whose time, not without serious considerations, has come. Precision medical education is next.
Definitions & Context
Before exploring the changes precision medical education will bring to schools, let’s define what we’re talking about and put some context around this moment in time.
NOTE: Much of the content that follows flows directly from conversations I’ve had over the past two years with Dr. Douglas Miller of the Medical College of Georgia. Dr. Miller has published widely on topics related to precision health and precision medical education. See examples here, here, here, and here. He is an expert on the topic of AI in education and medicine; I owe him a great debt of thanks for his contribution to the ideas in this article.
What is Precision Medical Education?
Precision Medical Education, or PME, is a shifting set of technologies without, as yet, a clear definition. Let me venture one here:
Precision Medical Education is an ethical, science-based approach to optimizing and individualizing the training and career outcomes of medical professionals based on data, data science, and research.
As this is one of the first articles to propose a definition for PME, Dr. Miller and I chose this definition carefully. To be in any way useful to its ultimate beneficiaries, our belief is that Precision Medical Education must be:
- ethical: A foundational element of a precision service is the assumption that vast quantities of individuals’ data will be used by algorithms to make individualized predictions. This is certainly the case with PME, but this data-centric view of individuals is rife with problems. Ethics, and an adaptive view of ethics when it comes to algorithmic bias, is essential in any working definition of PME.
- science-based: Medical education is built on education research. In some ways, the development of PME algorithms is like the “translational research” of medical education. Data science can translate educational research into practice in novel ways. As such, the duty to be open to criticism, to collaborate and test your work, and to live by a scientific code is critical when doing PME.
- aimed at improving training outcomes: Training can be rated in terms of time, quality, and cost. The goal of PME is to decrease the cost of training, to reduce the time of training if possible, while increasing quality.
- aimed at optimizing career outcomes: PME aims to positively influence the choices that confront medical trainees in medical school, residency, and beyond. There are obvious and non-obvious links between an individual’s choice quality and her future resiliency and likelihood of burnout, job satisfaction, and earning potential. PME seeks to tease out these links and provide a clear decision-support system to aid those individuals.
- individualized: PME seeks to provide a personalized experience to each trainee based on that individual’s particular attributes and preferences, as reflected in their data and by their actions.
- based on data, data science, and research: Today, PME and other precision initiatives rely mostly on data and data science to provide personalized insights. This is the core technical change that makes PME possible today. At the same time, a heavy reliance on “just data” is too narrow. A working definition of PME should leave the door open to other, non-data-centric aspects of the learner experience — whatever they may be and whenever they arise.
Who is Precision Medical Education For?
Precision Medical Education is designed to benefit the learner and the patients she serves. Full stop. To varying degrees, PME will also impact medical schools, healthcare providers, industry associations, and other stakeholders. This is normal and predictable with any paradigm change. Due to the pervasiveness of technology, it’s also unavoidable. But we must never forget who we’re serving.
Just as the patient must be at the center of personalized medicine, the learner and the patients she serves must be at the heart of precision medical education.
Why Precision Medical Education now?
In order to make a mass personalization effort work, you need:
- Computing power
- Prediction and personalization algorithms (machine learning, deep learning, AI)
- Appropriate data sets for training of algorithms
- Capable people to responsibly interpret data and guide algorithmic results
Today, computing power is basically free. The algorithms are mostly free, too. Open source tools like TensorFlow from Google are constantly being released and updated. The technology is here now.
But does medical education have appropriate data for PME? Are there people with sufficient knowledge of medical education AND data science that can responsibly bring some of these changes about? Medical education is approaching critical mass on addressing these two key questions.
Medical Education Data Is Primed for Algorithmic Use
Medical education has a long history with standards: accreditation standards, standard competency sets, work standards, curriculum standards. Medical education data tied to standards has largely been captured digitally into internal and external data instruments over the last 20 years. Due to the accreditation process, it’s mostly available for on-demand reporting right now.
Machine learning algorithms need structured data sets to train on in order to generate intelligent insights. Medical education has been building comprehensive, structured digital datasets for the past 20 years. As more medical schools are impelled to make better use of their data, their operational datasets will become increasingly suitable for advanced analytics and precisely modeled recommendations.
Research and researchers are emerging that will inform PME
To do precision health, you need health data (mostly genomics) and physicians working alongside data scientists. Instead of health data, PME requires medical education data. Instead of physicians working with data scientists, PME requires medical education researchers and practitioners working with data scientists.
As medical education research goes, so too do the technology-infused tools and approaches to problems in the med-ed industry. Just like the research of Olle ten Cate and others on Entrustable Professional Activities (EPAs) is leading to a wholescale reorientation towards more competency-based medical education (CBME), medical education data science research will lead to an embrace of the educational methods required for PME.
A few examples of recent papers highlight a growing trend:
A Call to Investigate the Relationship Between Education and Health Outcomes Using Big Data (link)
Saad Chahine, Kulamakan Mahan Kulasegaram, Sarah Wright, Sandra Monteiro, Lawrence E M Grierson, Cassandra Barber, Stefanie S Sebok-Syer, Meghan McConnell, Wendy Yen, Andre De Champlain, Claire Touchie.
Published in 2018, this paper (and three related papers) call for more investigation into the link between education and health outcomes using advanced analytics. The papers highlight the growing volumes of healthcare data that can be tied back to individual clinicians. The intersection of clinical data and training data will be a key area of focus for anyone interested in PME.
Signatures of medical student applicants and academic success (link)
Tal Baron, Robert I. Grossman, Steven B. Abramson, Martin V. Pusic, Rafael Rivera, Marc M. Triola, Itai Yanai
Published in 2020, this paper uses machine learning approaches like K-Means and logistic regression to cluster students into distinct groups with unique characteristics. From the abstract (emphasis mine): “The big data approach presented here can more sensitively uncover success potential since it takes into account the inherent heterogeneity within the student population.” This highlights another future aspect of PME: tying student characteristics and behaviors to eventual learning outcomes.
Use of a Machine Learning Algorithm to Classify Expertise: Analysis of Hand Motion Patterns During a Simulated Surgical Task (link)
Watson, Robert A., MBChB, FRCS(Eng)
This third paper, published in 2014, highlights a different set of meta-data from simulation and skill-building that may be useful for PME.
These papers are examples of research into three key areas of PME:
- The link between education choices and healthcare outcomes
- The link between student attributes when entering medical school with future academic outcomes
- The potential for more and more automated forms of performance assessment to drive more data into the system
They also show a good cross section of both medical expertise, medical research expertise, and data science expertise.
What these papers imply is that schools are starting to recognize the importance and opportunity of PME-related work. They are increasingly looking at their data as an important, structured asset. They are collaborating with (or training) people with advanced data capabilities.
We’re at a moment where the people and the data sets necessary to achieve something approaching precision medical education are coming together. PME will soon be at the forefront of the discussion in medical education. Let’s turn our attention to how medical schools should prepare for the PME future today.
Implementing Precision Medical Education
The technology trends today are inexorable. Precision medical education is just around the corner. Your institution needs to be ready. Here are the most important transitions Dr. Miller and I believe medical education programs will experience as we move to a future of personalized medical education.
Data Integration & Decision Support
The first transition most schools will make towards PME is from data silos and “what’s available” decision-making to data integration and decision support.
Training programs collect a lot of data. The advent of web- and mobile-enabled systems has made collecting data on all aspects of the learner experience increasingly easier over the past two decades. But, for most programs, we’ve done little more than replace paper stacks with browser tabs.
The accreditation process is still the canonical example of this: a self-survey or full accreditation is an incredibly labor-intensive process: extracting data from source systems, gathering ad-hoc data, cleaning data, manually tagging and linking data. Medical education programs undergoing accreditation often hire extra staff and spend huge amounts of time and money just to get data ready for pre-visit analysis and review. The process is challenging because there is a lot of data, but also because accreditation standards (rightly so!) seek to get a balanced view from multiple data sources. This requires extra work manually massaging and cross-linking data.
Imagine if the data required for accreditation, which is already collected and properly structured by schools, could be coded to accreditation standards and centralized into a clean data warehouse every day. The accreditation process could move from a one-time, stressful experience, to a real-time dashboard focused on guided analysis. That same dashboard could be used to inform decisions on curriculum review, assess learner performance, preview program outcomes and eventually even guide financial analysis and research agendas.
This is the central benefit of data integration: management decision support. Data integration is the key first step on the road to PME because a concerted effort to use integrated data to inform decisions forces a large set of habit changes inside an organization. It forces staff to improve their data literacy, it forces cyclical review processes to incorporate more sophisticated points of view, and it causes the types of conversations necessary for the change from reactive, “months after the fact” decision-making to real time adaptation. It makes required continuous quality improvement (CQI) truly continuous.
One45 built our data analytics platform in part to help further this transition.
Data Management & Governance
A concerted effort to use integrated data for decision making will cause one issue to raise its head almost immediately: data management & governance. Data management is the set of activities necessary to keep data healthy and well structured. Data governance is the set of activities designed to control access to data and to define sources of truth and real-world data definitions.
Both of these processes are underdeveloped in organizations that largely rely on data from operational systems, as is the case in most medical schools. This makes sense: most software systems in medical education are well evolved and have good permissioning systems for different users.
But: as data starts to flow from these systems to a single point of integration, it becomes much more likely that data sources will conflict with each other. Privacy and security risks rise, as well, as the base level for user access becomes the data view vs. the operational software system.
These issues are a natural outgrowth of an increasing focus on data, and there are well established processes for dealing with them. If you’re interested in learning a bit more about how one school did this, I recommend reading the background on Indiana University’s Decision Support Initiative.
Recognizing this issue ahead of time and acting proactively can, in our experience, save a lot of headaches.
Strategic Integration & A Point of View
This transition is the most important. In our work developing data warehouses and dashboarding systems for medical education, One45 has seen very successful data integration and decision support projects sponsored and led by IT, the Office of Medical Education, and Student Affairs.
In fact, it’s quite possible for a single group in a medical school to reap the benefits of data integration without buy-in from their peers. Most individual offices, on their own, deal with at least 5-10 software systems. There are clear benefits from local data integration and decision support projects.
This is certainly a workable model, and it avoids some of the issues around data management and governance raised above. But it will not lead to the adaptations required to support precision medical education.
Precision medical education needs to be a central pillar in a medical school’s strategic plan — just as precision health is the strategic focus of many academic medical centers.
Institutional leaders need to develop a progressive point of view on value of the emerging technologies referenced here, even if that point of view varies across organizations. Why? If implemented according to the principles above, PME is a worthy goal for benefiting learners, patients, and those who support them. But the technology trends which make PME possible also make aspects of it inevitable. Most staff and faculty will be asked to make more decisions based on data. Your learners will be exposed to data-informed decision making tools and methods in your curriculum. Teams will still start their own data integration projects unbeknownst to you.
Each organization must have a strategic priority assigned to these issues, one that makes it clear how investing precious organizational resources will address them.
The next transition is a cultural one: from data literacy to data fluency.
One thing I enjoy most about working with medical educators is the high level of data literacy that each person possesses. Nearly everyone has a working knowledge of statistics; conversations can be nuanced and extremely intellectually rigorous. So, for most schools, the transition from data literacy to data fluency is an evolutionary one, not a revolutionary one.
The distinction between data literacy and data fluency in my mind is one of point in time analysis to real-time adaptation. Being data literate means being able to analyze a set of data and draw useful conclusions about what happened. Being data fluent means being able to turn those conclusions into testable hypotheses and predictions about what will happen and to do so “live”. The “improvement” arm of continuous quality improvement (CQI) under PME is all about data fluency.
Becoming data fluent means the rate of change in your institution will go up. It also means that your staff will need to be conversant in algorithms vs static statistical analysis. It means training, a change in hiring priorities, and (as covered above) strategic support.
Leaders have a fantastic resource to draw upon here: researchers. Research is fundamentally about hypothesis testing and study design. Those skills need to be adapted for the faster pace of PME — but this is a difference in degree not in kind. A strong, data fluent research organization can help drive cultural change.
Data Sharing and Real Precision Medical Education
The final transition is the actual work required to bring PME to life. PME, truly learner-centric PME, will only be achieved through data sharing among schools and across levels of training, between medical education and healthcare and industry partners. Data sharing, as it pertains to healthcare and basic science research, is a networking model familiar to medical schools and research universities. In medical education, however, institutions often see their data as a protected asset; as a result, it is sub-optimally used for collaborative science designed to further learner outcomes.
Data sharing is the final frontier: it means that a school is secure enough in its own data policies, management, and capabilities that it is willing to put its data to work for the benefit of all learners.
The future of medical education is personalized. It is rooted in data and data science, but must be ethical and open to honest scientific inquiry. It is focused on improving training and career outcomes for individual learners. It recognizes that learners and the patients they serve are the sacred heart of this future and it puts them first, always.
The future of medical education will only come through transformation. Medical schools and residency programs must undertake the hard work of becoming data-informed decision makers. They must address the critical issues of data governance and data management. Leaders must make technology, data, and data science core to their strategy. They must lead their organizations from data literate to data fluent. And, finally, they must open their data to others for the benefit of all learners.
The future of medical education is precision medical education.