Rob Oakes
Apr 22, 2021

Understanding AI in Healthcare

What are the types of machine learning/AI and how are they used in the clinic?

AI can be categorized into two main types: narrow AI and general AI. "General AI" is described as the ability of computers to be self-sufficient and have cognitive function similar with human ability. "Narrow AI" is used to perform specific, applied tasks such as identifying new drugs or analyzing images for the presence (or absence) of specific features. While General AI is the stuff of science fiction, Narrow AI is widely used within healthcare, pharmaceuticals, biotechnology, finance, and the energy sector.

Machine Learning

Machine learning (ML) is a specific type of Narrow/Applied AI and represents a family of techniques able to discover patterns within data by linking inputs to outputs through the use of an algorithm (called a learning model). There are three main forms of machine learning: supervised, unsupervised, and reinforcement learning. Supervised learning uses "labeled" information (for which an outcome is known) to generate outcomes on new data using a model. Unsupervised learning looks for patterns within data and attempts to organize observations into categories based on the similarities and differences within the information.

In a classical ML pipeline, image features such as the textures, histograms, geometries, and morphologies are extracted explicitly as features before the data is passed to a classification model.

Deep Learning

Deep learning is a special type of ML that uses multilayer neural networks in the classification of data and is used widely in medical imaging and computer vision. Neural networks are distinct from other forms of machine learning in that they can detect non-linear relationships between the input variables and the outcomes using various "hidden layers." Their architecture allows for neural networks to detect more abstract patterns than other ML algorithms, and successfully navigate more complex tasks.

In deep learning pipelines, features are extracted automatically using convolution filters and pooling. The computer is able to learn complex patterns directly from the data without requiring explicit feature extraction, which allows for the model to recognize abstractions similar to a trained expert.

Deep Learning models are well suited to analyze the types of data created in modern medical workflows: raw medical images, annotations, and reports. Through the use of electronic health records, it is possible to use tens of millions of records representing billions of data points without any lapse in attention and apply the models to make predictions in a uniform fashion.

The types of patterns that Deep Learning models are capable of detecting can be very subtle, and go beyond "mere" statistical inference. While training on the data, is is common for the model to recognize patterns in images or text that manifest in the form of shapes, three-dimensional structures, feeling/emotion, or other correlations that can't readily be explained.

The image below shows two samples from a model trained to detect objects in photos and a visualization generated from a library called SHAP (SHapley Additive exPlanations). SHAP is a tool used to explore Deep Learning models and try to understand why they return specific results. The two most likely predictions are shown for images of a dowitcher and meerkat, along with a visualization explaining why those labels were chosen. Red pixels represent positive contributions to a label while blue pixels represent negative contributions. The model has identified features of the two animals which distinguish them from other labels. The dark and slender bill of the dowitcher and the distinctive dark markings of the meerkat are powerful positive contributors for those labels, but are absent from the red backed sand-piper and mongoose (the second most likely labels).

Figure: Interpreting Deep Learning Features Using Shap
Deep Learning models are capable of detecting subtle patterns that go beyond statistical inference. The model above is able to distinguish between a meerkat and a mongoose by looking for the distinctive dark fur which surrounds a meerkat's eyes, but is absent from those of the mongoose.
Scott Lundberg | SHAP
Figure: SHAP Scores for Chest X-Ray
The image above shows a SHAP visualization a model trained on a large dataset of Chest X-Ray images released by the NIH in 2017. Deep Learning models are able to detect patterns in the that are indicative of complex lung diseases such as pneumonia (shown) and COVID-19.

Working at Scale

The role that AI plays in most technology systems is that of "data processor." It has a voracious appetite for data, the more the better, and is the fundamental technology that allows for the processing and classification of large volumes of information. Because models learn "directly from the data" and do not need to be explicitly programmed, they have a mechanism to "improve with experience" as they encounter new information.

Rob Oakes Apr 22, 2021
More Articles by Rob Oakes


Unable to find related content


Unable to retrieve data due to an error
No results found
Back to All Comments