5 Questions is a periodic feature created by Cornerstone Research that asks our experts, senior advisors or affiliate experts to answer five questions.
We interview Mike DeCesaris, Vice President of Cornerstone Research’s Data Science Center, about the benefits of working with GPUs and how they can improve artificial intelligence (AI) and machine learning (ML) technologies.
What are GPUs?
Dedicated graphics processing units (GPUs), as the name suggests, were originally designed decades ago for the efficient performance of simple operations related to processing images and videos. These processes mostly involve matrix-based mathematical calculations. People are more familiar with central processing units (CPUs), which are commonly found in laptops, phones and smart devices, and can perform a wide variety of operations.
In the early 2000s, researchers realized that GPUs could provide a more efficient alternative to CPU-based computation for machine learning, as machine learning algorithms often involve the same computations as graphics processing algorithms. Despite availability and cost limitations regarding CPUs in recent years, GPU-based computation has become the de facto standard for machine learning or neural network training.
What are the benefits of using GPUs?
The main advantage is efficiency. The computing power provided by GPUs does more than streamline the analytical process. This facilitates more extensive model training for greater accuracy, expands the scope of the model search process to protect against alternative specifications, makes previously impossible specific models possible, and allows additional sensitivities on alternative datasets to ensure robustness.
How do GPUs support expert testimony?
AI-based systems replace human decisions with data-based ones. It reduces subjectivity and error when processing large volumes of complex information. We use AI and ML to drive automation of increasingly complex tasks and unlock new approaches to analysis, including using supervised and unsupervised learning. These technologies are supported by our internal GPUs.
How will the data science center impact GPU computing?
We use GPUs at all stages of the case lifecycle, from discovery to financial analysis and for all types of data, from standard tabular data to text and images. Some of these applications rely on popular applications of GPU computing, such as neural networks, while others rely on customized analytical frameworks. A few examples follow.
GPUs help us perform custom matrix arithmetic at a faster speed. For example, in antitrust matters, we often need to calculate the distance between all suppliers and all consumers (coordinate pairs). Shifting this calculation from CPUs to GPUs can calculate distances between approximately 100 million coordinate pairs per second.
Deep Neural Networks
Much of the excitement surrounding GPU-based computing focuses on neural networks. Although capable of handling general classification and regression problems, additional task-specific neural network architectures provide a framework for specialized analyzes of text, images, and sound. Due to the complexity of these models and the amount of data required to generate reliable results, their use is not feasible without GPU computing resources. When training a popular multi-class image model on a GPU, we experienced a 25,000% speedup compared to running the same process on a single CPU. We leverage this capability in content analytics for consumer fraud topics, where we design text and image classifiers to classify the intended audience of at-issue marketing materials.
As GPU computing becomes more ubiquitous, popular machine learning software packages increasingly include GPU-based computation options in their offerings. We often use spanning trees in regression and classification problems. These models combine several simple decision trees sequentially into one larger, more accurate learner. Compared to deep neural networks that can have hundreds of millions of parameters, these models are small and therefore require less data and training time to make generalizable inferences. These advantages make them more useful than deep neural networks for many types of analysis that we often encounter. Switching to GPU-based training processes allows us to train models for these tasks almost 100 times faster than the corresponding CPU specification.
Linguistic models, often based on one or more deep learning techniques, can categorize, parse, and structure text. We use large linguistic models to replace traditional word-based features in text classification problems such as extracting specific information, extracting relationships between entities, identifying semantic relationships, and quantifying social media sentiment regarding a public entity in defamation matters.
Unsurprisingly, given all these models can do, processing documents through these models by CPU can cause significant delays in the analytical process. With just a single GPU, we can split documents into individual parts and fully process several hundreds of sentences per second.
What developments can we expect in this space in the future?
GPUs and GPU-related software continue to evolve. Newer hardware may have more cores, faster cores, and more memory to accommodate larger models and data batches. New software may make it even easier to share models and data across multiple GPUs.
Other developments may involve completely different devices. To address some of the inefficiencies still present in GPU computing, machine learning practitioners have increasingly turned to application-specific integrated circuits (ASIC) and field-programmable gate arrays (FPGAs). For example, Google’s Tensor Processing Unit (TPU) is an ASIC designed specifically to perform calculations for its TensorFlow software package for machine learning. FPGAs offer more flexibility and are commonly used to implement machine learning models in production environments that require low latency, high bandwidth, and minimal power consumption.
We monitor developments in this space to ensure we continue to provide best-in-class service to our clients and professionals.