DeepView Architecture Overview¶

This page contains an overview of the DeepView architecture and how pieces work together. Please see the following three pages for more detailed information:

Architecture¶

An animated diagram illustrating the DeepView pipeline. A single batch at a time is fed through the entire pipeline, from Producer to Introspector.

DeepView begins with a Producer that is in charge of generating Batches of data for the rest of DeepView to process or consume.

DeepView only loads, processes and consumes data when it needs to. This is known as lazy evaluation. Using lazy evaluation avoids processing, running inference and analyzing the full dataset in one fell swoop, as these operations can easily require more than the system’s available memory.

Instead, a Producer generates small Batches of data, which can be processed and analyzed with the available memory as part of a DeepView pipeline. A pipeline is a composition of Batch transformations that we call PipelineStages.

Example PipelineStages could be various pre- or post- Processors, or Model inference. These all apply transformations to a Batch and output a new Batch (typically model responses).

Finally, DeepView’s Introspectors will analyze input Batches (usually Batches of model responses) to provide insights about the dataset and/or model.

Below shows a generic and example DeepView pipeline setup.

GENERIC

EXAMPLE

A picture of a specific sample DeepView pipeline. Starting with an Image Producer yields batches of images (one batch at a time). The Batch then goes through various Pipeline Stages: First an Image Resizer, then a Mean / Std Normalizer, ResNet Model inference, and a Pooler. At this point, the Batches contain pooled model responses, and they are (one at a time) fed into the IUA introspector.

See the next sections in these How To Guides for information about each of DeepView’s components.