Zentree Labs - Smart Model Optimization Tool for Edge Devices

Model Optimizer Tool

The very new paradigm of solving computer vision problems involves complex Convolutional Neural Networks (CNNs). When we try to solve problems in real-time, there lie different challenges. It would be even more challenging when we have to run the models on constrained environments like embedded edge devices. The key challenges are,

Deriving a new model based on the new objectives. It is majorly governed by application and deployment specifications
To fit the models given memory and bandwidth
Running multiple models for meaningful decisions
Low-bit representation of models without losing accuracy

Achieving these objectives involves high skill set and time consuming. The critical steps to make any model possible running on edge platforms meeting the practical requirements involve the following critical steps,

Re-design the model for the need
Pruning or Optimization of the model
Quantization of the model

Customizing the models to any specific need, based on the requirements, is incredibly challenging to get the models converged. We need an amazingly fast knowledge distillation procedure.

Model optimization is one of the major challenges when one wants to build applications on the edge side. The key objectives of the pruning is the reduction in the complexity of the model, which includes:

Number of parameters count should be brought down
Number of computes, GFlops should be reduced
Original Accuracy should be maintained or otherwise, ensure deviation within an acceptable range.

Model Cleaver is a very smart Resources & Context Aware Model Optimization tool, it offers automated knowledge distillation, optimization, & quantization.

Features

The features supported on the model optimizer tool are:

Automated CNN model optimization
Automated Model Quantization
ALMT – Automated Lean Model Training
Resource-aware optimization and quantization
On-demand data annotations by crowdsourcing
On-demand data collection through crowdsourcing

Advantages

Fully Automated Solution (Lean Model as output)
Smaller memory footprint & Power efficient solution
Seamless porting of models on to edge devices
Minimal loss and guaranteed accuracy
Right First Time

Who Needs this? Best fit for Model Optimizer Tool

Companies involved in:

Designing HW accelerators – Wants to support their clients with optimal models
Camera companies – Wants to support different applications on their devices
Vision analytics companies – Wants to achieve speed, and to fit on target and acceptable accuracy.
Embedded vision companies – Looking for Optimal models

Also, teams facing the challenges with the following issues needed this solution:

Not able to fit the model on their targets
Not able to control the losses in the quantization
Who wants to build hybrid models
Multiple inferences from the same input
Same inference from multiple inputs
Facing challenges adopting models for proper interfaces for HD, FHD, etc.
Companies optimizing scheduling the CNN block on their accelerators

Finally, this solution is needed for the teams:

That want to adopt the open-source models for their practical use.