ModelAccuracyChecker Class Reference

#include <ModelAccuracyChecker.hpp>

Public Member Functions
	ModelAccuracyChecker (const std::map< std::string, std::string > &validationLabelSet, const std::vector< LabelCategoryNames > &modelOutputLabels)
	Constructor for a model top k accuracy checker. More...
float	GetAccuracy (unsigned int k)
	Get Top K accuracy. More...
template<typename TContainer >
void	AddImageResult (const std::string &imageName, std::vector< TContainer > outputTensor)
	Record the prediction result of an image. More...

Detailed Description

Definition at line 45 of file ModelAccuracyChecker.hpp.

Constructor & Destructor Documentation

ModelAccuracyChecker()

ModelAccuracyChecker	(	const std::map< std::string, std::string > &	validationLabelSet,
		const std::vector< LabelCategoryNames > &	modelOutputLabels
	)

Constructor for a model top k accuracy checker.

Parameters

[in]	validationLabelSet	Mapping from names of images to be validated, to category names of their corresponding ground-truth labels.
[in]	modelOutputLabels	Mapping from output nodes to the category names of their corresponding labels Note that an output node can have multiple category names.

Definition at line 17 of file ModelAccuracyChecker.cpp.

    : m_GroundTruthLabelSet(validationLabels)
    , m_ModelOutputLabels(modelOutputLabels)
{}

Member Function Documentation

AddImageResult()

void AddImageResult ( const std::string &  imageName, std::vector< TContainer >  outputTensor  )

inline

Record the prediction result of an image.

Parameters

[in]	imageName	Name of the image.
[in]	outputTensor	Output tensor of the network running imageName.

Definition at line 73 of file ModelAccuracyChecker.hpp.

    {
        // Increment the total number of images processed
        ++m_ImagesProcessed;
 
        std::map<int, float> confidenceMap;
        auto& output = outputTensor[0];
 
        // Create a map of all predictions
        mapbox::util::apply_visitor([&confidenceMap](auto && value)
                             {
                                 int index = 0;
                                 for (const auto & o : value)
                                 {
                                     if (o > 0)
                                     {
                                         confidenceMap.insert(std::pair<int, float>(index, static_cast<float>(o)));
                                     }
                                     ++index;
                                 }
                             },
                             output);
 
        // Create a comparator for sorting the map in order of highest probability
        typedef std::function<bool(std::pair<int, float>, std::pair<int, float>)> Comparator;
 
        Comparator compFunctor =
            [](std::pair<int, float> element1, std::pair<int, float> element2)
            {
                return element1.second > element2.second;
            };
 
        // Do the sorting and store in an ordered set
        std::set<std::pair<int, float>, Comparator> setOfPredictions(
            confidenceMap.begin(), confidenceMap.end(), compFunctor);
 
        const std::string correctLabel = m_GroundTruthLabelSet.at(imageName);
 
        unsigned int index = 1;
        for (std::pair<int, float> element : setOfPredictions)
        {
            if (index >= m_TopK.size())
            {
                break;
            }
            // Check if the ground truth label value is included in the topi prediction.
            // Note that a prediction can have multiple prediction labels.
            const LabelCategoryNames predictionLabels = m_ModelOutputLabels[static_cast<size_t>(element.first)];
            if (std::find(predictionLabels.begin(), predictionLabels.end(), correctLabel) != predictionLabels.end())
            {
                ++m_TopK[index];
                break;
            }
            ++index;
        }
    }

GetAccuracy()

float GetAccuracy

(

unsigned int

k

)

Get Top K accuracy.

Parameters

[in]

k

The number of top predictions to use for validating the ground-truth label. For example, if k is 3, then a prediction is considered correct as long as the ground-truth appears in the top 3 predictions.

Returns

The accuracy, according to the top k th predictions.

Definition at line 23 of file ModelAccuracyChecker.cpp.

{
    if (k > 10)
    {
        ARMNN_LOG(warning) << "Accuracy Tool only supports a maximum of Top 10 Accuracy. "
                              "Printing Top 10 Accuracy result!";
        k = 10;
    }
    unsigned int total = 0;
    for (unsigned int i = k; i > 0; --i)
    {
        total += m_TopK[i];
    }
    return static_cast<float>(total * 100) / static_cast<float>(m_ImagesProcessed);
}

References ARMNN_LOG, and armnn::warning.

---

The documentation for this class was generated from the following files:

• src/armnnUtils/ModelAccuracyChecker.hpp
• src/armnnUtils/ModelAccuracyChecker.cpp