Attention modeling: a huge range of applications.
The applications of saliency maps are numerous and they can occur in many domains. For some applications, the saliency maps and their analyses are the final goal, while for others saliency maps are only an intermediary step. We propose three categories of applications in order to make a classification.
The first category of applications directly takes advantage of the detection of surprising, thus abnormal areas in the signal. We can call this class of applications “Abnormality detection”. Surveillance or events/defects detection are examples of applications domains in this category.
The second category will focus more on the opposite of the first one: as the attention maps provide us with an idea about the surprising parts of the signal, one can deduce where the normal (homogenous, repetitive, usual, etc…) signal is. We will call this category “Normality modeling”. The main application domains are in signal compression or re-targeting.
Finally, the third application category is related to the surprising parts of the signal but will go further than a simple detection. This application family will be called “Abnormality processing” and it will need to compare and further process the most salient regions. Domains such as robotics, object retrieval or interfaces optimization can be found in this category.
Applications based on abnormality detection.
In this section, applications are related to surveillance or defect detection. Some authors took into account the concept of “usual motion” either by using accumulation of motion features from videos in given regions which provide a “normality” of the motion in those regions [3] or using more complex systems as Hidden Markov Models (HMMs) to predict future normal motion [4].
While abnormal motion has been mostly used for crowd scenes, some authors like in [9] provide models which work on any general scene containing motion. Some saliency models were used [10][11] with audio data to spot unusual sounds in classical contextual sounds like a gunshot in the middle of a metro station audio ambiance.
In [13], saliency models are used for defect detection and were applied first to automatic fruit grading. In [9], in addition to video surveillance, their model can also apply to static images and find generic defects on those images. Saliency models are applied for defect detection on a wide variety of applications such as the semiconductor manufacturing and electronic production [14], metallic surfaces [15] or wafer defects [16].
In this category, we could add the use of saliency in computer graphics [37] or quality metrics [49] where the abnormal regions of the image are used to optimize graphical representation or to provide different weight to the quality metric depending on the pixels. In the next chapter, we will see the two other categories of applications of saliency models in engineering: normality detection and abnormality processing.
References:
3. Mancas, M. and Gosselin, B. (2010) Dense crowd analysis through bottom-up and 12 top-down attention. Proc. of the Brain Inspired Cognitive Sytems (BICS).
4. Jouneau, E. and Carincotte, C. (2011) Particle-based tracking model for automatic anomaly detection, in Image Processing (ICIP), 13 2011 18th IEEE International Conference on, IEEE, pp. 513–516.
9. Boiman, O. and Irani, M. (2007) Detecting irregularities in images and in video. International Journal of Computer Vision, 74 (1), 17–31.
10. Couvreur, L., Bettens, F., Hancq, J., and Mancas, M. (2007) Normalized auditory attention levels for automatic audio surveillance, in Int. Conf. on Safety and Security Engineering.
11. Mancas, M., Couvreur, L., Gosselin, B., Macq, B. et al. (2007) Computational attention for event detection, in Proc. Fifth International Conf. Computer Vision Systems.
13. Mancas, M., Unay, B., Gosselin, B., and Macq, D. (2007) Computational attention for defect
localisation, in Proceedings of ICVS Workshop on Computational Attention & Applications.
14. Bai, X., Fang, Y., Lin, W., Wang, L., and Ju, B.F. (2014) Saliency-based defect detection in industrial images by using phase spectrum. Industrial Informatics, IEEE Transactions on, 10 (4), 2135–2145.
15. Bonnin-Pascual, F. and Ortiz, A. (2014) A probabilistic approach for defect detection based on saliency mechanisms, in Emerging Technology and Factory Automation (ETFA), 2014 IEEE, IEEE, pp. 1–4.
16. Mishne, G. and Cohen, I. (2014) Multi-channel wafer defect detection using diffusion maps, in Electrical & Electronics Engineers in Israel (IEEEI), 2014 IEEE 28th Convention of, IEEE, pp. 1–5.
37. Longhurst, P., Debattista, K., and Chalmers, A. (2006) A gpu based saliency map for high-fidelity selective rendering, in Proceedings of the 4th international conference on Computer graphics, virtual reality, visualisation and interaction in Africa, ACM, pp. 21–29.
49. Ninassi, A., Le Meur, O., Le Callet, P., and Barbba, D. (2007) Does where you gaze on an image affect your perception of quality? Applying visual attention to image quality metric, in Image Processing, 2007. ICIP 2007. IEEE International Conference on, vol. 2, vol. 2, pp. II –169 –II –172, doi:10.1109/ICIP.2007.4379119.
