Computational Analysis of Paintings
Description
Evidently, the computational analysis of paintings must begin with a high-resolution digitized image of the painting. The digital image consists of individual pixels with a unique colour (or grayscale) value. This ensemble of individual pixels can subsequently be manipulated mathematically in such a way that patterns become visible.
The video lecture by Ingrid Daubechies (see the next tab on this page) explains the principle of one of the possible mathematical tools used in the computational analysis of paintings and also shows several convincing and interesting examples from her own research. These complex mathematical procedures for transforming the numerical data allow the scientists to answer several types of art historical questions such as the following:
1. Authentication of paintings
Building and analyzing a database of known paintings by a certain painter and then authenticate an unknown painting based on the patterns ascertained from the database. An example of such patterns could be the brushstrokes or marks of a special painting tool (1). Farid et al. (1) investigated eight genuine drawings by Pieter Bruegel the Elder and five imitations by different painters included in the catalog of the Metropolitan Museum of Art (4). All drawings are identified in the reference (1) by their number and can be found in the catalog (4). The delicate lines and shading of the drawings make them very suitable for mathematical analysis and it was hoped that the patterns in Bruegel’s work would differ from those of his imitators. The example below is Pastoral Landscape by Bruegel is currently in the collection of the National Gallery of Norway. The drawing is listed in the Metropolitan Museum Catalog (4) on page 90-91.
The results of the investigation were very encouraging and they are depicted in the image below. Blue circles represent genuine drawings by Bruegel, the red squares correspond to the imitations. It can be clearly seen that all the investigated parameters for the genuine drawings are grouped inside a relatively small sphere, while the imitations show much wider scattering and lie outside the sphere defined by the originals.
Results of analyzing 8 authentic Bruegel drawings (blue circles) and 5 imitations (red squares).
Image courtesy of the authors.
2. Original and copy by the same artist
Sometimes two versions of the painting exist. In this case, the question to be answered by computational analysis would be which of the two versions was the original and which was painted later as a copy (2).
3. Paintings by several painters
Some artists had large workshops with several apprentices. Could it be possible to establish which part of the painting was painted by the master and which parts were painted by somebody else? (1).
4. Identification of a painter and/or style
Analysis of a large database of paintings and identifying the painter and/or the style of the painting solely by computational analysis.
References
(1) Lyu, S., Rockmore, D., Farid, H, A digital technique for art authentication, Proceedings of the National Academy of Sciences of the United States of America, Volume 101, Issue 49, 7 December 2004, Pages 17006-17010. Available as pdf.
(2) Ingrid Daubechies, Developing Mathematical Tools to Investigate Art, Duke University. Available as pdf.
(3) Daniel Kim, Seung-Woo Son & Hawoong Jeong, Large-Scale Quantitative Analysis of Painting Arts, Nature, Scientific Reports 4, Article number: 7370 (2014), doi:10.1038/srep07370. Available online.
(4) Orenstein, Nadine M., ed., with contributions by Nadine M. Orenstein, Manfred Sellink, Jürgen Müller, Michiel C. Plomp, Martin Royalton-Kisch, and Larry Silver, Pieter Bruegel the Elder: Drawings and Prints. The Metropolitan Museum of Art, 2001. Available as pdf.
Procedure
Video 'Studying Art with Image Analysis' by Dr. Ingrid Debauchies
“Can image analysis detect the hand of the master? Wavelets and applications to the analysis of art paintings.” This lecture, presented by Dr. Ingrid Debauchies, describes wavelets, a mathematical tool used for the analysis and compression of images (including for digital cinema). Debauchies discusses how they have been used recently for the study of paintings by, for example, Van Gogh, Goossen van der Weyden, Gauguin, and Giotto.
Examples of Use
Pietro di Cristoforo Vannucci (Perugino) Virgin with Child, 1495-96
Painting in The Hood Museum of Art at Dartmouth College. Perugino was quite successful and maintained workshops in Perugia and in Florence. This painting was investigated by Farid et al. (1) using statistical methods in order to ascertain which parts of the painting were painted by Perugino himself and which parts were done by apprentices in his workshop. Such attributions are examples of the so-called “many hands problem”.
The high-resolution digital image of the painting was first converted to grayscale and the regions of the six faces were localized as shown in the above image. All six regions were then statistically evaluated by mathematical methods described in the reference (1). The results are summarized in the image below and they suggest that the three left-most faces numbered 1,2 and 3 are clustered together and were thus painted by one painter and that each of the remaining three faces numbered 4,5 and 6 was painted by different painters.
References
(1) Lyu, S., Rockmore, D., Farid, H, A digital technique for art authentication, Proceedings of the National Academy of Sciences of the United States of America, Volume 101, Issue 49, 7 December 2004, Pages 17006-17010. Available as pdf.
Paintings by Vincent van Gogh
Fabrizio Lamberti, Andrea Sanna and Gianluca Paravati, Computer-assisted analysis of painting brushstrokes: digital image processing for unsupervised extraction of visible features from van Gogh’s works, EURASIP Journal on Image and Video Processing 2014, 2014:53 doi:10.1186/1687-5281-2014-53. Available online.
Berezhnoy, I.E., Postma, E.O., van den Herik, H.J, Authentic: Computerized Brushstroke Analysis, Conference: Proceedings of the 2005 IEEE International Conference on Multimedia and Expo, ICME 2005, July 6-9, 2005, Amsterdam, The Netherlands, pp. 1586 – 1588. Available online.
J Li, L Yao, E Hendriks, JZ Wang, Rhythmic brushstrokes distinguish van Gogh from his contemporaries: findings via automated brushstroke extraction, IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 34, Nr 6, pp. 1159-1176. Available as pdf.
Haixia Liu, Raymond H. Chan, Yuan Yao, Geometric Tight Frame based Stylometry for Art Authentication of van Gogh paintings, Applied and Computational Harmonic Analysis, July 2014. Available online.
Berezhnoy, I., Postma, E., van den Herik, J, Computer analysis of Van Gogh’s complementary colours, Pattern Recognition Letters, Volume 28, Issue 6, 15 April 2007, Pages 703-709. Available as pdf.
Paintings by Jackson Pollock
Paintings by Jackson Pollock
Lior Shamir, What makes a Pollock Pollock: a machine vision approach, Journal: Int. J. of Arts and Technology, 2015 Vol.8, No.1, pp.1 – 10.
Quote from the abstract of the above publication:
Jackson Pollock introduced a revolutionary artistic style of dripping paint on a horizontal canvas. Here we study Pollock’s unique artistic style by using computational methods for characterising the low-level numerical differences between original Pollock drip paintings and drip paintings of other painters who attempted to mimic his signature drip painting style.
Yan Zheng, Xuecheng Nie, Zhaopeng Meng, Wei Feng, Kang Zhang, Layered modeling and generation of Pollock’s drip style, The Visual Computer, May 2015, Volume 31, Issue 5, pp 589-600, DOI 10.1007/s00371-014-0985-7. Available as pdf.
R.P. Taylor, R. Guzman, T.P. Martin, G.D.R. Hall, A.P. Micolich, D. Jonas, B.C. Scannell, M.S. Fairbanks, C.A. Marlow, Authenticating Pollock paintings using fractal geometry, Pattern Recognition Letters, Volume 28, Issue 6, 15 April 2007, Pages 695–702.
Andy Warhol, Flower Series
AAHDS Symposium: “Computational Analysis of Andy Warhol’s Flowers“
Marian Mazzone, Professor of Art History, College of Charleston Thomas Brady, B.S. Student in Computer Science, College of Charleston, “Computational Analysis of Andy Warhol’s Flowers” at the Archives of American Art History Digital Symposium (AAHDS).
Further Reading
References
(1) Jafarpour, S., Polatkan, G., Brevdo, E., Hughes, S., Brasoveanu, A., Daubechies, Stylistic analysis of paintings using wavelets and machine learning (Conference Paper), European Signal Processing Conference 2009, Pages 1220-1224. 17th European Signal Processing Conference, EUSIPCO 2009; Glasgow; United Kingdom; 24 August 2009 through 28 August 2009; Code 91099.
(2) Daniel Kim, Seung-Woo Son & Hawoong Jeong, Large-Scale Quantitative Analysis of Painting Arts, Nature, Scientific Reports 4, Article number: 7370 (2014), doi:10.1038/srep07370. Available online.
(3) Ingrid Daubechies, Developing Mathematical Tools to Investigate Art, Duke University. Available as pdf.
(4) David G. Stork, Computer image analysis of paintings and drawings: An introduction to the literature, Proc. of the Image processing for Artist Identification Workshop, van Gogh Museum, 2008. Available as pdf.
(5) Lyu, S., Rockmore, D., Farid, H, A digital technique for art authentication, Proceedings of the National Academy of Sciences of the United States of America, Volume 101, Issue 49, 7 December 2004, Pages 17006-17010. Available as pdf.
(6) Stork D.G., Computer Vision and Computer Graphics Analysis of Paintings and Drawings: An Introduction to the Literature. In: Jiang X., Petkov N. (eds) Computer Analysis of Images and Patterns. CAIP 2009. Lecture Notes in Computer Science, vol 5702. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03767-2_2