Panoramic , Macro and Micro Multispectral Imaging : An Affordable System for Mapping Pigments on Artworks

Introduction Reflectance imaging spectroscopy is used within art examinations to visually enhance old documents (Kim et al. 2011; Lettner et al. 2008; Padoan et al. 2008); to map and identify pigments (Delaney et al. 2014; Melessanaki et al. 2001; Ricciardi et al. 2009), as well as binders such as animal glue and egg tempera (Dooley et al. 2013); and to detect damage and areas of retouching (Pelagotti et al. 2008a). When pigments are mixed or glazed, reflectance spectroscopy does not always provide conclusive identification unless the pigments used have very clear and unique spectral features. In the opposite case, analytical examinations are recommended to produce detailed diagnostic information. Nevertheless, reflectance imaging spectroscopy provides important information on the materials present and can assist with the conservation decision-making process. Multispectral imaging equipment is commonly composed of a monochromatic camera: a CCD camera for the UV-VIS-NIR range or a much more expensive InGaAs camera for the SWIR (900–2500 nm) range. Some authors have also explored the possibility of using a colour digital camera (Blazek et al. 2013; Zhao et al. 2008). In general, the reflectance spectral features in the UV-VIS-NIR range are due to the electronic transitions responsible in part for the colour of the pigments, while those in the SWIR range are linked to the vibrational overtones. A wavelength selection system is added to the camera so that it can capture images of an object in a series of spectral bands. Once the images are registered and calibrated, they are combined to form a reflectance image cube, where the images are represented by the Xand Y-axes, and where the Z-dimension denotes the wavelength of each image. From the cube it is then possible to reconstruct the reflectance spectrum of each pixel. These systems are called multispectral if the number of spectral images produced is less than, or in the order of, a dozen (Pelagotti et al. 2008a; Pelagotti et al. 2008b; Toque et al. 2009) and hyperspectral if the number is higher. The multispectral group generally uses bandpass filters, which are the simplest wavelength selection method. In contrast, hyperspectral imagers can implement more advanced components such as liquid-crystal tunable filters (LCTFs) (Attas et al. 2003), acousto-optical tunable filters (AOTFs) (Liang et al. 2010) or grating spectrometers (Delaney et al. 2010) to provide hundreds of spectral images. (In order to avoid confusion, it must be noted that in conservation science the term ‘multispectral imaging’ is also used for the image documentation of art works with a collection of broad spectral band images realized with different sensors and lighting sources such as ultraviolet fluorescence photos, infrared reflectograms and X-ray radiographs (Cosentino 2013).) RESEARCH ARTICLE


Introduction
Reflectance imaging spectroscopy is used within art examinations to visually enhance old documents (Kim et al. 2011;Lettner et al. 2008;Padoan et al. 2008); to map and identify pigments (Delaney et al. 2014;Melessanaki et al. 2001;Ricciardi et al. 2009), as well as binders such as animal glue and egg tempera (Dooley et al. 2013); and to detect damage and areas of retouching (Pelagotti et al. 2008a).When pigments are mixed or glazed, reflectance spectroscopy does not always provide conclusive identification unless the pigments used have very clear and unique spectral features.In the opposite case, analytical examinations are recommended to produce detailed diagnostic information.Nevertheless, reflectance imaging spectroscopy provides important information on the materials present and can assist with the conservation decision-making process.
Multispectral imaging equipment is commonly composed of a monochromatic camera: a CCD camera for the UV-VIS-NIR range or a much more expensive InGaAs camera for the SWIR (900-2500 nm) range.Some authors have also explored the possibility of using a colour digital camera (Blazek et al. 2013;Zhao et al. 2008).In general, the reflectance spectral features in the UV-VIS-NIR range are due to the electronic transitions responsible in part for the colour of the pigments, while those in the SWIR range are linked to the vibrational overtones.A wavelength selection system is added to the camera so that it can capture images of an object in a series of spectral bands.Once the images are registered and calibrated, they are combined to form a reflectance image cube, where the images are represented by the X-and Y-axes, and where the Z-dimension denotes the wavelength of each image.From the cube it is then possible to reconstruct the reflectance spectrum of each pixel.These systems are called multispectral if the number of spectral images produced is less than, or in the order of, a dozen (Pelagotti et al. 2008a;Pelagotti et al. 2008b;Toque et al. 2009) and hyperspectral if the number is higher.The multispectral group generally uses bandpass filters, which are the simplest wavelength selection method.In contrast, hyperspectral imagers can implement more advanced components such as liquid-crystal tunable filters (LCTFs) (Attas et al. 2003), acousto-optical tunable filters (AOTFs) (Liang et al. 2010) or grating spectrometers (Delaney et al. 2010) to provide hundreds of spectral images.(In order to avoid confusion, it must be noted that in conservation science the term 'multispectral imaging' is also used for the image documentation of art works with a collection of broad spectral band images realized with different sensors and lighting sources such as ultraviolet fluorescence photos, infrared reflectograms and X-ray radiographs (Cosentino 2013).) This paper discusses the application of a simple multispectral imaging system composed of a CCD monochromatic camera and a set of 12 bandpass filters.A larger set of narrower filters would provide higher spectral resolution (Kubik 2007) but would also be more costly and require more intense illumination, constraints often undesirable when analysing works of art.The bandpass filters were selected with different bandwidths not equally spaced across the recorded spectrum, as is generally the case (Delaney et al. 2014;Ricciardi et al. 2009).Other studies have already used a combination of narrow and large bandpass filters (Liang et al. 2005), and, for this work, the centre wavelength and the bandwidth of the set of filters have been chosen in order to better represent the spectral features of 54 historical pigments (Cosentino 2015).
This paper first addresses the documentation of a relatively large panel painting and then examines the coupling of the system with a macro photography lens and with a stereo or a compound microscope for the study of paint cross-sections and single grains of pigment.These case studies illustrate the technical solutions devised and show for the first time that all of these examinations can be conducted using the same multispectral system coupled with appropriate hardware and software tools.Multispectral documentation of large works of art, such as frescoes, has already been achieved (Liang 2012;Martinez et al. 2002), and specific technical solutions have been provided using custom-made panoramic heads, large X-Y scanning stages and easels, and proprietary software.The coupling of the proposed multispectral imaging system with a stereo and a compound microscope is utilized specifically for the examination of cultural heritage artefacts, including cross-sections and slide-mounted paint samples.
This system provides a qualitative reconstruction of the reflectance spectra of the pigments for the sole purpose of segmenting the images of polychrome works of art.It is suggested for conservators wanting to identify areas of interest for further analytical investigation in order to achieve conclusive results.
Calibration of the images acquired for panoramic and macro photography was conducted using the AIC (American Institute of Conservation) photo target for in-scene reflectance.Its white, its black and its four grey patches were used to calibrate the images by applying a multi-point third degree polynomial calibration curve using ImageJ (Schneider et al. 2012).These patches are manufactured by X-Rite, and they are identical to those used in the X-Rite ColorChecker and the ColorChecker Passport.The patches are identified by the following designations (white to black): white; N8; N6.5; N5; N3.5; and black.In the Munsell notation their corresponding chromas are 9.5,8,6.6,5,3.5,2,their sRGB values are 243,200,160,122,85,52, and the reflectance across the 400-805 nm range covered by the interferential filters is uniform.The multi-point third degree polynomial calibration using the sRGB values of the six swatches made it possible to correct for the spectral response of the CCD across the spectrum and to normalize the spectral images based on the white swatch.
The images were then registered using ImageJ.Dark current subtraction was not necessary because of the high sensitivity of the camera and of the short exposure due to the availability of high-intensity lighting.Flat field correction was not applied because the specific task of this study was to evaluate this equipment for mapping historical pigments and segmenting images of polychrome artworks of very different sizes: it would have been impracticable to set a flat field calibration procedure for the largest of the artworks tested.Therefore, the decision was made simply to set the lights appropriately to best achieve uniform illumination.
For the calibration of the images acquired with the stereomicroscope and compound microscope, the same AIC photo target was used, but because of the high magnification, only the images of the N6.5 patch were acquired for simple linear calibration.
HyperCube (US Army Geospatial Center) imaging spectroscopy software was used for the analysis of the multispectral images.The reflectance spectra reconstructed from the 12 spectral images is here referred to with the acronym MSI-12, and the spectra are represented in the figures with dots.The filters cover only the VIS-NIR range and consequently standard halogen lamps without UV emission were used.All of the FORS (Fiber Optics Reflectance Spectroscopy) spectra of pigments presented in this paper belong to the downloadable online FORS spectra of historical pigments (Cosentino 2014a).Technical photography was also performed on the panel painting case study to complement the information provided by the multispectral camera.For this a Nikon D800 DSLR camera (36 MP, CMOS sensor) modified for full-spectrum acquisition (built-in IR filter removed) was employed.This camera was used for visible (VIS), infrared (IR), infrared false colour (IRFC), infrared fluorescence (IRF), ultraviolet fluorescence (UVF) and ultraviolet reflected (UVR) photography (Figure 1).These technical photographic methods and the related imaging equipment and calibration procedures are described elsewhere, including the details of filters and lighting set-up (Cosentino 2014b;Cosentino et al. 2014).Infrared reflectography (IRR) (Cosentino 2014c) was performed, within the same case study, with an InGaAs camera (320×256 pixels) Merlin NIR by Indigo Systems.

Results and Discussion
This work illustrates the application of a multispectral system for the examination of art and historical objects of different sizes.First, the spectral documentation of a large panel painting is discussed, involving adaptation of the panoramic photographic method.Then, a series of smaller case studies are examined using a macro lens, a stereomicroscope and, finally, a compound microscope.

Panoramic Photography of a Large Panel Painting
Multispectral imaging documentation was conducted on a large panel painting (163 (w) cm, 170 (h) cm, the top lunette 21 (h) cm): the Visitation between Saint Joseph and Saint Zachary (1480-1490) by Antonino Giuffrè (Figure 2).Giuffrè was a Sicilian painter active mostly in Messina during the final decades of the 15 th century and was a follower of Antonello da Messina (Bongiovanni 2001: 56).He and other local artists were involved in replicating and diffusing the style of the Master with a number of replicas.This panel painting is the only one confidently attributed to Giuffrè and is housed in Taormina (Basilica di San Nicolò), Sicily.There is an additional series of panel paintings which have been attributed to him but the attribution was based only on stylistic considerations.This painting was examined during its restoration treatment, which took place in the Angelo Cristaudo Conservation Studio in Acireale, Sicily.
Multispectral imaging documentation of large objects requires a large X-Y scanner (Martinez et al. 2002) or the acquisition of a high number of images and their mosaicking.A system for automatic acquisition and stitching of multispectral images has been suggested, requiring the employment of telescope optics (Liang 2012).The solution illustrated in this section instead considers a multispectral camera (SpectroCam VIS CCD) coupled with an automatic panoramic head (GigaPan Epic Pro) to acquire the images using the panoramic photographic method (Cosentino 2013) (Figure 3).A 20 mm lens was used; this provided an equivalent focal length of about 75 mm when mounted on the SpectroCam.(The equivalent focal length of a lens is calculated by multiplying its focal length by the crop factor of the imaging sensor with which the lens is being used (about 3.7 for the SpectroCam).)The camera was set at a distance of 3.5 m and two 400 W halogen lamps were used, allowing the painting to be mosaicked using four sections, and achieving a resolution of 2 pixels/ mm sufficient for pigment mapping and segmentation on large works of art (Dyer et al. 2013;Liang 2012).
The painting was also examined with 7 technical photographs (resolution: 3 pixels/mm) to complement   the information provided by the multispectral imaging system.
The technical photos provide valuable preliminary diagnostics of the painting.Figure 4 shows that the face of Saint Joseph (Area A in Figure 2) has been heavily restored, as evident in the UVF image.UV fluorescence photographs are used to identify different varnishes and over-paintings and are a supplementary technique for identifying pigments and binders (Dyer et al. 2013).The UVF image also shows a strong pale-white UV fluorescence in correspondence to the original white parts of the painting, suggesting tempera binder.There is also a weak reddish fluorescence, attributable to vermilion laid with tempera, corresponding with the red garments.UVR photography is used to map retouching made with two modern white pigments, zinc white and titanium white, which strongly absorb UV light and appear dark, contrasting with lead white which reflects UV (Bacci et al. 2007).The same figure shows that the white areas that appear to be retouchings because of the darker tone in the UVF image also appear black in the UVR, suggesting a restoration performed with titanium white.Zinc white is ruled out because its characteristic yellow UV fluorescence is not observed.The flat reflectance spectra of the original white paint (Point 13) and the restoration paint (Point 12) in Figure 5 rule out lithopone (a mixture of barium sulphate and zinc sulphide), which has characteristic absorption bands in the infrared (Cosentino 2014b).The red pigment in Point 4 is original while the red paint in Point 3 is in-painting with modern cadmium red as revealed by its infrared fluorescence in the IRF image (Figure 4).The original and retouched red paints cannot be differentiated in any of the other technical photos, nor can they be differentiated by their reflectance spectra, which are very similar (Figure 5).Both of these red paints appear brighter in the 620 nm image than in that at 578 nm, and their reflectance spectra show a sharp inflection point at 600 nm, which suggests vermilion for the original paint (Figure 5).
Area B is representative of the sky, which has original blue paint, as well as numerous inpaints that become evident in the spectral image at 800 nm (Figure 6).The reconstructed reflectance spectrum of the original blue pigment (Point 1) suggests azurite, and the areas retouched (Point 2) are easily detected thanks to their sharp reflectance increase in the infrared region (Figure 6).
Area C is located on the landscape and has original green paint (Point 5) and much retouching (Point 6), which also become apparent in the spectral image at 800 nm (Figure 7).The reconstructed reflectance spectrum of the original paint (Point 5) features the broad reflectance maximum of malachite while the areas of retouching (Point 6) show a maximum at 532 nm other than the sharp reflectance increase in the infrared region (Figure 7).
The Madonna's mantle, Area D, has been heavily restored to fill numerous lacunae.The reconstructed    reflectance spectrum of the original paint (Point 8) suggests both malachite and retouchings (Point 9) are easily detected because of their sharp reflectance increase in the infrared region (Figure 8).The yellow vest of Saint Joseph, within Area A, has been extensively retouched.
The reconstructed reflectance spectrum of the original paint (Point 7) seems to indicate yellow ochre because of the characteristic S-shape of ochre pigments (Figure 9).Spectral images allow retouching to be better localized and mapped than in the technical photographs.Figure 10 shows that the lacunae on the vest of Saint Zachary are more easily distinguished in the 800 nm spectral image, since with the high pass filter used for the infrared photography and with the IRR, both the original paint and the retouching become transparent.The paints have much greater contrast in the 800 nm spectral image as shown by their reflectance spectra (Figure 9).

Macro Photography of a Postage Stamp
This multispectral imaging system allows for the documentation of small objects using standard macro photography lenses.Figure 11 shows the SpectroCam coupled with the Nikon Nikkor Micro 105 mm f2.8 D. As previously discussed, because the CCD imaging sensor has a high crop factor the resulting magnification is higher than that produced with a full-frame digital camera.Calibration of the spectral images was performed using inscene small black and white reflectance references.The    Art. 6, page 9 of 17 current filter-wheel can accommodate only 8 filters and for the panoramic method discussed above the 12 filters were changed manually.For this and the following photographic methods only the first 8 filters were used.Indeed, manually changing the filters causes shaking of the camera that has little impact on the panoramic shooting, but that is unacceptable for macro and micro photography.A bigger filter-wheel or another technical solution is necessary to accommodate all 12 filters.To test the usefulness of the spectral imaging system combined with a macro lens, a postage stamp was examined.This stamp belongs to a private collection and was issued by the Kingdom of Italy a few years before WWII.
Figure 11 shows the spectral images of the section (about 1 cm wide) of the stamp examined.The spectral images at 578 nm and 620 nm indicate that the red ink has a sharp inflection point, and the reconstructed reflectance spectrum (Point 1, Figure 12) suggests the presence of madder lake, a common component of inks for postage stamps documented for this period (Chenciner 2011: 201).
The SpectroCam was coupled to a stereomicroscope to test its application for cases requiring higher magnification than macro photography (Figure 13).The test   subject was a cross-section taken from the painting Madonna and Child attributed to Anton Raphael Mengs (1728-1779) and conserved at KODE, Art Museums of Bergen (Figure 14).The spectral images (Figure 15) allow for the reconstruction of the reflectance spectrum of the red ground, which shows part of the S-shape characteristic of ochre (Figure 16). Figure 17 shows the spectral images acquired with the stereomicroscope of a small section (about 2 mm wide) of the postage stamp already examined with the macro lens.The reconstructed reflectance spectrum (Figure 16) provides the same conclusion as the macro photography.FORS reference spectra of red ochre and madder lake.

Micro Photography of Pigment Particles
The SpectroCam was coupled to a trinocular polarizing microscope equipped with Epi-illumination (illumination from one side) and transmitted illumination.The camera was mounted on the trinocular head using a c-mount adapter, while a digital camera (Nikon D3200) was applied on one of the eyepieces using an adapter for Nikon digital cameras in order to record RGB images (Figure 13).This set-up can obtain multispectral images of slide mounts useful for pigment identification.This application is particularly valuable for the analysis of pigment mixtures since the reflectance spectra can be used to separate the components of the mixtures.Slide mounts (Cargille Meltmount) of 5 historical pigments were tested.
Figure 18 shows the RGB image acquired with the digital camera of an azurite particle and its 8 spectral images.
The reconstructed reflectance spectrum successfully features the azurite maximum at about 450 nm (Figure 19).
Figure 20 shows the spectral images of a particle of smalt and Figure 19 reports its reconstructed reflectance spectrum, which represents satisfactorily the smalt broad absorption band between the blue and infrared regions.Malachite, another mineral pigment with relatively large particles (Figure 21), could be identified by its broad maximum (Figure 22).Analogously, the system proved successful with vermilion, a mineral pigment with much smaller particles (Figure 23).Its reconstructed reflectance spectrum features its characteristic inflection point at 600 nm (Figure 22).The system was also able to reconstruct the reflectance spectrum of chrome green, a modern industrial pigment, showing its sharp maximum in the green region (Figure 22).

Conclusions
This paper has illustrated the application of a simple multispectral imaging system for the documentation and examination of works of art and historical objects of a wide range of dimensions.This system can mount normal photographic lenses; consequently all of the photographic methods typically used for art documentation can potentially be implemented.The illustrated case studies show that the reflectance spectra of the original and retouching pigments could be differentiated and that historical pigments could be tentatively identified.The spectral images were also useful to map the filled lacunae and in one instance performed even better than the infrared technical photography because of the higher contrast in the selected infrared band.
The study then addressed the implementation of macro photography for the examination of a monochromatic postage stamp and continued with the coupling of the multispectral camera to a stereomicroscope     in order to reconstruct the reflectance spectrum of a painting cross-section from the Madonna and Child by Anton Mengs.Finally, the application of the camera on a compound microscope to obtain reflectance spectra of single pigment particles was discussed.These findings indicate the efficacy of coupling the proposed multispectral imaging system with a stereo and a compound microscope for the examination of cultural heritage artefacts.The microscopy element of this tool is able, tentatively, to identify pigments in cross-section at much lower cost than techniques like SEM EDX (scanning electron microscopy with energy dispersive X-ray spectroscopy), FT-IR (Fourier transform infrared spectroscopy) and Raman mapping spectroscopies.While the latter techniques produce more accurate results, it is proposed that the multispectral imaging system may be used to widen accessibility to equipment, enabling initial investigation of pigments, for example, during preliminary conservation investigations and to identify areas for more detailed analysis.
This paper has demonstrated the versatility of this simple multispectral imaging equipment and its capacity to reconstruct the reflectance spectra of pigments observed using different photographic and microscopic configurations to allow the mapping of historical pigments and their tentative identification.The results of this work encourage further study to develop the microscopy applications of this method and enhance the quality of the reconstructed spectra with a larger set of filters and more extensive tests on real artworks.

Competing Interests
The author declares that they have no competing interests.well as sensitive scientific cameras, and their sensitivity range is between about 350 nm and 1100 nm.

Glossary
InGaAs camera.These cameras are sensitive to the near infrared in the 900-1700 nm range and they are the most common imaging device used to perform infrared reflectography on works of art.Their name comes from the Indium, Gallium and Arsenic which are present in the imaging sensor.

Figure 3 :
Figure 3: Left, SpectroCam mounted on the GigaPan Epic Pro panoramic head.Centre, the spectral images acquired for the four sections of the painting.Right, the stitched, registered and calibrated 12 spectral images of the Visitation.

Figure 5 :
Figure 5: Visitation between Saint Joseph and Saint Zachary.Area A in Figure 2. Left, spectral images at 578 nm and nm.Right, MSI-12 spectra (dots) of Points 3, 4, 12 and 13 and FORS reference spectra of vermilion and cadmium red.

Figure 6 :
Figure 6: Visitation between Saint Joseph and Saint Zachary.Area B in Figure 2. Left, spectral images at 620 nm and nm.Right, MSI-12 spectra (dots) of Points 1 and 2 and FORS spectrum of azurite.

Figure 7 :
Figure 7: Visitation between Saint Joseph and Saint Zachary.Area C in Figure 2. Left, spectral images at 532 nm and nm.Right, MSI-12 spectra (dots) of Points 5 and 6 and FORS reference spectrum of malachite.

Figure 8 :
Figure 8: Visitation between Saint Joseph and Saint Zachary.Area D in Figure 2. Left, spectral images at 620 nm and 800 nm.Right, MSI-12 spectra (dots) of Points 8 and 9 and FORS reference spectrum of malachite.

Figure 9 :
Figure 9: Visitation between Saint Joseph and Saint Zachary.Area A in Figure 2. MSI-12 spectra (dots) of Points 7, 10 and 11 and FORS reference spectrum of yellow ochre.

Figure 10 :
Figure 10: Visitation between Saint Joseph and Saint Zachary.Area A in Figure 2. Technical photographs and spectral image at 800 nm.

Figure 11 :
Figure 11: Left, SpectroCam equipped with a macro lens (Nikon Nikkor Micro 105 mm f2.8 D).Right, spectral images at 578 nm and 620 nm of the postage stamp in the insert.

Figure 12 :
Figure 12: Postage stamp.Left, spectral images.Right, reconstructed reflectance spectrum (dots) of the ink (Point 1)and FORS reference spectrum of madder lake.

Figure 13 :
Figure 13: Left, SpectroCam coupled with the trinocular head of a compound microscope and a digital camera applied on one of the eyepieces.Right, SpectroCam mounted on one of the eyepieces of a stereomicroscope.

Figure 14 :
Figure 14: Anton Raphael Mengs, Madonna and Child.Courtesy of KODE, Art Museums of Bergen.

Figure 15 :
Figure 15: Madonna and Child, cross-section, Sampling Point 13 in Figure 14.SpectroCam mounted on a stereomicroscope, RGB image and spectral images.

Figure 17 :
Figure 17: SpectroCam mounted on a stereomicroscope.Detail of the postage stamp in Figure 11.RGB image and spectral images.

Figure 16 :
Figure 16: MSI-12 spectra (dots) of the cross-section's red ground, Point 1 in Figure 14, and of the postage stamp ink.FORS reference spectra of red ochre and madder lake.

Figure 19 :
Figure 19: Reconstructed reflectance spectra (dots) of particles of azurite and smalt on slide mounts and FORS reference spectra of azurite and smalt.

Figure 22 :
Figure 22: Reconstructed reflectance spectra (dots) of malachite, vermilion and chrome green particles and corresponding FORS reference spectra.
CCD camera.CCD (Charge-Coupled Device) cameras were the first imaging sensors used in digital photography and are now replaced by CMOS imaging devices.CCD technology is used in consumer digital photographic cameras as