Introduction
Package ‘photobiologyInOut’ defines functions for importing spectral
data from different instruments, data repositories and simulation
models, and for data exchange with R packages ‘hyperSpec’, ‘colorSpec’
and ‘pavo’ by reading or writing objects of classes defined in them (see
Tables at the start of each section).
As of version 0.4.21 packages ‘hyperSpec’ and ‘pavo’ need to be
installed only if the corresponding functions will be used. This avoids
bloat when these functions are not needed. If the vignette is built or
examples attempted to run in a system where these packages are not
installed, some output will be missing but no error will be
triggered.
All functions that read data from files attempt to decode and store
as metadata the information present in file headers. In addition, in
most cases the unchanged header of the file is stored unaltered as a
comment in the constructed objects.
It should be remembered, though, that this package has been developed
based on the example files I had access to. Files from the same
instruments with different hardware configurations, different firmware
versions, or even settings may differ substantially. In many cases the
output is produced by software in a host computer rather by the
instrument itself, adding further uncertainties and possible differences
due to for example the operating system of the host computer. A further
complication is that in some cases the format of dates, times and
numbers depends on the locale settings in use at the time of data
acquisition, or analysis. For all those reasons, do expect to have to do
some debugging, and most importantly always validate the imported data
against the original file (remembering to run a new validation each time
there is a software or firmware update) or update of this package as I
test each version before release only with the example files I have
available, which are not many.
Spectrometer output files
In the examples we use function system-file to reliably
locate the example files included in the package. For reproducing the
examples with these files, this call is needed. When using user’s files
only the path as a character string needs to be passed as argument.
In most cases the quantities in the files can be identified from the
file contents allowing a single function to work adaptively. In cases
when the format returned by different models of spectrometers from the
same supplier is largely consistent, then a single function can read
them all. In some cases the format depends on the computer software used
to acquire the data, of which a single supplier may have had more than
one over the years, thus requiring multiple functions. Some of the
instruments supported have long been discontinued but in their time were
very popular, like the LI-COR LI-1800 spectrometer from the late 1980’s.
The coverage of brands and models is limited by the instruments I have
had access to, or have had at least access to a sample of files. As of
version 0.4.24 data can be imported from files created by spectrometers
and/or software from six different suppliers.
Functions for importing measured spectral data.
| read_oo_ssirrad() |
Ocean Optics spectrom. |
SpectraSuite |
source_spct |
| read_oo_ssdata() |
Ocean Optics spectrom. |
SpectraSuite |
raw_spct |
| read_oo_jazirrad() |
Ocean Optics Jaz |
instrument |
source_spct |
| read_oo_jazpc() |
Ocean Optics Jaz |
instrument |
filter_spct |
| read_oo_jazpc() |
Ocean Optics Jaz |
instrument |
reflector_spct |
| read_oo_jazdata() |
Ocean Optics Jaz |
instrument |
raw_spct |
| read_oo_pidata() |
Ocean Optics spectrom. |
STS DK (Raspbian) |
raw_spct |
| read_wasatch_csv() |
Wasatch Phot. spectrom. |
Enlighten |
raw_spct |
| read_wasatch_csv() |
Wasatch Phot. spectrom. |
Enlighten |
filter_spct |
| read_wasatch_csv() |
Wasatch Phot. spectrom. |
Enlighten |
reflector_spct |
| read_wasatch_csv() |
Wasatch Phot. spectrom. |
Enlighten |
source_spct |
| read_cid_spectravue_csv() |
CID Bio-Science CI-710s |
instrument |
filter_spct |
| read_cid_spectravue_csv() |
CID Bio-Science CI-710s |
instrument |
reflector_spct |
| read_cid_spectravue_csv() |
CID Bio-Science CI-710s |
instrument |
object_spct |
| read_avaspec_csv() |
Avantes spectrom. |
instrument? |
source_spct |
| read_macam_file() |
Macam |
instrument |
source_spct |
| read_li180_txt() |
LI-COR LI-180 |
instrument |
source_spct |
| read_m_li180_txt() |
LI-COR LI-180 |
instrument |
source_mspct |
| read_licor_prn() |
LI-COR LI-1800 |
PC1800 (MS-DOS) |
source_spct |
| read_licor_prn() |
LI-COR LI-1800 |
PC1800 (MS-DOS) |
filter_spct |
| read_licor_prn() |
LI-COR LI-1800 |
PC1800 (MS-DOS) |
reflector_spct |
| read_m_licor_prn() |
LI-COR LI-1800 |
PC1800 (MS-DOS) |
source_mspct |
| read_m_licor_prn() |
LI-COR LI-1800 |
PC1800 (MS-DOS) |
filter_mspct |
| read_m_licor_prn() |
LI-COR LI-1800 |
PC1800 (MS-DOS) |
reflector_mspct |
Ocean Optics Jaz
Raw detector counts
Reading a raw data file generated by Ocean Optics’ Jaz spectrometer.
The light source was the Jaz PX pulsed Xenon light module.
The first few lines of the file look like this, with W for
wavelength, D for dark, R for reference, S for sample and P for
processed (all spectral data values are raw detector counts):
Jaz Data File
++++++++++++++++++++++++++++++++++++
Date: Mon Apr 25 12:49:11 2016
User: jaz
Dark Spectrum Present: Yes
Reference Spectrum Present: Yes
Processed Spectrum Present: Yes
Spectrometers: JAZA3098
Integration Time (usec): 748000 (JAZA3098)
Spectra Averaged: 1 (JAZA3098)
Boxcar Smoothing: 0 (JAZA3098)
Correct for Electrical Dark: No (JAZA3098)
Strobe/Lamp Enabled: Yes (JAZA3098)
Correct for Detector Non-linearity: No (JAZA3098)
Correct for Stray Light: No (JAZA3098)
Number of Pixels in Processed Spectrum: 2048
>>>>>Begin Processed Spectral Data<<<<<
W D R S P
190.313904 0.000000 0.000000 0.000000 0.000000
190.695511 0.000000 0.000000 0.000000 0.000000
191.077087 1138.953125 1123.134277 1102.795898 228.570541
191.458633 1184.149658 1227.086426 1059.859131 -289.473419
191.840149 1175.110352 1193.188965 1132.173584 -237.500336
...
jaz.raw.file <-
system.file("extdata", "spectrum.jaz",
package = "photobiologyInOut", mustWork = TRUE)
jazraw.spct <- read_oo_jazdata(file = jaz.raw.file)
jazraw.spct <- trim_wl(jazraw.spct, range = c(250, 900))
Plotting the spectrum.
The metadata stored in attributes can be accessed with functions. It
is clear, that not all settings can be recovered from the file. However,
we store the record will all the fields which would have been filled if
the data had been acquired directly from R using package
‘ooacquire’.
getWhenMeasured(jazraw.spct)
## [1] "2016-04-25 12:49:02 UTC"
getInstrDesc(jazraw.spct)
## Data acquired with 'Jaz' s.n. JAZA3098
## grating 'NA', slit 'NA'
getInstrSettings(jazraw.spct)
## integ. time (s): 0.748
## total time (s): 0.748
## counts @ peak (% of max):
Spectral energy irradiance
Reading an “Absolute Irradiance File” (sic) generated by Ocean
Optics’ Jaz spectrometer results in a source_spct object.
In this example, the light source measured was a `white’ fluorescent
tube.
The first few lines of the file look like this:
Jaz Absolute Irradiance File
++++++++++++++++++++++++++++++++++++
Date: Tue Feb 03 09:44:41 2015
User: jaz
Dark Spectrum Present: Yes
Processed Spectrum Present: Yes
Spectrometers: JAZA1065
Integration Time (usec): 193000 (JAZA1065)
Spectra Averaged: 3 (JAZA1065)
Boxcar Smoothing: 5 (JAZA1065)
Correct for Electrical Dark: Yes (JAZA1065)
Strobe/Lamp Enabled: No (JAZA1065)
Correct for Detector Non-linearity: Yes (JAZA1065)
Correct for Stray Light: No (JAZA1065)
Number of Pixels in Processed Spectrum: 2048
Fiber (micron): 3900
Collection Area: 0.119459
Int. Sphere: No
>>>>>Begin Processed Spectral Data<<<<<
W D S P
188.825226 0.000000 0.000000 0.000000
189.284851 0.000000 0.000000 0.000000
189.744415 -89.659378 -90.917900 -0.000000
190.203964 -106.165916 -96.419785 0.000000
...
jaz.s.irrad.file <-
system.file("extdata", "spectrum.JazIrrad",
package = "photobiologyInOut", mustWork = TRUE)
jaz.spct <- read_oo_jazirrad(file = jaz.s.irrad.file)
jaz0.spct <- jaz.spct
jaz.spct <- trim_wl(jaz.spct, range = c(290, 800))
Plotting the spectrum.
Cleaning spectral data
We can see that the data have problems. We get a warning because the
data include negative values for spectral irradiance. We will use some
methods from package ‘photobiology’ to correct the problem. As the data
are noisy we cannot just shift the scale so that the most negative value
becomes zero. Neither can we replace all negative values with zeros, as
this would create bias.
In the following code chunk we will use a region of the spectrum in
which spectral irradiance is known to be equal to zero as reference to
shift the scale zero. Afterwards we discard data ``known’’ to be zero,
and for which the instrument calibration is not valid, and finally we
plot the spectrum.
jaz.spct <- fshift(jaz0.spct, range = c(255, 290), f = "mean")
jaz.spct <- trim_wl(jaz.spct, range = c(290, 800))
autoplot(jaz.spct)
We can next try to smooth the spectrum as it is very noisy outside
the visible region.
jaz.spct <- smooth_spct(jaz.spct)
## 413 possibly 'bad' values in smoothed spectral response
Photon and energy irradiances.
e_irrad(jaz.spct, PAR()) # W m-2
## E_PAR
## 26691.58
## attr(,"time.unit")
## [1] "second"
## attr(,"radiation.unit")
## [1] "total energy irradiance"
All in one statement.
autoplot(read_oo_jazirrad(file = jaz.s.irrad.file))
As above but limiting the wavelength range plotted.
autoplot(read_oo_jazirrad(file = jaz.s.irrad.file),
range = c(250,850))
Adding our custom ``adaptive’’ smoothing.
autoplot(smooth_spct(read_oo_jazirrad(file = jaz.s.irrad.file)),
range = c(250,850))
## 629 possibly 'bad' values in smoothed spectral response
Other modular spectrometers from Ocean Optics
Now a file from an Ocean Optics’ Q6500 spectrometer, with data
processed with the Spectra Suite software.
Format of the header is similar, but not identical. The first few
lines of the file look like this:
SpectraSuite Data File
++++++++++++++++++++++++++++++++++++
Date: Mon May 06 15:13:40 CEST 2013
User: User
Dark Spectrum Present: Yes
Reference Spectrum Present: No
Number of Sampled Component Spectra: 1
Spectrometers: QEB1523
Integration Time (usec): 100000 (QEB1523)
Spectra Averaged: 1 (QEB1523)
Boxcar Smoothing: 0 (QEB1523)
Correct for Electrical Dark: No (QEB1523)
Strobe/Lamp Enabled: No (QEB1523)
Correct for Detector Non-linearity: No (QEB1523)
Correct for Stray Light: Yes (QEB1523)
Number of Pixels in Processed Spectrum: 1044
>>>>>Begin Processed Spectral Data<<<<<
199.08 0.0000E00
199.89 0.0000E00
200.70 0.0000E00
...
q.raw.file <-
system.file("extdata", "spectrum.SSIrrad",
package = "photobiologyInOut", mustWork = TRUE)
autoplot(read_oo_ssirrad(file = q.raw.file))
Array spectrometers from Wasatch Photonics
Enlighten can save spectral data in a variety of file formats. The
function read_wasatch_csv() reads CSV files with spectral
data in columns. It is designed so that it can read any variation of
this file format. In Enlighten it is possible to select which columns
are included in the file so their number can vary. However, the header
is rich in information and this allows in many but not all cases to
guess based on the “Technique” used in Enlighten the type of data being
imported. This means that in some cases the user needs to pass an
argument to parameter s.qty. This is also the case when the
column to be extracted is not that with heading “Processed” in the CSV
file. The metadata is as for Ocean Insight/Ocean Optics spectrometers
copied to attributes in the returned object.
Format of the header is simple and rather easy to parse. The header
of the file and the first few lines look like this:
ENLIGHTEN Version,2.2.7
Measurement ID,20211003-134004-612972-WP-00591
Serial Number,WP-00591
Model,WP-UV-VIS-C-S-25
Label,13:40:04 WP-00591
Declared Match,
Declared Score,0
Scan Averaging,1
Boxcar,0
Technique,Scope
Baseline Correction Algo,
ROI Pixel Start,0
ROI Pixel End,1023
Slit Width,25
Vignetted,False
Interpolated,False
Raman Intensity Corrected,False
Deconvolved,False
Integration Time,60
Timestamp,2021-10-03 13:40:04.612972
Note,Dark substracted
Temperature,-15.327480412352088
CCD C0,247.9385986328125
CCD C1,0.5131465792655945
CCD C2,-0.00012270470324438065
CCD C3,7.728250039917839e-08
CCD Offset,0
CCD Gain,1.9
Laser Wavelength,0.0
Laser Enable,False
Laser Power,100
Laser Temperature,0
Pixel Count,1024
Pixel,Wavelength,Processed
0,247.94,175.00
1,248.45,178.00
file.name <-
system.file("extdata", "enlighten-wasatch-scope.csv",
package = "photobiologyInOut", mustWork = TRUE)
wasatch.raw.spct <-
read_wasatch_csv(file = file.name, extra.cols = "drop")
summary(wasatch.raw.spct)
## Summary of raw_spct [1,024 x 2] object: wasatch.raw.spct
## Wavelength range 247.94-709.67 nm, step 0.43-0.52 nm
## Label: File: enlighten-wasatch-scope.csv
## Measured on 2021-10-03 13:40:04 UTC
## Data acquired with 'WP-UV-VIS-C-S-25' s.n. WP-00591
## grating 'default', slit '25'
## integ. time (s): 0.06
## total time (s): 0.06
## counts @ peak (% of max): NA
## w.length counts
## Min. :247.9 Min. : 173
## 1st Qu.:372.4 1st Qu.: 1604
## Median :487.6 Median :27621
## Mean :484.8 Mean :25832
## 3rd Qu.:598.8 3rd Qu.:43249
## Max. :709.7 Max. :61251
autoplot(wasatch.raw.spct)
Modular spectrometers from Avantes
Avantes’ two column .csv files can also be imported.
ava.raw.file <-
system.file("extdata", "spectrum-avaspec.csv",
package = "photobiologyInOut", mustWork = TRUE)
autoplot(read_avaspec_csv(file = ava.raw.file),
range = c(280, 900), unit.out = "photon")
Scanning spectrometer from Macam
Macam’s single column DTA files can also be imported.
The first few lines of the file look like this with all data in a
single column with alternate rows for wavelengths (in nm) and
irradiances, and a very terse header:
@19/5/1997
@17:44:58
#No Title
2.5000000000E+02
0.0000000000E+00
2.5100000000E+02
0.0000000000E+00
2.5200000000E+02
0.0000000000E+00
...
macam.raw.file <-
system.file("extdata", "spectrum.DTA",
package = "photobiologyInOut", mustWork = TRUE)
autoplot(read_macam_dta(file = macam.raw.file))
LI-180 array dector spectrometer from LI-COR
The LI-COR LI-180 is a portable, self-contained instrument, with
relatively low spectral resolution. This instrument measures spectral
irradiance in the range 380 nm to 780 nm. There is only one hardware
configuration but firmware updates have been released.
This instrument can save the spectral data in different formats. The
“XXX” format includes a rather long header, followed by spectral data,
then followed by a file footer. The long header can contain various
summaries computed from the spectrum, as well as date and time, and
information on the instrument.
The top of the file we will read looks like this:
Model Name LI-180
Serial Number A18M0157
Time 2021/03/02_09:24:26
PPFD 129.879440
PFD 171.178452
PFD-UV 2.336031
PFD-B 33.417435
PFD-G 46.992531
PFD-R 49.469433
PFD-FR 38.963074
Custom1(655~665nm) 5.057602
Custom2(725~735nm) 4.957998
Custom3(650~670nm) 10.137251
Custom4(720~740nm) 9.965228
UV% 1.364676
B% 19.521980
G% 27.452354
R% 28.899324
FR% 22.761662
Custom1% 2.954579
Custom2% 2.896391
Custom3% 5.922037
Custom4% 5.821544
R:B 1.480348
R:FR 1.269649
R:G 1.052708
B:G 0.711122
UV:B 0.069905
UV:FR 0.059955
B:G:R 0.000000
B:R:FR 0.000000
UV:B:G:R:FR 0.000000
Ratio1(Custom1:Custom2) 1.020090
Ratio2(Custom3:Custom4) 1.017262
Ratio3 0.000000
Ratio4 0.000000
LambdaP 495.000000
LambdaPV 106.211479
LambdaD 554.000000
LUX 7366.741699
IRR 35.163498
fc 684.641418
I-Time 92.000000
380nm 27.503876
381nm 27.968113
382nm 28.452974
383nm 28.665405
...
And the footer containing chromaticity data:
...
778nm 82.020149
779nm 82.078781
780nm 82.042412
CCT 5465.000000
Duv 0.004667
x 0.333318
y 0.351069
u' 0.203671
v' 0.482666
deltax 0.000063
deltay 0.009303
deltau' -0.003491
deltav' 0.004648
Purity 5.337072
CRI 98.779282
R1 98.665459
R2 99.470703
R3 98.974945
R4 98.248100
R5 98.830368
R6 99.473335
R7 98.902351
R8 97.668961
R9 94.101921
R10 99.232346
R11 98.246521
R12 99.336227
R13 98.899147
R14 99.324699
R15 97.831963
Function read_licor_espd() will automatically extract
the spectral data, date and time, and serial number.
licor_espd.file <-
system.file("extdata", "LI-180-irradiance.txt",
package = "photobiologyInOut", mustWork = TRUE)
li180.spct <- read_li180_txt(file = licor_espd.file)
In all cases as much information as possible is decoded, and the data
file headers are preserved as comments in the source.spct objects.
## Object: source_spct [401 x 2]
## Wavelength range 380-780 nm, step 1 nm
## Label: File: LI-180-irradiance.txt
## Measured on 2021-03-02 09:24:26 UTC
## Time unit 1s
##
## # A tibble: 401 × 2
## w.length s.e.irrad
## <dbl> <dbl>
## 1 380 0.0275
## 2 381 0.0280
## 3 382 0.0285
## # … with 398 more rows
## # ℹ Use `print(n = ...)` to see more rows
## LICOR LI-180 file 'LI-180-irradiance.txt' imported on 2022-08-13 11:03:53 UTC
## Model Name LI-180
## Serial Number A18M0157
## Time 2021/03/02_09:24:26
## PPFD 129.879440
## PFD 171.178452
## PFD-UV 2.336031
## PFD-B 33.417435
## PFD-G 46.992531
## PFD-R 49.469433
## PFD-FR 38.963074
## Custom1(655~665nm) 5.057602
## Custom2(725~735nm) 4.957998
## Custom3(650~670nm) 10.137251
## Custom4(720~740nm) 9.965228
## UV% 1.364676
## B% 19.521980
## G% 27.452354
## R% 28.899324
## FR% 22.761662
## Custom1% 2.954579
## Custom2% 2.896391
## Custom3% 5.922037
## Custom4% 5.821544
## R:B 1.480348
## R:FR 1.269649
## R:G 1.052708
## B:G 0.711122
## UV:B 0.069905
## UV:FR 0.059955
## B:G:R 0.000000
## B:R:FR 0.000000
## UV:B:G:R:FR 0.000000
## Ratio1(Custom1:Custom2) 1.020090
## Ratio2(Custom3:Custom4) 1.017262
## Ratio3 0.000000
## Ratio4 0.000000
## LambdaP 495.000000
## LambdaPV 106.211479
## LambdaD 554.000000
## LUX 7366.741699
## IRR 35.163498
## fc 684.641418
## I-Time 92.000000
## Data acquired with 'LI-180' s.n. A18M0157
## grating 'NA', slit 'NA'
getInstrSettings(li180.spct)
## integ. time (s): 0.092
## total time (s): NA
## counts @ peak (% of max): NA
autoplot(li180.spct, unit.out = "photon")
LI-1800 scanning spectrometer from LI-COR
The LI-COR LI-1800 was developed in the early 1980’s and remained
available for many years. Some units are still in use although the
technology has been superceeded. If re-calibrated these instruments are
still useful.
Spectral photon irradiance output files generated by LI-COR’s PC1800
program for the LI-1800 spectroradiometer. These files have a relatively
detailed header, but it lacks year information. Files can contain either
energy or photon based spectral irradiances, and this is signalled in
the header. In this example photon (= quantum) spectral irradiance is
returned. The first few lines of the file look like this:
"FILE:FL2"
"REM: TLD 36W/865 (QNTM)"
"LIMS: 300- 900NM"
"INT: 1NM"
"DATE:08/23 16:32"
"MIN: 300NM 1.518E-04"
"MAX: 546NM 7.491E-01"
300 1.518E-04
301 3.355E-04
302 2.197E-04
303 3.240E-04
...
Function read_licor_prn will automatically detect
whether the data is energy or photon based. In all cases as much
metadata information as possible is decoded, and the data file headers
are preserved as comments in the source.spct objects. The
missing information for year is set to zero in the
when.measured attribute, with month, day, hours and minutes
as decoded from the header. The time zone defaults to UTC and will need
in general to be passed as an argument to tz in the
function call.
licor.file <-
system.file("extdata", "spectrum.PRN",
package = "photobiologyInOut", mustWork = TRUE)
licor.spct <- read_licor_prn(file = licor.file, tz = "EET")
Even when using to correct argument for tz the time will
still default to UTC when the spectrum is printed or plotted, but this
time expressed in UTC may still be shifted from the correct time in the
time zone where measurements were acquired as without year information
it is impossible to adjust for daylight saving time and other shifts in
local times that have changed over the years as a result of changes in
legislation.
## Object: source_spct [601 x 2]
## Wavelength range 300-900 nm, step 1 nm
## Label: File: spectrum.PRN
## Measured on 0000-08-23 14:32:00 UTC
## Time unit 1s
##
## # A tibble: 601 × 2
## w.length s.q.irrad
## <dbl> <dbl>
## 1 300 1.52e-10
## 2 301 3.36e-10
## 3 302 2.20e-10
## # … with 598 more rows
## # ℹ Use `print(n = ...)` to see more rows
## LICOR LI-1800 file 'spectrum.PRN' imported on 2022-08-13 11:03:54 UTC
## "FILE:FL2"
## "REM: TLD 36W/865 (QNTM)"
## "LIMS: 300- 900NM"
## "INT: 1NM"
## "DATE:08/23 16:32"
## "MIN: 300NM 1.518E-04"
## "MAX: 546NM 7.491E-01"
autoplot(licor.spct, unit.out = "photon")
It is also possible to use the same function to import reflectance,
and transmittance spectra acquired by the LI-1800.
And a spectral reflectance output file generated by LI-COR’s PC1800
program for the LI-1800 spectroradiometer is used next.
The first few lines of the file look like this:
"FILE:RGD1"
"REM: REFL GREEN AD 1 "
"LIMS: 350- 800NM"
"INT: 2NM"
"DATE:05/30 13:50"
"MIN: 358NM 4.628E-02"
"MAX: 776NM 4.693E-01"
350 5.135E-02
352 4.713E-02
354 5.324E-02
356 4.740E-02
...
Function read_licor_prn cannot automatically detect the
spectral quantity in the file, and when the irradiance default is not
correct, users need to override it with an explicit argument for
parameter s.qty.
licor.file <-
system.file("extdata", "reflectance.PRN",
package = "photobiologyInOut", mustWork = TRUE)
licor.spct <- read_licor_prn(file = licor.file, s.qty = "Rfr")
In all cases as much information as possible is decoded, and the data
file headers are preserved as comments in the source.spct objects.
## Object: reflector_spct [226 x 2]
## Wavelength range 350-800 nm, step 2 nm
## Label: File: reflectance.PRN
## Measured on 0000-05-30 13:50:00 UTC
## Reflectance of type 'total'
##
## # A tibble: 226 × 2
## w.length Rfr
## <dbl> <dbl>
## 1 350 0.0514
## 2 352 0.0471
## 3 354 0.0532
## # … with 223 more rows
## # ℹ Use `print(n = ...)` to see more rows
## LICOR LI-1800 file 'reflectance.PRN' imported on 2022-08-13 11:03:55 UTC
## "FILE:RGD1"
## "REM: REFL GREEN AD 1 "
## "LIMS: 350- 800NM"
## "INT: 2NM"
## "DATE:05/30 13:50"
## "MIN: 358NM 4.628E-02"
## "MAX: 776NM 4.693E-01"
The leaf spectrometer from CID Bio-Science
This all-in-one spectrometer measures spectral reflectance, spectral
transmittance or spectral absorbance of plant leaves or in fact any thin
film. There is a single configuration available and being based on a
microcontroller this instrument is used autonomously and spectral data
saved internally can be exported as CSV files.
Reflectance and transmittance generate a single spectrum per
measurement while absorbance generates three: spectral absorbance,
spectral reflectance and spectral transmittance. The last two are needed
to compute absorbance, and, wisely, they are also returned. In the case
of absorbance measurements it is possible to import the data into an
object_spct containing variables Rfr and
Tfr or as separate objects.
file.name <-
system.file("extdata", "cid-spectravue-Rpc-Measurements.csv",
package = "photobiologyInOut", mustWork = TRUE)
cid_Rpc.spct <- read_cid_spectravue_csv(file = file.name)
## Off-range reflectance values [0.00..32.42] instead of [0..1]
## Summary of reflector_spct [1,235 x 2] object: cid_Rpc.spct
## Wavelength range 380.30945-1099.8846 nm, step 0.5594482-0.6019592 nm
## Label: Reflectance.
## Measured on 2022-03-08 15:02:47 UTC
## Reflectance of type 'total'
##
## w.length Rfr
## Min. : 380.3 Min. :0.0258
## 1st Qu.: 564.8 1st Qu.:0.2392
## Median : 746.7 Median :0.4474
## Mean : 744.5 Mean :0.3626
## 3rd Qu.: 925.2 3rd Qu.:0.4935
## Max. :1099.9 Max. :0.5188
autoplot(smooth_spct(cid_Rpc.spct, method = "supsmu"),
range = c(400, 1000), annotations = "") %+%
ylim(0, 0.55)
Output from simulation models
Functions for importing simulated spectral data.
| read_tuv_usrout() |
TUV (S. Madronich) |
version 5.0 |
source_spct |
| read_tuv_usrout2mspct() |
TUV (S. Madronich) |
version 5.0 |
source_mspct |
| read_qtuv_txt() |
TUV (S. Madronich) |
version 5.2 |
source_spct |
| read_uvspec_disort() |
libRadtran |
irradiance |
source_spct |
| read_uvspec_vesa() |
(T. & V. Kotilainen) |
irradiance |
source_spct |
| read_fmi_cum() |
(A. Lindfors) |
daily cumulated |
source_spct |
| read_m_fmi_cum() |
(A. Lindfors) |
daily cumulated |
source_mspct |
TUV
The output from the TUV model can be imported either by editing it
before import, or by making a simple edit to the output routine of TUV.
This function is known to work with TUV version 5.0 output. The output
from TUV can contain a variable number of spectra in ‘’parallel’’
columns, which are melted into a single column, with a factor
with letters as levels, a numeric variable with the zenith angle and a
POSIXct column with times. A date needs to be always supplied as the
output file from TUV has only time of day information.
tuv.file <-
system.file("extdata", "usrout.txt",
package = "photobiologyInOut", mustWork = TRUE)
tuv.spct <- read_tuv_usrout(file = tuv.file,
date = ymd("2014-03-21"))
summary(subset(tuv.spct, spct.idx == "A"))
## Summary of source_spct [482 x 5] object: anonymous
## containing 8 spectra in long form
## Wavelength range 280.5-761.5 nm, step 1 nm
## Label: TUV spectral simulation File: usrout.txt
## Measured on 2014-03-21 12:00:00 UTC
## Measured on 2014-03-21 13:00:00 UTC
## Measured on 2014-03-21 14:00:00 UTC
## Measured on 2014-03-21 15:00:00 UTC
## Measured on 2014-03-21 16:00:00 UTC
## Measured on 2014-03-21 17:00:00 UTC
## Measured on 2014-03-21 18:00:00 UTC
## Measured on 2014-03-21 19:00:00 UTC
## Time unit 1s
##
## w.length spct.idx s.e.irrad
## Min. :280.5 Length:482 Min. :0.000
## 1st Qu.:400.8 Class :character 1st Qu.:1.216
## Median :521.0 Mode :character Median :1.483
## Mean :521.0 Mean :1.322
## 3rd Qu.:641.2 3rd Qu.:1.680
## Max. :761.5 Max. :1.947
## angle date
## Min. :1.829 Min. :2014-03-21 12:00:00
## 1st Qu.:1.829 1st Qu.:2014-03-21 12:00:00
## Median :1.829 Median :2014-03-21 12:00:00
## Mean :1.829 Mean :2014-03-21 12:00:00
## 3rd Qu.:1.829 3rd Qu.:2014-03-21 12:00:00
## Max. :1.829 Max. :2014-03-21 12:00:00
## Object: source_spct [3,856 x 5]
## containing 8 spectra in long form
## Wavelength range 280.5-761.5 nm, step 1 nm
## Label: TUV spectral simulation File: usrout.txt
## Measured on 2014-03-21 12:00:00 UTC
## Measured on 2014-03-21 13:00:00 UTC
## Measured on 2014-03-21 14:00:00 UTC
## Measured on 2014-03-21 15:00:00 UTC
## Measured on 2014-03-21 16:00:00 UTC
## Measured on 2014-03-21 17:00:00 UTC
## Measured on 2014-03-21 18:00:00 UTC
## Measured on 2014-03-21 19:00:00 UTC
## Time unit 1s
##
## # A tibble: 3,856 × 5
## w.length spct.idx s.e.irrad angle date
## <dbl> <chr> <dbl> <dbl> <dttm>
## 1 280. A 3.04e-15 1.83 2014-03-21 12:00:00
## 2 282. A 1.16e-13 1.83 2014-03-21 12:00:00
## 3 282. A 1.82e-12 1.83 2014-03-21 12:00:00
## # … with 3,853 more rows
## # ℹ Use `print(n = ...)` to see more rows
It is possible to extract individual spectra with subset, or as done
here plot them in different panels.
autoplot(tuv.spct, annotations = c("colour.guide")) +
facet_wrap(~as.character(date), ncol = 2)
The output is a single
source_spct} object that can be easily converted into asource_mspct}
object containing the individual spectra as members of the
collection.
tuv.mspct <- subset2mspct(tuv.spct)
summary(tuv.mspct)
## Length Class Mode
## A 4 source_spct list
## B 4 source_spct list
## C 4 source_spct list
## D 4 source_spct list
## E 4 source_spct list
## F 4 source_spct list
## G 4 source_spct list
## H 4 source_spct list
## Warning in autotitle(object = object, object.label =
## object.label, time.format = time.format, : Multiple
## spectra in long form: title annotation not supported.
A file can be directly read into a collection using
read_tuv_usrout2mspct() which is a simple wrapper.
With the default of lubridate::today() date times are
‘mapped’ to the current local date using the time zone of the computer
as visible to R. This is unlikely to be correct!
tuv_nd.spct <- read_tuv_usrout(file = tuv.file)
when_measured(tuv_nd.spct)
## [1] "2022-08-13 12:00:00 UTC" "2022-08-13 13:00:00 UTC"
## [3] "2022-08-13 14:00:00 UTC" "2022-08-13 15:00:00 UTC"
## [5] "2022-08-13 16:00:00 UTC" "2022-08-13 17:00:00 UTC"
## [7] "2022-08-13 18:00:00 UTC" "2022-08-13 19:00:00 UTC"
Quick TUV calculator
The files output by the online calculator based on the TUV model,
contain at most one spectrum, and arguments to only some parameters can
be set by users. However, it is convenient to use when we only need a
few simple simulations.
Function read_qtuv_txt() can extract spectra and the
corresponding metadata from these files. These files do contain date
time information and geolocation data when they are supplied as
arguments to the calculator interface, otherwise only zenith angle is
available.
qtuv.file <-
system.file("extdata", "qtuv.txt",
package = "photobiologyInOut", mustWork = TRUE)
qtuv.spct <- read_qtuv_txt(file = qtuv.file)
summary(qtuv.spct)
## Summary of source_spct [140 x 7] object: qtuv.spct
## Wavelength range 280.5-419.5 nm, step 1 nm
## Label: Quick TUV spectral simulation File: qtuv.txt
## Measured on 2015-06-30 UTC
## Time unit 1s
##
## w.length s.e.irrad s.e.irrad.dir
## Min. :280.5 Min. :0.0000 Min. :0.00000
## 1st Qu.:315.2 1st Qu.:0.1930 1st Qu.:0.05189
## Median :350.0 Median :0.4600 Median :0.16540
## Mean :350.0 Mean :0.4439 Mean :0.18938
## 3rd Qu.:384.8 3rd Qu.:0.6097 3rd Qu.:0.26338
## Max. :419.5 Max. :1.0390 Max. :0.53710
## s.e.irrad.diff.down s.e.irrad.diff.up angle
## Min. :0.0000 Min. :0.00000 Min. :45
## 1st Qu.:0.1387 1st Qu.:0.01930 1st Qu.:45
## Median :0.2900 Median :0.04600 Median :45
## Mean :0.2545 Mean :0.04439 Mean :45
## 3rd Qu.:0.3467 3rd Qu.:0.06097 3rd Qu.:45
## Max. :0.5109 Max. :0.10390 Max. :45
## date
## Min. :2015-06-30
## 1st Qu.:2015-06-30
## Median :2015-06-30
## Mean :2015-06-30
## 3rd Qu.:2015-06-30
## Max. :2015-06-30
## Object: source_spct [140 x 7]
## Wavelength range 280.5-419.5 nm, step 1 nm
## Label: Quick TUV spectral simulation File: qtuv.txt
## Measured on 2015-06-30 UTC
## Time unit 1s
##
## # A tibble: 140 × 7
## w.length s.e.irrad s.e.irra…¹ s.e.ir…² s.e.ir…³ angle
## <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 280. 1.34e-14 3.89e-15 9.49e-15 1.34e-15 45
## 2 282. 4.33e-13 1.24e-13 3.09e-13 4.33e-14 45
## 3 282. 5.86e-12 1.66e-12 4.2 e-12 5.86e-13 45
## # … with 137 more rows, 1 more variable: date <dttm>,
## # and abbreviated variable names ¹s.e.irrad.dir,
## # ²s.e.irrad.diff.down, ³s.e.irrad.diff.up
## # ℹ Use `print(n = ...)` to see more rows, and `colnames()` to see all variable names
libRadtran
By default libRadtran’s uvspec writes only spectral irradiances to a
text file as output. This is different from ‘TUV’ which by default
includes an extensive header with the parameter settings used for the
simulation. It is easy to read this simple output file with R’s
functions. However, we provide functions, that simplify reading of the
files. The output from uvspec varies depending on its input. The main
source of differences is the solver routine used. We will provide a
separate function for each solver.
For reading this simple output, no special function is needed. We can
use read.table from base R. Here we read a file with two
columns with wavelengths and global spectral energy irradiance (named
“eglo” in libRadtran) in \(mW\,m^{-2}\,nm^{-1}\). The file was created
with one of the ‘uvspec’ examples included with libRadtran, but reducing
the output to two columns.
The first few lines of the file look like this:
250.000 0.000000e+00
251.000 0.000000e+00
252.000 0.000000e+00
253.000 0.000000e+00
...
uvspec.2col.file <-
system.file("extdata", "uvspec-plain-2col.dat",
package = "photobiologyInOut", mustWork = TRUE)
lrt.df <- read.table(file = uvspec.2col.file,
col.names = c("w.length", "s.e.irrad"))
uvspec.01.spct <- source_spct(w.length = lrt.df$w.length,
s.e.irrad = lrt.df$s.e.irrad * 1e-3)
summary(uvspec.01.spct)
## Summary of source_spct [3,751 x 2] object: uvspec.01.spct
## Wavelength range 250-4000 nm, step 1 nm
## Time unit 1s
##
## w.length s.e.irrad
## Min. : 250 Min. :0.000000
## 1st Qu.:1188 1st Qu.:0.003808
## Median :2125 Median :0.023999
## Mean :2125 Mean :0.244545
## 3rd Qu.:3062 3rd Qu.:0.264351
## Max. :4000 Max. :1.744596
cat(comment(uvspec.01.spct))
autoplot(uvspec.01.spct, range = c(250, 2500), unit.out = "photon")
An example using solver disort and our function
read_uvspec_disort() follows.
The first few lines of the file look like this:
290.000 0.000000e+00 0.000000e+00 0.000000e+00 0.000000e+00 0.000000e+00 0.000000e+00
291.000 1.046525e-11 4.800521e-06 1.674293e-07 2.374267e-12 5.342789e-07 2.664720e-08
292.000 5.888299e-10 2.813865e-05 9.814190e-07 1.335888e-10 3.162808e-06 1.561977e-07
293.000 4.383296e-09 5.764524e-05 2.010660e-06 9.944455e-10 6.522616e-06 3.200065e-07
...
uvspec.disort.file <-
system.file("extdata", "uvspec-disort.dat",
package = "photobiologyInOut", mustWork = TRUE)
uvspec.02.spct <- read_uvspec_disort(uvspec.disort.file)
## Warning: 611 parsing failures.
## row col expected actual file
## 1 -- 7 columns 8 columns 'C:/Users/Aphalo/AppData/Local/Temp/Rtmpq0JZWz/Rinst3b3c1c4d116a/photobiologyInOut/extdata/uvspec-disort.dat'
## 2 -- 7 columns 8 columns 'C:/Users/Aphalo/AppData/Local/Temp/Rtmpq0JZWz/Rinst3b3c1c4d116a/photobiologyInOut/extdata/uvspec-disort.dat'
## 3 -- 7 columns 8 columns 'C:/Users/Aphalo/AppData/Local/Temp/Rtmpq0JZWz/Rinst3b3c1c4d116a/photobiologyInOut/extdata/uvspec-disort.dat'
## 4 -- 7 columns 8 columns 'C:/Users/Aphalo/AppData/Local/Temp/Rtmpq0JZWz/Rinst3b3c1c4d116a/photobiologyInOut/extdata/uvspec-disort.dat'
## 5 -- 7 columns 8 columns 'C:/Users/Aphalo/AppData/Local/Temp/Rtmpq0JZWz/Rinst3b3c1c4d116a/photobiologyInOut/extdata/uvspec-disort.dat'
## ... ... ......... ......... .............................................................................................................
## See problems(...) for more details.
## Summary of source_spct [611 x 4] object: uvspec.02.spct
## Wavelength range 290-900 nm, step 1 nm
## Label: libRadtran spectral simulation File: uvspec-disort.dat
## Time unit 1s
##
## w.length s.e.irrad.dir s.e.irrad.diff
## Min. :290.0 Min. :0.0000 Min. :0.00000
## 1st Qu.:442.5 1st Qu.:0.2693 1st Qu.:0.01647
## Median :595.0 Median :0.3781 Median :0.04204
## Mean :595.0 Mean :0.3269 Mean :0.06485
## 3rd Qu.:747.5 3rd Qu.:0.4440 3rd Qu.:0.11234
## Max. :900.0 Max. :0.4928 Max. :0.19479
## s.e.irrad
## Min. :0.0000
## 1st Qu.:0.3237
## Median :0.4236
## Mean :0.3918
## 3rd Qu.:0.5049
## Max. :0.6058
cat(comment(uvspec.02.spct))
## libRadtran file 'uvspec-disort.dat' imported on 2022-08-13 11:04:02 UTC
autoplot(uvspec.02.spct, unit.out = "photon")
The data contains also estimates of diffuse and direct spectral
irradiance. Here we plot the total energy irradiance with a solid line
and the diffuse component with a dashed line.
ggplot(uvspec.02.spct) +
geom_line() +
geom_line(aes(y = s.e.irrad.diff), linetype = "dashed")
The uvspec file used to generate the spectrum read above is:
data_files_path uvspec_home/data/
atmosphere_file uvspec_home/data/atmmod/afglms.dat
source solar uvspec_home/data/solar_flux/kurudz_1.0nm.dat
rte_solver disort
mol_abs_param lowtran
deltam on
number_of_streams 6
wavelength 290 900
day_of_year 287
altitude 0.012
albedo_library IGBP
brdf_rpv_type 5
mol_modify O3 288 DU
mol_modify H2O 10 MM
sza 69.4662
sur_temperature 273
If we plan to save and reuse the spectral object, it is recommended
to append the input file to the comment.
uvspec.disort.inp.file <-
system.file("extdata", "uvspec-disort.inp",
package = "photobiologyInOut", mustWork = TRUE)
comment(uvspec.02.spct) <- paste(comment(uvspec.02.spct),
read_file(uvspec.disort.inp.file),
sep = "\n\n")
cat(comment(uvspec.02.spct))
## libRadtran file 'uvspec-disort.dat' imported on 2022-08-13 11:04:02 UTC
##
## data_files_path uvspec_home/data/
## atmosphere_file uvspec_home/data/atmmod/afglms.dat
## source solar uvspec_home/data/solar_flux/kurudz_1.0nm.dat
## rte_solver disort
## mol_abs_param lowtran
## deltam on
## number_of_streams 6
## wavelength 290 900
## day_of_year 287
## altitude 0.012
## albedo_library IGBP
## brdf_rpv_type 5
## mol_modify O3 288 DU
## mol_modify H2O 10 MM
## sza 69.4662
## sur_temperature 273
We give two additional examples, which will most likely need some
adjustment by users, as these are for output from libRadtran
post-processed to add additional information. These are included in the
package because myself and collaborators use these formats heavily. In
fact users could develop shell scripts or Perl scripts using the same
output format.
Output enriched with time and date data
In this case the file to be read is similar as above, but including
separate columns for direct and diffuse components of the spectral
energy irradiance. In addition two columns, one with date strings in ISO
format and one with times have been added. The file instead of
containing a single spectrum, contains several spectra in long form.
The first few lines of the file look like this:
290.000 2015-05-19 11_00_00 0.000000e+00 0.000000e+00
291.000 2015-05-19 11_00_00 0.000000e+00 0.000000e+00
292.000 2015-05-19 11_00_00 0.000000e+00 0.000000e+00
293.000 2015-05-19 11_00_00 1.893645e-05 3.439497e-05
294.000 2015-05-19 11_00_00 1.648530e-04 2.764368e-04
...
A function is included for reading data saved in a text file in this
format. It also automatically converts \(mW\,m^{-2}\,nm^{-1}\) into \(W\,m^{-2}\,nm^{-1}\).
uvspec.multi.file <-
system.file("extdata", "uvspec-multi.dat",
package = "photobiologyInOut", mustWork = TRUE)
lbr.multi.spct <- read_uvspec_disort_vesa(uvspec.multi.file)
print(lbr.multi.spct, n = 5)
## Object: source_spct [3,055 x 5]
## containing 5 spectra in long form
## Wavelength range 290-900 nm, step 1 nm
## Label: libRadtran spectral simulation File: uvspec-multi.dat
## Measured on 2015-05-19 11:00:00 UTC
## Measured on 2015-05-19 11:01:00 UTC
## Measured on 2015-05-19 11:02:00 UTC
## Measured on 2015-05-19 11:03:00 UTC
## Measured on 2015-05-19 11:04:00 UTC
## Time unit 1s
##
## # A tibble: 3,055 × 5
## datetime w.length s.e.irrad.dir s.e.irra…¹
## <dttm> <dbl> <dbl> <dbl>
## 1 2015-05-19 11:00:00 290 0 0
## 2 2015-05-19 11:00:00 291 0 0
## 3 2015-05-19 11:00:00 292 0 0
## 4 2015-05-19 11:00:00 293 1.89e-11 3.44e-11
## 5 2015-05-19 11:00:00 294 1.65e-10 2.76e-10
## # … with 3,050 more rows, 1 more variable:
## # s.e.irrad <dbl>, and abbreviated variable name
## # ¹s.e.irrad.diff
## # ℹ Use `print(n = ...)` to see more rows, and `colnames()` to see all variable names
Scripts developed by Anders Lindfors
A model for the simulation of the solar spectrum was developed at the
Finnish Meteorological Institute (FMI) by Dr. Anders Lindfors and
collaborators and uses functions from ‘libRadtran’ as its engine, but
saves some additional metadata to the output file. The main addition is
related to the estimation of the effect of clouds.
Functions read_fmi_cum() and
read_m_fmi_cum() can be used to read text files output as
daily integrated spectral irradiance. In other words cummulated daily
data. Function read_fmi2mspct() reads spectral irradiance,
extracting multiple sequential spectra from a single file.
The first few lines of the files with cummulated data look like
this:
# date number_of_scans start_scan stop_scan max_time_gap max_sza_gap warnings
# 20140821 15 3:30:00 17:30:00 60 7.4
# wavelength exposure(J/m2/nm)
2900 0.00000000e+00
2910 2.93132235e-05
2920 7.23526379e-04
...
We can read an individual file into a source_spct object
while adding some metadata read from the file header. In this case
values are for daily global spectral energy exposures rather than
irradiances. Wavelengths are expressed in Angstroms instead of
nanometres.
fmi.file <-
system.file("extdata", "2014-08-21_cum.hel",
package = "photobiologyInOut", mustWork = TRUE)
z.spct <- read_fmi_cum(fmi.file)
class_spct(z.spct)
## [1] "source_spct" "generic_spct"
## [1] "2014-08-21 UTC"
## Object: source_spct [511 x 2]
## Wavelength range 290-800 nm, step 1 nm
## Label: File: 2014-08-21_cum.hel
## Measured on 2014-08-21 UTC
## Time unit 86400s (~1 days)
##
## # A tibble: 511 × 2
## w.length s.e.irrad
## <dbl> <dbl>
## 1 290 0
## 2 291 0.0000293
## 3 292 0.000724
## # … with 508 more rows
## # ℹ Use `print(n = ...)` to see more rows
With function read_m_fmi_cum with an ``m’’ in the name
we can read several files each containing a single spectrum. The
returned object is a collection of source spectra.
fmi.files <-
system.file("extdata", c("2014-08-21_cum.hel", "2014-08-21_cum.hel"),
package = "photobiologyInOut", mustWork = TRUE)
z.mspct <- read_m_fmi_cum(fmi.files)
class(z.mspct)
## [1] "source_mspct" "generic_mspct" "list"
## # A tibble: 1 × 2
## spct.idx when.measured
## <fct> <dttm>
## 1 2014_08_21_cum.hel 2014-08-21 00:00:00
## Object: source_mspct [1 x 1]
## --- Member: 2014_08_21_cum.hel ---
## Object: source_spct [511 x 2]
## Wavelength range 290-800 nm, step 1 nm
## Label: File: 2014-08-21_cum.hel
## Measured on 2014-08-21 UTC
## Time unit 86400s (~1 days)
##
## # A tibble: 511 × 2
## w.length s.e.irrad
## <dbl> <dbl>
## 1 290 0
## 2 291 0.0000293
## 3 292 0.000724
## # … with 508 more rows
## # ℹ Use `print(n = ...)` to see more rows
##
## --- END ---
Above we gave the names of the files explicitly, but as we show here,
one can build on-the-fly a list of file names matching some pattern. The
example below is not run, as the location of example files may vary. The
string "." should be replaced with the path to the folder
where the files to be read are located.
fmi.files <- list.files(".", "*cum.hel")
fmi.files <- paste(".", fmi.files, sep = "")
z1.mspct <- read_m_fmi_cum(fmi.files)
class(z1.mspct)
getWhenMeasured(z1.mspct)
z1.mspct
One also add a geocode at the time of import (or later).
# because of Google's query limits call will frequently fail without a key
# my.geocode <- ggmap::geocode("Kumpula, Helsinki, Finland", source = "google")
my.geocode <- data.frame(lon = 24.96474, lat = 60.20911)
z2.mspct <-
read_m_fmi_cum(fmi.files,
geocode = my.geocode)
class(z2.mspct)
## [1] "source_mspct" "generic_mspct" "list"
getWhenMeasured(z2.mspct)
## # A tibble: 1 × 2
## spct.idx when.measured
## <fct> <dttm>
## 1 2014_08_21_cum.hel 2014-08-21 00:00:00
getWhereMeasured(z2.mspct)
## # A tibble: 1 × 4
## spct.idx lon lat address
## <fct> <dbl> <dbl> <chr>
## 1 2014_08_21_cum.hel 25.0 60.2 <NA>
## Object: source_mspct [1 x 1]
## --- Member: 2014_08_21_cum.hel ---
## Object: source_spct [511 x 2]
## Wavelength range 290-800 nm, step 1 nm
## Label: File: 2014-08-21_cum.hel
## Measured on 2014-08-21 UTC
## Time unit 86400s (~1 days)
##
## # A tibble: 511 × 2
## w.length s.e.irrad
## <dbl> <dbl>
## 1 290 0
## 2 291 0.0000293
## 3 292 0.000724
## # … with 508 more rows
## # ℹ Use `print(n = ...)` to see more rows
##
## --- END ---
Files with spectral irradiance contain data for multiple spectra
stored as text. Each spectrum is delimited at the top by a header line
with metadata and at the end by "end" in a line by
itself.
The first few and last lines for each spectrum look like this:
# 20130501 3:30:00 3:30:00 82.656
210.00 2900 0
210.00 2910 0
210.00 2920 0
...
210.00 8480 103.73018
210.00 8490 104.63495
210.00 8500 90.18384
end
...
The number of spectra, range of wavelengths and the length of each
spectrum can vary. Function read_fmi2mspct() uses matching
to the delimiters to read all the data in all cases.
fmi.file <-
system.file("extdata", "2013-05-01.hel",
package = "photobiologyInOut", mustWork = TRUE)
z3.mspct <- read_fmi2mspct(fmi.file)
class(z3.mspct)[1:2]
## [1] "source_mspct" "generic_mspct"
getWhenMeasured(z3.mspct[[1]])
## [1] "2013-05-01 03:30:00 UTC"
## [1] 15
## [1] "2013-05-01 03:30:00" "2013-05-01 04:30:00"
## [3] "2013-05-01 05:30:00" "2013-05-01 06:30:00"
## [5] "2013-05-01 07:30:00" "2013-05-01 08:30:00"
## [7] "2013-05-01 09:30:00" "2013-05-01 10:30:00"
## [9] "2013-05-01 11:30:00" "2013-05-01 12:30:00"
## [11] "2013-05-01 13:30:00" "2013-05-01 14:30:00"
## [13] "2013-05-01 15:30:00" "2013-05-01 16:30:00"
## [15] "2013-05-01 17:30:00"
getWhenMeasured(z3.mspct[[1]])
## [1] "2013-05-01 03:30:00 UTC"
getWhatMeasured(z3.mspct[[1]])
## [1] "File: 2013-05-01.hel"
Online public repositories of spectral data
Functions for importing spectral data downloaded from
repositories.
| read_FReD_csv() |
Floral Reflectance db. |
2017-03-19 |
reflector_spct |
| read_ASTER_txt() |
ASTER spectral lib. |
version 2.0 ASCII |
reflector_spct |
FReD Floral Reflectance Database
The files downloaded from FReD do not contain a header, but the first
column indicates the flower ID.
157, 300, 0.0627119
157, 301, 0.0654036
157, 302, 0.0677941
157, 303, 0.0670396
...
fred.file <-
system.file("extdata", "FReDflowerID_157.csv",
package = "photobiologyInOut", mustWork = TRUE)
fred.spct <- read_FReD_csv(file = fred.file,
label = "Gazania heterochaeta",
geocode = data.frame(lat = -28.8751, lon = 17.2293))
In this case as there is no metadata present in the file, it needs to
be supplied by the user.
## Object: reflector_spct [401 x 3]
## Wavelength range 300-700 nm, step 1 nm
## Label: File: FReDflowerID_157.csv
## Gazania heterochaeta
## Reflectance of type 'total'
##
## # A tibble: 401 × 3
## flower.id w.length Rfr
## <dbl> <dbl> <dbl>
## 1 157 300 0.0627
## 2 157 301 0.0654
## 3 157 302 0.0678
## # … with 398 more rows
## # ℹ Use `print(n = ...)` to see more rows
## FReD file 'FReDflowerID_157.csv' imported on 2022-08-13 11:04:03 UTC
ASTER spectral database
The files downloaded from ASTER contain a 25-lines-long header, but
at the moment only the first field is decoded, as the whole header
copied as a comment.
Name: Dry grass
Type: Vegetation
Class: Grasses
Subclass: Dry grass
Particle Size: Solid
Sample No.: drygrass.doc
Owner: JHU
Wavelength Range: All
Origin: The entire spectral range was measured at Johns Hopkins University.
Description: Dry grass. Spectra were assembled from two segments; the
bidirectional VNIR and SWIR comprising segment one, and the hemispherical
MWIR and TIR comprising segment two. The VNIR/SWIR spectrum was
measured in the laboratory at JHU with a GER IRIS Mark IV, using a large piece
of sod. The grass was illuminated from directly above and measured at a
reflectance angle of 60 degrees to avoid viewing the thatch.
Measurement: Bidirectional and directional hemispherical reflectance.
First Column: X
Second Column: Y
X Units: Wavelength (micrometers)
Y Units: Reflectance (percent)
First X Value: 0.38049
Last X Value: 14.011
Number of X Values: 2559
Additional Information: None.
0.38049 14.249
0.38299 14.251
0.38544 14.546
0.38791 14.694
...
aster.file <-
system.file("extdata", "drygrass-spectrum.txt",
package = "photobiologyInOut", mustWork = TRUE)
aster.spct <- read_ASTER_txt(file = aster.file)
The label and comment are set from the file header.
## Object: reflector_spct [2,559 x 2]
## Wavelength range 380.49-14011 nm, step 0.8-38 nm
## Label: Name: Dry grass
## File: drygrass-spectrum.txt
## Reflectance of type 'total'
##
## # A tibble: 2,559 × 2
## w.length Rfr
## <dbl> <dbl>
## 1 380. 0.142
## 2 383. 0.143
## 3 385. 0.145
## # … with 2,556 more rows
## # ℹ Use `print(n = ...)` to see more rows
## ASTER database file 'drygrass-spectrum.txt' imported on 2022-08-13 11:04:04 UTC
## Name: Dry grass
## Type: Vegetation
## Class: Grasses
## Subclass: Dry grass
## Particle Size: Solid
## Sample No.: drygrass.doc
## Owner: JHU
## Wavelength Range: All
## Origin: The entire spectral range was measured at Johns Hopkins University.
## Description: Dry grass. Spectra were assembled from two segments; the
## bidirectional VNIR and SWIR comprising segment one, and the hemispherical
## MWIR and TIR comprising segment two. The VNIR/SWIR spectrum was
## measured in the laboratory at JHU with a GER IRIS Mark IV, using a large piece
## of sod. The grass was illuminated from directly above and measured at a
## reflectance angle of 60 degrees to avoid viewing the thatch.
## Measurement: Bidirectional and directional hemispherical reflectance.
## First Column: X
## Second Column: Y
## X Units: Wavelength (micrometers)
## Y Units: Reflectance (percent)
## First X Value: 0.38049
## Last X Value: 14.011
## Number of X Values: 2559
## Additional Information: None.
##
Other R packages
A general way of exchanging data with other R packages or for use
with base R functions is to create a matrix from a collection of spectra
with as.matrix(), or a collection of spectra from a matrix
with one of the as.xxxx_mspct() methods such as
as.source_spct. Such methods are defined in package
‘photobiology’ as well as method join_mspct() for
conversion of collections of spectra into wide data frames.
However, a matrix is only guaranteed to contain numeric data and a
"dim" attribute, while conversion to a data frame preserves
only part of the metadata. These generic conversions cannot be reversed
without loss of information. When possible use the package specific
functions as they automate much of the recovery and preservation of
metadata.
Functions for exchanging data with foreign R packages.
| hyperSpec2spct() |
’hyperSpec’ |
import |
source_spct |
| spct2hyperSpec() |
’hyperSpec’ |
export |
hyperSpec |
| hyperSpec2mspct() |
’hyperSpec’ |
import |
source_mspct |
| mspct2hyperSpec() |
’hyperSpec’ |
export |
hyperSpec |
| colorSpec2spct() |
’colorSpec’ |
import |
source_spct |
| spct2colorSpec() |
’colorSpec’ |
export |
colorSpec |
| colorSpec2mspct() |
’colorSpec’ |
import |
source_mspct |
| mspct2colorSpec() |
’colorSpec’ |
export |
colorSpec |
| chroma_spct2colorSpec() |
’colorSpec’ |
export |
colorSpec |
| rspec2mspct() |
’pavo’ |
import |
source_mspct |
To ‘hyperSpec’
Can export to ''hyperSpec'' objects only collections of
spectra where all members have identical w.length vectors,
as objects of class hyperSpec store a single vector of
wavelengths for the whole collection of spectra.
z2.hspct <- mspct2hyperSpec(z2.mspct, "s.e.irrad")
## Warning in .local(.Object, ...): Spectra in data are
## overwritten by argument spc.
## [1] "hyperSpec"
## attr(,"package")
## [1] "hyperSpec"
From ‘hyperSpec’
Can import only data with wavelength in nanometres. Other quantities
and units are not supported by the ‘photobiology’ classes for spectral
data. See package ‘hyperSpec’ vignette “laser” for details on the data
and the conversion of the original wavelength units into nanometres.
## [1] "hyperSpec"
## attr(,"package")
## [1] "hyperSpec"
## hyperSpec object
## 84 spectra
## 3 data columns
## 36 data points / spectrum
## wavelength: lambda/nm [numeric] 404.5828 404.6181 ... 405.8176
## data: (84 rows x 3 columns)
## 1. t: t / s [numeric] 0 2 ... 5722
## 2. spc: I / a.u. [matrix, array36] 164.650 179.724 ... 112.086
## 3. filename: filename [character] rawdata/laser.txt.gz rawdata/laser.txt.gz ... rawdata/laser.txt.gz
We assume here, that the quantity for the spectral emission of the
laser is spectral energy irradiance, expressed in \(mW\,m^{-2}\,nm^{-1}\). This is likely to be
wrong but for the sake of showing how the conversion takes place is
irrelevant. The parameter multiplier can be passed a
numeric argument to rescale the original data. The default multiplier is
1.
wl(laser) <- list (
wl = 1e7 / (1/405e-7 - wl (laser)),
label = expression (lambda / nm)
)
laser
## hyperSpec object
## 84 spectra
## 3 data columns
## 36 data points / spectrum
## wavelength: lambda/nm [numeric] 411.7467 411.7473 ... 411.7677
## data: (84 rows x 3 columns)
## 1. t: t / s [numeric] 0 2 ... 5722
## 2. spc: I / a.u. [matrix, array36] 164.650 179.724 ... 112.086
## 3. filename: filename [character] rawdata/laser.txt.gz rawdata/laser.txt.gz ... rawdata/laser.txt.gz
plot(laser)
laser.mspct <-
hyperSpec2mspct(laser, "source_spct", "s.e.irrad", multiplier = 1e-3)
ggplot(laser.mspct[[1]]) +
geom_line() +
stat_peaks(geom = "text", vjust = -1, label.fmt = "%.6g nm", color = "red")
From ‘colorSpec’
# bug that needs to be fixed
fluorescent.mspct <- colorSpec2mspct(colorSpec::Fs.5nm)
print(fluorescent.mspct, n = 3, n.members = 3)
## Object: source_mspct [12 x 1]
## --- Member: F1 ---
## Object: source_spct [81 x 2]
## Wavelength range 380-780 nm, step 5 nm
## Time unit 1s
##
## # A tibble: 81 × 2
## w.length s.e.irrad
## <dbl> <dbl>
## 1 380 1.87
## 2 385 2.36
## 3 390 2.94
## # … with 78 more rows
## # ℹ Use `print(n = ...)` to see more rows
## --- Member: F2 ---
## Object: source_spct [81 x 2]
## Wavelength range 380-780 nm, step 5 nm
## Time unit 1s
##
## # A tibble: 81 × 2
## w.length s.e.irrad
## <dbl> <dbl>
## 1 380 1.18
## 2 385 1.48
## 3 390 1.84
## # … with 78 more rows
## # ℹ Use `print(n = ...)` to see more rows
## --- Member: F3 ---
## Object: source_spct [81 x 2]
## Wavelength range 380-780 nm, step 5 nm
## Time unit 1s
##
## # A tibble: 81 × 2
## w.length s.e.irrad
## <dbl> <dbl>
## 1 380 0.82
## 2 385 1.02
## 3 390 1.26
## # … with 78 more rows
## # ℹ Use `print(n = ...)` to see more rows
## ..........................
## 9 other member spectra not shown
##
## --- END ---
colorSpec2mspct(colorSpec::Hoya)
## Object: filter_mspct [4 x 1]
## --- Member: R-60 ---
## Object: filter_spct [46 x 2]
## Wavelength range 300-750 nm, step 10 nm
## Transmittance of type 'total'
## Rfr (/1): NA, thickness (mm): NA, attenuation mode: NA.
##
## # A tibble: 46 × 2
## w.length Tfr
## <dbl> <dbl>
## 1 300 0
## 2 310 0
## 3 320 0
## # … with 43 more rows
## # ℹ Use `print(n = ...)` to see more rows
## --- Member: G-533 ---
## Object: filter_spct [46 x 2]
## Wavelength range 300-750 nm, step 10 nm
## Transmittance of type 'total'
## Rfr (/1): NA, thickness (mm): NA, attenuation mode: NA.
##
## # A tibble: 46 × 2
## w.length Tfr
## <dbl> <dbl>
## 1 300 0
## 2 310 0
## 3 320 0
## # … with 43 more rows
## # ℹ Use `print(n = ...)` to see more rows
## --- Member: B-440 ---
## Object: filter_spct [46 x 2]
## Wavelength range 300-750 nm, step 10 nm
## Transmittance of type 'total'
## Rfr (/1): NA, thickness (mm): NA, attenuation mode: NA.
##
## # A tibble: 46 × 2
## w.length Tfr
## <dbl> <dbl>
## 1 300 0
## 2 310 0
## 3 320 0
## # … with 43 more rows
## # ℹ Use `print(n = ...)` to see more rows
## --- Member: LB-120 ---
## Object: filter_spct [46 x 2]
## Wavelength range 300-750 nm, step 10 nm
## Transmittance of type 'total'
## Rfr (/1): NA, thickness (mm): NA, attenuation mode: NA.
##
## # A tibble: 46 × 2
## w.length Tfr
## <dbl> <dbl>
## 1 300 0.00003
## 2 310 0.0058
## 3 320 0.081
## # … with 43 more rows
## # ℹ Use `print(n = ...)` to see more rows
##
## --- END ---
fluorescent.spct <- colorSpec2spct(colorSpec::Fs.5nm)
autoplot(fluorescent.spct, annotations = "")
colorSpec2chroma_spct(colorSpec::xyz1931.5nm)
## Object: chroma_spct [81 x 4]
## Wavelength range 380-780 nm, step 5 nm
##
## # A tibble: 81 × 4
## x y z w.length
## <dbl> <dbl> <dbl> <dbl>
## 1 0.0014 0 0.0065 380
## 2 0.0022 0.0001 0.0105 385
## 3 0.0042 0.0001 0.0201 390
## # … with 78 more rows
## # ℹ Use `print(n = ...)` to see more rows
To ‘colorSpec’
sun.cspec <- spct2colorSpec(sun.spct)
plot(sun.cspec, col = "blue")
spct2colorSpec(yellow_gel.spct)
##
## colorSpec object. The organization is 'vector'. Object size is 11632 bytes.
## the object describes 1 transparent materials, and the quantity is 'transmittance'.
## Wavelength range: 190 to 800 nm. Step size is 1 nm.
##
## 1 spectra
## 611 data points / spectrum
##
## Material Min Max LambdaMax Integral
## 1 spct_1 1e-05 0.9017857 768 260.3202
chroma_spct2colorSpec(beesxyzCMF.spct)
##
## colorSpec object. The organization is 'matrix'. Object size is 14560 bytes.
## the object describes a responder to light with 3 output channels, and the quantity is 'power->neural'.
## Wavelength range: 300 to 700 nm. Step size is 1 nm.
##
## 3 spectra
## 401 data points / spectrum
##
## Channel Min Max LambdaMax E.response
## 1 x 0.006 1 560 135.358
## 2 y 0.000 1 435 103.696
## 3 z 0.000 1 340 68.365
Wrappers on ‘colorSpec’ functions.
Functions spct_CCT(), spct_CRI() and
spct_SSI() call the respective compute
functions after converting source_spct objects. CCT, CRI
and SSI are meaningful for human vision although they are used not only
for general illumination but also in photography and cinematography.
They can be generalized by overriding the default references from human
vision with those of cameras or other visual systems.
spct_CCT(white_led.source_spct) # correlated color temperature
## [1] 2715.785
spct_CRI(white_led.source_spct) # color rendition index
## [1] 80.76436
spct_CRI(white_led.source_spct, named = TRUE)
## white_led.source_spct_1
## 80.76436
spct_SSI(white_led.source_spct, sun.spct) # spectral similarity index
## [1] 37
From ‘pavo’
In this example we convert an rspec object from package
‘pavo’ into a collection of spectra and then we plot it with
ggplot methods from package
ggspectra' (an extension toggplot2’). The data are the
spectral reflectance of the plumage from seven different individual
birds of the same species, measured in three different body parts.
data(sicalis)
class(sicalis)
## [1] "rspec" "data.frame"
## [1] "wl" "ind1.C" "ind1.T" "ind1.B" "ind2.C"
## [6] "ind2.T" "ind2.B" "ind3.C" "ind3.T" "ind3.B"
## [11] "ind4.C" "ind4.T" "ind4.B" "ind5.C" "ind5.T"
## [16] "ind5.B" "ind6.C" "ind6.T" "ind6.B" "ind7.C"
## [21] "ind7.T" "ind7.B"
We convert the data into a collection of spectra, and calculate
summaries for three spectra.
sicalis.mspct <- rspec2mspct(sicalis, "reflector_spct", "Rpc")
summary(sicalis.mspct[[1]])
## Summary of reflector_spct [401 x 2] object: anonymous
## Wavelength range 300-700 nm, step 1 nm
## Reflectance of type 'total'
##
## w.length Rfr
## Min. :300 Min. :0.001798
## 1st Qu.:400 1st Qu.:0.008288
## Median :500 Median :0.031709
## Mean :500 Mean :0.052848
## 3rd Qu.:600 3rd Qu.:0.098775
## Max. :700 Max. :0.114807
summary(sicalis.mspct[[2]])
## Summary of reflector_spct [401 x 2] object: anonymous
## Wavelength range 300-700 nm, step 1 nm
## Reflectance of type 'total'
##
## w.length Rfr
## Min. :300 Min. :0.006783
## 1st Qu.:400 1st Qu.:0.030112
## Median :500 Median :0.096994
## Mean :500 Mean :0.105449
## 3rd Qu.:600 3rd Qu.:0.179691
## Max. :700 Max. :0.183823
summary(sicalis.mspct[[3]])
## Summary of reflector_spct [401 x 2] object: anonymous
## Wavelength range 300-700 nm, step 1 nm
## Reflectance of type 'total'
##
## w.length Rfr
## Min. :300 Min. :0.001191
## 1st Qu.:400 1st Qu.:0.022293
## Median :500 Median :0.085235
## Mean :500 Mean :0.116253
## 3rd Qu.:600 3rd Qu.:0.212554
## Max. :700 Max. :0.224162
We convert the subset of the collection corresponding to the first
individual into a single spectra object for plotting with
ggplot.
ggplot(rbindspct(sicalis.mspct[1:3])) +
aes(linetype = spct.idx) +
ylim(0,0.3) +
geom_line()
Here we extract the ``crown’’ data from all individuals and plot
these spectra in a single plot.
print(sicalis.mspct[c(TRUE, FALSE, FALSE)])
## Object: reflector_mspct [3]
## --- Member: ind1.C ---
## Object: reflector_spct [401 x 2]
## Wavelength range 300-700 nm, step 1 nm
## Reflectance of type 'total'
##
## # A tibble: 401 × 2
## w.length Rfr
## <int> <dbl>
## 1 300 0.00759
## 2 301 0.00773
## 3 302 0.00829
## # … with 398 more rows
## # ℹ Use `print(n = ...)` to see more rows
## --- Member: ind2.C ---
## Object: reflector_spct [401 x 2]
## Wavelength range 300-700 nm, step 1 nm
## Reflectance of type 'total'
##
## # A tibble: 401 × 2
## w.length Rfr
## <int> <dbl>
## 1 300 0.00297
## 2 301 0.00233
## 3 302 0.00323
## # … with 398 more rows
## # ℹ Use `print(n = ...)` to see more rows
## --- Member: ind3.C ---
## Object: reflector_spct [401 x 2]
## Wavelength range 300-700 nm, step 1 nm
## Reflectance of type 'total'
##
## # A tibble: 401 × 2
## w.length Rfr
## <int> <dbl>
## 1 300 0.000595
## 2 301 0
## 3 302 0.00119
## # … with 398 more rows
## # ℹ Use `print(n = ...)` to see more rows
## --- Member: ind4.C ---
## Object: reflector_spct [401 x 2]
## Wavelength range 300-700 nm, step 1 nm
## Reflectance of type 'total'
##
## # A tibble: 401 × 2
## w.length Rfr
## <int> <dbl>
## 1 300 0.00375
## 2 301 0.00347
## 3 302 0.00413
## # … with 398 more rows
## # ℹ Use `print(n = ...)` to see more rows
## --- Member: ind5.C ---
## Object: reflector_spct [401 x 2]
## Wavelength range 300-700 nm, step 1 nm
## Reflectance of type 'total'
##
## # A tibble: 401 × 2
## w.length Rfr
## <int> <dbl>
## 1 300 0.00423
## 2 301 0.00536
## 3 302 0.00655
## # … with 398 more rows
## # ℹ Use `print(n = ...)` to see more rows
## --- Member: ind6.C ---
## Object: reflector_spct [401 x 2]
## Wavelength range 300-700 nm, step 1 nm
## Reflectance of type 'total'
##
## # A tibble: 401 × 2
## w.length Rfr
## <int> <dbl>
## 1 300 0.000633
## 2 301 0.000614
## 3 302 0.000193
## # … with 398 more rows
## # ℹ Use `print(n = ...)` to see more rows
## --- Member: ind7.C ---
## Object: reflector_spct [401 x 2]
## Wavelength range 300-700 nm, step 1 nm
## Reflectance of type 'total'
##
## # A tibble: 401 × 2
## w.length Rfr
## <int> <dbl>
## 1 300 0.00168
## 2 301 0.00104
## 3 302 0.00170
## # … with 398 more rows
## # ℹ Use `print(n = ...)` to see more rows
##
## --- END ---
ggplot(rbindspct(sicalis.mspct[c(TRUE, FALSE, FALSE)])) +
aes(linetype = spct.idx) +
ylim(0,0.15) +
geom_line() +
ggtitle("'crown' reflectance spectra")
We calculate the mean reflectance in wavebands corresponding to ISO
colors obtaining a data frame. We then add to this returned data frame a
factor indicating the body parts.
refl.by.band <- reflectance(sicalis.mspct, w.band = list(Red(), Green(), Blue(), UVA()))
refl.by.band$body.part <- rep(c("crown", "throat", "breast"), 7)
refl.red <- reflectance(sicalis.mspct, w.band = Red())
names(refl.red)[2] <- "red.reflectance"
refl.red$body.part <- rep(c("crown", "throat", "breast"), 7)
ggplot(refl.red, aes(x = body.part, y = red.reflectance)) +
stat_summary(fun.data = "mean_se", color = "red") +
geom_point(alpha = 0.5)
