MORSE Driver Table

11JUL24

When MORSE is run it reads a file in the local directory called morse.drv. This 'driver table' is an editable file determining other input/output files and how MORSE is run.

In this sense it is similar to the RFM, although in the RFM most optional sections require the relevant 3-character flag in the *FLG section to 'enable' the option, while MORSE automatically uses any optional section in the driver table, without any need for an associated flag.

Structure of MORSE Driver Table
Mandatory sections (in this order)
1. *HDR Comment record written to output file headers
2. *FLG Option flags
3. *L1C Input spectra
4. *ALT Vertical grid levels
5. *RTV> Target quantities to be retrieved
6. *MIC Microwindow list
7. *ATM Atmospheric profiles
8. *ILS Instrument Line Shape files, or
*SRF Spectral Response Function files
9. *FOV Field of View Shape (omitted if nadir-viewing)
Optional sections (in any order)
*ACC Accuracy settings
*ACV A Priori Covariance
*ASD A Priori Standard Deviation
*CLD Cloud Detection Criteria
*CNV Convergence Criteria
*FIN Internal spectral fine grid
*GRD Irregular Spectral Grid
*HIT HITRAN Database
*LUT Look Up Table Filenames
*NTE Vibrational Temperatures for non-LTE forward model
*OUT Specify Output Files
*PIX Limit range of pixels to be retrieved
*SFC Surface parameters
*SHP Specify line shapes for line-by-line calculations
*SVD SVD-compressed Look Up Tables
*XSC Directories for Molecular Cross Sections
Termination Record (last record that will be read)
*END

Sections
The driver table consists of a number of sections identified by records starting with an asterisk followed by 3 characters
*ABC
where ABC is some code (case-insensitive) defining the contents of the section. These 4 characters have to occupy the first 4 positions in the record.

The driver table is terminated with

*END
(remember to add a 'carriage return' <CR> after the *END)

The first 8 sections (9 for limb-viewing) are mandatory and the sequence is fixed. Thereafter sections are optional and in any order, terminated with the *END record.

Only the first 4 characters of any record starting with * are read, so comments may be added to these records without any exclamation mark if you need to remind yourself what each does.

Records
Each section consists of one or more records:
  • Anything beyond 200 characters is ignored (set by value LENREC in module lenrec_sub.f90)
  • Anything beyond an exclamation mark '!' is ignored — useful for adding comments or 'commenting out' records
  • Empty records (ie containing only space characters) are also ignored.

Fields
Each record is usually divided into one or more fields separated by spaces:
  • Case is not significant except where referring to filenames or directories
  • Spaces are not significant except to separate fields
  • Character fields do not require quotation marks ( ' or " )
  • The various fields can be distributed over an arbitrary number of records
  • Order may be significant — depends on the type of section.
Keyword Field
A special class of field is of the form PARAM=VALUE (no spaces around the '=' sign). Here PARAM is one of a predefined set of keywords (depending on the section) and VALUE is the string or numerical value assigned to it.

Variable Types
The following notation is used when describing the type of variable represented by each Field in the driver table. Storage values are nominal, and actually defined by the kind_dat.f90 module.
Type Description Storage
I Integer 4 bytes
R Real 4 bytes
D Double Precision 8 bytes
Cn Character string, length n n bytes

Tokens
To avoid having to write multiple driver tables to process different filenames, in many cases the input/output filenames can use 'tokens', which are expanded according to various parameters extracted from the L1C input file (for that reason the L1C input filename itself cannot use tokens).
Tokens can also be used as part of filename templates, for example replacing all the absorbing species with '#GAS'
Tokens are of the form '#ABC' (always uppercase) and are expanded as follows:
Token Expansion Description
L1C parameter tokens
#HMS C6 [hhmmss] Hours, minutes, seconds of orbit start
#YMD C8 [yyyymmdd] Year, month, day of orbit start
#ORB ≤C10 Orbit number, truncated (eg '12345')
#INS ≤C10 Instrument identifier, truncated (eg 'IASI-A')
#SAT ≤C10 Satellite identifier, truncated (eg 'MetOp-A')
Filename template tokens
#GAS ≤C7 Absorbing molecule, lower case (eg 'co2', 'f11')
#MIC ≤C8 Microwindow label, as specifed in *MIC section.

Examples
morse_nh3.drv (IASI NH3 retrieval - nadir-viewing)
morse_pt.drv (MIPAS Pressure/Temperature Retrieval - limb-viewing)
morse_h2o.drv (MIPAS Water Vapour Retrieval - limb-viewing)

The rest of this document describes the contents of each section.

*RTV Section (#5)

Description
Retrieval grid (first record) and list of parameters to be retrieved.

Format
Single record defining grid level indices for retrieval, then multiple fields of single type, any order.
Section structure
GRD1 GRD2 ... GRDn Retrieval Grid, single record
RTV1 RTV2[i:j:k] ... Retrieval Parameters [and retrieval grid subsets], multiple records
... RTVn
Field Type ValueDescriptionUnitsRestrictions
GRD[1] Cn * Retrieve on all (NLev) grid levels Only item in record
0 Column retrievals Only item in record
i Retrieve on level i 1 : NLev
i:j Retrieve on all levels i:j inclusive 1 : NLev
i:j:k Retrieve on levels i:j with step k 1 : NLev
RTV C14 TEM Temperature [2] K
PRE Pressure [2] hPa Not on *PRE levels
CTM Continuum [3] km-1
AEROSOL Aerosol [4] km-1
SFCTEM Surface Temperature K
OFF Radiometric Offset [3] nW/(cm2.sr.cm-1)
[gas] Molecule VMR[5] ppmv from List of gases

Notes
  1. GRD lists the indices of the forward model grid (defined in the previous section) on which the retrieved profiles are represented (although individual species may be retrieved on subsets of this grid). Index 1 represents the lowest altitude level. Apart from the special cases of * and 0 this can be a list of increasing integers, a range expression (i:j or i:j:k), or any mix as long as these increase monotonically.

  2. The hydrostatic constraint is assumed whenever temperature (TEM) and pressure (PRE) are retrieved together in limb mode, or if temperature is retrieved on pressure surfaces in nadir mode.

  3. CTM and OFF are retrieved independently for each microwindow and therefore not included in the output unless the MWO Flag is set.

  4. AEROSOL differs from CTM in that a single profile is retrieved for all microwindows, rather than an independent profile for each microwindow separately. Effectively AEROSOL is treated just like any other retrieved gas. Note that the a priori covariances for AEROSOL and CTM differ, so even if AEROSOL is retrieved for a single microwindow over the same altitude range as CTM the results will differ.

  5. [gas] Molecules can either be specified by the chemical formula, HITRAN index or (for CFCs) 'F' number. However, internally these are stored as formulae or (for CFCs) 'F' number, all lower case.

Examples
Nadir-viewing (IASI)
*RTV 0 ! Column retrieval SFCTEM NH3 H2O ! Joint surface temperature, NH3 & H2O columns
Limb-viewing (MIPAS)
*RTV * ! Retrieve on all (17 here) tangent heights TEM PRE CTM[6:17] OFF ! Joint pT, MW continuum (scans 6-17 only) and offset

*ILS Section (#8)

Description
Apodised Instrument Line Shape

Format
Multiple fields of single type, any order.
Field Type Description
ILSFIL C200 Name(s) of .ils file(s)

Notes
  1. A generic lineshape can be supplied for all microwindows and/or separate files for particular spectral regions (the valid spectral range is included in the .ils header)

Example
*ILS
  iasi.ils ! IASI ILS file

*SRF Section (#8)

Description
Spectral Response Functions

Format
Multiple fields of single type, any order.
Field Type Description
SRFFIL C200 Name(s) of .srf file(s)

Notes
  1. Spectral response functions are required for filter radiometers.

Example
*SRF
  HSDI_01.srf ! HSDI Channel#1

*FOV Section (#9) (Limb Only)

Description
Field of View function
This section is omitted for nadir-viewing spectra (as determined by the L1C file).

Format
Single field
Field Type Description
FOVFIL C200 Name of .fov file

Notes
  1. It is assumed that the shape has no spectral dependence so only a single shape is used for all bands
  2. By default, the FOV is allowed to distort with refraction at low altitudes. This can be switched off by adding the FVZ flag in the *FLG section.
  3. Since only one sweep is processed at a time, there is no particular advantage in having a FOV representation which matches the retrieval grid spacing.

Example
*FOV
  mipas_5pt.fov ! 5 pt representation of MIPAS FOV 

*ACC Section (Optional)

Description
Change Accuracy settings

Format
Multiple PARAMETER=VALUE fields, any order
Field Type Description Default
NGJACC= NGJACC I Number of gases for which Jacobians are computed No. rtvd species
PGCACC= PGCACC R Fractional change in pressure requiring new C-G path calc 0.01
TGCACC= TGCACC R Change in temperature [K] requiring new C-G path calc 0.1
JCPACC= JCPACC R Fraction of Jacobian perturbation requiring additional path 0.01
JCCACC= JCCACC R Fraction of Jacobian perturbation requiring new C-G path calc 0.5

Notes
  1. Default values are set in module acccom_dat.f90, except for NGJACC which is set in drvrtv_sub.for
  2. For the default setting of NGJACC, TEM and PRE both count as 'retrieved species' so Jacobians are only calculated for the primary (and secondary, in the case of TEM+PRE) absorbing molecules in each microwindow
  3. Default values can be made more stringent simply by adding the ACC Flag in the *FLG section, in which case NGJACC becomes the total number of absorbers and the other parameters are all reduced by a factor 10. Any settings in this section will then override these new defaults.

Example
*ACC
    NGJACC=4  TGCACC=0.2

*ASD Section (Optional)

Description
Change A Priori Covariance Matrix

Format
Multiple PARAMETER=VALUE records, any order
Field Type Description Units Range Default
CORREL= CORREL R Correlation length [km] 0:100 50km
TEMASD= TEMASD R Temperature SD [K] 0.001:1000 10K
SFCTEMASD= SFCTEMASD R Surface Temperature SD [K] 0.001:1000 10K
PREASD= PREASD R Pressure SD [%] 0.001:1000 50%
[gas]ASD= VMRASD R VMR SD [%] 0.001:1000 100%
EXTASD= EXTASD R Extinction SD [km-1] 0.001:1000 0.005
POIASD= POIASD R Pointing SD [km] 0.001:1000 0.15

Notes
  1. Default values are set in module asdcon_dat.f90
  2. [gas] has to be given as 'F' number for CFCs, formula for other species

Example
*ASD
    CORREL=0 ! Uncorrelated A Priori
    CH4=10   ! Change CH4 to 10% uncertainty

*CLD Section (Optional)

Description
Change Cloud Detection Criteria

Format - Nadir
Multiple PARAMETER=VALUE fields, any order
Field Type Description Default
PIXCLD=PIXCLD R Maximum Pixel Cloud percentage (0:100) 0.0 [%]

Format - Limb
Multiple PARAMETER=VALUE fields, any order
Field Type Description Default
CIXCLD=CIXCLD R Maximum Cloud Index (-1 or positive) 1.8
RADCLD=RADCLD R Minimum Cloud Radiance (-1 or positive) 125.0 [nW/(cm2.sr.cm-1)]
TOPCLD=TOPCLD R Maximum Cloud Altitude (-1 or positive) 30.0 [km]

Notes
  1. These settings only have any effect on the retrievals if the CLD Flag in the *FLG section is set, otherwise the only effect is on the thresholds for which warning messages on possible cloud contamination are sent to the morse.log file.
  2. PIXCLD (for nadir-viewing) sets the maximum pixel cloud cover, as defined in the IASI L1C file, for a pixel to be used for the retrieval.
  3. CIXCLD is the maximum value of the U.Leicester Cloud Index for which the sweep will be assumed cloud-contaminated (ie assumed cloud free for higher values). Note that a Cloud Index of 0.0 in the L1C input file is assumed cloud-free.
  4. RADCLD is the minimum radiance in the cloud detection channel (960.7cm-1) for which the sweep will be assumed cloud contaminated, (ie assumed cloud-free for lower values).
  5. Clouds are flagged if either the Cloud Index or the Cloud Radiance tests indicate a cloud. Set values of -1 to turn off either or both tests.
  6. TOPCLD sets the maximum altitude for which a cloud can be flagged. This is necessary because Cloud Index values become a bit random at high altitudes due to noise. Setting this to -1 turns off any altitude limit, but is not recommended unless CIXCLD=-1 also.
  7. Default values are set in module cldcom_dat.f90 and reported at the start of the morse.log file.

Example
*CLD
    RADCLD=-1    ! Remove Cloud Radiance threshold test
    CIXCLD=2.2   ! Set more stringent Cloud Index threshold

*CNV Section (Optional)

Description
Change Convergence Criteria

Format
Multiple PARAMETER=VALUE records, any order
RecordType Description Default
MAXITR= MAXITR I Maximum number of iterations (GE 1) 10
CHILIM= CHILIM R Minimum value of ChiSq for convergence 1.0
DELCHI= DELCHI R Minimum Change in ChiSq for convergence 0.1
GAMMAX= GAMMAX R Maximum value for Levenberg-Marquardt parameter 1000.0

Notes
  1. Default values are set in module cnvcom_dat.f90, but can also be modified collectively by setting the ACC Flag in the *FLG section: MAXITR multiplied by 10, CHILIM and DELCHI divided by 10.
  2. The ChiSq statistic is the sum of the contributions from the A priori and Measurements, divided by the number of measurements, evaluated in module swprtv_sub.f90, i.e. should equal about 1 if only the instrument noise is the limiting factor.
  3. The retrieval is STOPPED if any of the above criteria are satisfied (tested in swprtv_sub.f90) and defined as CONVERGED if GAMMA = 1 (Levenberg Marqardt parameter minimised) and
    • Either change in ChiSq < DELCHI * min(ChiSq),
    • Or ChiSq < CHIMIN

Example
*CNV
    CHILIM=2.0 
    MAXITR=5

*GRD Section (Optional)

Description
Irregular Spectral Grid for internal forward model calculations

Format
Multiple records, any order
FieldTypeDescription
GRDFIL C200 Name of irregular grid file

Notes
  1. MORSE can use either of the RFM irregular grid file formats: .grd files or irregular spectral files
  2. The irregular grid filename may contain a '*' character. After all explicit files are read, the '*' is expanded as the label of any microwindow which does not yet have an irregular grid file assigned.

Example

*HIT Section (Optional)

Description
Spectroscopic Database file (eg HITRAN binary)

Format
Single Field
FieldTypeDescription
HITFIL C200 Name of binary spectroscopic database file

Notes
  1. A spectroscopic database file is required if any absorbing molecules have HITRAN ID less than 100 (which will usually be the case) AND corresponding Look Up Tables are not available.
  2. The binary file is the same format as used by the RFM, and can be created from a HITRAN ASCII file using program hitbin
  3. Unlike the RFM, MORSE can only use HITRAN files converted to binary form. I'll get around to allowing direct reading of the original 120 character .par files eventually ...

Example
*HIT
    ../morse_files/hitran_mipas.bin

*LUT Section (Optional)

Description
Filenames for Look Up Tables of absorption coefficient

Format
Multiple fields, arbitrary order
Field Type Description
LUTFIL C200 Name of LUT file

Notes
  1. One LUTFIL string may include #MIC and/or (usually both) #GAS tokens, which will be used as a filename template. Once all the explicitly named LUT files are checked, the code will search for files containing the remaining #MIC/#GAS combinations.

Example
*LUT
[need to add some usable examples]

*NTE Section (Optional)

Description
Vibrational temperature filenames

Format
Multiple fields, single type, order is not significant.
Field Type Description
NTEFIL C200 Name of .nte file

Notes
  1. MORSE will accept vibrational temperatures using either the RFM .nte files (entered in this section) or, more simply, within standard .atm files using the *gas(iso)(vib) label.
  2. Look Up Tables cannot be used in conjunction with non-LTE calculations.

Example
*NTE
 co2_day_200km.nte  ! mid-lat daytime vib.temps

*OUT Section (Optional)

Description
Specify Output Files

Format
Multiple PARAMETER=VALUE fields, any order.
Field Type Description Default
OUTDIR=OUTDIR C193 Output directory [local directory]
ATMFIL=ATMFIL C193 Retrieved Atmosphere no output
DIAFIL=DIAFIL C193 Diagnostics no output
RESFIL=RESFIL C193 Residual Spectra no output
RSMFIL=RSMFIL C193 Resume file morse.rsm
RTVFIL=RTVFIL C193 Retrieval Vector morse.rtv
SWPFIL=SWPFIL C193 Sweep Diagnostics no output

Notes
  1. Apart from morse.log, only the .rtv file is written by default, and .rsm is written if the RSM Flag is set. Other output files are created by assigning names using this section (including re-assigning the name of the .rtv file).
  2. The OUTDIR value (a trailing '/' character is added if it is not supplied) simply added to the start of all the output filename strings, so what works and what doesn't depends slightly on the operating system. Used on its own, it will simply redirect the standard MORSE output files to a different directory.
  3. Output files are listed in the morse.log file when this section of the driver table is read.
  4. The module outcom_dat.f90 contains the default and assigned output filenames.
  5. This section cannot be used to change the name or location of the log file, which is always morse.log written in the current directory (the reason being that this file is opened before the driver table is read - see module morse.f90 if you really do need to change it).
  6. Output file names can contain tokens, eg #YMD and #ORB (upper case) which will be expanded to the 8-digit date, eg '20181208', and 5-digit orbit number eg, '05234' in the output filenames.

Example
*OUT
    RTVFIL=morse_#ORB.rtv
    ATMFIL=morse_#ORB.atm
    outdir=../Morse_Outputs/

*PIX Section (Optional)

Description
Limit range of pixels (=profile locations) to be retrieved
Default is to retrieve every pixel in L1C file

Format
1 or 2 fields, order is significant
Field Type Description Range
IPIX1 I 1st pixel to be retrieved 1:NPIX
[IPIX2] I (Optional) last pixel to be retrieved IPIX1:NPIX
NPIX = number of pixels in L1C file

Notes
  1. If only IPIX1 is specified then the processing starts with this pixel number and continues to the end of the L1C file
  2. If IPIX2 is specified with a value greater than the number of pixels in the file then a warning message is printed to the log file and processing continues to the last pixel in the L1C file
  3. To process just a single pixel, specify identical values for IPIX1 and IPIX2

Example
*PIX
   1 10  ! Process just the first 10 pixels

*SHP Section (Optional)

Description
Specify alternative (ie non-Voigt) line shapes for line-by-line calculation

Format
Multiple records, any order
Record structure: SHPNAM GAS(1) GAS(2) ... GAS(N)
FieldTypeDescription
SHPNAM C3 Lineshape to be used
GAS C7 Molecule to which lineshape is applied, or '*'
Allowed values for SHPNAM
FieldDescription
VOI Voigt Lineshape
LOR Lorentz Lineshape
DOP Doppler Lineshape
VVW or VAN Van Vleck Weisskopf Lineshape
CHI Voigt plus Chi-factor

Notes
  1. As RFM *SHP section.

Example
An unrealistically complicated example ...
*SHP
     Chi-factor  CO2    ! Use CO2 sub-Lorentzian wings
     LORENTZ  H2O CH4   ! Use Lorentz shape for H2O and CH4
     LORENTZ  N2O       ! Also N2O
     VOI         *      ! Use Voigt shape for all other absorbers [default anyway]
     DOPP CO            ! Doppler shape for CO

*SVD Section (Optional)

Description
Filenames for (MIPAS) SVD-compressed Look Up Tables

Format
Multiple fields, arbitrary order
Field Type Description
SVDFIL C200 Name of SVD-compressed LUT file CS*DAT

Notes
  1. The SVDFIL filename may contain a '*' character, which is then expanded for the microwindow and absorber ID for any missing data.

Example
*LUT
    ./examples/CS_PT__0511_01.DAT ! pT MW#501 H2O LUT
    ./examples/CS_PT__0511_02.DAT ! pT MW#501 CO2 LUT
    CS_*DAT            ! all other PT MWs & absorber LUTs

*XSC Section (Optional)

Description
Filenames for molecular Cross Section files

Format
Character strings, order is not significant.
Field Type Description
XSCFIL C200 Filenames for .xsc files

Notes
  1. If XSCFIL contains the '*' character, this is expanded to the molecule name (lower case, with 'f' numbers for CFCs/HCFCs, eg 'f12', 'f23') of any cross-section molecules which do not have an explicitly assigned .xsc file.

Example
*XSC
    ./examples/aerosol.xsc         ! just aerosol.xsc in current directory
    ../xscfiles/*.xsc    ! directory for all other .xsc files