cwepr.io.niehs module

Importers for the NIEHS PEST file formats.

Contents

The US National Institute of Environmental Health Sciences (NIEHS) provides a collection of public epr software tools, abbreviated PEST. With those software tools comes a collection of file formats used internally, both binary and text-based.

For a general overview of PEST, see its homepage:

All these file formats have in common that they contain rather few metadata. Therefore, it is highly recommended that users provide metadata by other means, at best in machine-readable format. One possibility would be to use infofiles, as described in the respective aspecd.infofile module.

Overview of file formats

The PEST homepage provides an overview of the different file formats available, for details see:

For convenience, the most important information is provided below:

File extension

Description

.lmb

NIEHS exclusive binary format

.exp

ASCII text format with various different realisations

.dat

ASCII HP-PC interchange format

Just to make things easier, the lmb format exists in two flavours, with extensions “.lmb” and “.sim”, the latter for simulated spectra.

Equally, the ASCII experimental spectra can exist in a plethora of variations, according to the official documentation: with and without header, with one or two columns, and with the first column being magnetic field (in Gauss), time (in seconds), or g value. Interestingly enough, though, the accepted input format is always two columns with first column interpreted as magnetic field values.

Details of the individual file formats are given below. Eventually, it is planned to support all three mentioned file formats.

NIEHS exclusive binary format (.lmb)

This file format is a binary format exclusively implemented to be used with the NIEHS PEST suite. While it is somewhat documented, the best documentation is still the C source code of the importer used to read the format. Therefore, below a few more details will be given, together with the C code used to implement the respective importer.

For authoritative details, the reader is referred to the following online sources:

General file architecture

The sequence of contents of a file in this format is as follows:

Type

Length

Description

identifier

4 byte

either ESRS or ESR2

parameters

20x 4 byte

parameters, e.g., for constructing field axis

data

Nx 4 byte

actual data, number of points in parameters[2]

comment

60 byte

comment (in case of ESR2 format, see below)

metadata

20x 12 byte

metadata, e.g., microwave frequency

comments

2x 60 byte

additional comments in case of ESR2 format

Parameter values

The parameters are not all used. The table below is extracted from the documentation available at the NIEHS website. Note: Due to a slightly strange way of using the loops in the original C source code (see below for details), the number of the parameters in the table below is offset by one. Furthermore, only those parameters with relevance for the import are described.

#

Description

0

sweep width (in Gauss)

1

center field (in Gauss)

2

number of points

3-8

unused here

9

scan time in seconds

10-20

unused here

Metadata

The metadata entries are not all used. The table below is extracted from the documentation available at the NIEHS website. Note: Due to a slightly strange way of using the loops in the original C source code (see below for details), the number of the metadata entries in the table below is offset by one. Furthermore, only those metadata with relevance for the import are described.

#

Description

0-1

unused here

2

modulation amplitude

3

modulation frequency

4

time constant

5

receiver gain

6-7

unused here

8

microwave power

9

microwave frequency

10

date (unclear which one: start, end, saving of data)

11

time (unclear which one: start, end, saving of data)

12

scan time in seconds

13

sample temperature

14-20

unused here

Note

Parameters 0-9 are typed in manually, therefore, never trust these values. Furthermore, due to their human origin, sometimes, value and unit are inconsistently separated (no separation, space, …). Parameters 10-20 are read from the spectrometer control software. Nevertheless, at least for the temperature, this does not mean that the field necessarily contains sensible values.

C code of NIEHS lmb importer (reference implementation)

The C code shown below is abbreviated (only the important parts of the files are shown), and it is only for reference. For details, see the original sources available online from the NIEHS PEST website:

Furthermore, the copyright of the source code remains with its original authors, namely Dave Duling (duling@niehs.nih.gov).

Abbreviated contents of the files.c file containing the PEST importer for lmb files. The source code of the two header files used is given below.
/*
   files.c
   basic routines for reading and writing binary epr data files

   updated to use the new binary file format with expanded comments
*/

#include <stdio.h>
#include <string.h>
#include <math.h>

#include "constant.h"
#include "filename.h"

/**************************** READ FILE *****************************

 Read data for an import.  Deals with fpar, fstring, fnote0, and data

 return:   1 if error,   0 if no error.

*/
int read_file( char *fname, double *ddata, float *fpar,
               char fstring[][STRSIZE], char *fnote0,
               char *fnote1, char *fnote2 )
{

  FILE  *fp;
  int   i, ip;
  float fvalue;
  char  gstring[13];

  /* file not able to be read from */
  if ((fp = fopen(fname, "rb")) == NULL)  return(1);

  fread(gstring, 4, 1, fp);
  gstring[4] = '\0';

  if( strcmp(gstring, FILE_1)!=0 && strcmp(gstring, FILE_2 )!=0 ) {
     fclose(fp);
     return(1);                     /* Not correct info. so exit */
  }

  for(i = 1; i <= MAX_PAR; ++i)     /* read data parameters */
    fread(&fpar[i], 4, 1, fp);

  if( (int) fpar[3] > MAX_PTS ) {   /* check # data points  */
    fclose(fp);
    return(1);
  }

  for(i = 1; i <= (int)fpar[3]; ++i) {      /* read data values     */
     fread(&fvalue, 4, 1, fp);
     ddata[i] = (double) fvalue;
  }

  fread(fnote0, 60, 1, fp);         /* read comment         */

  for(i = 1; i < STR_NUM ; ++i )    /* read strings         */
    fread( fstring[i], 12, 1, fp );

  if( strcmp(gstring, FILE_2)==0 ) {        /* read extra comments  */
    fread( fnote1, 60, 1, fp );
    fread( fnote2, 60, 1, fp );
  }

  fclose(fp);           /* close file           */
  return(0);            /* return success flag  */

}  /* end of OPEN FILE */
Abbreviated contents of the constant.h file used in files.c shown above.
/* constant.h

*/

/* Define the number of datapoints based on the DOS restriction
   of 64KB of memory in one allocation unit.  Otherwise, use
   many more data points !
*/
#ifdef DOS_LIMITED
  #define MAX_PTS     4096
  #define MAX_PTS2    2048
#else
  #define MAX_PTS     16384
  #define MAX_PTS2    8192
#endif

/* this data not platform dependent */

#define MAX_PAR     20
#define STR_NUM     20
#define STRSIZE     13
#define COMMENTSIZE 61
Abbreviated contents of the filename.h file used in files.c shown above.
/* filename.h

 Filename(s) & info constants
*/

#define  FILE_1         "ESRS"
#define  FILE_2         "ESR2"

To import files in this format, use the NIEHSLmbImporter. Within a recipe, the cwepr.io.factory.DatasetImporterFactory should automatically select the correct importer for you.

NIEHS ASCII text format (.exp)

This text file format is actually a series of different formats, just to make life easier. The most basic format simply contains (usually two) columns with numerical data, but there are other versions as well with additional information on top.

Two example files available from the NIEHS PEST website are documented below, representing at least two different variants of this format (and currently the only two variants supported by the respective importer class).

Plain text format containing only columns of data. This format is obviously trivial to read. The only tricky part for the importer is to distinguish it from the more detailed format shown below.
3290.000    -1.5689E+01
3290.945    1.5251E+01
3291.890    2.8689E+00
Text format with additional parameters at the top. Presumably the first block contains some description of the data, e.g. related to a simulation. However, the NIEHS PEST website does not provide too much details.
[EPR]
N1: DMPO_OOH_OH spectrum S/N=5 with J.Bolton parameters
N2: DMPO_OH:  aN= 15.3, aH= 15.3, 0.61, 0.25
N3: DMPO_OOH: aN= 14.9, aH= 14.9

[DATA]
3295.10     -0.283
3295.16     1.523
3295.22     -0.964

To import files in this format, use the NIEHSExpImporter. Within a recipe, the cwepr.io.factory.DatasetImporterFactory should automatically select the correct importer for you.

ASCII HP-PC interchange format (.dat)

This text file format is a rather simple file format with four header lines and only the intensities stored, but actually no metadata at all. An example is given below:

ESRFILE
50
3359.27
8192
-12.234
   -5.48376
   -2.22887
   0.284066
   -0.698693

The first four lines have special meanings and are referred to as “header”, the following lines (up to the end of the file) are the actual intensity values of the stored EPR spectrum.

Content

Description

ESRFILE

Identifier of the file type

50

scan range in Gauss (here: 50 G)

3359.27

center field in Gauss (here: 3359.27 G)

8192

number of data points (here: 8192) - usually a power of two

As mentioned above, all following lines contain just the intensity values. Therefore, the importer needs to reconstruct the field axis from the center field and scan range values provided. No further metadata (such as microwave frequency) are available, therefore, the spectra usually cannot be analysed appropriately unless the missing information is provided by other means.

To import files in this format, use the NIEHSDatImporter. Within a recipe, the cwepr.io.factory.DatasetImporterFactory should automatically select the correct importer for you.

Module documentation

class cwepr.io.niehs.NIEHSDatImporter(source=None)

Bases: aspecd.io.DatasetImporter

Importer for NIEHS PEST dat (text) files.

The text file format dealt with here is a rather simple file format with four header lines and only the intensities stored, but actually no metadata at all.

For details of the file format, see the section ASCII HP-PC interchange format (.dat).

Note

Due to the lack of any metadata of this format, the resulting dataset is quite limited in its metadata. Only the axes measures and units are added (by informed guessing). Therefore, you are highly encouraged to provide the lacking metadata by other means.

New in version 0.3.

class cwepr.io.niehs.NIEHSLmbImporter(source=None)

Bases: aspecd.io.DatasetImporter

Importer for NIEHS PEST lmb (binary) files.

This file format is a binary format exclusively implemented to be used with the NIEHS PEST suite.

For details of the file format, see the section NIEHS exclusive binary format (.lmb).

Todo

Assign further metadata. Note that some metadata are entered manually, and therefore, parsing values with units with or without space between value and unit can be quite tricky.

New in version 0.3.

class cwepr.io.niehs.NIEHSExpImporter(source=None)

Bases: aspecd.io.DatasetImporter

Importer for NIEHS PEST exp (text) files.

The text file format dealt with here is actually a series of different formats, just to make life easier. The most basic format simply contains (usually two) columns with numerical data, but there are other versions as well with additional information on top.

In case of the latter format with additional information on top of the file, this information is added as comment annotation to the dataset.

For details of the file format, see the section NIEHS ASCII text format (.exp).

Note

Due to the lack of any metadata of this format, the resulting dataset is quite limited in its metadata. Only the axes measures and units are added (by informed guessing). Therefore, you are highly encouraged to provide the lacking metadata by other means.

New in version 0.3.