ODI (Open Data Interface) Client
The Open Data Interface (ODI) provides a common backend and database system for space environment data processing systems at ESA. It provides a generic/common interface to space environment datasets and allows for the ingestion of new datasets. The data is stored in a MySQL database and the ODI system is based on software that allows the migration to different platforms, although it is mainly targeted for a Linux platform.
The Open Data Interface (ODI) is a software system that stores time series space environment data and metadata. The metadata syntax follows the CDF/ISTP and the COSPAR PRBEM guide lines. The ODI system relies on a MySQL database to store its data and metadata. There are MySQL database drivers for all major computer platforms (Linux, Mac OS X, Unix, Windows) and libraries have been implemented for all major programming languages (e.g. C, C++, IDL, Java, Mathematica, Matlab, Perl, PHP). This means that data in ODI are accessible to a wide variety of software systems.
The ODI system contains:
software to ingest data and metadata for a number of specific datasets from plain text files;
software to ingest data and metadata from ISTP compliant CDF files;
software to export data into plain text files or CDF files;
a set of command line tools to administer and explore the database;
scripts to automatically download datasets;
scripts to automatically ingest datasets;
scripts to set up scheduled download/ingest (based on cron);
a REST and HAPI interface for internet access;
Python, IDL, Matlab, PHP clients to facilitate access to the data.
The distribution consists of two parts: one for the client, and one for the server.
The core ODI system has been developed in PHP, providing platform independence. As the format of datasets originating from plain text files is not standardized, custom text parsing code is necessary for such files. Several parsers are included in ODI and new parsers can be added to allow the ingestion of additional datasets. New parsers are automatically identified by the system.
Datasets currently with ingestion configuration details includes:
ACE/EPAM
ACE/MAG
ACE/MFI
ACE/SEPICA
ACE/SIS
ACE/SWEPAM
ACE/SWE
ACE/ULEIS
CRRES/PROTEL
CRRES/MEA
DSCOVR/FC
DSCOVR/MAG
GOES/MAG
GOES/PART
GOES/PCHAN
GOES/XRAY
GOES/SEM/XRS (GOES-01 >> GOES-15)
GOES/SEM/EPS (GOES-05 >> GOES-12)
GOES/SEM/MAGNETO (GOES-05 >> GOES-15)
GOES/SEM/MAGNETO (GOES-05 >> GOES-15)
GOES/SEM/HEPAD (GOES-06 >> GOES-15)
GOES/SEM/EPEAD (GOES-13 >> GOES-15)
GOES/SEM/MAGPD (GOES-13 >> GOES-15)
GOES/SEM/MAGED (GOES-13 >> GOES-15)
GOES/SEM/MAGPD (GOES-13 >> GOES-15)
GPS (ns41, ns48, ns53, ns54, ns55, ns56, ns57, ns58, ns59, ns60, ns61, ns62, ns63, ns64, ns65, ns66, ns67, ns68, ns69, ns70, ns71, ns72, ns73)
helios a & b: e6 & e7 datasets
ESA SREM (L1, L2 and V0):
proba1
integral
herschel
planck
rosetta
strv1c
Himawari-8 SEDA (electron & proton)
IMP8/CPME: electrons, helium, ions and protons
IMP8/CRNC
IMP8/GME
Magnetospheric indices: AE, AP, CP, C9, Dst, EISN, F10, ISN, Kp, OMNI2, OMNI/HRO
POES/SEM: METOP02, NOAA15-19
PROBAV/EPT
RBSP/HOPE L2/SA, L3/MO, L3/PA
RBSP/MAGEIS L2, L3
RBSP/REPT L2, L3
RBSP/RPS L2
SEPEM Reference Data Set (h_wbkg, he_nbkg, h_nbkg)
WIND/EPACT
WIND/MFI
WIND/SWE
XMM/ERMD