ODI (Open Data Interface ) Server
The Open Data Interface (ODI) provides a common backend and database system for space environment data processing systems at ESA. It provides a generic/common interface to space environment datasets and allows for the ingestion of new datasets. The data is stored in a MySQL database and the ODI system is based on software that allows the migration to different platforms, although it is mainly targeted for a Linux platform.
The Open Data Interface (ODI) is a software system that stores time series space environment data and metadata. The metadata syntax follows the CDF/ISTP and the COSPAR PRBEM guide lines. The ODI system relies on a MySQL database to store its data and metadata. There are MySQL database drivers for all major computer platforms (Linux, Mac OS X, Unix, Windows) and libraries have been implemented for all major programming languages (e.g. C, C++, IDL, Java, Mathematica, Matlab, Perl, PHP). This means that data in ODI are accessible to a wide variety of software systems.
The ODI system contains:
- software to ingest data and metadata for a number of specific datasets from plain text files;
- software to ingest data and metadata from ISTP compliant CDF files;
- software to export data into plain text files or CDF files;
- a set of command line tools to administer and explore the database;
- scripts to automatically download datasets;
- scripts to automatically ingest datasets;
- scripts to set up scheduled download/ingest (based on cron);
- a REST and HAPI interface for internet access;
- Python, IDL, Matlab, PHP clients to facilitate access to the data.
The distribution consists of two parts: one for the client, and one for the server.
The core ODI system has been developed in PHP, providing platform independence. As the format of datasets originating from plain text files is not standardized, custom text parsing code is necessary for such files. Several parsers are included in ODI and new parsers can be added to allow the ingestion of additional datasets. New parsers are automatically identified by the system.
Datasets currently with ingestion configuration details includes:
- ACE/EPAM
- ACE/MAG
- ACE/MFI
- ACE/SEPICA
- ACE/SIS
- ACE/SWEPAM
- ACE/SWE
- ACE/ULEIS
- CRRES/PROTEL
- CRRES/MEA
- DSCOVR/FC
- DSCOVR/MAG
- GOES/MAG
- GOES/PART
- GOES/PCHAN
- GOES/XRAY
- GOES/SEM/XRS (GOES-01 >> GOES-15)
- GOES/SEM/EPS (GOES-05 >> GOES-12)
- GOES/SEM/MAGNETO (GOES-05 >> GOES-15)
- GOES/SEM/MAGNETO (GOES-05 >> GOES-15)
- GOES/SEM/HEPAD (GOES-06 >> GOES-15)
- GOES/SEM/EPEAD (GOES-13 >> GOES-15)
- GOES/SEM/MAGPD (GOES-13 >> GOES-15)
- GOES/SEM/MAGED (GOES-13 >> GOES-15)
- GOES/SEM/MAGPD (GOES-13 >> GOES-15)
- GPS (ns41, ns48, ns53, ns54, ns55, ns56, ns57, ns58, ns59, ns60, ns61, ns62, ns63, ns64, ns65, ns66, ns67, ns68, ns69, ns70, ns71, ns72, ns73)
- helios a & b: e6 & e7 datasets
- ESA SREM (L1, L2 and V0):
- proba1
- integral
- herschel
- planck
- rosetta
- strv1c
- Himawari-8 SEDA (electron & proton)
- IMP8/CPME: electrons, helium, ions and protons
- IMP8/CRNC
- IMP8/GME
- Magnetospheric indices: AE, AP, CP, C9, Dst, EISN, F10, ISN, Kp, OMNI2, OMNI/HRO
- POES/SEM: METOP02, NOAA15-19
- PROBAV/EPT
- RBSP/HOPE L2/SA, L3/MO, L3/PA
- RBSP/MAGEIS L2, L3
- RBSP/REPT L2, L3
- RBSP/RPS L2
- SEPEM Reference Data Set (h_wbkg, he_nbkg, h_nbkg)
- WIND/EPACT
- WIND/MFI
- WIND/SWE
- XMM/ERMD