Partners header

Integration Plan:
Data Standards

Software development, prototyping, testing and validation, software tools, common data formats, networking, client-server architecture.
DRAFT Release 0.1, 17 November 1996
Author: Kurt Fedra

Data Exchange Formats

For the data exchange between the project participants, the following data standards are recommended:

  • Text files:
    • plain ascii
    • HTML
    • Word (*.doc files): uuencoded, as mail attachments, or by binary ftp transfer
    • RTF files (ascii)
  • Imagery and graphics:
    • GIF files (please note that larger files should NOT be sent as mail attachments, but transferred by ftp;
    • TIFF (5.0, compressed) for 24-bit data
  • GIS data:
    • Arc/Info export format
    • DLG (USGS Digital Line Graph format)
    • DXF (least desirable)
    • satellite imagery and raster data in any convenient format INCLUDING a format description.
  • Model Data:
    • ascii data (with an appropriate format description, and where appropriate with the necessary reading routines)
    • binary data: together with description and the appropriate reading routines;

Client-Server Data Exchange

Client-server data exchange is based on the http protocol. The Demonstrator uses initially simple ascii files based on the protocol specifications for model scenarios and model result transfer. (see also: Architecture Specifications

This uses a simple protocol of keyword:value pairs, for example:
        StartTime:199710250000 # yyyymmddhhmm 
        TimeStep:1480    # in minutes
        RunLength:96     # in time steps
        NoxIndustry:1.2  # multipliers for data matrices

Meta-Data: The HDF Hierarchical Data Format

HDF is a library and platform independent data format for the storage and exchange of scientific data. It includes Fortran and C calling interfaces, and utilities for analyzing and converting HDF data files.

HDF is developed and supported by NCSA, and is available in the public domain. HDF is used world-wide in many fields, including Environmental Science, Neutron Scattering, Non-Destructive Testing, and Aerospace, to name a few. Scientific projects that use HDF include NASA's Mission to Planet Earth, and the DOE's Accelerated Strategic Computing Initiative. A detailed description of HDF is available from the NCSA's web server .

The primary intended application of HDF in ECOSIM is for the storage and Internet publishing of model scenarios

The main relevant features are:

  • HDF stands for Hierarchical Data Format. It is a multi-object file format for the transfer of graphical and numerical data between machines.

  • HDF is a versatile file format. It supports six different data models. Each data model defines a specific type of data and provides a convenient interface for reading, writing, and organizing a unique set of data elements.

  • HDF is a self-describing format, allowing an application to interpret the structure and contents of a file without any outside information.

  • HDF is a flexible file format. With HDF, you can group sets of related objects together and then access them as a group or as individual objects. There are pre-defined sets for raster images and floating point multidimensional arrays. Users can also create their own grouping structures using an HDF feature called vgroups.

  • HDF is an extensible file format. It can easily accommodate new data models, regardless of whether they are added by the HDF development team or by HDF users.

  • HDF is a portable file format. HDF files can be shared across platforms. An HDF file created on one computer, say a Cray supercomputer, can be read on another system, say IBM PC, without modification.

  • HDF is available in the public domain.

Copyright 1995-2002 by:   ESS   Environmental Software and Services GmbH AUSTRIA