Arctic Data Centre

Information to be provided.

Information to be provided.

Information to be provided.

The Arctic Data Centre catalogue contains data both generated and owned by the Norwegian Meteorological Institute as well as data hosted on behalf of project partners and links to data hosted by partners, but made searchable through the Arctic Data Centre.

For further information on the data policy applying for data owned by the Norwegian Meteorological Institute, please visit https://www.met.no/en/free-meteorological-data/Licensing-and-crediting.

The Arctic Data Centre strongly encourage the concept of ethically free and open data in accordance with the IASC data policy. This also implies that the Arctic Data Centre strongly encourage data consumers to cite the data they use.

Introduction

A dataset is a collection of data. In the context of ADC it is not important whether the dataset is stored in relational databases or as physical files. Typically a dataset represent a number of variables in time and space. In order to best serve the data through the web services developed the following guidance exist for datasets within ADC:

1. A dataset can be a collection of variables stored in a relational database or flat files (e.g. NetCDF/CF).
2. A dataset is defined as a number of spatial and/or temporal variables.
3. A dataset should be defined by the information content and not the production method.

Some examples

For gridded data (e.g. numerical simulations or remote sensing products), FIXME

The software for automatic extraction of metadata (ACDD) to MMD (XML-files) is available at https://svn.met.no/projects/nc_to_mmd/nc_to_mmd.pl.

FIXME - more information to come...

Below, some compliance checkers that can be used to check datasets are listed.

NetCDF/CF checkers:

• http://cfconventions.org/compliance-checker.html
• http://puma.nerc.ac.uk/cgi-bin/cf-checker.pl
• Source code (standalone Python) https://pypi.python.org/pypi/cfchecker
• https://podaac-uat.jpl.nasa.gov/mcc/ (Both CF and ACDD)
• https://data.ioos.us/compliance/index.html
• https://github.com/ioos/compliance-checker(Both CF and ACDD)

ACDD checkers:

• https://podaac-uat.jpl.nasa.gov/mcc/ (Both CF and ACDD)
• https://data.ioos.us/compliance/index.html
• https://github.com/ioos/compliance-checker(Both CF and ACDD)

While speaking about metadata, it is important to remember that different types of metadata exist. From a data provider perspective, the two main categories to relate to are

• Discovery metadata are powering human and machine search efforts for data. This is similar to the index card in a library. It will tell where a certain dataset is located, who owns it, its content, how to access it and what you can do with it.
• In order to actually use the metadata to something useful, use metadata is required. Use metadata tell you the actual meaning of a number in the dataset. It links numbers to variables (whether temporal or geographical), tells how missing values are encoded, units of the variables etc.
• MM2, MMD, DIF, ISO19115 and ACDD are all discovery metadata.
• CF is use metadata.
• It is important to remember these differences and their application areas.

Some relevant information resources are listed below

• Information on discovery metadata standards
• http://gcmd.nasa.gov/add/standards/index.html
• GCMD Science Keywords (CSV file)
• How to generate a data management plan
• http://www.dcc.ac.uk/resources/data-management-plans

Documentation of data

1. Automatic extraction of discovery metadata from NetCDF/CF files adhering to the Attribute Convention for Data Discovery using ncdigest
2. A metadata editor providing information in MMD format.
1. This is currently only available inhouse, but will be integrated in the next portal solution to be deployed within 2017.

Hosting of data

1. Primarily for activities within the Norwegian Meteorological Institute, but relevant dataset from other data owners may also be handled following a request.
2. Data hosted should as a general rule be shared with all users of ADC, although exceptions may be discussed.
3. When data are hosted on disk, the purpose is to make them available to users using standardised metadata and data exchange protocols.
4. Data should preferably be in NetCDF following the Climate and Forecast Convention v 1.6 or higher.
5. Data should preferably have discovery metadata embedded following the Attribute Conventions  Dataset Disovery.
6. Data can be hosted on NIRD (NorStore) and made available through ADC.

Sharing of data

1. Data are made available through a THREDDS Data Server if hosted locally.
2. Data can be hosted elsewhere and made searchable in the catalogue provided correct dscovery metadata are available.
1. Accepted formats are
1. Local formats MM2 and MMD (compliance documents under development)
2. GCMD DIF
3. ISO19115 using GCMD Science Keywords for descriptions of parameters
4. If data contains ACDD (see above), discovery metadata can be extracted automatically
5. FGDC
2. Metadata can be harvested from other catalogues if in the formats above and served through OAI-PMH.
3. The preferred interfaces to data (where most functionality can be provided) are
1. OPeNDAP
2. OGC WMS

Searchable catalogue

1. Human and machine (OAI-PMH) interfaces.

The Arctic Data Centre (ADC) is working towards a distributed data management framework, in line with the activities of the combined SAON/IASC Arctic Data Committee. In order to publish your data in ADC, some background information is provided in this page. Further information will provided in sub pages.

On a general basis the process has the following steps:

1. Produce your data
1. Document your data
2. Do it right from the beginning
2. Share your data
1. Make it available through application servers offering interoperability interfaces (e.g. THREDDS)
2. Update documentation
3. Ingest metadata in the catalog

The fast lane

1. Document and format your data using NetCDF following the CF convention and using ACDD for discovery elements (links below).
1. Remember to add GCMD Science keywords (online resource further down) for the keywords attribute.
2. Contact (adc-support@met.no) to initiate the process of publishing data

Future plans

• Release of new human frontend and metadata harvesting backend.
• Enable support for PIDs/DOIs
• Upload and configuration service for THREDDS