Parameter: Difference between revisions

From PANGAEA Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
Line 10: Line 10:
# Clearly separate parameters from methods or any other specifications. [[Method]]s are defined in the method table; the relation between a data series and the method or data specific comments is set during the import.
# Clearly separate parameters from methods or any other specifications. [[Method]]s are defined in the method table; the relation between a data series and the method or data specific comments is set during the import.
# New parameters are defined by the [[user:Sschumacher|data librarian]]. Please use the
# New parameters are defined by the [[user:Sschumacher|data librarian]]. Please use the
  [http://epic.awi.de/30028/3/InpParameter.xlsx '''PARAMETER IMPORT FORM''']
  [hdl:10013/epic.38464.d003 '''PARAMETER IMPORT FORM''']
for new definitions and submit as '''excel or text/zipped format''' via the '''[http://www.pangaea.de/submit ticket system]'''. There are two forms of parameter submission issues: 1. Project ''PANGAEA Data Archiving & Publication'', Issue Type ''Parameter Submission'' for general parameter requests. 2. An issue-related parameter request, choose the drop-down menu ''More Actions'' in the main data submission issue. Go to ''Create Sub-Task'' and chosse the Issue Type ''New Parameter''.  
for new definitions and submit as '''excel or text/zipped format''' via the '''[http://www.pangaea.de/submit ticket system]'''. There are two forms of parameter submission issues: 1. Project ''PANGAEA Data Archiving & Publication'', Issue Type ''Parameter Submission'' for general parameter requests. 2. An issue-related parameter request, choose the drop-down menu ''More Actions'' in the main data submission issue. Go to ''Create Sub-Task'' and chosse the Issue Type ''New Parameter''.  



Revision as of 2016-09-30T07:10:08

Window for parameter definition/editing

The parameter table contains all parameters with unit and ID, grouped by categories (Parameter group, see also discussion page).

When defining new parameters, please keep in mind:

  1. First check for existing parameters by using the 4D-client or the Parameter Dictionary. When using the 4D-client also use the search function -> Parameter contains subset of parameter name or Abbreviation is equal to parameter abbreviation. E.g., a search for 0.063-0.032 mm fraction [%] -> Parameter contains 0.063 mm will list all parameters containing 0.063 mm. Search for Fe % -> Abbreviation is equal to Fe will list all Iron-parameters with the abbreviation Fe (not Fe2+, Fe peak area etc.).
  2. Avoid duplicate definitions of parameters at any time! If the parameter already exists with a unit different from the one needed and the data can easily be converted, a new parameter should NOT be defined and the data must be converted prior to or during import. (It is one of the major challanges of Pangaea, that it delivers data in a consistent format, which also means it uses standard units as far as defined in science.)
  3. Do not define parameters with user specific 'qualifiers', e.g. in species names something like Thalassiosira sp. F. In this case the data should be linked to the parameter Thalassiosira sp. and the data series comment should contain sp. F (see also import; for the use of abbreviations in taxonomic names see Taxon).
  4. Do not define parameters containing two different individuals, e.g. Convallina logani/dawsoni. Instead use Convallina logani and add to the data series comment including Convallina dawsoni. Do not define any mixed parameters accordingly.
  5. Clearly separate parameters from methods or any other specifications. Methods are defined in the method table; the relation between a data series and the method or data specific comments is set during the import.
  6. New parameters are defined by the data librarian. Please use the
[hdl:10013/epic.38464.d003 PARAMETER IMPORT FORM]

for new definitions and submit as excel or text/zipped format via the ticket system. There are two forms of parameter submission issues: 1. Project PANGAEA Data Archiving & Publication, Issue Type Parameter Submission for general parameter requests. 2. An issue-related parameter request, choose the drop-down menu More Actions in the main data submission issue. Go to Create Sub-Task and chosse the Issue Type New Parameter.

Also request of changes of a parameter definition must be submitted as ticket, use the general parameter request Parameter Submission.

A nearly unlimited number of parameters can be used in a data set. An example set containing 550 parameter (columns) is doi:10.1594/PANGAEA.614749


Field description of the parameter table

Mandatory fields are underlined; unused columns and lines in import files should be deleted.

  • Parameter Name contains the full name of the parameter. Parameter Name in combination with the unit must be unique in Pangaea. Parameter names have the most important specification in the beginning, followed by all describing therms in hierarchic order. Example: when dissolved organic carbon is measured, carbon is the main parameter information, followed by organic and the description of it's condition dissolved = Parameter Name Carbon, organic, dissolved. Do not use abbreviations in the parameter name. Compare the new parameter name with existing parameters and follow their syntax.
  • Abbreviation or short name of the parameter; is used in the header of data sets. The Abbreviation gives a short form of the parameter name, it never contains comma. Delta notation is given by d, standard deviation and standard errors are std dev and std e. Example: abbreviation of the parameter Carbon, organic, dissolved is DOC.
  • Unit should be given for all numeric-parameters and should follow standard use i.a. already existing parameter should only be defined with an other unit if the values can not be converted from one to an other. The unit does not contain chemical formulas, elements or the sampling milieu. This must be given in the Parameter name. Text-parameters have no unit, delete the unit column in import form or leave empty. Example: parameter Carbon, organic, dissolved has the unit µmol/kg not µmol C/kg.
  • Data group (DataGroup) describes the data status. Give the full therm, e.g. Primary data in the import file, Primary is not valid. Example: Carbon, organic, dissolved in µmol/kg is Secondary data.
    • Primary data are raw data, e.g. counts of microfossils or weight of samples.
    • Secondary data are those calclulated from the raw data and are the most commonly used, e.g. concentrations in %, mg/kg, #/cm**3 etc.
    • Tertiary data are mostly calculated data on a higher level of scientific interpretation, e.g. paleotemperatures.
    • not classified can be used for technical parameters.
  • LowerLimit/UpperLimit can be used to define the numeric range of values in which a certain parameter will occur. An internal routine will check during the import of data for outliers and will flag them as not valid. Delete columns for text-parameters in import form or leave empty.
  • Default format some predefined formats are offert by a menue but can be eddited by hand. The format should follow the general precision and will be used by the system as the default. Format can be changed during (or after) the import of data on the config card.
  • Default data type of a parameter can be numeric (1) or text (2). If a text parameter is defined, no unit, format and min/max values should be given. A field of a text parameter may contain up to 255 characters. Example: Carbon, organic, dissolved [µmol/kg] is a numeric parameter.
  • Default method (DefaultMethodID) is a relational field to the Method table where a required method has to be defined first. Methods defined in this field are shown during import of data by default. The default can be changed during the import procedure. Use the ID of the method for import. Example: Carbon, organic, dissolved is calculated, Method ID 50.
  • Reference (ReferenceID) can be given in case a parameter was defined through a publication; relational to Reference. Use the ID of the referenceI for import. There reference is not shown in the dataset. it is better to use the URL field
  • URL may contain a link to a more detailed explanation/definition of the parameter, e.g. in Wikipedia or in a paper. You can give a link or a DOI. This definition should be of general use. For species parameter the URL field will automatically filled with a link to a taxonomic database (e.g. ITIS, WORMS).
  • Comment (Description) may be used for any descriptions, helpfull to other curators to understand its meaning. This is an internal info field, its content does not appear in datasets!
  • Keywords may be used to define a certain parameter group for special purpose, projects or users. Keyword-related parameter lists can be produced using the DDI tool to set up a dynamic link; relational to the Thesaurus. Field not included in the parameter import form. Keywords need to be added by hand for each parameter.


Parameter import file for the Example: Carbon, organic, dissolved [mmol/kg]


Some useful tips for parameter names:


parameter type help for definition Parameter Name in PANGAEA Abbreviation in PANGAEA Unit in PANGAEA
ratio of elements, compounds etc. given with / and ratio in name Iron/manganese ratio Fe/Mn no unit, or µmol/mol
capitalization first word of parameter and proper noun are capitalized Calcium carbonate CaCO3
first word or phrase of abbreviation Flood activity Flood act
complexe names, chemistry give full names, no abbreviations in parameter names Calcification rate of calcium carbonate Calc rate CaCO3

Find missing parameter in an import file with Split2Events

  1. Install Split2Events Split2Events
  2. Prepare a local list of all parameters called ParameterDB.pdb. For this step use Tools->Refresh parameter database or Tools->Merge new parameters to parameter database. The name of the parameter database can be given first with File->General options....
  3. Open your import file.
  4. Create a metadata file with the option use metadata file; find parameter by name. Split2Events identifies the ID for each parameter by using the parameter database. If a parameter is unknown, the ID is set to unknown. If write parameter import file on the Options tab was checked, a list of the unknown parameters is written to imp_Parameters.txt. Proof carefully if the unknown parameters are real new parameters or if they exist with an other spelling. After checking the new parameters, completing the parameter import file create an issue (http://issues.pangaea.de) and upload it to the issue.
  5. After the parameters are imported, go to Split2Events and use Tools->Merge new parameters to parameter database.
  6. Continue with step 3.

How is a parameter recognized during import?

  • ID of parameter
  • Parameter name with unit in square bracket, e.g. Equivalent dose [mSv]
  • Parameter abbreviation with unit, e.g. H [mSv]. Be careful the combination of abbreviation and unit is not unique. Example: C. wuellerstorfi [%] can be Cibicidoides or Cibicides wuellerstorfi.

In principle, parameter definitions should be unique but the import using parameter (short) names may still produce some ambiguous parameter messages.


Check taxonomic parameters in the pangaea parameter table - the script (by Robert Huber) matches the pangaea parameter table with the species catalog of UBIO