[CF-metadata] Using add_offset & scale_factor with coordinate variables

Randy Horne rhorne at excaliburlabs.com
Wed Apr 11 11:23:14 MDT 2012


Jim:

In our use case, software is simplified, not because of the removal of a multiply and an addition, but as a result of the coordinate variable values always having values starting at 0, and ascending monotonically and sequentially (i.e. 0, 1, 2, 3 …n), and just pulling specific add_offsets and scale_factors out of a table where the table has an entry for each type of product.

vey respectfully,

randy


On Apr 11, 2012, at 1:05 PM, Jim Biard wrote:

> Hi.
> 
> I'm not against expanding the use of scale and offset, per se, but the reasons that you would like to use them seem (to me) to not be generally applicable.  Argument 1) implies that you obtain some sort of measurable benefit from saving one multiplication and one addition in your production software.  If you have optimized your code so well that this represents a major improvement, you are to be congratulated, but that won't be true for most.  Argument 2) does not appear to be something you can count on in every circumstance, nor do I think it advisable to imbue the scale_factor attribute with any such interpretation as a general rule.
> 
> I am finding it often the case for swath data that coordinate variables end up occupying a larger portion of a file than data variables.  I think it's a great idea to address this issue.  I must admit I'm not sure what the best way is.
> 
> Grace and peace,
> 
> Jim
> 
> Jim Biard
> Research Scholar
> Cooperative Institute for Climate and Satellites
> Remote Sensing and Applications Division
> National Climatic Data Center
> 151 Patton Ave, Asheville, NC 28801-5001
> 
> jim.biard at noaa.gov
> 828-271-4900
> 
> On Apr 11, 2012, at 10:50 AM, Randy Horne wrote:
> 
>> 
>> The allure of using add_offset & scale_factor with coordinate variables goes beyond saving space.   For example, in our use case …
>> 
>> (1)
>> Makes the s/w producing the product simpler.
>> 
>> (2)
>> The scale_factor also tells you what the horizontal spatial resolution is in a representation that aligns with the coordinate space.
>> 
>> 
>> very respectfully,
>> 
>> randy
>> 
>> 
>> 
>> On Apr 10, 2012, at 3:11 PM, Etienne Tourigny wrote:
>> 
>>> If netcdf-4 is an option for you, it might be simpler to compress
>>> (deflate) the coordinate variables, this works transparently.
>>> 
>>> Etienne
>>> 
>>> On Tue, Apr 10, 2012 at 2:40 PM, David Hassell
>>> <d.c.hassell at reading.ac.uk> wrote:
>>>> Hello Randy,
>>>> 
>>>> I, too, recently thought about this when working on some CF processing
>>>> software (cf-python), but couldn't think of a use case so didn't
>>>> pursue it - and now you have one!
>>>> 
>>>> It seems to me that the use of scale_factor and add_offset is intended
>>>> for (but not restricted to ...?) the reduction of dataset size
>>>> (chapter 8). Is the requirement in your example for space saving, or
>>>> some other convienience? I personally think that the latter case is
>>>> better served by an appropiate setting of the units attribute (such as
>>>> units="0.555555555555556 K @ 459.67" if we have converted from Kelvin
>>>> to Fahrenheit).
>>>> 
>>>> Anyway, I can't think of a reason not to allow these attributes on
>>>> coordinates - these variables can be very large, after all.
>>>> 
>>>> All the best,
>>>> 
>>>> David
>>>> 
>>>> ---- Original message from Randy Horne (01PM 05 Apr 12)
>>>> 
>>>>> Date: Thu, 5 Apr 2012 13:05:45 -0400
>>>>> From: Randy Horne <rhorne at excaliburlabs.com>
>>>>> To: cf-metadata at cgd.ucar.edu
>>>>> X-Mailer: <IMail v8.21>
>>>>> Subject: [CF-metadata] Using add_offset & scale_factor with coordinate
>>>>> variables
>>>>> 
>>>>> Folks:
>>>>> 
>>>>> Appendix A, Table A.1 in the CF conventions state that add_offset and scale_factor can be used with data variables, but not coordinate variables.
>>>>> 
>>>>> For the data producing system I am working (GOES-R Ground Segment) it is particularly convenient to make use of add_offset and scale_factor with our coordinate variables.
>>>>> 
>>>>> Is there an issue with changing the conventions to allow this ?
>>>>> 
>>>>> very respectfully,
>>>>> 
>>>>> randy
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> ..............End of Message ...............................-->
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> CF-metadata mailing list
>>>>> CF-metadata at cgd.ucar.edu
>>>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>>>> 
>>>> 
>>>> --
>>>> David Hassell
>>>> National Centre for Atmospheric Science (NCAS)
>>>> Department of Meteorology, University of Reading,
>>>> Earley Gate, PO Box 243,
>>>> Reading RG6 6BB, U.K.
>>>> 
>>>> Tel   : 0118 3785613
>>>> Fax   : 0118 3788316
>>>> E-mail: d.c.hassell at reading.ac.uk
>>>> _______________________________________________
>>>> CF-metadata mailing list
>>>> CF-metadata at cgd.ucar.edu
>>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>>> 
>> 
>> 
>> ____________________________________
>> 
>> Randy C. Horne (rhorne at excaliburlabs.com)
>> Principal Engineer, Excalibur Laboratories Inc.
>> voice & fax: (321) 952.5100
>> url: http://www.excaliburlabs.com
>> 
>> 
>> 
>> 
>> _______________________________________________
>> CF-metadata mailing list
>> CF-metadata at cgd.ucar.edu
>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> 
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata


____________________________________

Randy C. Horne (rhorne at excaliburlabs.com)
Principal Engineer, Excalibur Laboratories Inc.
voice & fax: (321) 952.5100
url: http://www.excaliburlabs.com




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20120411/708a6153/attachment-0001.html>


More information about the CF-metadata mailing list