GitHub

CDFDatasets

DOI version

CDFDatasets.jl is a Julia package for reading CDF (Common Data Format) files, commonly used in space physics and other scientific domains. It provides a Julia interface to CDF files using the CommonDataModel.jl interface. See CDF reader benchmarks for comparison with other CDF readers.

Installation

using Pkg
Pkg.add("CDFDatasets")

Quickstart

Here's a quick example using OMNI solar wind data:

using CDFDatasets

# Open a CDF dataset
omni_file = joinpath(pkgdir(CDFDatasets), "data/omni_coho1hr_merged_mag_plasma_20200501_v01.cdf")
ds = cdfopen(omni_file)
Dataset: /home/runner/work/CDFDatasets.jl/CDFDatasets.jl/data/omni_coho1hr_merged_mag_plasma_20200501_v01.cdf
Group: omni_coho1hr_merged_mag_plasma

Data variables
  heliographicLatitude (744) dims=Epoch [HelioGraphic Inertial (HGI) latitude; deg]
  heliographicLongitude (744) dims=Epoch [HGI longitude; deg]
  BR (744) dims=Epoch [BR in RTN (Radial-Tangential-Normal) coordinate system; nT]
  BT (744) dims=Epoch [BT in RTN coordinate system; nT]
  BN (744) dims=Epoch [BN in RTN coordinate system; nT]
  ABS_B (744) dims=Epoch [Field Magnitude Average |B  1/N SUM |B|; nT]
  V (744) dims=Epoch [Bulk Flow Speed; km/s]
  elevAngle (744) dims=Epoch [Proton flow elevation angle / latitude (RTN); Deg]
  azimuthAngle (744) dims=Epoch [Proton flow azimuth angle / longitude (RTN); Deg]
  N (744) dims=Epoch [Ion Density; N/cm3]
  T (744) dims=Epoch [Temperature; Deg K]
Support variables: Epoch
Global attributes
  28 attributes: Project, Discipline, Source_name, Data_type, Descriptor, Data_version, TITLE, TEXT, ...

Explore the dataset

julia> println("Variables: ", keys(ds))Variables: ["Epoch", "heliographicLatitude", "heliographicLongitude", "BR", "BT", "BN", "ABS_B", "V", "elevAngle", "azimuthAngle", "N", "T"]
julia> println("Attributes: ", keys(ds.attrib))Attributes: ["Project", "Discipline", "Source_name", "Data_type", "Descriptor", "Data_version", "TITLE", "TEXT", "MODS", "PI_name", "PI_affiliation", "Generation_date", "Acknowledgement", "ADID_ref", "Rules_of_use", "Instrument_type", "Generated_by", "Time_resolution", "Logical_file_id", "Logical_source", "Logical_source_description", "LINK_TEXT", "LINK_TITLE", "HTTP_LINK", "alt_logical_source", "Mission_group", "spase_DatasetResourceID", "DOI"]
julia> ds.attrib["Descriptor"]1-element Vector{String}:
 "merged magnetic field and plasma data from cohoweb"

Access variables

julia> ds["Epoch"]Epoch (744) dims= [Epoch Time; DD-MMM-YYYY_hr:mm]
julia> ds["Epoch"][[1,end]]2-element Vector{Epoch}:
 2020-05-01T00:00:00
 2020-05-31T23:00:00
julia> ds["BR"]BR (744) dims=Epoch [BR in RTN (Radial-Tangential-Normal) coordinate system; nT]
julia> ds["BR"].attribCommonDataModel.Attributes{CDFVariable{Float32, 1, CommonDataFormat.CDFVariable{Float32, 1, CommonDataFormat.VDR{Int64}, CommonDataFormat.CDFDataset{Int64}}, String, CDFDataset{CommonDataFormat.CDFDataset{Int64}, Nothing}, CommonDataFormat.LazyVAttrib{CommonDataFormat.CDFDataset{Int64}, Int32}}} with 11 entries:
  "FIELDNAM"     => "BR (RTN)"
  "CATDESC"      => "BR in RTN (Radial-Tangential-Normal) coordinate system"
  "VALIDMIN"     => Float32[-1000.0]
  "VALIDMAX"     => Float32[1000.0]
  "UNITS"        => "nT"
  "FORMAT"       => "f6.1"
  "FILLVAL"      => Float32[-1.0f31]
  "VAR_TYPE"     => "data"
  "DISPLAY_TYPE" => "time_series"
  "DEPEND_0"     => "Epoch"
  "LABLAXIS"     => "BR (RTN)"
# Calculate magnetic field magnitude
br = ds["BR"]
bt = ds["BT"]
bn = ds["BN"]
b_mag = sqrt.(br.^2 + bt.^2 + bn.^2) |> collect
744-element Vector{Float32}:
 3.544009
 2.9342802
 3.442383
 3.8961518
 0.64031243
 5.3721504
 5.4990907
 5.9514704
 4.3301272
 4.634652
 ⋮
 3.3301651
 2.9949956
 3.226453
 3.0380914
 3.828838
 3.3301651
 3.3674917
 3.749667
 3.8026307

API Reference

CDFDatasets.CDFDataset — Method
CDFDataset(file; backend = :julia)

Load the CDF dataset at the file path. The dataset supports the API of the JuliaGeo/CommonDataModel.jl.

backend controls the backend used to load the CDF dataset. Two options are available: :julia and :PyCDFpp. The default is :julia.

For PyCDFpp backend, we use lazy_load = true by default. If lazy_load = false, all variable values are immediately loaded.

source
CDFDatasets.cdfopen — Method
cdfopen(file; kw...) :: CDFDataset
cdfopen(files; kw...) :: CDFDataset

Opens CDF file(s) as a AbstractCDFDataset.

source
CDFDatasets.materialize — Method
materialize(var)::CDFVariable

Load variable data from disk into memory, preserving name, dataset link, and metadata.

source