API Reference

AbstractCoordinateSystem

Base abstract type for all coordinate system implementations.

AbstractCoordinateVector

Base abstract type to represent coordinates in a coordinate systems.

A variable v of a type derived from AbstractDataVariable should at least implement:

• Base.parent(v): the parent array of the variable

Optional:

• times(v): the timestamps of the variable
• units(v): the units of the variable
• meta(v): the metadata of the variable
• name(v): the name of the variable
• dim(v, i): the i-th dimension of the variable
• dim(v, name): the dimension named name of the variable

A specific realization of a reference system.

A Reference System is a scheme for orienting points in a space and describing how they transform to other systems.

See also: AbstractReferenceFrame

Abstract base for representing a point in a 3D coordinate system.

Reference:

• https://docs.astropy.org/en/stable/coordinates/representations.html
• https://github.com/JuliaEarth/CoordRefSystems.jl/blob/main/src/crs.jl

An iterator over continuous segments of a time array, where consecutive times differ by at most max_dt.

DataSet <: AbstractDataSet

A concrete dataset with a name, data (parameters), and metadata.

Construct a DataSet from a name and data, with optional metadata.

DataSet(ld::LDataSet; kwargs...)

Create a concrete DataSet from a Dataset template with specified data.

See also: LDataSet

Instrument <: AbstractInstrument

Fields

• name: The name of the instrument
• metadata: Additional metadata
• datasets: Collection of datasets

Construct an Instrument from a dictionary.

keyword-based constructor

LDataSet <: AbstractDataSet

A template for generating datasets with parameterized naming patterns.

Fields

• format: Format string pattern for the dataset name
• data: Dictionary of variable patterns
• metadata: Additional metadata

Examples

using SPEDAS.MMS

# Access FPI dataset specification
lds = mms.datasets.fpi_moms

# Create a concrete dataset with specific parameters
ds = DataSet(lds; probe=1, data_rate="fast", data_type="des")

The format string and variable patterns use placeholders like {probe}, {data_rate}, which are replaced with actual values when creating a concrete DataSet.

NoMetadata

Indicates an object has no metadata. But unlike using nothing, get, keys and haskey will still work on it, get always returning the fallback argument. keys returns () while haskey always returns false.

Project <: AbstractProject

A representation of a project or mission containing instruments and datasets.

Fields

• name: The name of the project
• metadata: Additional metadata
• instruments: Collection of instruments
• datasets: Collection of datasets

keyword-based constructor

Frames can depend on epoch and planetary orientation models

An iterator over a series of time ranges that are window long, with a cadence of cadence.

Create a new product with the composed function

dim(x, i)
dim(x, name)

Get the i-th dimension of object x, or the dimension associated with the given name. The default implementation returns axes(x, i).

A dimension may be time-varying or dependent on other dimensions; in such cases, the effective size of the corresponding array dimension ndims can be greater than 1.

getcsys(x)

Get the coordinate system of x. Ideally, this should return both the reference frame and coordinate representation.

If x is a instance of AbstractCoordinateSystem, return x itself. If x is a type of AbstractCoordinateSystem, return an instance of the coordinate system, i.e. x().

This is a generic function, packages should extend it for their own types.

getmeta(x, key, default=nothing)

Get metadata value associated with key for object x, or default if key is not present.

getmeta(x)

Get metadata for object x. If x does not have metadata, return NoMetadata().

setmeta(x, key => value, ...; symbolkey => value2, ...)
setmeta(x, dict::AbstractDict)

Update metadata for object x for key key to have value value and return x.

setmeta!(x, key => value, ...; symbolkey => value2, ...)
setmeta!(x, dict::AbstractDict)

Update metadata for object x in-place and return x. The metadata container must be mutable.

The arguments could be multiple key-value pairs or a dictionary of metadata; keyword arguments are also accepted.

Examples

setmeta!(x, :units => "m/s", :source => "sensor")
setmeta!(x, Dict(:units => "m/s", :quality => "good"))
setmeta!(x; units="m/s", calibrated=true)

Throws an error if the metadata is not mutable. Use setmeta for immutable metadata.

cotrans(out, A, [times]; in=get_coord(A), backend=GeoCotrans)
cotrans(in => out, A, [times]; backend=GeoCotrans)

Transform data to the out coordinate system.

By default, this uses Julia's GeoCotrans. Use backend = IRBEM after loading IRBEM.jl to call Fortran's IRBEM implementation.

References:

• IRBEM-LIB: compute magnetic coordinates and perform coordinate conversions (Documentation, IRBEM.jl)
• SPEDAS Cotrans

Julia-based Space Physics Environment Data Analysis Software

See the Documentation for more information.

amap(f, a, b)

Apply f to the intersection of a and b along shared selectors.

https://github.com/rafaqz/DimensionalData.jl/issues/914

cotrans(out, A, [times]; in=get_coord(A), backend=GeoCotrans)
cotrans(in => out, A, [times]; backend=GeoCotrans)

Transform data to the out coordinate system.

By default, this uses Julia's GeoCotrans. Use backend = IRBEM after loading IRBEM.jl to call Fortran's IRBEM implementation.

References:

• IRBEM-LIB: compute magnetic coordinates and perform coordinate conversions (Documentation, IRBEM.jl)
• SPEDAS Cotrans

current_density(B, V)

Calculate the current density time series from magnetic field (B) and plasma velocity (V) time series.

Assume 1-D structure along the z-direction. Remember to transform the coordinates of B and V first (e.g. using mva

fac_mat(vec::AbstractVector; xref=[1.0, 0.0, 0.0])

Generates a field-aligned coordinate (FAC) transformation matrix for a vector.

Arguments

• vec: A 3-element vector representing the magnetic field

fill_gaps(times, data; resolution, margin)

Given a sorted vector of time stamps times and corresponding data values, this function inserts missing time stamps with a value of NaN if the gap between consecutive time stamps is larger than resolution + margin.

• If the gap is only slightly larger (within margin of the resolution), no gap is inserted.
• The function supports numeric times or DateTime (with appropriate resolution types).

Arguments

• times: Sorted vector of time stamps.
• resolution: The expected time difference between consecutive time stamps.
• margin: Allowed deviation from resolution before inserting missing time stamps.

Returns

A tuple (full_times, full_values) where:

• full_times is a vector containing all time stamps (original and inserted).
• full_values is a vector of data values with NaN for inserted gaps.

References

• https://pyspedas.readthedocs.io/en/latest/modules/pytplot/tplotmath/degap.html

resample(arr, n; dim=1, verbose=false)

Resample an array along the dimension dim to n points. If the original length is less than or equal to n, the original array is returned unchanged.

rotate(da, mats)

Rotate a dimensioned array using a vector of rotation matrices aligned to its time axis. Requires DimensionalData to be loaded.

rotate(ts::AbstractMatrix, mat::AbstractMatrix)

Coordinate-aware transformation of vector/matrix by rotation matrix(s) mat(s). Assume ts is a matrix of shape (n, 3).

select_rotate(da, mats; selectors=Near())
select_rotate(da, mats, coord; kwargs...)

Rotate a dimensioned array using nearest-neighbor matched rotation matrices. Requires DimensionalData to be loaded.

Set the coordinate system.

Updates legend names and axis labels if they include the coordinate system. Also updates the dimension name if it contains the coordinate system.

Reference:

• https://pyspedas.readthedocs.io/en/latest/modules/pytplot/dataattgetterssetters.html#set_coords

tlingradest(fields, positions)

Interpolate and Compute spatial derivatives such as grad, div, curl and curvature using reciprocal vector technique.

tresample(da, n; dim=nothing)

Resample a dimensioned array along its time dimension (or dim) to n points.

@load_project_config(file)

Load configuration from a file and export all key-value pairs as constants. The macro evaluates in the calling module's context.

TimeseriesUtilities

A collection of utilities to simplify common time series analysis.

From data cleaning to arithmetic operations (e.g. linear algebra) to common time series operations (e.g. resampling, filtering).

Data Cleaning

• find_outliers, find_outliers_median, find_outliers_mean
• replace_outliers, replace_outliers!

Query

• times, time_grid
• timerange, common_timerange

(Windowed) Statistics

• Base: tstat - tstat(f, x, [dt]; dim)
• NaNStatistics wrappers: tmean, tmedian, tsum, tvar, tstd, tsem

Algebra

• tcross, tdot, tnorm
• tsproj, tproj, toproj
• tsubtract, tderiv

Time-Domain Operations

• tselect
• tclip, tclips
• tview
• tmask and tmask!
• tshift
• tsplit
• tgroupby
• Resampling: tinterp, tsync

Time-Frequency Domain Operations

• tfilter, pspectrum

ContinuousTimeRanges(times, max_dt)

Iterator of (t_start, t_stop) spans over times, splitting wherever consecutive values differ by more than max_dt.

DiffQ(v, t; dim=1)

Difference quotient of v with respect to t.

Date/DateTime differences are converted to seconds before division.

GroupByDynamic(times, every, period=every, start_by=:window)

Iterator over (index_range, window_start) pairs for sliding windows on sorted times.

IntervalRange{T, D}

Lazy iterator of (t_start, t_end) pairs splitting [t0, t1) into dt-sized windows.

WindowedView(data, coords, windows; dim=1)
WindowedView(data, coords, before, after; dim=1)

AbstractVector of views into data, one per time window. wv[i] returns the slice of data along dimension dim whose coords fall within the i-th window.

windows is any indexable source of (t_start, t_stop) pairs (e.g. TimeRanges). The before/after form builds a symmetric window around each point in coords. Requires coords to be sorted.

common_timerange(x1, xs...)

Get the common time range (intersection) across multiple arrays. If there is no overlap, returns nothing.

dimnum(x, dim)

Get the ordinal of the dimension dim in x.

dropna(A; dim=nothing)

Remove slices containing NaN values along dimension dim.

find_continuous_timeranges(x, max_dt)

Find continuous time ranges for x, where max_dt is the maximum time gap between consecutive times.

find_outliers(A, [method, window]; dim = nothing, kw...)

Find outliers in data A along the specified dim dimension.

Returns a Boolean array whose elements are true when an outlier is detected in the corresponding element of A.

The default method is :median (other option is :mean), which uses the median absolute deviation (MAD) to detect outliers. When the length of A is greater than 256, it uses a moving window of size 16.

See also: find_outliers_median, find_outliers_mean, isoutlier - MATLAB

find_outliers_mean(x::AbstractVector, window; threshold = 3)

Find outliers that are defined as elements more than three standard deviations from the mean.

This method is faster but less robust than find_outliers_median.

find_outliers_median(x, window; threshold=3)

Find outliers that are defined as elements more than threshold=3 times the scaled median absolute deviation (MAD) from the median.

When window is set to a integer, a moving window of that size is used to compute local MAD. Otherwise, global statistics are used.

References

• Median absolute deviation - Wikipedia

groupby_dynamic(x, every, period=every, start_by=:window)

Eagerly collect (group_idx, starts) vectors for sorted x.

interpolate_outliers!(x, t, outliers)

Interpolate outliers in x using interpolation of neighboring non-outlier values.

Vector rejection

proj(a, b)

Vector projection of a vector a on (or onto) a nonzero vector b.

References: Wikipedia

See also: sproj, oproj

pspectrum(A, times; dim=ndims(A), nfft=256, noverlap=div(nfft, 2), window=DSP.hamming)
pspectrum(A; dim=nothing, freqdim=:frequency, kwargs...)

Compute a time-frequency power spectrum with DSP.spectrogram.

For containers that implement the common axis API, times are inferred from the selected axis and the result is rebuilt with frequency and spectrogram time axes followed by the remaining axes.

replace_outliers!(A, s, [find_method, window]; kwargs...)

Finds outliers in A and replaces them with s (by default: NaN).

See also: find_outliers, filloutliers - MATLAB

replace_outliers!(A, method, [find_method, window]; kwargs...)
replace_outliers!(A, method, outliers; kwargs...)

Replaces outliers in A with values determined by the specified method.

Outliers can be detected using find_outliers with optional find_method and window parameters or specified directly as a Boolean array outliers.

method can be one of the following:

• :linear: Linear interpolation of neighboring, nonoutlier values
• :previous: Previous nonoutlier value
• :next: Next nonoutlier value
• :nearest: Nearest nonoutlier value

See also: filloutliers - MATLAB

replace_outliers(A; args...; kw...)

Non-mutable version of replace_outliers!.

smooth(data, times, window; dim=ndims(data))
smooth(data, window; dim=ndims(data))

Smooths a time series by computing a moving average over a sliding window. Edge windows are truncated, so the output has the same size as the input.

Each window covers the half-open interval [coord - before, coord + after). A scalar window is interpreted as a coordinate span along the smoothed axis.

Arguments

• dim=ndims(data): Dimension along which to perform smoothing
• op=nanmean: Function used to aggregate each window

Scalar projection

tclip(A, t0, t1; dim=nothing)

Clip A to time range [t0, t1] along dimension dim.

dim should be sorted before clipping (see tsort).

tclips(xs...; trange=common_timerange(xs...))

Clip multiple arrays to a common time range trange.

If trange is not provided, automatically finds the common time range across all input arrays.

tcross(x, y; dim=nothing)

Compute the cross product of two (arrays of) vectors along dimension dim.

tderiv(A, times; dim=1)
tderiv(A; dim=nothing)

Compute the time derivative of A. Set lazy=true for lazy evaluation.

See also: deriv_data - PySPEDAS

tdot(x, y; dim=nothing)

Dot product of two arrays x and y along dimension dim.

tfilter(A, times, Wn1, Wn2=nothing; dim=ndims(A), designmethod=nothing)
tfilter(A, Wn1, Wn2=nothing; dim=nothing, designmethod=nothing)

Bandpass filter A between Wn1 and Wn2. The upper cutoff defaults to the Nyquist frequency.

References

• https://docs.juliadsp.org/stable/filters/
• https://www.mathworks.com/help/signal/ref/filtfilt.html
• https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.filtfilt.html

tgroupby(x, every, period = every, start_by = :window)

Group x into windows based on every and period.

time_grid(x, dt)

Create a time grid from the minimum to maximum time in x with the step size dt.

Examples

# Create hourly time grid
time_grid(x, Hour(1))
time_grid(x, 1u"hr")

# Create 1-s intervals
time_grid(x, Second(1))
time_grid(x, 1u"second")
time_grid(x, 1u"Hz")

timerange(x)
timerange(x1, xs...)

Get the time range of time series data x.

For a single argument, returns a tuple (tmin, tmax) containing the minimum and maximum times. For multiple arguments, returns the common time range (intersection) across all arrays - equivalent to common_timerange(x1, xs...).

Examples

# Single time series
times = [1, 2, 3, 4, 5]
timerange(times)  # (1, 5)

# Multiple time series - find common range
x1_times = [1, 2, 3, 4]
x2_times = [2, 3, 4, 5]
timerange(x1_times, x2_times)  # (2, 4)

See also: common_timerange, tminimum, tmaximum

times(x)

Get time coordinate of x.

tinterp(A, old_times, new_times; interp=LinearInterpolation)

Interpolate time series A at new time points new_times.

The interp constructor must accept (u, t; kws...) and return a callable object. Its syntax is compatible with DataInterpolations.jl.

Examples

# Interpolate at a single time point
tinterp(time_series, DateTime("2023-01-01T12:00:00"))

# Interpolate at multiple time points using cubic spline interpolation
new_times = DateTime("2023-01-01"):Hour(1):DateTime("2023-01-02")
tinterp(time_series, new_times; interp = CubicSpline)

tinterp_nans(A; dim = nothing, kwargs...)

Interpolate only the NaN values in A along dimension dim.

tmask!(A, t0, t1; dim=nothing)
tmask!(A, its; dim=nothing)

Mask all data values within the specified time range(s) (t0, t1) / its with NaN.

tmask(A, args...; kwargs...)

Non-mutable version of tmask!. See also tmask!.

tmaximum(x)

Get the maximum timestamp of x.

tmean(x, [dt]; dim=nothing)

Calculate the arithmetic mean of x along dimension dim, optionally grouped by dt.

dim accepts integer index, dimension type/instance, or nothing (defaults to the time dimension).

tmedian(x, [dt]; dim=nothing)

Calculate the median of x along dimension dim, optionally grouped by dt.

dim accepts integer index, dimension type/instance, or nothing (defaults to the time dimension).

tminimum(x)

Get the minimum timestamp of x.

tnorm(A; dim=nothing)

Compute the norm of each slice in A along dimension dim.

See also: tnorm_combine

tnorm_combine(x; dim=nothing, name=:magnitude)

Calculate the norm of each slice along dim and combine it with the original components.

toproj(A, B; dim=nothing)

Compute vector rejection (orthogonal projection) of A from B along dimension dim.

tproj(A, B; dim=nothing)

Compute vector projection of A onto B along dimension dim.

tresample(A, old_times, freq; kw...)

Resample time series A onto a regular time grid with the specified frequency freq.

See also: tinterp, time_grid

tselect(A, t; dim=nothing)
tselect(A, t, δt; dim=nothing)

Select the value of A at time nearest to t.

With δt, restricts the search to [t-δt, t+δt] and returns missing if that window is empty.

tsem(x, [dt]; dim=nothing)

Calculate the standard error of the mean of x along dimension dim, optionally grouped by dt.

dim accepts integer index, dimension type/instance, or nothing (defaults to the time dimension).

tshift(x, t0 = nothing; dim=nothing)

Shift the dim of x by t0.

tsort(A; dim=nothing, rev=false)

Sort A along dimension dim.

tsplit(t0, t1, n::Int)
tsplit(t0, t1, dt)
tsplit(t0, t1, dtType::Type{<:Period})

Split the range from t0 to t1 into n parts, dt-sized parts, or by period type (e.g., Month, Day).

tsproj(A, B; dim=nothing)

Compute scalar projection of A onto B along dimension dim.

tstat(f, x, [dt]; dim = nothing)

Calculate the statistic f of x along the dim dimension, optionally grouped by dt.

See also: groupby_dynamic

tstd(x, [dt]; dim=nothing)

Calculate the standard deviation of x along dimension dim, optionally grouped by dt.

dim accepts integer index, dimension type/instance, or nothing (defaults to the time dimension).

tsubtract(x, op=nanmedian; dim=nothing)

Subtract a statistic op along time dimension (or dim) from x.

tsum(x, [dt]; dim=nothing)

Calculate the sum of x along dimension dim, optionally grouped by dt.

dim accepts integer index, dimension type/instance, or nothing (defaults to the time dimension).

tsync(A, Bs...)

Synchronize multiple time series to have the same time points.

This function aligns time series Bs... to match time points of A by:

1. Finding common time range between all time series
2. Extracting subset of A within common range
3. Interpolating each series in Bs... to match the time points of the subset of A

Examples

A_sync, B_sync, C_sync = tsync(A, B, C)

See also: tinterp, common_timerange

tvar(x, [dt]; dim=nothing)

Calculate the variance of x along dimension dim, optionally grouped by dt.

dim accepts integer index, dimension type/instance, or nothing (defaults to the time dimension).

tview(A, t0, t1; dim=nothing)

View A in time range [t0, t1] along dimension dim.

unwrap(x)

Return the innermost object (array) of wrapped x with similar behavior (e.g. same size, same type, etc.)

window_bf_sizes(window)

Converts a window specification to backward and forward window sizes.

When window is a positive integer scalar, the window is centered about the current element and contains window-1 neighboring elements. If window is even, then the window is centered about the current and previous elements.