DVOS Data Object Library

Contents



Data Variable Object Scheme

The purpose of the new data handling core (DVOS) is to remove performance bottlenecks inm QDOS. At the same time it should simplify coding within QSAS and plugins, and make the whole package easier to maintain when it moves wholely into public domain maintenance.

Specifically, in contrast to QDOS, data objects are a single class and not a hierarchy of classes handling different data types and dimensionality. Objects are still accessed through var pointers, which allows for data to be self-destroying when out of scope, but it is no longer necessary to narrow objects to specific sub-classes (scalar, matrix, scalar sequence, matrix sequence etc).

The single class handles all necessary data conversions and dimensions transparently, which in turn means less coding in the calling modules, and hence easier maintenance. The reduction in layers of abstraction also make it easier for the community to understand and maintain the core code itself. As many operations as possible are handled by the DVOS core, and this includes intelligent handling of fill values and metadata. All operations supported are safe against the underlying data types and invalid operations will do nothing (rather than throw an exception), so calling a supported method is always safe. The only exception to this are a few explicit de-reference methods used for fast access directly into the data valarrays. They are to be used only for looping explicitly through the data when dimensionality and data type has been checked. Safe (but slower) options for each of these are also supported.

The data itself is held in a valarray, and the object knows how to unpack the records and matrix dimensions appropriately. This ensures that copying data and constructing objects that are sequences of arrays is very much faster than in QDOS where matrix sequences were sequences of var pointers to matrices. Most methods and operators are handled at the top level by the object so as to minimise the number of operations that require stepping through the valarray data itself. Where possible valarray supported methods are used to avoid explicit valarray element access, and for this the fill values are set to NaN for double data to ensure they remain unaltered by arithmetic operations.



Dvar 'Pointers'

The Dvar template class provides a wrapper for data objects that ensures they are deleted when they go out of scope. They are not strictly pointers, but are a class containing a pointer to the data object and a dereference operator (->) defined such that they may be used like pointers to the data object.

Every time a var pointer is created, the reference counter for the target object is incremented, and whenever a Dvar object is deleted (or goes out of scope) the reference counter is decremented. When the counter is zero the data object is deleted.

Explicitly the QSAS data variable object class, DvObject, may be accessed via the Dvar pointer class for it, DvObject_var. From here on we shall discuss the DvObject_var derived class. The class contains a pointer to a DvObject class data object which is nesver null.

There are three constructors:

DvObject_var () constructs an 'empty' DvObject (see the DvObject class) and returns is_nil() as true.

DvObject_var (const DvObject_var &T) constructs an object that points to the same object that T points to. The object is not copied or changed other than to increment its reference pointer.

DvObject_var ( DvObject *P) constructs an object that points to the object P. The object is not copied or changed other than to increment its reference pointer.

There is an assignment operator =, that takes a DvObject pointer as argument, and this is used to assign a Dvar pointer to an object created with the new command. Additionally there is a similar assignment operator =, that takes a DvObject_var reference as argument.

For example, where (...) refers to any argument list taken by DvObject constructors:

DvObject_var dobj; // creates a pointer to an 'empty' data object (is_nil() returns true)
dobj = new DvObject(...); // creates a new data object and points dobj to it (deleting orphaned object automatically, e.g. 'empty' object above)
DvObject_var dobj2 = dobj; // creates a new Dvar pointer to the object in dobj
DvObject_var dobj3(dobj); // also creates a new Dvar pointer to the object in dobj
dobj3 += dobj; // adds the content of dobj to that of dobj3 (and ensures units are the same, converting a copy of dobj on the fly as needed)

Note that a new object is usually created using the assignment operator (through the explicit 'new') and these take any of the constructors for the DvObject class as the RHS.

The ptr() method gives direct access to the pointer to the DvObject data object inside Dvar. 

The de-reference operator ->,  gives direct access to the methods available in the DvObject data class, while the operator, *, gives access to the data object itself. Hence dobj->seqSize() and (*dobj).seqSize() both return the length of the data sequence provided by the DvObject method seqSize().

The comparison operators == and != compare the data object_id values. Each non-empty object has a unique id, which can be found from dobj->get_id(), and these comparisons determine whether the Dvar pointers refer to exactly the same target data object. To compare data values other operators are provided in the DvObject data class, for example *dobj1 == *dobj2 compares the data arrays in each object.
'Empty' objects all have id = -1, and are therefore always equal to each other.

The bool methods  is_nil() and  is_ok() return the equivalent methods from the data class, so dobj->is_nil() and dobj.is_nil() are equivalent. If there is data in the target data object, then is_nil() is false and is_ok() is true, and vice versa. The pointer to the enclosed data object is never actually nil, and all methods can be called safely. Data object methods that are not applicable to the underlying data type do nothing but attach an error message to any returned object.

Note, however, that the data arrays inside the DvObject are valarrays, and these do not do array bound checking. Under most circumstances the DvObjects can perform whatever operations are required through defined methods and operators, and only when accessing data element by element will it be necessary to determine the data type and valarray length. See the DvObject class.

A DvObject_var may be used as a return value from a method. It may also be passed in an argument by value or reference.

If passed by value a new DvObject_var is created which points to the same object target. Changes to this object affect the object in the calling function, but if the DvObject_var is changed to point to a new target, that change will not propagate back to the calling function.

Passing by reference is slightly faster and the same pointer and target are involved, and all changes affect the calling function.

Most operators supported by DvObject are also available directly from DvObject_var for convenience. Operators normally result in a new object being created and assigned to the target DvObject_var, the exception being operators of the form +=, -=, /=, *= which modify the target object directly and are thus slightly faster than the equivalent binary operators, +, -, /, * which create a new object.



DvObject Class

The DvObject class provides a single class for data objects that contains the data in a valarray of appropriate type and all metadata (xrefs) as a vector of DvNode objects. Every object of class DvObject has a unique id which may be accessed via
get_id(). If two var pointers refer to the same data object then obj1->get_id() == obj2->get_id() will be true.


DvObject objects are normally handled via the DvObject_var wrapper class that ensures data is deleted when it goes out of scope. It is possible to use DvObjects directly for local temporary storage where objects are going to be destroyed explicitly [if created using 'new', DvObject *d= new DvObject(...) ] or by going out of scope if created directly as an instance, DvObject(...).

Consequently, methods that return a pointer to a new DvObject (DvObject *) must be used only with a var pointer, e.g.
DvObject_var obj = this->subset(0, 10);
and not
DvObject *ptr = this->subset(0, 10);
even though subset returns DvObject *.

The DvObject may contain data of one of the following types:
double
int
DvString
DvTime
DvEvent (a time interval type)
and a valarray is resized to hold the data when allocated through the operator = or one of the constructors.

The methods
is_dbl(),
is_int(),
is_str(),
is_time() and
is_event() may be used to determine the type data contained. It is only necessary to determine the data type when accessing elements directly as all methods (unless otherwise stated) are safe to use with all data types, and will do nothing if not appropriate to the data type held. DVOS itself will never throw an exception.

The equivalent negated methods are also available,
not_dbl() etc.

Additionally, the methods
is_ok() and
is_nil()
can be used to determine if any data is available, or if the object is 'empty'. Note that operations on empty objects (except direct access of data elements in the valarrays) are always safe as methods operate only on non-empty valarrays (and similarly lists of xrefs). An object that is_nil() will also have id = -1.

The data are treated as though a sequence of arrays of any rank (number of dimensions). There are short-cut access routines up to rank 4, but higher dimension data may be accessed by performing array index arithmetic explicitly. If the array dimension vector is empty then the data is treated as a sequence of single values, and if the sequence length is one the data is taken to be a single array (or single value if the dimension vector is zero). Each record in the sequence must have the same rank. Within the valarrays the last dimension varies the fastes and record number varies the slowest so that the data is packed as a sequence of C arrays.

Array Dimensions and Sequence sizes
The methods:
size_t seqSize() returns the length of the sequence
vector <size_t> & Dims() returns the dimensions of the arrays (the same for all records in the sequence)
size_t nDims() is a shortcut to Dims().size() and gives the number of dimensions (rank)
size_t arraySize() is the total number of elements in each array = Dims()[0] * Dims()[1] * ... * Dims()[ nDims() - 1 ]
size_t totalSize() the total size of the valarray = seqSize() * arraySize()

Destuctor
The destructor should never be called. Objects should always be pointed to using DvObject_var pointers,
in which case the object will self destruct when no longer in scope anywhere. It is dangerous to explicitly delete a DvObject
as it may be in use elsewhere.
Constuctors
There are general constructors and specific constructors for each data type (also see subSequence() operators below which may act as copy constructors):

Empty constructor:
DvObject () constructs a DvObject with a single double (0.0). This is the object created for an 'empty' DvObject_var() constructor. It has object_id=-1 and so returns is_nil() as true, but is otherwise safe. It returns seqSize() as 0, but totalSize() and arraySize() will be 1. These objects are not intended to be used other than safe place holders. Use DvObject (0.0) to construct a valid single double with value zero.

Copy Constructors
DvObject(DvObject_var obj, bool withXrefs=true) constructs a copy of the object pointed to by obj, including xrefs if withXrefs is true (default)
DvObject(DvObject &obj, bool withXrefs=true) constructs a copy of the object obj, including xrefs if withXrefs is true (default)
DvObject(DvObject *obj, bool withXrefs=true) constructs a copy of the object obj, including xrefs if withXrefs is true (default)

DvObject(DvObject_var &obj, size_t nRecs, bool withXrefs=true)
DvObject(DvObject *obj, size_t nRecs, bool withXrefs=true) both construct a copy of the first record in obj repeated nRecs times, includes xrefs if withXrefs is true (default). It is provided for internal use to repeat single values across a sequence, e.g.
DvObject_var d0 = getDep0(); // get Depend_0
 DvObject_var tags = new DvObject(d0.ptr(), (size_t) 1); // create new object of the first record
or more simply
DvObject_var d0 = getDep0()->subSequence(0); // get Depend_0 first record
Data type specific constructors:
Sequence, of nRecs, set to value, default is single value
  DvObject(double value, size_t nRecs=1);
  DvObject(int value, size_t nRecs=1);
  DvObject(DvString value, size_t nRecs=1);
  DvObject(DvTime value, size_t nRecs=1);
  DvObject(DvEvent value, size_t nRecs=1);
 
Sequence of single values with length of valarray size
  DvObject(valarray <double> &from);
  DvObject(valarray <int> &from);
  DvObject(valarray <DvString> &from);
  DvObject(valarray <DvTime> &from);
  DvObject(valarray <DvEvent> &from);

Sequence, nRecs, of dimensions dims,  all initialised with value
  DvObject(double value, vector <size_t> &dims, size_t nRecs=1);
  DvObject(int value, vector <size_t> &dims, size_t nRecs=1);
  DvObject(DvString value, vector <size_t> &dims, size_t nRecs=1);
  DvObject(DvTime value, vector <size_t> &dims, size_t nRecs=1);
  DvObject(DvEvent value, vector <size_t> &dims, size_t nRecs=1);

Construct from valarrays of data, length of valarrays must match (or exceed) product of array dimensions and sequence length
  DvObject(valarray <double> &from, vector <size_t> &dims, size_t nRecs=1);
  DvObject(valarray <int> &from, vector <size_t> &dims, size_t nRecs=1);
  DvObject(valarray <DvString> &from, vector <size_t> &dims, size_t nRecs=1);
  DvObject(valarray <DvTime> &from, vector <size_t> &dims, siDvMaskze_t nRecs=1);
  DvObject(valarray <DvEvent> &from, vector <size_t> &dims, size_t nRecs=1);

Assignment
  void assign_regular(double h) leaves the first value in the valarray unchanged and creates each subsequent entry by adding h incrementally. It has no effect if the data type is not a double or DvTime. If the object is a time sequence, h is assumed to be in seconds. This is convenient for creating regular Depend_0 variables, e.g.
DvObject_var d0 = getDep0(); // get existing Depend_0
DvObject_var tags = new DvObject(d0.ptr(), n); // create new object with first record of this repeated n times
tags->assign_regular(hReg); // replace data with regular spaced data off the first value
Note that this works exactly the same for a scalar or time sequence for Depend_0.

  void assign_regular(int h) leaves the first value in the valarray unchanged and creates each subsequent entry by adding h incrementally. It has no effect if the data type is not a int or DvTime. If the object is a time sequence, then h is assumed to be in nanoseconds. Use assign_regular(double) for seconds.

  void assign_regular(DvInt8 h) leaves the first value in the valarray unchanged and creates each subsequent entry by adding h incrementally. It has no effect if the data type is not a DvTime, and h is assumed to be in nanoseconds. Use assign_regular(double) for seconds.

The assignment operator:
operator =,  takes a DvObject as argument, and assigns the contents of the object to that of the argument (including xrefs). Note this differs from the assignment operator for DvObject_var. For example:
DvObject_var dobj1 = new DvObject(3.0);
DvObject_var dobj2 = new DvObject(7.0);
dobj1 = dobj2;  // points dobj1 to the contents of dobj2 and increments the reference counter of *dobj2
cout << *dobj1 << endl; // = 7.0, the data object holding 3.0 deleted itself
cout << (dobj1->get_id() == dobj2->get_id()) << endl; // true, both vars point to same object
*dobj1 = *dobj2;  // assigns the data and xrefs in dobj1 to be the same as in dobj2 (neither reference counter changes)
cout << dobj1->dbl() << endl; // = 7.0, see below
cout << (dobj1->get_id() == dobj2->get_id()) << endl; // false, two different objects holding the same data values (copy)


Fast Access
Fast data access routines are provided for stepping through the data arrays (valarrays). These DO NOT do array bound or data type checking which is left to the calling function (see examples).
double &dbl(size_t nRec, size_t i, size_t j, size_t k, size_t m) returns the reference to the double held in a 4D array at the specified element [i,j,k,m] for record nRec.
double &dbl(size_t nRec, size_t i, size_t j, size_t k) returns the reference to the double held in a 3D array at the specified element [i,j,k] for record nRec.
double &dbl(size_t nRec, size_t i, size_t j) returns the reference to the double held in a 2D array at the specified element [i,j] for record nRec.
double &dbl(size_t nRec, size_t i) returns the reference to the double held in a 1D array at the specified element [i] for record nRec.
double &dbl(size_t posn) returns the reference to element at position posn in the valarray. It is used for explicit navigation of the data array, for example when the array rank is greater than 4 or the matrix dimension is irrelevant (stepping through all elements). If no argument is given the first data point is returned, and this is used for access of single scalar values.
double &dbl(size_t nRec, vector <size_t> &index) returns the reference to the element at record nRec and array position given by the vector index. The size of index must match the data array nDims().

Note that a single array of rank greater than 1 it is necessary to specify that record zero is used, e.g. obj->dbl(0, i, j), since otherwise the i will be used as record number, obj->dbl(i,j) is record i and element j of the array stored in c order (last index varies fastest).
However, the special case of a single rank 1 array it will be safe to just step along the dimension since obj->dbl(i) and obj->dbl(0,i) will return the same element.

 Identical methods are available for all supported data types:
int &itg(...) returns the reference to the integer at the appropriate valarray element, where (...) are the same as for dbl(...) above.
DvString &str(...) returns the reference to the DvString at the appropriate valarray element, where (...) are the same as for dbl(...) above.
DvTime &time(...) returns the reference to the DvTime at the appropriate valarray element, where (...) are the same as for dbl(...) above.
DvEvent &event(...) returns the reference to the DvEvent at the appropriate valarray element, where (...) are the same as for dbl(...) above.
Thus
valarray <double> dd(6);
vector <size_t> size;
size.push_back(3);
size.push_back(2); // Note last index varies fastest
dd[0] = 11;
dd[1] = 12;
dd[2] = 21;
dd[3] = 22;
dd[4] = 31;
dd[5] = 32;
dobj = new DvObject(dd, size); // create 1 record holding 3x2 array of doubles
for(int i=0; i<size[0]; i++){
   for(int j=0; j<size[1]; j++){
      cout << dobj->dbl(0, i, j) << "  "; // nRec = 0 for single record
   }
   cout << endl;
};
         
output is
11  12
21  22
31  32

valarray <double> record(size_t r)  returns a valarray of doubles to give fast access to the double data elements in the data array at the specified record. If the object does not hold double data or the requested record is out of range a valarray containing a single array of zeros is returned. Thus

nrec = 0;
valarray <double> val = dobj[nrec]; // fetch 1 record of doubles
for(int i=0; i<val.size(); i++)cout << val[i] << ", ";

output is
11,  12,  21,  22,  31,  32, which is the inverse of the above example.

valarray <double>& d_valarray()  gives direct access to the valarray of doubles for the special case of double data. This circumvents the data arrays being protected, and should be used with caution and only when speed is important. Intended for internal use, but described for completeness.



Safe Fast Access
A safe version of fast data access routines are also provided. These are naturally slower as they do type and array bound checking. For example double &dblS(double safe, size nRec, size i, size j, size k, size m) is the same as double &dbl(size nRec, size i, size j, size k, size m)
except that if the data type is not double or the request is out of array bounds a static local double is returned containing the value safe.
The access methods are...
  double & dblS(size nRec, size i, size j, size k, size m, double safe=0.);
  double & dblS(size nRec, size i, size j, size k, double safe=0.);
  double & dblS(size nRec, size i, size j, double safe=0.);
  double & dblS(size nRec, size i, double safe=0.);
  double & dblS(size posn=0, double safe=0.);  element at this location in valarray, single scalar can use dblS()

  int & itgS(size nRec, size i, size j, size k, size m, int safe);
  int & itgS(size nRec, size i, size j, size k, int safe);
  int & itgS(size nRec, size i, size j, int safe);
  int & itgS(size nRec, size i, int safe);
  int & itgS(size posn=0, int safe=0);  element at this location in valarray, single scalar can use itgS()

  DvString & strS(size nRec, size i, size j, size k, size m, DvString safe=Dvstring());
  DvString & strS(size nRec, size i, size j, size k, DvString safe=Dvstring());
  DvString & strS(size nRec, size i, size j, DvString safe=Dvstring());
  DvString & strS(size nRec, size i, DvString safe=Dvstring());
  DvString & strS(size posn=0, DvString safe=Dvstring());  element at this location in valarray, single string can use strS()

  DvTime & timeS(size nRec, size i, size j, size k, size m, DvTime safe=DvTime());
  DvTime & timeS(size nRec, size i, size j, size k, DvTime safe=DvTime());
  DvTime & timeS(size nRec, size i, size j, DvTime safe=DvTime());
  DvTime & timeS(size nRec, size i, DvTime safe=DvTime());
  DvTime & timeS(size posn=0, DvTime safe=DvTime());  element at this location in valarray, single time can use timeS()

  DvEvent & eventS(size nRec, size i, size j, size k, int m, DvEvent safe=DvEvent());
  DvEvent & eventS(size nRec, size i, size j, size k, DvEvent safe=DvEvent());
  DvEvent & eventS(size nRec, size i, size j, DvEvent safe=DvEvent());
  DvEvent & eventS(size nRec, size i, DvEvent safe=DvEvent());
  DvEvent & eventS(size posn=0, DvEvent safe=DvEvent());  element at this location in valarray, single event can use eventS()

Subsetting Sequences
Methods are provided that extract data by record.
DvObject_var subSequence(size_t nRec) returns a new object containing the single record at nRec, with all xrefs (subset if needed)
DvObject_var subSequence(size_t fromRec, size_t toRec) returns a new object containing the record between fromRec and toRec inclusive, with all xrefs (subset if needed)
DvObject_var subSequence(DvEvent &range) returns a new object containing the records for which timetags are within range.
DvObject_var subSequence(slice sl, size_t len=0) returns a new object containing the data valarray sliced using sl, with all xrefs (subset if needed)
void apply_mask(DvMask &msk) applies the mask to the object and its xrefs, and resizes it accordingly. The xrefs are coppied before the mask is applied to them. Records with msk true are kept.

Create Similar Objects
Methods are provided that create an object of the same type, but with new dimensions or sequence length.
DvObject *create(vector <size_t> dims, int nRec) returns a new object of the same data type but with nRec records in the sequence and array dimensions dims. All elements are filled with the default object of this type.
DvObject *create( int nRec) returns a new object of the same data type and array dimensions, but with nRec records in the sequence. All elements are filled with the default object of this type.

Replacing Data in Objects
These methods delete the existing data (of any type) within an object and replace it with the valarray data and with the same type as the valarray.
void replaceData(valarray <double> &data)
void replaceData(valarray <int> &data)
void replaceData(valarray <DvString> &data)
void replaceData(valarray <DvTime> &data)
void replaceData(valarray <DvEvent> &data)
Dimensions and sequence length are left unchanged, so they must be explicitly corrected if changed, e.g. using
obj->seqLen = records;
obj->dims.clear();
obj->dims.push_back(len);
Intervals
Methods are provided specifically for locating time or scalar Depend_0 bounding records. If the search fails, recL and recU return -1.
void getBounds(int &recL, int &recU, DvTime &t) locates the records that bound the specified time t in a monotonic time  sequence)
void getBounds(int &recL, int &recU, double d) locates the records that bound the specified value d in a monotonic scalar sequence)
void getRecBounds(int &recL, int &recU, double start, double end) locates the records that bound the specified scalar interval between start and end in a monotonic scalar sequence)
void getEventRecBounds(size_t &recL, size_t &recU, DvEvent &range) locates the records that bound the specified time interval range in a monotonic time  sequence)


Record Operators
An operator that allows access to the elements of the data array at a record, [r], works through an intermediary class, DvRecord. This class has a set of operators defined on it that apply directly to the specified record. It is rarely necessary to use the DvRecord class itself as it is provided purely as a convenience class to allow records to be used as both l and r values. For example
 seq[2] = DvNaN; // DvNaN holds the local value for NaN
 seq[0] = seq[2];
will set all the elements of record 2 to NaN, and then copy these values into record 0. Note that the DvRecord class always gives access to the actual record in the DvObject. To take a copy of the data array it should be copied to a valarray. Note also that the record(int r) method (above) is a specific shortcut to get a copy of the double data valarray at the specified record in objects containing double data.
Example, to interpolate data records of arbitrary dimensions onto a DEPEND_0, target:
  // create n records containing first record of this
DvObject_var result = new DvObject(this, n);
...
 // interpolate
 double factor = (target[i] - ssThis[i_this]) / (ssThis[i_this] - ssThis[i_this-1]);
 result[i] = (*this)[i_this];
 result[i] -= (*this)[i_this-1] ;
 result[i] *= factor;
 result[i] += (*this)[i_this-1];

Operations between records are thus handled as well as operations between records and single values. The above sample works for objects containing double, int and DvTime data types (int types will truncate at each step, so should be treated with caution). The DvTime type is safe, but the intermediate values of += and -= are not valid date_times, and resolution can be lost when /= and *= are used with values far from unity.

Operators supported are =, +=, -=, *=, /= with arguments of double, int, DvTime, DvString and DvEvent and valarrays and DvRecord containers of the same. Not all arguments are valid for all data types. When the operation is meaningless, the result is unchanged. Note however, that sensible conversions between DvString and DvTime are made where possible.

Note that if the requested record is off the end of the data sequence, then the first record is always returned. This has the effect that a sequence of a single element (or array) will behave like a sequence of any length with the first element repeated. This can be convenient if it is not known in advance whether a variable is record varying or not (e.g. DELTA_PLUS or DEPEND_1).

void setRecord(size_t nRec, size_t oRec, DvObject* obj) sets record [nRec] of this object to that of record [oRec] of object obj.
 No bounds checking is done, but convenient for merging or shuffling records between objects.

Conversions
The "toType" conversions return a valarray holding the same data, but converted to the appropriate data type. They may be used inside a constructor if the converted data is to be kept, but are normally expected to be for local use (hence the valarray rather than a DvObject). An optional default value may be provided, and if not set, the standard default for the type is used - zero for double and int, and empty string or MJD 2000 for time.
If the conversion is not possible the valarray contains all default values.
valarray <double> toDouble(double safeDefault)
valarray <int> toInt(int safeDefault)
valarray <DvString> toStr(DvString safeDefault )
valarray <DvTime> toTime(DvTime safeDefault)

The "asType" conversions will retrieve a single specified element from the data array converted to the required data type.
If the conversion is not possible the safe default is returned. This form is not efficient with large data arrays, but convenient for small data objects such as attributes or single values.
 double     asDouble(size_t posn=0, double safeDefault=0)
double     asDouble(size_t posn, size_t i, double safeDefault=0)
double     asDouble(size_t posn, size_t i, size_t j, double safeDefault=0)
double     asDouble(size_t posn, size_t i, size_t j, size_t k, double safeDefault=0)
double     asDouble(size_t posn, size_t i, size_t j, size_t k, size_t m, double safeDefault=0)
int             asInt(size_t posn=0, int safeDefault=0) etc
DvString   asStr(size_t posn=0, DvString safeDefault=DvString() ) etc
DvTime     asTime(size_t posn=0, DvTime safeDefault=DvTime()) etc
It is thus possible to work with data in a specific data type.

DvObject_var   convertToDbl() is a convenience for handling data of an unknown type by converting to a double object, for example where an operation may not be possible for an integer type, or where rescaling could cause truncation. It returns a new object with the data as doubles.    

DvString  getTypeAsText() returns a text string specifying the data type stored. Possible values are "CHAR", "DOUBLE", "INT", "ISO_TIME", "ISO_TIME_RANGE".   

Operators
Explicit operators are supported when appropriate to the underlying data type.  Operators create a new object containing the result (or an empty object if the operation is not meaningful for the data types) and hence return a DvObject *. The exceptions are operators of the form +=, *=, etc which modify the target object, and hence return DvObject &. Unit handling (and conversion on the fly if necessary and possible) using SI_conversion and vector frame checking is automatic. Joining using linear interpolation is also automatic if necessary. Xrefs are modified and attached to the result where possible. The data type and dimension is always taken from first argument. e.g.
 valarray <double> x(2);
x[0] = 0.0;
x[1] = 1.0;
DvObject_var y = new DvObject(2.0);
 DvObject_var z = x + y; // holds the double values (2.0, 3.0)

Note also that
DvObject_var x = new DvObject(100.);
DvObject_var y = new DvObject(" 10.^2 is");
 DvObject_var s = y + x; // holds the DvString value "  10.^2  is 100."
DvObject_var z = x + y; // holds the double value 200. as the DvString.toDouble() understands latex as well as C syntax numbers

Modifying Operators (single value args)
These all return DvObject& as they change the object itself (this). Each of these operations are applied to each element in the object independently.
These operators are of the type DvObject obj operator+=(double, int, DvString etc) where the RHS is a single value are not accessible directly from the var pointer, which must be dereferened to use them (e.g. DvObject_var obj; *obj += i;) This differs from operators of the type DvObject obj operator+=(DvObject&) which can be used via the var pointer (e.g. DvObject_var obj; DvObject_var obj2; obj += obj2;) .

They are valid respectively for the specified data types below:

operator+=  double+=all (event adds duration, DvTime += DvTime gives a non-valid time, but is useful as an intermediate step in a chain of operations), int+=all, DvTime+=double (the time is incremented by double, default is seconds).

Note additionally that
operator+=  DvEvent and DvString are also provided and have the special meaning of appending either an event or string to a sequence of events or strings. This is a convenience for adding events to event tables or strings to lists of strings. Has no action if the base object is not of the same data type as the argument.
Note, however, that DvObject::operator+=(DvString) will append a string to the sequence of strings, while DvString::operator+=(DvString) will append to the DvString.

operator-=  double-=(double,int),  int-=(int,double), DvTime-=(double, DvTime)  (the time is decremented by double in seconds, DvTime -= DvTime gives a non-valid time, but is useful as an intermediate step in a chain of operations).

operator*=  double*=(double,int),  int*=(int,double), DvTime*=(double)  (useful as an intermediate step in a chain of operations).

operator/=  double/=(double,int),  int/=(int,double), DvTime/=(double)  (useful as an intermediate step in a chain of operations).

void apply_mask(DvMask &msk) method will remove records from the calling object (and its xrefs) according to whether the corresponding entry in msk is true (keep) or false (remove). If the size of the mask does not match the sequence length, then this method does nothing other than set the object's error status.

Modifying Operators (DvObject args)
TheOperators are also available to operate on DvObject arguments, where the following type conventions apply:

operator+=(DvObject&)  valid for double+=all (event adds duration, DvTime += DvTime gives a non-valid time, but is useful as an intermediate step in a chain of operations), int+=all, DvTime+=double (the time is incremented by double, default is seconds), DvString+=all (text equivalent appended to each string; this differs from +=DvString which adds string to the end of a sequence of strings).

operator-=(DvObject&)  valid for double-=(double,int),  int-=(int,double), DvTime-=(double, DvTime)  (the time is decremented by double in seconds, DvTime -= DvTime gives a non-valid time, but is useful as an intermediate step in a chain of operations).

operator*=(DvObject&) valid for double*=(double,int),  int*=(int,double), DvTime*=(double)  (useful as an intermediate step in a chain of operations). The resultant array dimension is the same as the first argument if the second is a single value. Otherwise the array dimensions must be conformal, and the resulting dimensions satisfy the rules of matrix multiplication.

operator/=(DvObject&) valid for double/=(double,int),  int/=(int,double), DvTime/=(double)  (useful as an intermediate step in a chain of operations). The resultant array dimension must be the same as this, so division by a single value only is allowed.

Binary Operators
These all return DvObject* with the same type and dimensions of the first object (except operator* which may change the result dimension).
Operations may be with a single value or array, or with a sequence. In the case of a sequence, the sequences must be of the same length or be able to be joined onto the first argument.

They are valid respectively for the specified data types below:

operator+  double/all (event adds duration, DvTime += DvTime gives a non-valid time, but is useful as an intermediate step in a chain of operations), int/all, DvTime/double (the time is incremented by double, default is seconds), DvString/all (text equivalent appended to string).

operator-  double/(double,int),  int/(int,double), DvTime/(double, DvTime)  (the time is decremented by double in seconds, DvTime -= DvTime gives a non-valid time, but is useful as an intermediate step in a chain of operations).

operator* double/(double,int),  int/(int,double), DvTime/(double)  (useful as an intermediate step in a chain of operations). The resultant array dimension is the same as the first argument if the second is a single value. Otherwise the array dimensions must be conformal, and the resulting dimensions satisfy the rules of matrix multiplication.

operator/ double/(double,int),  int/(int,double), DvTime/(double)  (useful as an intermediate step in a chain of operations). The resultant array dimension must be the same as this, so division by a single value only is allowed.

Vector Operators
These all operate on 3-vector objects.
 
 DvObject_var getVecFromRLP() Returns the RLP vector as a cartesian vector. Assumes input theta in range [0, pi/2) and phi in range [0, 2pi).These all operate on 3-vector objects.
 
 DvObject_var getVecFromRTP() Returns the RTP vector as a cartesian vector. Assumes input latitude in range [-pi/2,pi/2) and phi in range [0,2pi).

DvObject_var dot(DvObject_var &obj) Takes the dot (inner) product of records in this with those of obj (double three Vectors only).
DvObject_var vec(DvObject_var &obj) Takes the vector (cross) product of records in this with those of obj (double three Vectors only).
In both cases obj may be a single vector or a sequence of vectors. Sequences will be joined automatically if necessary.


Unary Operators
These methods all exist in two forms. One returns a copy if the object, this, with the data changed according to the standard mathematical definitions, while the other modifies the object itself and is declared void. Units are modified where possible, and if the operation requires the data to be unitless (e.g. log) the error flag is set in the resulting data if it has units, but the operation is completed anyway.

DvObject_var sqrt() Square root (double)
void sqrtThis() Square root this object

DvObject_var abs() Absolute value (double , int) positive part of scalar or elements of an array, but gives magnitude of vector.
void absThis() Absolute value this object positive part of scalar or elements of an array, magnitude of vector (object dimension changes).

DvObject_var log() Natural logarithm (double) of scalar or elements of an array. Sets the error text in the object has units
void logThis() Natural logarithm  of this object  of scalar or elements of an array.

DvObject_var log10() Logarithm to base 10 (double) of scalar or elements of an array. Sets the error text in the object has units
void log10This() Logarithm to base 10 of this object  of scalar or elements of an array.

DvObject_var inverse() Inverse (double) of scalar or elements of an array.
void inverseThis() Inverse of this object  of scalar or elements of an array.
DvObject_var chgSign() Changes sign (double, int) of scalar or elements of an array.
void chgSignThis() Changes sign of this object  of scalar or elements of an array.
DvObject_var exp() Take exponential of data elements (double).
void expThis() Take exponential of data elements (double) of this object.
DvObject_var trace() Takes trace of a 2D square matrix  (double, int).

DvObject_var minusStart() Subtracts the first record from every record. When applied to a sequence of DvTime objects (timetags) it will return seconds since start of the sequence as double objects.

double max() Returns the maximum value in the object.
double min() Returns the minimum value in the object.

Other Basic Operators
DvObject_var power(double p) raises data values(double) to power p,  for each scalar or element of an array.
void powerThis(double p) Raises data values(double) of this object to power p,  for each scalar or element of an array.
 DvObject_var power(int p) Raises data values (double or int) to power p,  for each scalar or element of an array.
void powerThis(int p) Raises data values (double or int) of this object to power p,  for each scalar or element of an array.

 DvObject_var remainder(double d) Remainder after dividing by d (double) for scalar or each element of an array.
void remainderThis(double d) Remainder after dividing this object  by d (double) for scalar or each element of an array.
DvObject_var remainder(DvObject_var &arg) Remainder after dividing by arg (double).
void remainderThis(DvObject_var &arg) Remainder after dividing this object  by arg (double).

Trig and Angle Operators
All trig functions assume radians if there is no SI_Conversion attribute available, otherwise the data are converted to radians if possible. Inverse Trig functions take a bool argument, indeg, specifying if the output is in radians (false, default) or degrees (true). All trig operators trap inputs out of valid range and set the result to DvNaN and an error message. Periodic functions are handled modulo 2 pi, but inverses are in the range (-pi/2, pi/2). The exception is atan2() which returns values in (-pi, pi).

DvObject_var sin() Takes trigonometric sine (double) of scalar or elements of an array.
void sinThis() Takes trigonometric sine (double) of this object for scalar or elements of array.

DvObject_var cos() Takes trigonometric cosine (double) of scalar or elements of an array.
void cosThis() Takes trigonometric cosine (double) of this object for scalar or elements of array.

DvObject_var tan() Takes trigonometric tangent (double) of scalar or elements of an array.
void tanThis() Takes trigonometric tangent (double) of this object for scalar or elements of array.

DvObject_var cot() Takes trigonometric cotangent (double) of scalar or elements of an array.
void cotThis() Takes trigonometric cotangent (double) of this object for scalar or elements of array.

DvObject_var asin(bool indeg) Takes inverse of trigonometric sine (double) of scalar or elements of an array.
void asinThis(bool indeg) Takes inverse of trigonometric sine (double) of this object for scalar or elements of array.

DvObject_var acos(bool indeg) Takes inverse of trigonometric cosine (double) of scalar or elements of an array.
void acosThis(bool indeg) Takes inverse of trigonometric cosine (double) of this object for scalar or elements of array.

DvObject_var atan(bool indeg) Takes inverse of trigonometric tangent (double) of scalar or elements of an array.
void atanThis(bool indeg) Takes inverse of trigonometric tangent (double) of this object for scalar or elements of array.

DvObject_var acot(bool indeg) Takes inverse of trigonometric cotangent (double) of scalar or elements of an array.
void acotThis(bool indeg) Takes inverse of trigonometric cotangent (double) of this object for scalar or elements of array.

DvObject_var sinh() Takes trigonometric sine (double) of scalar or elements of an array.
void sinhThis() Takes trigonometric sine (double) of this object for scalar or elements of array.

DvObject_var cosh() Takes hyperbolic cosine (double) of scalar or elements of an array.
void coshThis() Takes hyperbolic cosine (double) of this object for scalar or elements of array.

DvObject_var tanh() Takes hyperbolic tangent (double) of scalar or elements of an array.
void tanhThis() Takes hyperbolic tangent (double) of this object for scalar or elements of array.

DvObject_var coth() Takes hyperbolic cotangent (double) of scalar or elements of an array.
void cothThis() Takes hyperbolic cotangent (double) of this object for scalar or elements of array.

DvObject_var asinh() Takes inverse of hyperbolic sine (double) of scalar or elements of an array.
void asinhThis() Takes inverse of hyperbolic sine (double) of this object for scalar or elements of array.

DvObject_var acosh() Takes inverse of hyperbolic cosine (double) of scalar or elements of an array.
void acoshThis() Takes inverse of hyperbolic cosine (double) of this object for scalar or elements of array.

DvObject_var atanh() Takes inverse of hyperbolic tangent (double) of scalar or elements of an array.
void atanhThis() Takes inverse of hyperbolic tangent (double) of this object for scalar or elements of array.

DvObject_var acoth() Takes inverse of hyperbolic cotangent (double) of scalar or elements of an array.
void acothThis() Takes inverse of hyperbolic cotangent (double) of this object for scalar or elements of array.

DvObject_var toDeg() Returns this object converted to degrees if in radians, and sets metadata accordingly.
void toDegThis() Converts this object to degrees if in radians, and sets metadata accordingly.

DvObject_var toRad() Returns this object converted to radians if in degrees, and sets metadata accordingly.
void toRadThis() Converts this object to radians if in degrees, and sets metadata accordingly.

The following operators are convenience calls to convert between 2D and 3D vector representations and should be self explanatory. X, Y and Z are the usual cartesian components, while R is radius (magnitude), L is latitude, T is theta (colatitude) and P is phy (azimuth). Radians are to be assumed unless the method contains _deg. Going from polar to cartesian the method will use the SI_Conversion attribute to determine whether the input is in radians or degrees.

DvObject_var XYtoRP()
void XYtoRPThis()
DvObject_var XYtoRP_deg()
void XYtoRPThis_deg()
DvObject_var RPtoXY()
void RPtoXYThis()

DvObject_var XYZtoRPZ()
void   XYZtoRPZThis()
DvObject_var XYZtoRPZ_deg()
void   XYZtoRPZ_degThis()
DvObject_var RPZtoXYZ()
void   RPZtoXYZThis()
 
DvObject_var XYZtoRTP()
void   XYZtoRTPThis()
DvObject_var XYZtoRTP_deg()
void   XYZtoRTP_degThis()
DvObject_var RTPtoXYZ()
void   RTPtoXYZThis()
 
DvObject_var XYZtoRLP()
void   XYZtoRLPThis()
DvObject_var XYZtoRLP_deg()
void   XYZtoRLP_degThis()
DvObject_var RLPtoXYZ()
void   RLPtoXYZThis()


Array Operators
These modify the arrays at every record, and replace the DEPEND_i attributes appropriately.

DvObject_var subDimension(size_t i, bool withXref=true) Operates on a 1D array by removing the element at position i in the array. The length of the dimension is reduced by 1. If withXref is true (default) xrefs are attached.

DvObject_var subsetDim(size_t d, size_t from, size_t to) Takes the subset of the array along dimension d between from and to inclusively. If from > to, then the subarray is taken cyclicly, starting at from up to the end of the dimension then from zero to to. If from == to then sliceDim() is called and the number of dimensions will be reduced by one.


DvObject_var sliceDim(size_t d, size_t at) Takes a slice of the array at element at in dimension d. The number of dimensions will be reduced by one, and DIMENSION_i will be adjusted for any remaining dimensions if necessary.

DvObject_var sumDim(size_t d, size_t from, size_t to, bool stripBad=true) Takes the sum of the array along dimension d between from and to inclusively.  If from == to then sliceDim() is called. The number of dimensions will be reduced by one. If stripBad is true (default) then entries with fillvalues are not included in the sum.

DvObject_var averageDim(size_t d, size_t from, size_t to, bool stripBad=true) Takes the average of the array along dimension d between from and to inclusively.  If from == to then sliceDim() is called. The number of dimensions will be reduced by one. If stripBad is true (default) then entries with fillvalues are not included in the average (and the divisor adjusted accordingly).

DvObject_var unwrapDim(size_t d, size_t from, size_t to) Turns the dimension d between from and to inclusively into record entries spread evenly over the time interval. The array dimension is effectively aliased to time.  If from > to, the elements are selected cyclicly, first in the range (from, N) inclusive then (0, to) inclusive. The number of dimensions will be reduced by one and the number of records will be multiplied by the number of rows selected, usually (from - to + 1), but will be (N - from + to + 1) if from > to.

DvObject_var reverseDim(size_t d, size_t from, size_t to) Re-orders the array along dimension d between from and to inclusively, such that the dimension runs from to to from.  If from == to then sliceDim() is called. It requires that to > from, but application of a cyclic subset first allows for external range selection. The number of dimensions is unchanged, but dimension d has length (to - from + 1).

DvString commonText() returns a string representation of the common parts in all DvString elements of an array or sequence. If is_str() is false for this object (not string type) then an empty string is returned.

DvObject_var det() Returns the determinant of a square two dimensional array of doubles.

DvObject_var adjoint() Returns the adjoint of a square two dimensional array of doubles.

 DvObject_var transpose(bool withXref=true) Returns the transpose of any two dimensional array. An NxM array will become and MxN array. If withXref is true (default) xrefs are attached.
DvObject_var matrixInverse() Returns the inverse of a matrix of doubles.

 DvObject_var trace() Returns the trace (sum of diagonal elements) of a square 2D array (int and double data only).

 DvObject_var subMatrix(size_t i, size_t j, bool withXref=true) Returns the subMatrix formed by removing row i and column j. If withXref is true (default) xrefs are attached.

DvObject _var outerProduct(DvObject_var& arg) Returns the outer product of matrix A with matrix B. Result has dimensions R(i,j,..., k,l,...) where A(i,j,...) and B(k,l,...). Objects with frames (e.g. vectors) must be in the same frame.

Note that Inner product is the same as multiply and dot product as appropriate, and is not provided separately.

DvObject_var element(size_t i) Returns a scalar (sequence) comprising element i of the array (dimensions are handled in C ordering with last index varying fastest).
DvObject_var cycleMod(int nstart, int n, int modn) Returns an object with the modn elements starting at nstart moved cyclicly by n positions. The elements are moved only within the specified block. This will work on a matrix or scalar sequence and any data type. It performs only one cyclic move, and does NOT repeat for every record.

DvObject_var sum() Returns an object of the same dimensions, but summed over all records. The returned sequence size is 1.

DvObject_var multElements(DvObject_var& arg) Returns an object of the same dimensions as this with the elements of this multiplied by the corresponding element in arg. The dimensions of this and arg must be the same.

DvObject_var divideElements(DvObject_var& arg) Returns an object of the same dimensions as this with the elements of this divided by the corresponding element in arg. The dimensions of this and arg must be the same.

Utilities are also provided to return components and array reduced objects using the QSAS data slot syntax for component selection.
In general within QSAS these are used by the slots themselves, and slot->getObject() is used to return whatever is selected.
They are provided here for completeness.

vector <DvString> available(DvString constraint=DV_NONE) lists the possible component options for a data object for the provided constraint. This is for use in QSAS slots.
DvObject_var getObjComponent(DvString comp) general routine returns the object after component subsampling.
DvObject_var getVectorComponent(DvString comp) returns the selected component of the vector subsetted on records if necessary.
 DvObject_var getTimeComponent(DvString comp) returns a time object according to the component selection.
DvObject_var getIntervalComponent(DvString comp) returns the selected interval object according to the component selection.
DvObject_var getArrayComponent(DvString comp) returns an a object with reduced dimensions according to the component selection.
 DvObject_var getTimeSubset(DvString comp) returns the object subset according to the record selection.
DvObject_var getTimeAve(DvString comp) returns the object averaged according to the record selection.
DvEvent getTimeInterval(DvString comp) returns the time interval represented by the object and its component selection.

Component strings are predefined in DvObject.h, and include...

#define DV_RMAG "R (mag)"
#define DV_X "X"
#define DV_Y "Y"
#define DV_Z "Z"
#define DV_LAT_DEG "Lat (deg)"
#define DV_THETA_DEG "theta (deg)"
#define DV_PHI_DEG  "phi (deg)"
#define DV_LAT_RAD "Lat (rad)"
#define DV_THETA_RAD "theta (rad)"
#define DV_PHI_RAD "phi (rad)"
#define DV_LOCAL_T "Local time"
#define DV_T_START "Start"
#define DV_T_CENTRE "Centre"
#define DV_T_END "End"



 slice_array <double> getDSlice(slice sl) used internally to access double data for use with valarrays.
void setSlice(slice sl, slice_array <double>) used internally to set double data (used with getDSlice).
void setSliceDim(size_t d, size_t el, slice_array <double> val) used internally to set double data (used with getDSlice) to set element el of dimension d for every record using the slice_array val, which must have the same number of elements as seqSize().
Equivalent calls exist for int, DvString, DvTime and DvEvent data types, e.g. getSSLICE(std::slice) for DvStrings.
Additionally, two methods for double data type only are provided.
 slice_array <double> elementValarray(size_t i) used internally to access double data (used directly with double valarrays) to get element i of arrays for every record, will have the same number of elements as seqSize().
 valarray <double> detValarray(size_t arrayDim, size_t len) used internally, returns the determinant of a matrix as a valarray.

See also DvMatrixEig and DvMinVariance classes below for further matrix utilities.

Algorithms
These provide some more complex operations on objects, and generally take options controlling algorithm details.

DvObject_var integrate(DvString &gapMethod, DvString &integralMethod) Returns a new object holding the integral of this object. The argument gapMethod may be DV_NN,  DV_IGNORE_GAPS or DV_LINEAR, which respectively refer to filling data gaps with the nearest neighbour values (to the half way point from either side), leaving gaps out of the integral entirely and linear filling across gaps.  If integralMethod is set to DV_DEF_INTEGRAL the definite integral is taken over the entire supplied range, otherwise if integralMethod is set to DV_INDEF_INTEGRAL the return object is a sequence of the same length as this, and holds the accumulative integral from the start of the sequence to each record.

DvObject_var differentiate(DvString &derivMethod, double GapSize) Returns a new object holding the derivative of this object. The argument derivMethod may be either DV_DERIV_3PT (default) or DV_DERIV_5PT which result in 3 point or 5 point estimates respectively. The 5 point method requires regularly spaced data and will join on the fly onto a regular Depend_0 that it creates with the same data spacing as the first two records. The argument gapSize is taken as the separation between records that will be treated as a gap and excluded from the returned sequence. If gapSize is negative (default) then it is calculated as 1.5 times the separation between the first two records. The first and last records (and records either side of a gap) use a simple two point estimation.

DvObject_var regression() Returns a new object holding the straight line (2 data points and two time tags (or double depend_0 ) that best fits the data. The slope, intersept on the y-axis and the goodness of fit (correlation coefficient) are attached to the returned object as xrefs.

DvObject_var regression(DvObject_var &x ) Returns a new object holding the straight line (2 data points and two time tags (or double depend_0 ) that best fits the data taking the argument x as the x-data (and this object providing the y-data). The slope, intersept on the y-axis and the goodness of fit (correlation coefficient) are attached to the returned object as xrefs.

DvObject_var mergeWith(DvObject_var &dobj ) Returns a new object holding the data from this object merged with the data from dobj, interlaced if necessary, according to the values on the DEPEND_0 (usually time tags). If one data sequence entirely precedes the other, then they are simply concatenated. The number of returned records is the sum of the number of records in the two input data objects minus the number of duplicate records. If a record is duplicated (e.g. same timetag in both objects) then the data from this object is used.

DvObject_var stripDuplicates(double tolerence ) Returns a new object holding the data from this object with duplicates removed. Records are considered duplicates if their DEPEND_0 value for the two records differ by less than tolerence. The input DEPEND_0 is assumed monotonic (see makeMonotonic()). 

DvObject_var cleanObject() Returns a new object holding the data from this object with all records containing either a FILLVAL or NaN removed. This may have undesired effects in array data if one channel holds bad data, in which case subsetting on dimension first will be necessary (see, e.g. subsetDim() above).

 DvMask getCleanMask() Returns a mask of the records that would have been removed in cleanObject() above. This object is unchanged.

DvObject_var makeMonotonic() Returns a new object holding the data (and xrefs) sorted in increasing order of the Depend_0 xref (e.g. time tags). Duplicate time tags are not removed, see stripDuplicates() above.

DvObject_var applyOrder(vector <int> &order) This is used internally by makeMonotonic(). It returns a new object holding the data sorted into record order specified by the values of order. The size or order must match the length of this->seqSize(), and each record value between 0 and  this->seqSize() - 1 must occur exactly once.


Xref Handling
Specific utilities for handling xrefs (attributes) are provided, and the DvNode class (below) rarely needs to be used explicitly.

 bool xref_exists( DvString xref_name ) 
bool xref_exists( const char* xref_name ) both return true if a xref with the specified name exists in the object's list of xrefs.
DvNode *find_xref( DvString& xref_name )
DvNode *find_xref( const char* xref_name ) both return a pointer to the named xref object. If the object does not exist this pointer will be NULL.  It returns a pointer to the DvNode object with the specified name. Note, however, that the method get_xref() below is usually the simplest method to use as it returns the DvObject_var pointer contained in the node.
DvObject_var get_xref( DvString xref_name )
DvObject_var get_xref( const char* xref_name ) both return a copy of the DvObject_var referenced. This method is safe in that a valid object is always returned, though if the xref does not exist it will be empty of data and will give is_nil() true. This is the normal way to access a xref of known name. The list of xrefs in the parent object is not modified, if the xref does not exist the returned DvObject is not added to the xref list.
void delete_xref( DvString xref_name )
void delete_xref( const char* xref_name ) both remove the DvNode object from the parent object's list of xrefs and deletes it, so the contained DvObject will be destroyed automatically if not referred to elsewhere, for example via a DvObject_var in another object's list of xrefs. It is not necessary to delete a xref before using change_xref().
void list_xref_names( vector<DvString>& sl ) creates a vector containing all the xref names in the object's list. The vector sl must be created by the calling routine, and names are appended.
void change_xref( DvString xref_name, DvObject_var xref_obj)
void change_xref( const char* xref_name, DvObject_var xref_obj) both locate the named xref and change its DvObject_var pointer to xref_obj. If the xref does not already exist it is created and added to the object's list of xrefs. If the target object is in the xref's own xref list a copy of it will be attached without its own xrefs to avoid infinite loops.
void change_xref( const char* xref_name, DvString &s)
void change_xref( const char* xref_name, const char *cs)
void change_xref( const char* xref_name, double d)
void change_xref( const char* xref_name, int i) these overloaded versions are shortcuts and create the appropriate DvObject from the input data on the fly.
void copy_xrefs_from(DvObject_var& from_obj)
void copy_xrefs_from(DvObject& from_obj) both copy all the xrefs from the object from_obj into this object (calls change_xref()).
void copy_xref_from(DvString& name, DvObject_var& from_obj)
void copy_xref_from(const char* name, DvObject_var& from_obj)
void copy_xref_from(DvString& name, DvObject& from_obj)
void copy_xref_from(const char* name, DvObject& from_obj) all copy the single xref called name from the object from_obj into this object (all call change_xref()).
DvNode* first_xref() returns a pointer to the first DvNode object (xref) in this DvObject. Note that the DvNode class contains the pointer DvNode * next, to enable traversing the linked list of all xrefs in this DvObject. The last xref in the list has next set to NULL, e.g.
DvNode *xref = xref_obj.first_xref();
while(xref){   
  cout << xref->name() << endl;
   xref = xref->next;       
}

bool xref_in_graph(DvObject &xref_obj) returns true if this is in the argument's list of xrefs (recursively). Used by change_xref().
void ensureSIC( ) modifies the object xrefs to ensure that the essential numerical values have an SI_Conversion attribute (using that of this object if they are not already set). If this has no SI_Conversion it does nothing. Attributes affected are Delta_plus, Delta_minus, Scalemax, Scalemin, Validmax and Validmin.
Metadata Utilities
Specific utilities for handling specific useful metadata (CAA, CSDS and ISTP attributes) are provided. These are part of the DvObject class and used internally to ensure operations are allowed, and to perform conversions and metadata correction within operations. As such, when DvObject operators are used it is not necessary to call these methods explicitly as DvObject ensures metadata compliance where possible.

 DvObject_var getDep0() is simply a call to get_xref(DEPEND_0).
bool hasTimeTags() returns true if the Depend_0 xref points to a time type object.
 DvObject_var getTimeTags() returns a var pointer to this object if it is a time object, else Depend_0 if that is a time object else an empty var pointer.
DvEvent getTimeRange() returns the DvEvent corresponding to the start and stop time of the time tags if they exist, otherwise the default DvEvent().
 bool getDataRange(int recFrom, int recTo, double &min, double &max) returns the minimum and maximum value of the data between the specefied records for data of type double. Return is true if data is double and range found, else returns false.
bool getDataRange(double &min, double &max) finds the minimum and maximum value in the data object. Double and int data are just cast to double, and time objects return the Epoch2000 values (see DvTime). It returns true if successful, and false if the object is nil or the data type is not handled.
DvString getXrefText(DvString &name)
DvString getXrefText(const char *name) both return a DvString holding the content of the named attribute converted (if necessary) to text.
int get_iFILL()  returns the fill value as an integer.

 Unit and SI_conversion handling utilities are used within DvObject operators to ensure units are used and modified correctly. There are three different unit systems that data can be converted to/from. The SI_conversion string specifies the units of the data object by means of a standard SI unit string and a conversion factor that would convert the data into the specified SI units (by multiplying the data by the conversion factor). A third unit system is the Base SI units that all recognised SI units can be converted into. Units and conversions can be reduced to this Base SI form to ensure that the underlying units are the same and that conversions are between the same unit base.

DvString getSIC(int i=0) gets the SI_conversion string as a DvString, and if i is specified returns the ith element of the SI_Conversion string matrix (e.g. for an rlp vector, although this is rarely supported in data files).
DvString getBaseSI() returns the SI_conversion string in base SI form.
 double convFactor() returns the numeric SI_conversion factor to convert data into its SI form, that is the numerical value preceding the '>'.
double convFactorToBaseSI() returns the numeric SI_conversion factor to convert data into its base SI form. This is often the same as convFactor().
double convFactorFrom(DvString &SIC) returns the numeric SI_conversion factor to convert data from SIC units to this.
 bool sameBaseSI(DvObject_var &arg) returns true if this and arg have equivalend base SI units ( e.g returns true for V / km and nT m / s).
bool sameBaseSI(const char * SIstr) returns true if this and an SI_Conversion string SIstr have equivalend base SI units ( e.g returns true for V / km and nT m / s).
bool sameUnits(DvObject_var &arg) returns true if this and arg have the same base SI units and conversion factors.
 bool sameUnits(const char * SIstr, int i=0) returns true if this and an SI_Conversion string SIstr have the same base SI units and conversion factors. If i is specified the ith entry in the SI_Conversion string matrix of this object is used (e.g. for rlp vectors).
DvString getUnitsProduct(DvObject &arg) returns the concatenated UNITS string for this and arg.
DvString getUnitsRatio(DvObject &arg) returns the concatenated UNITS string for this and 1 / arg.
 DvString getUnitsPower(double p) returns the UNITS string for this raised to the power p as "(unit)^p".
DvString getUnitsInverse(double p) returns the UNITS string for 1 / this.
 DvString getSICProduct(DvObject &arg) returns the SI_conversion string for this times arg.
DvString getSICRatio(DvObject &arg) returns the SI_conversion string for this / arg.
 DvString getSICInverse() returns the SI_conversion string for the inverse of this object.
DvString getSICPower(double p) returns the SI_conversion string for this raised to the power p.
bool hasUnits() returns true if this has any SI units after reduction to Base SI form. Used to determine whether it is valid to take log(), exp() etc. If SI_Conversion is not found returns false.


All the following coordinate system methods look for the CSDS Frame attribute as well as CAA COORDINATE_SYSTEM, REPRESENTATION, REPRESENTATION_1, and TENSOR_ORDER attributes.
DvString getFrameAttr() returns the frame information in the CSDS syntax, e.g. vector>gse_xyz.
DvString getFrame() returns the coordinate system part of the frame, e.g. gse.
DvString getRep() returns the representation string, e.g. xyz, rlp or rtp.
int getOrder() returns the order of the object (0 for scalar or array, 1 for vector and 2 for rank 2 tensor.
bool sameFrame(DvObject_var &arg) returns true if this and arg are measured in the same reference frame.
bool sameFrame(DvString &frame) returns true if this is measured in the same reference frame as frame (in CSDS syntax, e.g. vector>gse_xyz).
 DvString setFrameAttr(DvString frameAttr) sets the FRAME, COORDINATE_SYSTEM, ORDER and REPRESENTATION from the frameAttr string in CSDS syntax, e.g. vector>gse_xyz.
DvString setFrameAttr(DvString frame, DvString rep, int order) sets the FRAME, COORDINATE_SYSTEM, ORDER and REPRESENTATION from the frame, representation and order.
DvString setFrameProduct(DvObject_var &arg) sets the FRAME, COORDINATE_SYSTEM, ORDER and REPRESENTATION appropriate after this object has been multiplied by arg.

int getOrder() returns the numeric value of the rank of a matrix. e.g. 3-vectors are rank 1.
 bool isThreeVector() returns true if the Frame attribute (if present) contains vector, or (if absent) the array size is 3.
bool isVectorXYZ() returns true if it is a cartesian three vector (uses getRep() and isThreeVector()).
void ensureVectorXYZ() converts this object into cartesian representation. Does nothing if already cartesian.
bool isDeg() returns true if this object is measured in degrees (uses sameUnits("1>deg")).
 bool isRad() returns true if this object is measured in radians (uses sameUnits("1>rad")).
bool isAngle() returns true if this object is measured in either degrees or radians. It checks both the SI_Conversion and Units attributes.
 bool isPhiAngle() returns true if this object is an azimuthal angle (range > pi or '"phi" or "azimut" used in describing it).
 bool isThetaAngle() returns true if this object is a colatitude (theta) angle (from metadatat).
 bool isLatAngle() returns true if this object is a latitude angle (from metadata).
DvString angleUnitStr returns a DvString containing one of "deg", "rad" or "".
bool isVectDeg() returns true if any component of this object is measured in degrees.
 bool isVectRad() returns true if any component of this object is measured in radians.
DvObject_var angleMod(int angMax, DvString rad_deg) returns this object with its range converted from one range to another depending on the value of angMax as follows:
angMax = 360 converts into the range 0, 360.
angMax = 180 converts into the range -180, 180.
angMax = 90 converts between the ranges 0, 180 and -90, 90 (the algorithm is the same, so repeated application converts back).

bool okDims(DvObject_var &arg) returns true if the arg array dimension matches that of this object, or the arg array has one element.
bool sameDims(DvObject_var &arg) returns true if this object and arg have arrays of identical dimensions.
bool conformalDims(DvObject_var &arg) returns true if this object is conformal for multiplication by arg. Returns true if either object has arraySize() = 1, or last dimension of this is the same as first dimension of arg.
bool squareMat() returns true if this object is a square matrix (2 dimensions and both the same).

Error Handling
All operators return a valid object (either the original object or a default single value of appropriate type), but if this happens an error message is appended to the returnd object's error list. Error handling methods are:

void clearError() empties the object's error list.
 void error(const char*flag) appends the message flag to the object's error list.
 DvString getError(size_t i) returns the error message at position i on the object's error list. Default is the last error appended.
 int nError() returns the number of error messages on the object's error list. Returns zero if no errors have been detected.


 Sequence and Joining Utilities
All DvObject operators check arguments are joined and will perform a default join automatically (if necessary) . However, joining first may be appropriate if something other than a default join is required, such as joining to regular timetags or boxcar averaging.

These are the same utilities that are used internally by the DvObject operators to ensure that data are correctly joined. In the following methods the DEPEND_0 variable may be either DvTime or double values. Data gaps are never removed in default operations as DVOS operators join on the fly if needed and binary operators require the result to be of the same length as the target.

The default gap size used for linear interpolation is 1.5 times the target spacing, and gaps are linear filled by default while ends use nearest neighbour.

The default boxcar uses a box of width twice the target spacing, and a minimum of 3 points in the box. If fewer than the minimum number of points fall inside the box, then the result is a gap. The gap handling default is linear inside a gap and nearest neighbour on the ends.

The Nearest Neighbour gap default is to use nearest neighbour inside gaps and off the ends.

The default gap and join options can be overridden by attaching suitable xrefs first (see Join Options below) in which case care must be taken when setting DV_GAP_OPTION to DV_REMOVE as the result may have fewer records than the target. If multiple objects are to be joined to the same target with gaps removed, then MultiJoin() should be used (see DvJoinList class below) to ensure they share a common timeline with gaps in all inputs removed. In all cases, if gap handling is set to remove gaps, then the end handling will also remove gaps.

DvObject_var Join(DvObject &dobj, bool withXrefs)  joins this object onto the Depend_0 of dobj if it exists, otherwise it uses dobj itself as target. It can thus be called with either a data object or DEPEND_0 object as argument. It uses the join option and gap handling most suitable to the data type and spacing. Linear join is used for numeric data types (double, int, time) unless the target spacing is more than twice the object's original DEPEND_0 spacing, in which case a boxcar is used. Nearest neighbour is used for string and event data types. The result is a var pointer to a new object resulting from this being joined onto target, which must be a sequence of either DvTime or double values. If withXrefs is true (default) the xrefs are attached and joined as necessary.

The defaults are safe for most plugin use, and the single method Join() below is all that is needed.

 DvObject_var Join(DvObject_var &dobj, bool withXrefs)  is a convenience call taking a var pointer as argument. It calls Join(DvObject &dobj, bool withXrefs)  above.

bool isRegular(double &spacing) returns true if the data are equally spaced (DEPEND_0 object evenly spaced, e.g. regular time tags). The argument spacing holds the value of this spacing on return.

double get_spacing() returns (usually) the minimum separation between values, e.g.
double spacing = dobj->getDep0()->get_spacing();
This is assumed to be the nominal data spacing with data gaps ignored. It retains the sign of the spacing, so a monotonic decreasing series has a negative spacing (e.g. if Depend_0 is t0 - t ). The algorithm sets the spacing to be the difference between the first two entries in DEPEND_0, and then scans the sequence and if an absolute separation smaller than the absolute value of this spacing is found more than once, then the second of these values is used as the new spacing. This continues until all records are checked. This reduces the likelihood of being confused by data glitches. Any spacing less than 1.0e-20 is assumed to be a duplicate timetag and is ignored. 

DvInt8 get_nanoSpacing() as above, but only applies to time sequences and the spacing is returned in nanoseconds.

bool  isJoined(DvObject &arg) uses the Depend_0 of arg as the argument if it has one, otherwise takes arg itself as the argument. Compares the id of the Depend_0 of this object with the id of the argument. Returns true if the id values are the same, otherwise if either has no Depend_0, then isJoined() returns true if the sequence length of this object and the argument are the same, or the argument has a sequence length of one. Otherwise (id's differ and both have a DEPEND_0) isJoined() does a fuzzy comparison with each value of the Depend_0 and the target and returns true only if they are all within a small tolerence (1.e-10, good to picoseconds), otherwise returns false.

bool  same(DvObject &arg) is the equivalent call to isJoined() except that this object and arg are themselves the DEPEND_0 object. This is a convenience call when testing against a target set of timetags (or scalar DEPEND_0), e.g. if( dep0->same(target_tt) )...

DvObject_var  linearJoin(DvObject &target, bool withXrefs, DvMask *Gmsk=NULL) joins this object onto the Depend_0 of dobj if it exists, otherwise it uses dobj itself as target. It can thus be called with either a data object or DEPEND_0 object as argument.  It returns a var pointer to a new DvObject containing this linearly joined onto the target object which must be a sequence of either DvTime or double values. The xrefs of the argument are attached to the result (joined if necessary) if withXrefs is true (the default). Options for gap width and gap handling may be set as xrefs in the object prior to joining and propagate into the resultant object for chaining operations (see Join Options below). If the optional DvMask pointer is provided, the DvMask must be created before linearJoin() is called, but it will be resized inside linearJoin(). If provided it returns a mask of the same length as the target with true for records to be kept and false for those flagged as gaps. It is used by MultiJoin. If not provided, then records are removed inside linearJoin() as necessary if either gap or end handling is set to DV_REMOVE.

DvObject_var  boxcarJoin(DvObject &target, bool withXrefs, DvMask *Gmsk=NULL) joins this object onto the Depend_0 of dobj if it exists, otherwise it uses dobj itself as target. It can thus be called with either a data object or DEPEND_0 object as argument.  It returns a var pointer to a new DvObject containing this object boxcar joined onto the target object which must be a sequence of either DvTime or double values. The xrefs of the argument are attached to the result (joined if necessary) if withXrefs is true (the default). Options for gap width and gap handling may be set as xrefs in the object prior to joining and propagate into the resultant object for chaining operations (see Join Options below). If the optional DvMask pointer is provided, the DvMask must be created before linearJoin() is called, but it will be resized inside linearJoin(). If provided it returns a mask of the same length as the target with true for records to be kept and false for those flagged as gaps. It is used by MultiJoin. If not provided, then records are removed inside linearJoin() as necessary if either gap or end handling is set to DV_REMOVE.

DvObject_var  nnJoin(DvObject &target, bool withXrefs, DvMask *Gmsk=NULL) oins this object onto the Depend_0 of dobj if it exists, otherwise it uses dobj itself as target. It can thus be called with either a data object or DEPEND_0 object as argument.  It returns a var pointer to a new DvObject containing this object nearest neighbour joined onto the target object which must be a sequence of either DvTime or double values. The xrefs of the argument are attached to the result (joined if necessary) if withXrefs is true (the default). Options for gap width and gap handling may be set as xrefs in the object prior to joining and propagate into the resultant object for chaining operations (see Join Options below). If the optional DvMask pointer is provided, the DvMask must be created before linearJoin() is called, but it will be resized inside linearJoin(). If provided it returns a mask of the same length as the target with true for records to be kept and false for those flagged as gaps. It is used by MultiJoin. If not provided, then records are removed inside linearJoin() as necessary if either gap or end handling is set to DV_REMOVE.

Note that xrefs will be joined using the same method as the data and this may not be appropriate for the data type. In rare cases it may be necessary
to join certain xrefs separately and attach after joining the data.


Join Options may be added as xrefs to the DvObjects to be joined. If they are not present defaults will be used.
The possibilities are:

DV_JOIN_METHOD can be DV_LINEAR, DV_NN or DV_BOXCAR, and this choice is propagated into the resulting object, so it may be specified at the start of a chain of operations, and will override the default algorithm in Join().

DV_GAP_WIDTH as a double is the gap tolerence for linear and nearest neighbour joins. If this xref is not set the gap will be 1.5 times the target spacing.

DV_BOX_WIDTH as a double is the boxcar width for a boxcar join. If this xref is not set the boxcar width will be 2 times the target spacing. For a boxcar any interval with too few points (see DV_MIN_BOXCAR) will be treated as a gap, and DV_GAP_WIDTH is ignored.
 
DV_MIN_BOXCAR as an int is the minimum number of values acceptable to make up the boxcar (for a boxcar join). The default is 3, and if fewer records are found inside the box at a record, then that record is taken to be a gap.

DV_GAP_OPTION determines what join should do with gaps in the input data. Options are DV_LINEAR, DV_NN, DV_REMOVE, DV_FILL (use fillvalue) or DV_ZERO_FILL.
The default option for boxcar join is DV_LINEAR
The default option for linear join is DV_LINEAR
The default option for nearest neighbour join is DV_NN

DV_END_OPTION determines what join should do when data start after or end before the target. Options are DV_LINEAR, DV_NN, DV_REMOVE, DV_FILL (use fillvalue) or DV_ZERO_FILL.
The default option for boxcar join is DV_NN
The default option for linear join is DV_NN
The default option for nearest neighbour join is DV_NN

DvObject_var interpAt(DvTime t) returns a var pointer with the value(s) of this object interpolated at time t and converted to double if possible (see asDouble() for conversions). Returns 0.0 if the object has no valid time tags. The returned object will have type double, sequence length 1 and array dimensions the same as the input object.

DvObject_var getCentres() returns a var pointer holding the values of this as though they were centred between Delta_plus and Delta_minus. Usually this is applied to the timeline and returns the centre time tags.

void applyLinearFill(DvMask &mask) applies the mask to this object. For records where mask is false the data value is linearly interpolated between the neighbouring valid data points. It permits linear filling of gaps for the boxcar where we require the interpolation to be between the boxcar averaged values, and which must therefore be done after the boxcar is complete. It is for internal use.
DvJoinList Class
The only purpose of this class is to permit joining of several DvObjects (Multijoin) onto the same target object. It is a DvList containing a linked list of DvNode objects which are the data objects to be joined and their names. Multijoin differs from joining objects independently only in that where gaps are removed from any resultant object, the same gaps are removed form all returned objects, so that all the MultiJoined objects are on exactly the same time tags (or scalar DEPEND_0).  

void append(DvString name, DvObject_var &obj) add an object and its name to the list (will be placed in a DvNode)
DvString MultiJoin( DvObject_var  &target, DvJoinList  &retList) does the join operation.  The target object must be a sequence of either DvTime or double values.

The DvString return value is either "Multi Join OK" or an error message.

The DvJoinList object calling MultiJoin() must hold the input objects for joining as the linked list of DvNodes. An empty DvJoinList is passed into MultiJoin and will contain the joined objects as a linked list of DvNodes on return.

Joining options may be added to the DvObjects to be joined as xrefs (see below). If they are not present the defaults will be used.
The possibilities are:

DV_JOIN_METHOD = DV_LINEAR, DV_BOXCAR or DV_NN
Note that only double data can be boxcar joined, otherwise the option is ignored and Nearest Neighbour is used.
Note also that only double, int and DvTime data can be linear joined, otherwise the option is ignored and Nearest Neighbour is used.
If not present, double, int and DvTime data default to linear join (DvString and DvEvents always use Nearest Neighbour).

DV_GAP_WIDTH as a double must be the boxcar width (for a boxcar join) or the gap tolerence for linear and nearest neighbour joins. If this xref is not set the boxcar width will be 10 times the data spacing, while the gap will be 1.5 times the data spacing.

DV_GAP_OPTION = DV_LINEAR, DV_NN, DV_REMOVE, DV_FILL or DV_ZERO_FILL
The default option for boxcar join is DV_ZERO_FILL
The default option for linear join is DV_LINEAR
The default option for nearest neighbour join is DV_ZERO_FILL
DvMask Class

 The DvMask class is a simple container for a valarray <bool>.

The DvObject method void apply_mask(DvMask &msk) will remove records from the calling object (and its xrefs) according to whether the corresponding entry in msk is true (keep) or false (remove). If the size of the mask does not match the sequence length, then this method does nothing other than set the object's error status.

There is an equivalent method DvObject *subSequence(DvMask &msk) in DvObject that returns a new object rather than modifying the object itself.

The DvMask class has the following methods

 DvMask(int n, bool ok=true)  This constructor creates a new mask of length n and sets all entries to ok. Default is ok = true (keep).
 DvMask(DvMask &msk)  This is a copy constructor that duplicates msk and its values.
size_t size()  Returns the length of the mask
size_t resultSize()  Returns the number of true entries in the mask
void resize(size_t n, bool ok=true)  Resizes the valarray to length n and sets them to ok. Default is true (keep).
 bool operator[ ](size_t i )  Provides access to the valarray value at i. This is the actual element, and allows modification of the value.
DvMask &operator+=(DvMask &m2)  adds the false (remove) values in m2 to this mask. The resulting mask will strip records identified by either mask.
DvNode and DvList Classes

The DvNode class is a container for a DvObject_var pointer and an associated name (the name of the object being held).
The DvList class contains a linked list of DvNode pointers, and methods to search and manipulate the list. It is used to hold the metadata (called xrefs, but attributes in CDF terminology) attached to a variable, the QSAS Working List, and any other list of objects for passing around the system (e.g. MultiJoin).

DvNode pointers work as a linked list, so that each node holds the location of the next node in the list, or a NULL.

The DvObject class stores its xrefs (metadata) via a DvList object called xrefs. The first DvNode in this list holds the first xref, its name and links to the next xref, and so on. Since metadata can itself have metadata, this linkage could become circular, so the DvObject::change_xref() method ensures that the target does not appear in the xref's list of xrefs (recursively). If a circular reference is found the a copy of the object without its list of xrefs is attached.

There are three equivalent constructors for convenience:
DvNode(const char *name, DvObject_var obj) where name is the node name (e.g. attribute name or Working List name) and obj is a var pointer to the data object
DvNode(string name, DvObject_var obj) where name is the node name and obj is a var pointer to the data object
DvNode(DvString name, DvObject_var obj) where name is the node name and obj is a var pointer to the data object
and an empty constructor.

The attributes are usually created, accessed and modified through the methods in DvObject and other classes with linked lists, and use of the following low level methods is rare.

The DvNode methods
DvString  name() returns the attribute name
DvObject_var  obj() returns the var pointer to the data object
void  setName(DvString name) sets (or changes) the attribute name
void  setObj(DvObject_var obj) sets (or changes) the var data pointer
DvNode  *next is the next node in the list (note it is a public pointer not a method) and is the normal way to step through a list.)
void  toXML(ofsteam &) is the method used for save to an XML file (for save/restore) and is an internal used by DvList.
void  fromXML(char *) is the method used for extract from an XML file (for save/restore) and is an internal used by DvList.


 The DvList methods
DvList() Creates an empty list.
DvNode  *first() returns a pointer to the first node in the linked list
int size() returns the number of nodes in the list
DvNode *makeNode() adds a new (empty) node to the start of the list and returns a pointer to it
void remove(DvString name) removes the named node from the list and relinks appropriately
DvString append(DvString name, DvObject_var &obj) adds a new node to the list (actually prepends) with this name. If the name exists an 'A' is added until a unique name found. The actual name used is returned.
DvNode *find(DvString &name) returns a pointer to the named node
DvNode *find(const char *name) returns a pointer to the named node
void clear() empties the list and sets the first node to NULL

void  toXML(ofsteam &ofp, DvString ListName) is the method used for save to an XML file (for save/restore) and is an internal used by DvObject and QSAS.
void  fromXML(char *) is the method used for extract from an XML file (for save/restore) and is an internal used by DvObject and QSAS. See below.

When the DvList goes out of scope it deletes the DvNodes in the list recursively.

Note: higher level methods on the relevant DvList and its DvNode contents are provided in the parent classes (DvObject, DvJoinList and DvWorkingList).
XML Methods

A DvObject can be written and read as XML text.  The two methods (which handle all supported data types) are:
void  toXML(ofsteam &ofp) writes the object and recursively its metadata to an XML file.
void  fromXML(char *) populates an object from XML text.

Since objects within QSAS are held on the Working List (a DvList) and their metadata are also (recursively) attached via a DvList,
it is never necessary to call these low level routines directly.

To save a DvList (such as the Working List) could involve code such as the following...
   
    DvList WL;
...
    std::ofstream ofp;
    ofp.open("file.xml", std::fstream::out );
    WL.toXML(ofp, DvString("WORKING_LIST") );
    ofp.close();

This in turn could be read via...

    DvList WL;

...
    WL.clear(); // remove old list unless data to be appended

    char * xml = readXML("file.xml"); // failure to read returns NULL
    size_t start;
    DvString tag;
    char *ptr = getNextTag(xml, start, tag);
    while(ptr){
        if(tag == "WORKING_LIST") {
            DvList WL_temp;
            WL.fromXML(xml+start);
           
        }
        // else if() read any other tags in the XML text
...
        // get next tag
        xml = ptr;
        ptr = getNextTag(xml, start, tag);
    }
    if(xml) delete xml;

In the case of QSAS itself the Working List is created at the same time as populating the GUI and so the XML list is read into a temporary DvList first, then unpacked into the real WL via the internal QT aware calls...

    DvNode *node = WL_temp.first();
    while(node){
        QString name = node->name().c_str();
        QwuiPutOnWL(node->obj(), name, false); // false is don't replace
        node = node->next;
    }

There are two read functions:
char *readXML(DvString nameAndPath) which opens the file and extracts the contents to a new char *. Note that there is a constructor for a DvString from a const char *, so calls using quoted text such as readXML("/path/file") are allowed.
char *getNextTag(char *xml, size_t &start, DvString &tag) which steps through the tags in the XLM string, returning a pointer to just beyond the closing tag. A null terminator is inserted after the tag contents which are returned via the offset pointer start. The DvString tag returns the tag name.
These functions are provided within the DVOS library. 

Note that the QSAS GUI save/restore operations are written using the QT supplied XML methods. However, all DVOS data objects and lists use only the methods contained within the DVOS library, which is thus self contained.



DvString Class

The DvString class inherits from std::string but has additional methods

The constructors are:
 DvString(void) constructs an empty string
 DvString( const string& s ) constructs a copy of s
 DvString( const DvString& ds ) constructs a copy of ds
 DvString( const char* cs ) constructs a copy of cs 
 DvString( const char c ) constructs a DvString of one character set to c

and are matched by equivalent assignment operators
DvString& operator=( const string& s ) etc.

There are further assignment operators that set the string from a "quoted" string (quotes are removed)
void setQuoted(  string& s )
void setQuoted(  const char * cs ) both of which set this string to the value of s or cs with enclosing quoted removed. If there are no enclosing quotes the string is set to the argument without change. Only the opening quote is tested, if present a closing quote is assumed and the last character removed.

The methods
 string str() gives access to the object explicitly as a std::string
bool blank() returns true if the string comprises only quote, space and tab characters
bool empty() returns true if the string holds no characters
bool isNumber() returns true if the string comprises only a numeric value (only numbers or  +, -, e, E, . )
bool isInt() returns true if the string comprises only an integer numeric value (only numbers or  +, - )
char last() returns the last character in the string
and the operator << allows the string to be streamed using cout and similar.

Substrings
DvString trimmed() returns a copy of the string with leading and trailing spaces removed.
DvString before(const char delim) returns the substring ending before the first occurence of delim. If delim is not found an empty DvString is returned.
 DvString before(const char *delim) returns the substring ending before the first occurence of delim. If delim is not found an empty DvString is returned.
 DvString before(size_t posn) returns the substring ending before posn. If posn is beyond the end, the whole string is returned.
DvString after(const char delim) returns the substring starting after the first occurence of delim . If delim is not found an empty DvString is returned.
DvString after(const char *delim) returns the substring starting after the first occurence of delim. If delim is not found an empty DvString is returned.
DvString after(size_t posn) returns the substring starting after posn. If posn is beyond the end, an empty DvString is returned.
DvString common(DvString &str) returns the common parts between this and str. Restarts comparison at space, "(" and "_" and concatenates common blocks. Removes all "|" characters otherwise mod operator messes up common stems in vectors.
DvString between(const char *delim1, const char *delim2) returns the substring starting after the first occurence of delim1 and ending before the first occurence of delim2. If either deilm is not found an empty string is returned.
 DvString back(const char *delim) returns the substring starting after the first occurence of delim. If delim is not found, the whole string is returned.
DvString front(const char *delim) returns the substring ending before the first occurence of delim. If delim is not found, the whole string is returned.
DvString replace(const char * str1, const char * str2) returns a copy of this string with the first occurence of str1 replaced by str2;
DvString replaceAll(const char * str1, const char * str2) returns a copy of this string with all occurences of str1 replaced by str2;
DvString replaceTokens(const char * str, const char delim) returns a copy of this string with the appropriate text from str as follows. If str contains a substring of the form ?str1=str2? where ? is the delim, then all occurences of str1 in this string are replaced by str2. The string str may hold more than one token of the form ?str1=str2? and each is processed in turn in the order they appear in str. The default value of delim is '?'.
DvString wrapOperator(DvString op) Returns a copy of this string with the operator wrapped around it. The operator should  have two space separated parts, e.g. "< >".
DvString cutToken(const char* tok1, const char* tok2, DvString &target) Returns a copy of this string with the text between tok1 and tok2 inclusive removed provided that this text does NOT appear in target. Used for removing non-common blocks of text. Used by common() above. If tok1 is an empty string the cut text is from the start to tok2. If tok2 is an empty string the cut text is from tok1 to the end.

Conversions
double toDouble() returns the text as a double
int toInt() returns the text as an int
DvTime toTime() returns the text as a DvTime object
DvEvent toEvent() returns the text as a DvEvent object
void set(double val) set the content of the DvString to the text equivalent of val.
void set(double val, int precision) sets the content of the DvString to the text equivalent of val, but limits the number of places after the decimal point to precision. Tries to use sensible fixed or exponential formats for appropriate value.
void append(double d, const char * spacer) appends the text spacer followed by value of double d as a string
void append(double d, const char * spacer, int precision) appends the text spacer followed by value of double d as a string, but limits the number of places after the decimal point to precision. Uses scientific notation for value below 0.1 or above 1000.
void append(int i, const char * spacer) appends the text spacer followed by value of int i as a string
void append(DvTime t, const char * spacer) appends the text spacer followed by value of DvTime t as a string
void append(DvEvent e, const char * spacer) appends the text spacer followed by value of DvEvent e as a string
The default spacer is an empty string. Appending to an empty string can be used in place of an assignment.
Note that the method s = t_var->getISOstring() can be used to create a DvString from a DvTime or DvEvent.
The operator += is equivalent to append with default empty spacer, and is overloaded to work for double, int, DvTime, DvEvent, DvString, string and const char*.
 DvObject_var changeUnitsTo(DvString SIC, DvString Units) returns a DvObject pointer containing a copy of this converted to the units specified in the SI_conversion string SIC. The Units argument is used for the Units attribute in the returned object and are free form text. To convert to km/s one
could specify changeUnitsTo("1.0e3>m s^-1", "km/s"). The conversions to DvString are done automatically at run time.

Case sensitivity
DvString toUpper() returns the same string converted to upper case. Characters with no upper case remain unchanged
The operator == returns true if the strings are the same when both converted to upper case (this is thus a case insensitive method of comparing strings which is useful for attributes). It tages either a DvString or const char * as target.
bool contains(DvString &vs)
bool contains(const char *cs) both return true if this string contains vs or cs respectively. Like the operator == compared strings are converted to upper case before comparison.
bool containsCS(DvString &vs)
bool containsCS(const char *cs) are the equivalent case sensitive methods.


DvTime Class

The DvTime class stores date and time as an integer (base_day), day in Modified Julian Date, and a DvInt8 (time_nsec) counting nano-seconds since the start of base_day in atomic time, TAI. The time_nsec value can hold up to at least 300 years after the base_day. See DvTime.h for more details of Modified Julian Date (MJD). For reference January 01, 2000 has base_day 51544, and is the default date.

DvInt8 is declared as a long long, and time is held to nano-second accuracy. The number of digits shown after the decimal point in the seconds field can be controlled for text format output.

Arithmetic is always done modulo hours, minutes and seconds, so, for example, setting a time with 1hr 62s will safely be treated as 2hr 2s.

The constructors are:
DvTime(double sec) constructs a time from any date and seconds offset from 2000-01-01 00:00:00.0000. It is the inverse of getSecTAI(), and is used internally within QSAS when time is needed as a double, for example when setting plot windows.
DvTime(const char* iso_time ) constructs a time from a UTC  date-time string in ISO format.
DvTime(const string& iso_time ) constructs a time from a UTC date-time string in ISO format.
 DvTime(int year, int month=1, int day=1, int hour=0, int min=0, double sec=0.0 ) constructs a time from UTC date_time elements.
 DvTime(int year, int month=1, int day=1, int hour=0, int min=0, DvInt8 nano=0 ) constructs a time from UTC date_time elements with nano-seconds.
 DvTime(const DvTime& ttime ) copy constructor.
DvTime(DvInt8 nano, int day=MJD2000 ) constructor from nano-second and start day in TAI.

Setting DvTime
void setFromUT(int year, int month, int day, int hour, int min, double sec) sets the time from individual elements in UTC.
 void setFromUT(int year, int month, int day, int hour, int min,  DvInt8 nsec) sets the time from individual elements in UTC.
void setFromDOY(int year, int dofy, int hour, int min, double sec) sets the time from individual elements in UTC using day of year.
void setFromBCE(int yearBCE, int month, int day, int hour, int min, double sec) sets the time from individual elements in UTC for year
before current epoch (formerly BC).
void setFromMJD(double ModifiedJulianDate) set from Modified Julian Date in TAI (includes fraction of day).
void setFromCDFepoch(double cdf_epoch) set from CDF epoch value.
void setFromCDFepoch16(double *cdf_epoch16) set from a pair of CDF epoch16 values.
void setFromJD(double JulianDate) set from Julian Date.
void setFromEpoch2000(double sec) set from seconds after Jan 01, 2000. This is the inverse of getEpoch2000().
void setFromISOstring(const char * ISOstring) set from ISO format UTC string.
 void setFromISOstring(string ISOstring) set from ISO format UTC string.
void setFromTT2000(DvInt8 nsec ) set from CDF TT2000 nanosec value.
void setFromTAI2000(DvInt8 nsec ) set fromTAI2000 nanosec value.
DvTime& set_today() sets the date to today and the time to the start of the day.

Accessing DvTime
int breakDownTAI(int *year, int *month, int *day, int *hour, int *min, double *sec) breaks a DvTime down into UTC date_time elements.
int breakDownUTC(int *year, int *month, int *day, int *hour, int *min, double *sec) used internally to break a DvTime down into the date_time elements after leap seconds have been removed. It is used only as an intermediate step as DvTime is in TAI. Do not use.
double getSecTAI() returns TAI as a double seconds from start of MJD2000. This is not true TAI or TT2000 which both start from noon, use getTAI2000().
double getMJD() returns MJD in TAI as a double (includes fraction of day).
double getJD() returns JD in TAI as a double (includes fraction of day).
double getDiffDays(const DvTime &arg) returns the difference in days between this DvTime and arg, (this - arg).
 double getDiffSecs(const DvTime &arg) returns the difference in seconds between this DvTime and arg (this - arg).
double getDiffNano(const DvTime &arg) returns the difference in nano-seconds between this DvTime and arg (this - arg).
double getCDFepoch() returns the time as a double CDF epoch value.
void getCDFepoch16(double *epoch16) returns the time as a pair of doubles as used in CDF epoch16.
double getEpoch2000() returns the time as a double seconds since Epoch2000 (Jan 01, 2000)
double getEpoch1970() returns the time as a double seconds since Epoch1970 (Jan 01, 1970)
DvString getISOstring(int delim) returns an ISO format standard string in UTC, if delim=0 (default) a space separates the date and time, if delim=1 a capital T separates them, if delim=2 a trailing Z is also appended.
const char * getDayOfWeek() returns the three letter abbreviation for the day of the week for this date in UTC.
const char * getLongDayOfWeek() returns the unabbreviated day of the week for this date in UTC.
const char * getMonth(int m) returns the abbreviated month for m.
const char * getLongMonth(int m) returns the unabbreviated month for m.
bool getYandD(int *year, int* day) finds the year and day of year and returns true if it is a leap year.
size_t strfMJD(char * buf, size_t len, const char *format) Formats a text string according to the format string. Uses the same syntax as strftime() but does not use current locale. The null terminator must be included in len which is the size of the buffer. Returns the length of the formatted string.
DvTime normalized() const returns the same date_time but with the time_nsec field being less than a day, and base_day modified accordingly.
void normalize() changes the internal storage such that the time field is less than a day, and base_day modified accordingly.
DvInt8 getTAI2000() returns the nanosecond value for the TAI offset from the year 2000. Reverses setFromTAI2000().
DvInt8 getTT2000() returns the nanosecond value for the TT offset from the year 2000 (CDF TT2000 value). Reverses setFromTT2000().

 
Setting Options
void setSecResolution(int resolution) sets the number of decimal places to be shown in seconds field in string format.
void setGregorianStartMJD(double GregorianMJD) sets the start date used for the Gregorian calendar. This was not uniform between cultures (or software). Default uses the Gregorian calendar after 4 Oct 1582 (Julian) i.e. from 15 Oct 1582 Gregorian.
Note unix libraries use Gregorian only from 14 Sept 1752.
   
Operators
DvTime& operator=( const DvTime& ttime )
DvTime& operator=(  const double dtime ) where dtime is a double time since Epoch2000.
DvTime& operator=(  const string& isotime ) where isotime is a string date_time in ISO format.
double operator=(  const DvTime& ttime ) returns the time in seconds since Epoch2000.

DvTime& operator*=(  const double factor ) provided only for intermediate steps in forming averages etc, may not yield a valid time.
DvTime& operator/=(  const double factor ) provided only for intermediate steps in forming averages etc, may not yield a valid time.
DvTime& operator+=(  const double factor ) provided only for intermediate steps in forming averages etc, may not yield a valid time.
DvTime& operator-=(  const double factor ) provided only for intermediate steps in forming averages etc, may not yield a valid time.
DvTime& operator+=(  const double sec ) adds seconds sec.
DvTime& operator-=(  const double sec ) subtracts seconds sec.

DvTime operator+(  const double sec ) returns the time plus seconds sec.
DvTime operator-(  const double sec ) returns the time minus seconds sec.
double operator-(  const DvTime& ttime ) returns the difference between times in seconds.

   
Logical Operators
bool operator==( const DvTime& ttime ) returns true if times are the same to an accuracy of .1 nanoseconds.
bool operator!=( const DvTime& ttime ) returns true if times are not the same to an accuracy of .1 nanoseconds.
bool operator<( const DvTime& ttime ) returns true if this is before ttime.
 bool operator<=( const DvTime& ttime ) returns true if this is before or equal to ttime to an accuracy of .1 nanoseconds.
bool operator>( const DvTime& ttime ) returns true if this is after ttime.
bool operator>( const DvTime& ttime ) returns true if this is after or equal to ttime to an accuracy of .1 nanoseconds.
   
Utilities
int getSecResolution() returns an estimate of the number of characters after the seconds decimal place. If the object is neither time or event then zero is returned.
DvTime & aveThis(DvTime ttime) sets this to the average value of this and ttime. Returns this.
DvTime ave(DvTime ttime) sets result to the average value of this and ttime. Returns a new DvTime object.
Dvint8 UTtoDV(int y, int m, int d) internal utility for finding leap seconds (in nanoseconds) that must be added to convert UTC to TAI for specified year, month and day.
Dvint8 UTtoDV() internal utility for finding leap seconds (in nanoseconds) that must be added to convert UTC to TAI, calculates y, m and d and calls above.
Dvint8 DVtoUT() internal utility for finding leap seconds (in nanoseconds) that must be subtracted to convert TAI to UTC before calling breakDownUTC().
Dvint8 DVtoUT(bool &inLeapSec) internal utility for finding leap seconds (in nanoseconds) that must be subtracted to convert TAI to UTC before calling breakDownUTC(). Additionally returns inLeapSec as true if the TAI time is actually within the leap second itself.
DvEvent Class

The DvEvent class stores time intervals as two DvTime values, _t1 and _t2. In the following, tests to "better than nanosecond accuracy" use fuzzy equality tests within 0.1 nanoseconds.


The constructors are:
DvEvent() constructs an interval from DvTime() to DvTime+1.0 seconds.
DvEvent(const DvEvent& event) copy constructor.
DvEvent(string &ISOstring) constructs from an ISO standard time interval string (two ISO date_time strings separated by '/').
DvEvent(const DvTime& st, const DvTime& ed, Sense option) construct from two DvTime objects. If option is NO_TIME_SENSE (default), then _t1 and _t1 are set from st and ed respectively. If option is TIME_SENSE_NEGATIVE or TIME_SENSE_POSITIVE the two times are ordered appropriately.

Methods
 DvTime start() returns _t1.
DvTime end() returns _t2.
 DvTime start_abs() returns earlier of _t1, _t2.
DvTime end_abs() returns later of _t1, _t2.
void set_start(const DvTime& st) sets the value of _t1 to st.
void set_end(const DvTime& ed) sets the value of _t2 to ed.
void set(const DvTime& st, const DvTime& ed, Sense option) sets the event from the two DvTime objects. If option is NO_TIME_SENSE (default), then _t1 and _t1 are set from st and ed respectively. If option is TIME_SENSE_NEGATIVE or TIME_SENSE_POSITIVE the two times are ordered appropriately.
void setFromISOstring(const char* ISOstring) sets the event from an ISO standard event string.
void setFromISOstring(string & ISOstring) sets the event from an ISO standard event string.

 std::string  getISOstring(int delim) returns event is ISO standard string format. If delim=0 (default) date and time elements are space separated, if delim=1 then they are separated with character 'T' and if delim=2 the date_time strings are also terminated with the character 'Z'.
Sense get_sense() returns one of NO_TIME_SENSE (if _t1 == _t2 to better than nanosecond accuracy), TIME_SENSE_NEGATIVE (if _t1 > _t2 ) or TIME_SENSE_POSITIVE (if _t1 < _t2 ).
void force_sense(Sense option) sets the ordering of _t1 and _t2 according to the value of option (default is TIME_SENSE_POSITIVE).
void reverse() swaps the values _t1 and _t2.
double duration() returns _t2 - _t1 in seconds.
double duration_abs() returns the absolute value of _t2 - _t1 in seconds.
std::string iso_srep() returns the interval as two ISO  date_time strings separated by " - " for display purposes.
DvTime max(DvTime &t1, DvTime &t2) returns the later of _t1 and _t2.
DvTime min(DvTime &t1, DvTime &t2) returns the earlier of _t1 and _t2.

DvEvent int_intersect(const DvEvent& ti) returns the intersect of this event with event ti.
DvEvent int_union(const DvEvent& ti) returns the union of this event with event ti.
bool contains( const DvTime& t ) returns true if the time t is inside this event interval.
bool intersects( const DvEvent& tivl ) returns true if the event tivl overlaps this event.
bool empty()  returns true if the start and end times are the same to better than nanosecond accuracy.

   
Operators
DvEvent& operator=(  const DvEvent event ) copy operator.
DvEvent& operator+=(  double offset ) adds the value offset (in seconds) to both start and end times.
DvEvent& operator-=(  double offset ) subtracts the value offset (in seconds) from both start and end times.
 double operator*(  const double multiplier ) returns the duration() of the interval multiplied by multiplier.
bool operator==(  const DvEvent& event ) returns true if the start and end times of this and event are the same to better than nanosecond accuracy.
bool operator!=(  const DvEvent& event ) returns true if the equality operator above returns false.
bool operator>(  const DvEvent& event ) returns true if the start() of this is after the start() of event, of if the starts are equal and the end() of this is after the end() of event. Otherwise returns false.




DvMatrixEig Class

The constructor for the DvMatrixEig class
DvMatrixEig(DvObject_var &m) evaluates the eigenvalues and eigenvectors for the matrix m, and holds them internally. The matrix must have rank 2, and the two dimensions must be equal (2D square matrix of dimension nxn) otherwise both eigenvectors and eigenvalues are empty objects.

Methods
 DvObject_var eigenvalues() returns the eigenvalues as a 1D array of n elements.
 DvObject_var eigenvectors() returns the eigenvectors as a 2D array of nxn elements, with vectors stored as rows. Note they are calculated as columns using d_eigen, and then transposed.




DvMinVariance Class

This class finds the minimum and maximum variance directions of a vector sequence.
 
The constructor for theDvMinVariance class
DvMinVariance(DvObject_var &vs, DvObject_var &constraint, const int method) determines the minimum for the vector sequence vs subject to an optional constraint constraint, and using the method specified by the integer method. If constraint is an empty object it is ignored. For constraint non-empty, see ISSI blue book and notes in the code.
 
The method may be one of:
method = 1 uses the classic covariance matrix and is the default
method = 2 uses the Weimer type covariance matrix
method = 3 uses the Siscoe type covariance matrix
method = 4 should not be used, and is retained only for completeness, it uses the extreme Weimer type covariance matrix (N -> infinity)



Methods
 DvObject_var eigenvalues() returns the eigenvalues as a sequence of 1D array of n elements.
 DvObject_var eigenvectors() returns the eigenvectors (Rotation matrix sequence).
 DvObject_var projection() returns the projection (nxn matrix sequence), see ISSI book.
 DvObject_var covariance() returns the covariance (nxn matrix sequence).
 DvObject_var eigenquality() returns the eigenvalue quality (as a sequence of 1D array of n elements).
 double orthogonality_quality() returns the orthogonality quality.



Accessible Internal Utilities

There are a few useful general functions used inside DVOS that can be accessed directly.
 
valarray <double> atan_yx(valarray <double> &y, valarray <double> &x) provides the function atan2(y,x) on a valarray, with the result in (0, 2pi).
double atan_yx( double y,  double x) provides the function atan2() with result in (0, 2pi).
bool isFill(double val, double fill) returns true if the value val is not a number or satisfies a fuzzy equality with fill.
bool notFill(double val, double fill) returns false if the value val is not a number or satisfies a fuzzy equality with fill.