DADiSP Online Help
Click here to see this page in full context
DADiSP Worksheet Functions > Function Categories > Numerical Formatting > SETPRECISION
Sets the number of significant digits after the decimal place to display for numerical values including tabular listings and scales.
SETPRECISION(prec)
|
prec |
- |
An integer, the number of digits of precision after the decimal point. |
An integer, the previous precision value.
setprecision(3)
sets the number of digits after the decimal place to 3.
setprecision(-1)
resets the display precision to the internal default of 6 digits.
p = setprecision(16);
disp(eps);
setprecision(p);
Displays EPS to 16 digits in a popup box and restores the original precision.
SETPRECISION affects only the display of a value. It does not affect the internal precision used for calculations.
Use setprecision (with no arguments) to display the current precision.
Because the tabular view of data automatically conforms to the specified precision, this function can be extremely useful for formatting a table Window.
SETPRECISON affects all numerical displays in each Window of a Worksheet, including Window annotation.
To observe the change in precision, the Window must be redrawn.
SETPRECISION can be abbreviated SETPREC.
The internal default precision set by setprecsion(-1) can be specified with the DEFAULT_PRECISION configuration parameter.
See SETXYTICPRECISION and SETYTICPRECISION to set the tic precision for individual Windows.
See SETFORMAT to specify a format string for additional control over numeric formatting.