Vocabulary/NumericPrecisions

From J Wiki
Jump to navigation Jump to search

Back to: Vocabulary

Numeric Precisions in J

J has several precisions for the various types of data, as described here. J automatically converts from one precision to another as needed.

This page explains the details for numeric type, and how you can influence J's choice of precision.

Use datatype to find the precision of any given constant (or noun in general) 

   datatype 'alpha'
literal
   datatype 0
boolean
   datatype _
floating

datatype is a

  • Standard Library word(s) residing in the 'z'-locale
  • Defined in the factory script stdlib.ijs which is located in  ~system/main/stdlib.ijs
  • View the definition(s) in a JQt session by entering:  open '~system/main/stdlib.ijs'

Note: verb datatype uses the Foreign (3!:0 y), which returns the precision of noun y as an integer. You can use (3!:0) instead of datatype 

   (3!:0) 'alpha'
2
   (3!:0) 0
1
   (3!:0) _
8

We show the results of both datatype and (3!:0) in the table below

Low Priority <-- Numeric Precisions --> High Priority
Exact Inexact Exact Inexact
Boolean Integer Extended-Integer Rational Floating-Point Integer2 Integer4 Long Floating-Point Complex
J default? * * * * * *
Range 0-1 machine

word size
(4byte)±231
(8byte)±263

(unlimited) (unlimited) IEEE

double precision
--> range
±1E±308

±215 ±231 IEEE

double precision
--> range
±1E±308

IEEE

double precision
for both real and
imaginary parts

Size of atom

in memory

1 byte (varies) (varies) (varies) 8 bytes 2 bytes 4 bytes 16 bytes 16 bytes
Overflows

to higher
precision

yes yes N/A N/A no no no no N/A
Precision

from (3!:0)

1 4 64 128 8 6 7 11 16
Precision

from datatype

boolean integer extended rational floating integer2 integer4 floating16 complex
Example 1 34 367x 1r2 1.2 1.2fq 1j2

Notes on the above table

The fixed-precision types integer2 and integer4 signal fixed-precision overflow if the result of an operation does not fit in the same size as the arguments.

Extended-integer constants (numbers ending in 'x') are always extended-integer precision, even if a lower priority precision can store the exact value in memory 

   datatype 1x
extended

Rational constants (numbers with embedded 'r') are extended-integer precision, provided it can store the exact value in memory. Otherwise they're rational precision

Rational precision stores a pair of [extended] integers which, when divided, give the number to be represented. 

   datatype 2r2
extended
   datatype 1r2
rational

Float16 values use 2 IEEE-754 floats to hold a single number. The two parts, hi and lo, are in canonical form defined by the following:

  • The value of the number is hi+lo
  • hi cannot be infinite
  • -1/2ULP of hi<=lo<=1/2ULP of hi
    • if |lo| is 0, the signs of hi and lo must be the same
    • if |lo| is 1/2ULP of hi, the signs of hi and lo must be different

Other numeric constants use the lowest-priority value that equals the given constant, unless a precision-defining character forces a higher precision. The precision-defining characters are j (complex), . (float), or leading 0 (integer).

  • 2 , 2.000, and 2j0 are integer, float, and complex constants with the value 2
  • 1 , 01, 1.000, 1.0fq, and 1j0 are boolean, integer, float, double-double, and complex constants with the value 1
   datatype 2
integer
   datatype 2.000
floating
   datatype 2j0
complex
   datatype 1
boolean
   datatype 1.000
floating
   datatype 1j0
complex

The inexact precisions approximate a number using floating-point arithmetic. Comparisons between inexact precisions are tolerant.

The exact precisions store an exact value in memory. Comparisons between exact-precision numbers are not tolerant.

The precision of a verb's result

The rules for determining result precision are:

  • the precision is based on the precisions of the arguments, ignoring their values. The result has the lowest-priority precision that can represent any result.
    • Exception: Integer computations that always produce exact integer results produce integer precision unless overflow is detected. Only then do they promote to floating-point.
    • Exception: operations on extended and rational arguments that may produce non-integral results do not promote to inexact precision unless the data values require it.

Demote the precision of a noun by using a verb guaranteed to have an exact (e.g. Boolean) result. This yields a noun that uses less memory and may run faster when used in a special combination phrase.

  • Use  <. y and  >. y to demote any inexact precision to integer
  • Use  1= y to demote any precision to Boolean.

The only primitive verbs that produce extended-integer results (on all-nonempty arguments) are Extended Precision (x:) and Anagram Index (A.) . Prime Factors (q:) produces an extended-integer result if required.

When J decides whether to give Boolean precision to a nonempty integer result, it ignores the result's actual value (e.g. 0 or 1) 

   datatype 0            NB. 0 is a boolean constant
boolean
   datatype 1            NB. 1 is a boolean constant
boolean
   datatype 4            NB. 4 is an integer constant
integer
   datatype 0 +. 1       NB. (boolean OR boolean) ought to give boolean result (0 or 1)
boolean
   0 +. 1                NB. ... yes it does!
1
   datatype 4 +. 1       NB. but (boolean GCD integer) MIGHT be integer, so result is integer
integer
   4 +. 1                NB. ... even though the result's actual value is 1
1

Verbs with extended-integer or rational arguments return an exact precision if possible 

   datatype ^ 0x         NB. exp(0) = 1
extended
   datatype 2x % 3       NB. 2x % 3 = 2r3
rational
   datatype %: 4 9 4r9   NB. Exact square roots stay exact
rational
   datatype %: 10x       NB. But inexact square roots become floating-point
floating

As shown in the last example above, a result that can't be stored in memory exactly will automatically be promoted to an inexact precision. You can prevent this by using  <.@v or  >.@v This tells J that an inexact precision is not needed, so J can keep the result in exact (extended-integer) precision 

   0 ": %: 2 * 10^100x   NB. The result is floating-point
141421356237309520000000000000000000000000000000000
   0 ": <.@%: 2 * 10^100x   NB. The result is extended
141421356237309504880168872420969807856967187537694

Automatic Promotion of Argument Precision

For computational dyads (e.g. x+y or x<y) J promotes both arguments to the same precision, i.e. J converts whichever argument (x or y) has the lower-priority precision into that of the other argument, according to the table above. This act of promotion is based solely on the precision of the two arguments x and y. It ignores their value. 

   datatype (1.2) + 23x       NB. 1.2 is a floating-point constant, so 23x gets promoted to floating-point
floating
   datatype (1.0) + 23x       NB. 1.0 is a Boolean constant, so it gets promoted to extended
extended
   datatype (0.5+0.5) + 23x   NB. (0.5+0.5)=1, but it's floating-point, so 23x gets promoted to floating-point
floating

Promotion takes place before the operation of the dyad is performed. This gives an easy way to make a given result have extended precision: add an x to the end of any contributing constant 

   ,. ! */\ 6 $ 2x
                                                                                         2
                                                                                        24
                                                                                     40320
                                                                            20922789888000
                                                      263130836933693530167218012160000000
126886932185884164103433389335161480802865516174545192198801894375214704230400000000000000

Not every argument causes automatic promotion!

  • Data arguments, such as the arguments of computational dyads or the value-contributing arguments of structural verb like x # y, influence the precision of the result.
  • Control arguments (e.g. the shape-contributing arguments of structural verbs like: #) do not influence the precision of the result.
   ,. ! */\ 6x $ 2
         2
        24
     40320
2.09228e13
2.63131e35
1.26887e89

Comprehensive details are given in the table below.

Surprising results are:

  • $ y and # y produce extended-integer result when y is extended or rational
  • integer %: extended produces extended if exact, but extended ^ rational produces floating-point whenever the denominator is not 1 (e. g. 27x ^ 1r3)
  • x %. y with integer y causes promotion of exact x to floating-point, because y is inverted before promotion is performed.
  • 1x ^. 1x produces extended-integer
  • x p. y and x p.. y promote Boolean and integer to floating-point (if you want to keep to integers, consider using x #. y to evaluate your polynomial)
  • Seeming tautologies, such as that x <. _ is equal to x, can be violated:
   <. 9223372036854765580 <. _  NB. With 64-bit integers
9223372036854765568
Precisions Produced By J Verbs
Valence Description Verbs Precision Produced
Monads Structural ~. |. |: , ,. ,: [ ]

{. {: }. }:

same as y
+: - #. same as y except never Boolean
> ; from contents of y
Computational,

fixed
precision

= ~: Boolean
I. p.. /: \: L. integer
*. floating-point
< { ;: Boxed
A. Extended-Integer
j. r. Complex
Computational,

variable
precision

+ *: -. same as y
<: >: same as y, except never Boolean
<. >. * same as y except converts

floating-point-->integer
rational-->extended

+. | same as y except converts

complex-->floating-point

i: same as y except converts

Boolean-->integer
complex-->floating-point
or
complex-->integer
depending on values

#: same as y except converts

floating-point-->Boolean
integer-->Boolean
if values permit,
But
floating-point 1-->integer 1)

o. floating-point, or complex if needed
$ # i. p: q: integer, except

extended if y is extended or rational

^ ^. -: % %. inexact (complex if needed),

preserves extended and rational
if result is exact

%: ! Preserves Boolean, converts

integer to inexact (complex if needed),
preserves extended and rational
if result is exact

? ?. Boolean, integer, floating-point, or extended

depending on value of y

C. Boxed or integer

depending on type of y

p. Depends on values of y.

Multiplier and roots have
independent precisions.

Dyads Comparisons = -: ~: e. > >: < <:

+: *:

Boolean
Structural $ |. # {. }. |: ] {

A. C.

same as y
-. [ same as x
, ,. ,: higher-priority of argument precisions
Computational,

automatic
promotion

>. <. +. * *. ! higher-priority of argument precisions
? ?. integer or extended

depending on higher-priority
of argument precisions

| higher-priority argument precisions

except
integer if x is integer
y is floating-point
result is (tolerantly) 0
x does not contain 0 or infinity.

#: higher-priority of argument precisions

except
integer if x is integer
y is floating-point
result is (tolerantly) 0
x does not contain 0 or infinity.
Also Boolean is promoted to integer

+ - #. higher-priority of argument precisions

but Boolean converted to integer

p. higher-priority of argument precisions

after Boolean and integer
converted to floating-point

p.. higher-priority of argument precisions

after Boolean and integer y
converted to floating-point

% %: higher-priority of argument precisions

but converts Boolean and integer to floating-point
(or complex if needed),
preserves extended and rational if result is exact

^ higher-priority of argument precisions

but converts integer y (only) to floating-point
(or complex if needed),
preserves extended and rational if y is integer

^!.n first converts inexact or Boolean y to integer;

then higher-priority of argument precisions

%. first calculates %.y

and uses that precision to perform promotion;
converts Boolean and integer to floating-point
(or complex if needed),
preserves extended and rational if result is exact

^. inexact (complex if needed),

or extended for 1x ^. 1x

Computational,

fixed
precision

i. i: I. integer
j. r. complex
; boxed

Overflow and Result-Dependent Promotion of Argument Precision

The result of an operation may be promoted to a higher precision if the value cannot be represented in the precision of the arguments.

  • arithmetic on booleans is promoted to integer
  • any non-integral result, for example (x ^ y, x % y) is promoted to floating-point
  • any complex result, for example (%: _1) is promoted to complex

Numeric overflow in a computation also causes promotion except for fixed-size precisions, which quietly discard upper precision.


Retrieved from "https://code.jsoftware.com/mediawiki/index.php?title=Vocabulary/NumericPrecisions&oldid=51357"