System/ReleaseNotes/J903

Changes to the J engine

Summary of Changes to the J Language in the release

New foreigns 9!:4 and 9!:5 support Nameref Caching, a way of avoiding name-lookup overhead especially useful when you have many locatives or long search paths.
New foreign u 4!:8 (adverb) produces a cached reference to u
Old foreigns 2!:2 and 2!:3 are removed from the language
+/!.0 y performs compensated summation (addition with improved accuracy)
x +/@:*"1!.0 y performs the computation at twice the precision of floats
x i.!.1 y assumes that x and y are both in nondescending order; faster when cells are integer lists. Supports IRS.
name:: is a self-effacing reference to name. name is deleted, and the reference is replaced by its value, made inplaceable if possible.

unreleased work in progress

J903 beta-q (the current beta)

virtual blocks allowed in boxes in more situations, see details about virtual blocks
256-bit instructions used for 1-byte frets in ([x] f;.12 y) (AVX2 only)
+/ y, >./ y, <./ y now use 32 accumulators on float arguments rather than 16 (AVX2 only)
128!:10 y calculates LU decomposition of y

J903 beta-p withdrawn

J903 beta-o

new feature: name:: is a self-effacing reference to name. name is deleted, and the reference is replaced by its value, made inplaceable if possible.
Parser revisited in detail, much faster (by 2x or more for sentences containing only singletons)
code for atomic singletons rewritten, lower overhead and with faster NaN-checking for floats
(x # y) rewritten to use 256-instructions on x, and faster copy for atomic y
lowest-level memory-allocation paths rewritten to reduce register spills
(AVX2 only) (x {"1 y) improved for integer and float y
(x ((> |) * ]) y (force near-0 IEEE values to 0) supported with special code when x is a float atom and y is float (AVX2 only)

J903 beta-n withdrawn

J903 beta-l

Inner loop of transpose (|:) unrolled, 50% faster
(+/ y) (+/\ y) (+/\. y) (and also for - and *) rewritten for integer y, faster
Complete rewrite of combinations created by hook/fork/@:/&:/&./&.: . Now blocks are freed immediately after the verb they are applied to, provided that verb did not save the value or modify it inplace. Also, in hooks (f g h), the cases where f or h are [ ] @[ @] are handled specially to remove overhead.

J903 beta-k

256-bit instructions used for (I. y) when y is boolean (AVX and AVX2 only)
256-bit instructions used for (+/ y) when y is a boolean list (AVX and AVX2 only)
256-bit instructions used for x | y on integers, when x is a power of 2 (64-bit only)
(AVX2 only) +/\ y, <./\ y, and >./\ y on float lists improved

J903 beta-j

bitwise booleans x m b. y implemented with 256-bit instructions (AVX and AVX2 only)
comparison combinations on bytes performed with word-wide comparisons
comparison combinations i.&0@:comp (also i:, also 1, also +/) use 256-bit instructions (AVX and AVX2 only)
sequential-search loops recoded (list i. atom) for AVX and AVX2, faster
special code for (i. <./) y etc. no longer faster, deleted
internal coding of types changed. There should be no visible effects, but if there are any they will be in sparse arrays
+/!.0 y hand-unrolled, 2x faster
x E. y on characters recoded, faster when matches are frequent (AVX and AVX2 only)

J903 beta-i

(64-bit only) x ([ -. -.) y is implemented as a member of the i.-family, performing only one hash
(AVX2 only) all loops for arithmetic primitives with heterogeneous arguments recoded, faster
(> y) rewritten to avoid needless copying of fill
(AVX2 only) all loops for arithmetic primitives rewritten with better addressing modes, faster
(64-bit only) special code for -.&n reinstated, applying also to ([ -. -.)&n
Precomputed searches (m&i., (e.&n, (i.&1@:>&n, etc) support comparison tolerance

J903 beta-h

(AVX2 only) +/\ y, <./\ y, and >./\ y on float lists rewritten
(AVX2 only) all loops for arithmetic primitives with homogeneous arguments recoded, much faster on inplace arguments
(AVX2 only) internal memory-copying routine rewritten, faster for all sizes, but especially for non-multiples of 32 bytes

J903 beta-g

search for private names faster
debut of PPPP: pre-parsed parenthesized primitives. Parenthesized expressions that contain only primitives and do not cause executions of a verb (thus (+/ % #) but not (i. 1e8)) are parsed when an explicit entity is defined, avoiding the overhead of parsing each time the sentence is executed
arithmetic dyads + - % * >. <. coded to use 256-bit instructions when one argument is float and the other is integer (AVX2 only)
+./\ y, *./\ y, =/\ y, ~:/\ y recoded for boolean arrays of rank >1
All atomic suffix scans f/\. y recoded for boolean arrays of rank >1
=/\ y and ~:/\ y recoded for boolean list arguments

J903 beta-f

fix blunder in assert.: sentence no longer executed if 0=9!:34''
cached namerefs expose some of the value's characteristics to combining modifiers: IRS and immediate-opening
x i.!.1 y assumes that x and y are both in nondescending order; faster when cells are integer lists. Supports IRS.
Name lookups using the search path recoded, reducing the number of searches; Bloom filter added to public locales to allow fast skipping of locales with few names

J903 beta-e

+/ y, <./ y , and >./ y for floating-point y with rank >1 rewritten to use less cache bandwidth, >2x faster
+/!.0 y performs compensated summation (addition with improved accuracy)
x +/@:*"1!.0 y performs the computation at twice the precision of floats

J903 beta-d

x ]\ y, x [\ y, and x ,\ y return a virtual result when x<0 and there is no shard
f/ y faster on boolean arrays of rank >1 when f is a primitive logical operation
+/ y rewritten for boolean y, faster
special code for x f/@:g y with atomic f & g rewritten. Most special cases removed, as they are no longer faster than the normal code. Special code is used only when g cannot be executed in place.
New foreigns 9!:4 and 9!:5 support nameref caching, a way of avoiding name-lookup overhead especially useful when you have many locatives or long search paths. Put ". :: 1:'9!:5(1)' at the top of your script to see if you can use the feature.
New foreign u 4!:8 (adverb) produces a cached reference to u
Old foreigns 2!:2 and 2!:3 are removed from the language
|. y executes in place where possible
u"n is identical to u when n equals the ranks of u. If you need to generate a compound, use u"v.
in for_xyz. loops, a virtual block is used for xyz, and xyz_index cannot be modified inside the loop
Code for atomic/"1 y when 1-cells of y have length 2 rewritten

J903 beta-c withdrawn

J903 beta-b withdrawn

J903 beta-a