Guides/Named Noun Internals

From J Wiki
Jump to: navigation, search

The following is an overview of the internals of J named nouns. The examples assume J32, where ints and pointers are both 4 bytes.

Memory Foreigns

The following definitions are used:

require 'jmf'

memr=: 15!:1        NB. read memory: address,byte_offset,count [,type]
memw=: 15!:2        NB. write memory
mema=: 15!:3        NB. allocate memory
symget=: 15!:6      NB. get address of locale entry for name
symset=: 15!:7      NB. set name to address header
allochdr=: 15!:8    NB. allocate header

Named Nouns

A J named noun is implemented in three parts: a symbol table entry, a header, and the data. The symbol table entry points to the header which points to the data.

Suppose a new noun abc is defined as:

abc=: 10 + i.2 3

Then, a symbol table entry is created, which starts with two pointers:

1. pointer to name abc 1. pointer to header

The header is list of 7 integers, followed by the shape which is itself a list of integers. In this case:

40 0 80 4 1 6 2 2 3
1. offset from header to data

1. internal flags 1. reserved size for data - at least large enough to store data 1. data type 1. reference count 1. data length (*/shape) 1. rank 1. shape (list of integers)

Note that in this case, the reserved size for the data is 80 bytes, which is larger than the actual size of 24 bytes.

The data is stored in ravelled order. Typically, it is stored immediately after the header, but this is not required. For example, it would not be the case when mapping files that only contain data.

The following verb returns the symbol table pointers, fixed header, shape and data:

viewnoun=: 3 : 0
s=. symget <y               NB. pointer to symbol table entry
sys=. memr s,0 2 4          NB. pointers to name, header
h=. 1{sys                   NB. header pointer
fheader=. memr h,0 7 4      NB. fixed header
'offset flag size type refcount len rank'=. fheader
shape=. memr h,28,rank,4    NB. shape
data=. memr (h+offset),0,len,type
sys;fheader;shape;data
)

For example:

   viewnoun 'abc'
┌───────────────────┬───────────────┬───┬─────────────────┐
│145020288 146347416│40 0 80 4 1 6 2│2 3│10 11 12 13 14 15│
└───────────────────┴───────────────┴───┴─────────────────┘

Now define def as copy of abc. Note that the header pointer for def is the same as for abc, i.e. both nouns point to the same memory; and that the reference count is incremented to 2:

   def=: abc
   viewnoun 'abc'
┌───────────────────┬───────────────┬───┬─────────────────┐
│145020288 146347416│40 0 80 4 2 6 2│2 3│10 11 12 13 14 15│
└───────────────────┴───────────────┴───┴─────────────────┘
   viewnoun 'def'
┌───────────────────┬───────────────┬───┬─────────────────┐
│146099384 146347416│40 0 80 4 2 6 2│2 3│10 11 12 13 14 15│
└───────────────────┴───────────────┴───┴─────────────────┘

Now redefine def as a vector. This creates a new memory area for def and sets the rank and shape.

   def=: ,def
   viewnoun 'def'
┌───────────────────┬───────────────┬─┬─────────────────┐
│146099384 145010112│32 0 24 4 1 6 1│6│10 11 12 13 14 15│
└───────────────────┴───────────────┴─┴─────────────────┘

Now add a new row to abc. The new data fits into the reserved size, so there is no data movement, i.e. the header pointer is unchanged:

   abc=: abc,7
   viewnoun 'abc'
┌───────────────────┬───────────────┬───┬───────────────────────┐
│145020288 146347416│40 0 80 4 1 9 2│3 3│10 11 12 13 14 15 7 7 7│
└───────────────────┴───────────────┴───┴───────────────────────┘

Creating a J noun

We can create a noun by creating the data, the header and the symbol table entry. For example, we create noun mydata with value i.2 3 :

   d=: mema 24
   (i.6) memw d,0 6 4
   h=: allochdr 40
   hdr=: (d-h),0 24 4 1 6 2 2 3
   hdr memw h,0 9 4
   mydata=: symset h
   mydata
0 1 2
3 4 5

   viewnoun'mydata'           NB. in this case, header and data are not contiguous
┌───────────────────┬─────────────────────┬───┬───────────┐
│149241240 146224904│_9536336 0 24 4 2 6 2│2 3│0 1 2 3 4 5│
└───────────────────┴─────────────────────┴───┴───────────┘

Mapped Files

A mapped noun is a noun where the data, or both the data and header, are mapped to a file. Changing the noun changes the file.

The verb map_jmf_ does this mapping - this verb handles several minor details related to mapping a noun, but the core logic is straightforward.

A mapped file can have one of two forms:

  • it contains both header and data, and hence is self-describing
  • it contains only the data, in which case a suitable header is constructed when mapping the file

When mapping a file, the verb map_jmf_ maps the file to get a memory address to it, and creates a new symbol table entry. Then,

  • for self-describing files, the symbol table entry points to the file memory address, which is the address of the header
  • for data-only files, a new header is created in memory, and assigned appropriate values. The address of this new header is set in the symbol table entry, while the memory address to the file is set as an offset in the new header.