User:Devon McCormick

From J Wiki
Jump to navigation Jump to search

Devon McCormick

Founder - New York City J Users Group

I curate the SIGAPL (Special Interest Group on APL) page under the auspices of ACM's SIGPLAN (Special Interest Group on Programming LANguages.)

Email: <<MailTo(devon AT SPAMFREE acm D0T org)>>

J Wiki Pages

Following are largely ones I've written but also includes those related to areas in which I'm interested.

o Trees: an exposition on the great utility of using the parent-index method of representing trees in J.

o Array-thinking: an explanation of the fundamental difference between array-thinking and scalar-thinking and how the former helps us think about algorithms.

o An attempt at a minimal subset of J to ease the shock of learning all that there is.

o An example of using J for beginning financial data analysis.

o OLEExcel - manipulate Excel worksheets using OLE interface: Scripts/OLEExcel

o WS - save J variables to file and retrieve them: Scripts/File J Variables

o An early (unsuccessful) attempt to use the J database "JDB" on the "Netflix Challenge" data. It appears to have been unsuccessful because this dataset was too large for JDB to handle.

o Start of an attempt to detail J equivalents of basic operations in Microsoft Excel.

o An essay contrasting visual understanding with symbolic understanding.

o BigFiles - work with files larger than 2 GB: Scripts/Working with Big Files. This is now largely obsolete with the advent of 64-bit versions of J that have extended the scope of the file primitives to handle larger files.

Parallelizing J

o Some general limitations to consider when parallelizing code.

o General discussion of multi-core parallelization in J; general ideas on running threads from J.

o Simple example of writing parallel code in J.

o Detailed example of how to simply parallelize code in J. This assumes the (common) sort of problem where coordination of the various parallel processes can be handled at the point where a "master" process spins them off and that the problem domain lends itself to "coarse" parallelism. I currently run two multi-core J routines almost every day using this technique and it's worked well for years now with substantial speed-ups and little difficulty.

o Complete examples of parallel J code presented at the APL2010 conference in Berlin, Germany, in September, 2010. These are the two examples of parallel code tasks, mentioned above, as well as an implementation of mutex (mutual exclusion), all written in J (version 6.02).

o Ideas on constructing arbitrarily-large datasets for comparing the effectiveness of different parallel implementations.

o "Red or Black" game simulation -> parallelization example NYCJUG/2011-04-12/RedOrBlackGameSimulation

Presentations About J

Feel free to take anything from these for your own presentation.

o Slides of my talk on "Array Thinking" at the 2014 J Conference in Toronto.

o An introduction to J presented to Hacker School in New York in September, 2014.

o "J: the Hard Parts" - a fairly lengthy talk explaining some of the parts of J that most people find the most difficult to comprehend, given to the N-Languages Meetup group on January 21, 2014.

o Presentation of J to a "Functional Programmers" Meetup group on September 20, 2011.

o Short (five minute) introduction of J to general programming group at a language slapdown which was supposed to be held on November 17, 2010 but was canceled.

o Development of code for "Diffusion-Limited Aggregation" from an initial version coded in a more conventional format to a more "J-like" version

o "Short is Sweet" - An exposition on the importance of succinct expression (from a talk for the Kdb User Group on 12/9/2009).

o Slides from my lightning J introduction for the "Seven Languages in Seven Months" Meetup on 1/21/2014 (pdfs) in two parts.

Other Links: Wiki Tips, Foreigns, Starting J, Command-line Options

Here are some tips for working with pages on this wiki.

This page lists all J foreign conjunctions. These are used to access resources external to the J language itself, things like I/O, times and timing, object characteristics, and such.

This is the list of steps J follows upon start up.

Here are command-line parameters for starting J with various options.

Non-breaking Space (NBSP)

Sometimes when loading a text file as a J script, we find that it contains characters which give a "spelling error". Here is an example of how to fix the file when it contains a non-breaking space. This character, also called "NBSP", is meant to prevent an automatic line-break on the character, which looks like a space (32{.a.), to prevent breaking up parts of a token represented by multiple space-delimited words.

It can get into text that was cut-and-pasted from a web page and can be annoying because it's hard to see what the problem is. Here is an example of discovering and fixing this problem in a file.


Here we see the initial error upon loading the file: it fails with a "spelling error".
The caret indicates an odd-looking character. Why is this a spelling error?
Fortunately, we are running J within an interactive emacs editor session, so we can easily select the odd-looking character that seems to have caused the problem.

Looking up this character in a., we see that the result is two numbers, showing us that it is a multi-byte character.

If we replace this pair of characters with a space and write the result to file, we are now able to load it without error.

This image is of J running under the emacs editor so we can see an indicator of the NBSP - it's the cyan lines which look like underscores between some of the characters on the line of J code. Since we are in a live editor session, we can also copy an instance of the bad character, put it in quotes and look it up in a. as we see above. Getting back two numbers when we do this shows us that this is a Unicode character represented in two bytes.

We can then replace this pair of bytes with a single space using the standard rplc function and write it back into a new file. The subsequent successful loading of this new file without error shows us that our fix worked.

Solving Problem Reading Images in JConsole

I've had a recurring issue with many J upgrades: they break the code I most often use. What breaks is the call to underlying verb wdreadimg, like this:

   herbs=. read_image 'Herbs1.jpg'
|domain error: cd
|   m=.    wdreadimg(utf8,y);wh=.2$2-2

The immediate cause of the problem is in the definition of wdreadimg:

'"jqt.dll" wdreadimg >+ x *c *i'&cd

In the 8.04 version of J where this code still worked, that verb was defined this way:

'"E:/Users/DevonMcC/j64-804/bin/jqt.dll" wdreadimg >+ x *c *i'&cd

However, the exact sequence by which this gets defined differently is obscure, and doing the equivalent assignment in J 8.07 revealed the absence of the jqt.dll file in the 8.07 bin sub-folder. Copying this there did not help.

The break-through came from looking at an older email chain where I'd written to the J Forum about my problems. In this chain, Ric Sherlock discovered that the underlying image reader from the jqt package, readimg_jqtide_, worked for .png files but not for .jpg files. He eventually came up with this fix:

from:	bill lam <> via 
to:	Programming forum <>
date:	Jun 15, 2017, 12:49 AM
subject:	Re: [Jprogramming] Location of "readimg"

I had tried again and might had found a solution. Apparently just
loading qtlib under jconsole failed to locate the qt plugin folder. It
needs to set env QT_PLUGIN_PATH to the path of J bin directory where
contains the Qt dll files.  This can be automated by adding a line
into jconsole.cmd

@rem load J console
@rem run from the J install directory
@cd /d %~dp0
@set QT_PLUGIN_PATH=%~dp0bin

then click on jconsole.cmd to launch

    $readimg_jqtide_ jpath '~temp/toucan.png'
144 200
    $readimg_jqtide_ jpath '~temp/toucan.jpg'
144 200

With this insight, I modified my J-launcher utility J64-807.bat to look like this:

Rem J64-807.bat: start 64-bit jconsole v8.07 of J (under emacs).
Rem cd "%BASEDSK%\Users\devon_mccormick\j64-807\bin"
cd "%BASEDSK%\Users\DevonMcC\j64-807\bin"
Rem "-interactive" signals not a batch run
..\jconsole.cmd %BASEDSK%\amisc\JSys\J8\DHMConfig.ijs -interactive -js "1!:44 '\amisc\work\neuralNets\'"

(The bit at the end following "-js" is something I change as my default working area changes)

To make this work properly by making jconsole.cmd process the command-line arguments I typically provide, I had to modify that file in the 8.07 installation directory to be like this:

@rem load J console
@rem run from the J install directory
@cd /d %~dp0
@set QT_PLUGIN_PATH=%~dp0bin
@bin\jconsole.exe %1 %2 %3 %4 %5 %6 %87 %8 %9

So, it looks like I now have a 8.07 installation that does not break my most-used code.

Feedback -- leave me /Comments