NYCJUG/2008-05-13

From J Wiki
Jump to: navigation, search

Error messages, common beginner questions, statistics and data visualization for schools, ideas from other languages

Location:: Offices of BEST in Hoboken, NJ


Meeting Summary

We discussed the difficulties and advantages of J error messages - most people had objections. On this topic and others, it may be profitable to begin compiling beginners' questions; those attached here represent a start of this compilation.

On the topic of displaying and saving numeric matrixes, we saw a partial solution that came up when putting together notes on the problem.

We reviewed a proposal on writing a statistics and data visualization package for (middle and high) schools in both J and APL. This proposal comes from a friend in Australia who has gotten some enthusiasm for the idea from teachers at his son's school.

Finally, we talked about the necessity of being open to good ideas from other languages. Some of us need to be reminded that J is not the be-all and end-all of programming languages.

Agenda

             Meeting Agenda for NYC JUG 20080513
             -----------------------------------
1. Beginner's regatta: are J error messages suitable for beginners?  See
"Error Messages For Beginners".

Some other beginner questions: see "Questions from a Beginner".


2. Show-and-tell: displaying numeric matrixes: saving pictures automatically -
see "Save Viewmats.doc".


3. Advanced topics: project idea: Statistics and Data Visualization package
for schools - see "Statistics Project Proposal.doc".


4. Learning and teaching J: an outreach effort: haunting software discussion
boards - see "Reply to Criticism about J code for Pascal Triangle.doc".

Proceedings

Beginner's Regatta

Error Messages for Beginners

Thomas suggested that the short and simple answer to "Are J error messages suitable for beginners?" is "No".

This discussion was prompted by a request on the J forum for help with a short expression that was returning a domain error. It was evident that the poster was looking at parts of the phrase that were not in error because he didn't understand a few basic underpinnings of J error messages. Specifically, he didn't understand that the error location is indicated by three extra spaces in the display of the offending line.

However, the more puzzling aspect of these messages are the general characterization of different errors as, e.g. "domain" or "syntax" error. This is somewhat puzzling to experienced J users as well, though more as a matter of a more abstract dispute about exactly what general type of error pertains in a given situation. For instance, sometimes a syntax error obscures what is more properly a value error. [Does anyone have a simple example of this to illustrate the problem?]

In practice, though, I think each of us is prone to our own characteristic errors and we learn what to look at for our most common ones.

FAQ for Beginners

There seems to be a few very common areas in which J novices have trouble. Some of these are task-oriented, like "How do I read a .csv file into a variable?" whereas others are more general, like "Why is the order of execution so weird?" I propose to start accumulating these questions with an eye to putting together a FAQ for beginners.

Here's a [[[Guides/BeginnerQuestionsCompilation]]: start on this].

Show-and-Tell

We discussed issues with generating and saving images based on numeric matrixes. There's a problem with displaying an image using the handy "viewmat" utility: a sequence of commands to display and save an image will work fine when entered in a J session but will fail to save the image when run from within a script. Apparently the image is not actually displayed until the J session returns to user-input mode so attempting to save it within a script gets only a blank image.

Here's a handout where I explored this problem.

Attempting to Save Viewmats

We can easily, interactively, generate, view, and save a picture displayed with "viewmat": SinSqr.png
   load 'playTileMat080402.ijs'
   require 'viewmat media/platimg'
   coinsert 'jgl2'
   glqall=: (|. $ [:glqpixels 0 0&,)@glqwh
   TD=: 'C:\amisc\J\NYCJUG\200805\'
   sm0=. (2-/\1 o. i:3j999)+/2 o. i:5j999
   viewmat sm0
   (glqall'') writeimg TD,'sinSqr.png'
516960
But there's a problem if we attempt to run a sequence of phrases like this in order to save a sequence of pictures programmatically: SinSqr1NotImmed.png
(glqall'') writeimg TD,'sinSqr2.png' [ viewmat sm2=. sm1 +/ . * |:sm1 [ wd 'reset' [ (glqall'') writeimg TD,'sinSqr1.png' [ viewmat sm1=. sm0 +/ . * |:sm0

Yes, nothing shows up in the file.

If I repeat the first sequence above interactively: SinSqr1.png
   viewmat sm1=. sm0 +/ . * |:sm0
   (glqall'') writeimg TD,'sinSqr1.png'
518400

Dan Bron had run into this same problem as we can see on his J wiki page DanBron/Snippets/PrettyPictures:


Oleg provided a mechanism to save viewmat images to PNG files, so I feel that the following should generate all the image files for this table. However, it does not; I just get a bunch of empty white images.

Debugging indicates that it is probably a repaint/selected window issue. I do not know how to fix it. -- Dan Bron 2007-03-16 13:42:32

Well this steps into the uncharted treacherous territory of sync programmatic execution and async GUI. So I guess to do it like that, the use of timer might help. But that's an overkill.

I recommend using something that is proven and supported like the animate addon. You can designate each picture as a step in the animation. And all will be saved nicely in a series of files. I guess you could take it further by making code that will convert your boxed list of picture verbs into a script for animate automatically. Or have just one new animate script that will accept a list of picture verbs in some form and derive number of steps, etc. Maybe an Options dialog. -- Oleg Kobchenko 2007-03-16 14:35:59


One Solution

One way to overcome this problem is to avoid using "viewmat":

Load a customized palette ADJPAL: ADJPAL.png
   load 'logo'
NB. Show the palette:
   ADJPAL viewmat i. 16 16
   (glqall'') writeimg TD,'ADJPAL.png'
810000
Now use this palette to show the picture the way we've been doing it: Mm0usual.png
   $mm0=. +/~1 o. (o. 0.5)+i:3j999
1000 1000
   ADJPAL viewmat mm0
   setsize_jviewmat_ 1000 1000
   (glqall'') writeimg TD,'mm0usual.png'
4000000
Now use this palette with functions from the "images" library: Mm0wi.jpg
NB. Need to scale matrix to integers 0 to 255:
   (<./,>./),mm0
_1.979985 1.999991
   sc00FF
[: <. 0.5 + 0 255 scaleNums ]
   $ADJPAL{~sc00FF mm0
1000 1000 3
   (ADJPAL{~sc00FF mm0) write_image TD,'mm0wi.jpg'
47094

A Better Solution

David Mitchell came up with a fix for this problem: redefine "vmrun" in the "jviewmat" namespace.

from	David Mitchell <davidmitchell@att.net>
reply-to	Programming forum <programming@jsoftware.com>
to	Programming forum <programming@jsoftware.com>
date	Tue, Dec 16, 2008 at 9:12 AM
subject	Re: [Jprogramming] savemat_jviewmat_
mailing list	programming.jsoftware.com Filter messages from this mailing list
	
Perhaps this is too simple or has side effects I haven't encountered, but it
seems to work for me

   open'viewmat'

change

NB. =========================================================
vmrun=: 4 : 0
...
wd 'pmovex ',": 0 >. fx,fy,dif + fw,fh
wd 'pshow'

to
NB. =========================================================
vmrun=: 4 : 0
...
wd 'pmovex ',": 0 >. fx,fy,dif + fw,fh
jvm_show '' NB. DM 2008 12 16
wd 'pshow'

Save and load viewmat.ijs .

Run test_viewmat'' .  On Vista 32, for me, Viewmat now displays correctly and
saves a good BMP.

--
Regards,
David Mitchell

This change can be made to the file "viewmat.ijs" in the ~system/classes/view directory. This change has not been thoroughly tested for absence of side-effects.

Advanced Topics

Learning and Teaching J

Lately, as part of my SIGAPL volunteer outreach effort, I've been haunting software discussion boards with the idea of raising the profile of array-processing languages, J in particular. It's interesting to see how much overhead people are used to even when doing very simple things, so I take the opportunity to point out how incredibly simple it would be in J.

In order to avoid being only obnoxious, I also try to help them with their problem in their language of choice (though it may be the language foisted on them by programming class requirements). Recently, I got a rather vehement response to my nudging and I attempted to reply to it as nicely as I could. The topic was how to generate rows of Pascal's triangle in order to have the coefficients of a polynomial expansion; the original attempt was in C.

I present here a summary of the discussion this provoked because it illustrates how thinking about the problem in J can help with the formulation of a solution even in another language. John raised the point here that we should keep in mind how J is not the Ultimate Solution and that good programmers come up with amazing solutions using all sorts of tools; he also had some critical remarks about the piece of Haskell code presented by the other party in this exchange.

Reply to J Criticism about Pascal’s Triangle Code

In a discussion forum about writing code to generate Pascal’s triangle, I used the opportunity to promote J. The following response was posted:
That's because J is deliberately cryptic and hides all the hard work behind pre-written combinators and functions, in this case ! being "choose". Given the equivalent of these two defined in Haskell:

comb n k = foldl' (\x y -> ((n - x + 1) * y) `div` x) 1 [1..k]
table f xs ys = foldr ((:) . (`map` ys) . f) [] xs[/code]

Then producing Pascal's triangle is as simple as  join (table comb) [0..]

But overall a much simpler solution is

iterate (\xs -> zipWith (+) (0:xs) (xs++[0])) [1]

My reply to this follows: I suppose anything is cryptic if you make no effort to understand it.

Even so, your statement is very odd. Apparently you are under the mis-apprehension that your failure to understand is the fault of a succinct, powerful notation and this means that the language was "deliberately" designed to be hard to understand.

Why would you even think something like this? I mean, why would someone deliberately make something difficult to understand? Most people I know strive for clarity. Do you think that "1+1" is cryptic because it uses a symbol "+" instead of the word "plus"? I suppose it is if you don't know math.

The other odd thing you seem to imply is that avoiding hard, unnecessary work is a bad thing. So, you're against the idea of carefully designed, well-tested, powerful, pre-written functions that you can combine in a clear, logically consistent manner?

This is all the more puzzling as your initial answer

join (table comb) [0..]

apparently very closely resembles the J solution of

i. !/ i.

because each "i." generates a vector of consecutive integers and "/" - similarly to (table comb) - combines these vectors in a table using "!". The "!" function, with a right and left arguments x and y returns the number of ways x things can be chosen from y possibilities - pretty much a direct statement of what you are doing when determining the coefficient of a polynomial expansion.

There are, of course other ways to do this more algorithmically and less mathematically as you appear to show in your final example. I say "appear" because that example is rather cryptic insofar as it uses the pre-written function "zipWith".

In J, I could write something I presume is similar to this:

   pascalNewRow=: 3 : '1,(2+/\y),1'

based on the following rule for generating a new row of the triangle from the previous one: start with 1; add together adjacent pairs for the middle section, and end with 1. The above expression "2+/\" adds together each adjacent 2 numbers across the first dimension of an array; the "3 :" defines a function. Once you know that, the code is a straightforward expression of the rule.

Similarly to the final expression in Haskell, we iterate the function "pascalNewRow" by using an iterator expressed by some symbols, "^:" instead of the word "iterate". So, the expression

   pascalNewRow^:10 ] 1

returns the first ten rows of Pascal's triangle (starting with a "seed" of 1).

I find this much less cryptic than the Haskell expression but, of course, I've made the effort to understand it.

In any case, this has wandered far off topic. Why don't I conclude by giving a solution to the original problem? Taking my J code as a template, start by defining a function to add adjacent pairs of a vector:

int *addAdjacentPairs( int vlen, int *vec)
{   int vc, win2[2];
    win2[0]=0;
    win2[1]=vec[0];
    for(vc=0;vc<=vlen;vc++)
    {   vec[vc]=win2[0]+win2[1];
        win2[0]=win2[1];
        win2[1]=vec[vc+1];
    }
// Danger: returned vector is 1 longer than input: allocate accordingly.
    return 0;
}

and simply use it in a loop:

// PascalTriangle.c: print some rows of Pascal's triangle.

#include<stdio.h>

int main(void)
{   int vec[21];
    int ii,vc,nn;
    printf("How many lines (up to 20) do you want? ");
    scanf("%d",&nn);
// Should verify result and handle errors but this is why I don't like these
// languages that don't handle arrays natively so I won't bother.
    printf("\n");

    for(ii=0;ii<nn;ii++)
    {   vec[ii]=1;
        for(vc=0;vc<=ii;vc++) printf("%i ",vec[vc]);
        printf("\n");
        addAdjacentPairs( ii, vec);
     }
    return 0;
}

This provides a nice example of how solving the problem with a high-level notation leads to a more elegant solution even in a lower-level language.

Scan of Meeting Notes

MeetNotes080513 30.jpg